ML Linear Regression

ML Linear Regression: Fitting Straight-Line Relationships

Linear regression models the relationship between an input feature x and a continuous numeric target y by fitting the best straight line: y = m * x + b. Here m is the slope (how much y changes per unit of x) and b is the intercept.

It is one of the fastest, most interpretable baseline models in Machine Learning for tasks like estimating delivery times from distance or sales from ad spend.

Computing Slope, Intercept, and R-Squared in Python

Python 3.10+ includes statistics.linear_regression(x, y) right in the standard library, and scikit-learn (LinearRegression) provides the full ML .fit() / .predict() API for larger workflows.

import statistics

ad_spend_k = [2, 4, 6, 8, 10]
revenue_k = [15, 24, 31, 42, 49]

model = statistics.linear_regression(ad_spend_k, revenue_k)
predicted_for_12 = model.slope * 12 + model.intercept
r_value = statistics.correlation(ad_spend_k, revenue_k)

print(f"Slope={model.slope:.2f}, Intercept={model.intercept:.2f}")
print(f"Predicted at 12k: {predicted_for_12:.2f}k | R^2={r_value**2:.3f}")

Checking Fit Quality

• R² (Coefficient of Determination): Closer to 1.0 means the line explains most variance in y; near 0.0 means a straight line fits poorly.
• Residuals: Inspect actual - predicted errors across x to confirm no curved pattern remains.

Common Mistakes to Avoid

• Extrapolating far outside training range: A straight line fit on ages 5–15 predicts nonsense for age 90.
• Confusing correlation with causation: A high R² shows two metrics move together, not that x causes y.
• Ignoring curved relationships: Fitting a straight line to U-shaped or exponential data leaves large systematic errors.

Frequently Asked Questions

What does Ordinary Least Squares (OLS) minimize?

OLS chooses the slope and intercept that minimize the sum of squared vertical distances (residuals) between actual points and the predicted line.

How do I run linear regression with Scikit-Learn?

Instantiate LinearRegression(), reshape X into a 2D array X.reshape(-1, 1), call model.fit(X_train, y_train), and call model.predict(X_test).

What if one outlier point skews my regression line?

Extreme outliers inflate squared errors heavily; inspect scatter plots first or use HuberRegressor for outlier-resistant fitting.