ML Multiple Regression

ML Multiple Regression: Predicting Outcomes from Multiple Features

Multiple regression predicts a numeric target from two or more input features at once: y = b0 + b1*x1 + b2*x2 + .... Real-world estimates rarely depend on a single factor—a house price depends on square footage, bedrooms, and neighborhood together.

Each learned coefficient bi measures how much the prediction changes when feature xi increases by one unit while holding every other feature constant.

Fitting Multiple Regression with Scikit-Learn

Pass a 2D feature matrix X (rows = samples, columns = features) and a 1D target array y into LinearRegression().fit(X, y).

from sklearn.linear_model import LinearRegression
import numpy as np

# Features: [area_sqm, bedrooms] -> Target: monthly_rent
X = np.array([
    [50, 1],
    [70, 2],
    [90, 2],
    [110, 3],
    [130, 3],
])
y = np.array([950, 1300, 1550, 1950, 2200])

model = LinearRegression().fit(X, y)
pred = model.predict([[85, 2]])

print("Coefficients:", np.round(model.coef_, 2))
print("Predicted rent for 85sqm, 2BR:", round(float(pred[0]), 2))

Handling Correlated and Categorical Features

If two input features carry duplicate information (for example, area_sqm and area_sqft), coefficients become unstable—drop redundant columns or apply Ridge regularization. For text categories like "city", encode them with OneHotEncoder(drop="first").

Common Mistakes to Avoid

• Passing raw unencoded strings into LinearRegression: Scikit-Learn expects numeric arrays; convert categorical strings with OneHotEncoder.
• Comparing raw coefficients without scaling: A feature measured in centimeters has a coefficient 100x smaller than one measured in meters; standardize features before ranking importance by magnitude.
• Using raw R² alone when adding many columns: Standard R² never decreases when you add junk columns; check adjusted R² or cross-validated RMSE on test splits.

Frequently Asked Questions

What is multicollinearity and why does it matter?

Multicollinearity happens when two or more predictor columns are strongly correlated with each other, making individual coefficient signs and sizes unreliable even if predictions look fine.

How do I evaluate multiple regression accuracy?

Report RMSE (Root Mean Squared Error) or MAE (Mean Absolute Error) on held-out test data because both metrics are expressed in the target's real units (dollars, minutes).

When should I switch from plain LinearRegression to Ridge?

Use Ridge regression whenever you have many correlated features or more columns than rows, because L2 regularization stabilizes weights.