ML Polynomial Regression

ML Polynomial Regression: Modeling Curved Trends Without Overfitting

Polynomial regression extends linear regression to curved relationships by adding powers of the input feature (x^2, x^3, and so on): y = b0 + b1*x + b2*x^2. Even though the curve bends, it remains linear in its model weights, so fast least-squares solvers handle it easily.

Use polynomial regression when scatter plots clearly show acceleration, saturation, or U-shaped curves that a straight line misses.

Fitting a Polynomial Curve in Python

With NumPy (np.polyfit) or Scikit-Learn (PolynomialFeatures + LinearRegression), you pick the polynomial degree and fit the curve cleanly.

import numpy as np

# Sample braking distance (meters) vs speed (km/h) — follows quadratic curve
speed = np.array([20, 40, 60, 80, 100])
distance = np.array([6, 18, 38, 66, 102])

coeffs = np.polyfit(speed, distance, deg=2)
poly_fn = np.poly1d(coeffs)

print("Predicted braking distance at 70 km/h:", round(float(poly_fn(70)), 2))

Avoiding High-Degree Overfitting

Setting deg=2 or deg=3 captures smooth bends. Cranking deg=12 on 15 data points forces the curve to wiggle wildly through every noise point—scoring great on training data while failing badly on new inputs.

Common Mistakes to Avoid

• Choosing a high degree blindly: Always compare validation error across degrees 1, 2, 3, 4 and pick the simplest curve that generalizes well.
• Extrapolating high-degree polynomials: High-order terms like x^5 explode rapidly outside the training range.
• Forgetting feature scaling with PolynomialFeatures: Squaring or cubing unscaled large numbers causes numerical instability; scale x first inside a Scikit-Learn Pipeline.

Frequently Asked Questions

Is polynomial regression still considered linear regression mathematically?

Yes. The relationship between x and y is curved, but the equation is linear with respect to the unknown coefficients b0, b1, b2.

How do I implement polynomial regression cleanly in Scikit-Learn?

Chain make_pipeline(StandardScaler(), PolynomialFeatures(degree=2), LinearRegression()) so scaling and feature expansion stay leak-free.

What is a better alternative for complex non-linear curves?

For jagged or multi-feature curves, tree ensembles (RandomForestRegressor, GradientBoostingRegressor) or splines often generalize more safely than high-degree polynomials.