ML Feature Scaling

ML Feature Scaling: Standardization and Normalization Without Leakage

Feature scaling puts numeric columns onto comparable ranges so large numbers like salary = 95000 do not drown out smaller numbers like age = 32. Algorithms that compute distances or gradient steps rely heavily on balanced feature scales.

Without scaling, K-Nearest Neighbors, K-Means clustering, SVMs, neural networks, and regularized linear models treat huge-scale columns as artificially more important.

StandardScaler vs MinMaxScaler

• StandardScaler (Z-score): Shifts each column to mean 0 and standard deviation 1 ((x - mean) / std). Works best for linear models, SVMs, and PCA.
• MinMaxScaler: Rescales values into [0, 1]. Works well for image pixels and bounded neural network inputs, though outliers squeeze normal values together.

from sklearn.preprocessing import StandardScaler
import numpy as np

X_train = np.array([[25, 45000], [35, 65000], [45, 95000]])
X_test = np.array([[30, 55000]])

scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)
X_test_scaled = scaler.transform(X_test)  # Transform only — never fit on test!

print("Scaled test row:", np.round(X_test_scaled, 3))

Preventing Data Leakage with Pipelines

Wrap your scaler and estimator inside sklearn.pipeline.make_pipeline(StandardScaler(), LogisticRegression()) so cross-validation automatically fits the scaler only on each training fold.

Common Mistakes to Avoid

• Calling fit_transform() on test data: Fitting the scaler on X_test or on the full dataset before splitting leaks test statistics into training.
• Scaling the target variable y unnecessarily: Usually you scale only input features X, not classification labels y.
• Spending time scaling features for Decision Trees: Decision Trees, Random Forests, and gradient-boosted trees split on thresholds (age > 30) and are completely invariant to monotonic scaling.

Frequently Asked Questions

Should I use StandardScaler or RobustScaler when my data has outliers?

Use RobustScaler, which centers using the median and scales by the interquartile range (IQR), preventing extreme spikes from squashing typical values.

Do I fit the scaler again when predicting new production rows?

No. Save the fitted scaler (or entire Pipeline) and call .transform() on incoming production inputs using the exact training mean and standard deviation.

Does MinMaxScaler guarantee test values stay inside [0, 1]?

No. If a new test row exceeds the training max or falls below the training min, its scaled value will lie outside [0, 1].