Scaling and Normalization
Put features on a comparable scale so no single feature dominates by magnitude.
Syntax
StandardScaler().fit_transform(X)Features often have wildly different ranges (age 0-100 vs income 0-100000). Many algorithms are thrown off by this, so we scale them.
Two common scalers
- StandardScaler — subtract the mean, divide by the standard deviation (mean 0, sd 1).
- MinMaxScaler — squeeze values into the range 0 to 1.
Distance-based models (KNN, k-means) and neural networks especially need this.
Example
from sklearn.preprocessing import StandardScaler
import numpy as np
X = np.array([[20, 40000],
[40, 80000],
[60, 120000]], dtype=float)
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)
print(X_scaled)
# each column now has mean 0 and sd 1
# [[-1.22 -1.22]
# [ 0. 0. ]
# [ 1.22 1.22]]When to use it
- An engineer applies StandardScaler to age and income features before training an SVM so that the 0-100 age range does not dominate the 10k-1M income range.
- A neural-network trainer scales all inputs to [0, 1] using MinMaxScaler to match the sigmoid output range, improving gradient flow.
- A data pipeline fits scalers on training data only and uses transform on the test set to prevent information from the test set leaking into preprocessing.
More examples
StandardScaler: zero mean, unit variance
Fits StandardScaler on training data only and applies the same learned mean/std to the test set, the correct way to avoid data leakage.
import numpy as np
from sklearn.preprocessing import StandardScaler
X_train = np.array([[25, 50000], [34, 80000], [47, 120000]])
X_test = np.array([[30, 65000]])
scaler = StandardScaler()
X_train_s = scaler.fit_transform(X_train)
X_test_s = scaler.transform(X_test) # fit already done
print(X_train_s)MinMaxScaler: range [0, 1]
Scales features to the [0, 1] interval by computing (x - min) / (max - min) per column, suitable for algorithms sensitive to feature magnitude.
from sklearn.preprocessing import MinMaxScaler
import numpy as np
X = np.array([[10, 200], [50, 400], [30, 100]])
scaler = MinMaxScaler()
X_scaled = scaler.fit_transform(X)
print(X_scaled)Scaler inside a Pipeline
Chains StandardScaler and KNN inside a Pipeline so scaling is always applied before prediction and cross-validation handles the fit/transform split automatically.
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.neighbors import KNeighborsClassifier
from sklearn.datasets import load_iris
X, y = load_iris(return_X_y=True)
pipe = Pipeline([
('scale', StandardScaler()),
('knn', KNeighborsClassifier(n_neighbors=5))
])
pipe.fit(X, y)
print(pipe.score(X, y))
Discussion