Cross-Validation

Test on several different splits to get a more reliable performance estimate.

Syntaxcross_val_score(model, X, y, cv=5)

A single train/test split can be lucky or unlucky. k-fold cross-validation splits the data into k folds, trains on k-1 and tests on the last, rotating through all folds.

Result

You get k scores; their mean is a far more trustworthy estimate, and their spread shows stability.

Example

Example · python
from sklearn.model_selection import cross_val_score
from sklearn.ensemble import RandomForestClassifier
import numpy as np

X = np.random.RandomState(0).rand(60, 4)
y = (X[:, 0] > 0.5).astype(int)

model = RandomForestClassifier(random_state=0)
scores = cross_val_score(model, X, y, cv=5)

print(scores)        # 5 accuracy scores
print(scores.mean()) # average performance

When to use it

  • A researcher uses 10-fold cross-validation instead of a single train/test split to get a stable accuracy estimate with a standard deviation on a small dataset.
  • An ML engineer uses cross_val_score inside a hyperparameter search loop to compare model variants without any data leakage.
  • A data scientist uses stratified k-fold to maintain class balance in each fold when evaluating a model on an imbalanced medical dataset.

More examples

Basic k-fold cross-validation

Runs 5-fold cross-validation and reports both the per-fold scores and the mean with standard deviation, giving a reliable estimate of generalisation.

Example · python
from sklearn.model_selection import cross_val_score
from sklearn.ensemble import RandomForestClassifier
from sklearn.datasets import load_iris
import numpy as np

X, y = load_iris(return_X_y=True)
model = RandomForestClassifier(n_estimators=50, random_state=0)

scores = cross_val_score(model, X, y, cv=5)
print('CV scores:', scores.round(3))
print(f'Mean: {scores.mean():.3f} +/- {scores.std():.3f}')

Stratified K-Fold for imbalanced data

Uses StratifiedKFold to ensure each fold contains the same proportion of positive and negative cases, critical for datasets with class imbalance.

Example · python
from sklearn.model_selection import StratifiedKFold, cross_val_score
from sklearn.linear_model import LogisticRegression
from sklearn.datasets import load_breast_cancer

X, y = load_breast_cancer(return_X_y=True)
skf = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)

scores = cross_val_score(LogisticRegression(max_iter=5000), X, y, cv=skf)
print('Stratified CV:', scores.round(3))
print('Mean:', scores.mean().round(3))

Cross-validated hyperparameter search

Searches over several max_depth values using cross-validation scores only, finding the optimal depth without ever exposing the test set.

Example · python
from sklearn.model_selection import cross_val_score
from sklearn.tree import DecisionTreeClassifier
from sklearn.datasets import load_digits
import numpy as np

X, y = load_digits(return_X_y=True)
best_depth, best_score = None, 0

for depth in [2, 4, 6, 8, 10, None]:
    clf = DecisionTreeClassifier(max_depth=depth, random_state=0)
    cv = cross_val_score(clf, X, y, cv=5).mean()
    print(f'max_depth={depth}: {cv:.3f}')
    if cv > best_score:
        best_score, best_depth = cv, depth

print('Best depth:', best_depth)

Discussion

  • Be the first to comment on this lesson.