Cross-Validation
Test on several different splits to get a more reliable performance estimate.
cross_val_score(model, X, y, cv=5)A single train/test split can be lucky or unlucky. k-fold cross-validation splits the data into k folds, trains on k-1 and tests on the last, rotating through all folds.
Result
You get k scores; their mean is a far more trustworthy estimate, and their spread shows stability.
Example
from sklearn.model_selection import cross_val_score
from sklearn.ensemble import RandomForestClassifier
import numpy as np
X = np.random.RandomState(0).rand(60, 4)
y = (X[:, 0] > 0.5).astype(int)
model = RandomForestClassifier(random_state=0)
scores = cross_val_score(model, X, y, cv=5)
print(scores) # 5 accuracy scores
print(scores.mean()) # average performanceWhen to use it
- A researcher uses 10-fold cross-validation instead of a single train/test split to get a stable accuracy estimate with a standard deviation on a small dataset.
- An ML engineer uses cross_val_score inside a hyperparameter search loop to compare model variants without any data leakage.
- A data scientist uses stratified k-fold to maintain class balance in each fold when evaluating a model on an imbalanced medical dataset.
More examples
Basic k-fold cross-validation
Runs 5-fold cross-validation and reports both the per-fold scores and the mean with standard deviation, giving a reliable estimate of generalisation.
from sklearn.model_selection import cross_val_score
from sklearn.ensemble import RandomForestClassifier
from sklearn.datasets import load_iris
import numpy as np
X, y = load_iris(return_X_y=True)
model = RandomForestClassifier(n_estimators=50, random_state=0)
scores = cross_val_score(model, X, y, cv=5)
print('CV scores:', scores.round(3))
print(f'Mean: {scores.mean():.3f} +/- {scores.std():.3f}')Stratified K-Fold for imbalanced data
Uses StratifiedKFold to ensure each fold contains the same proportion of positive and negative cases, critical for datasets with class imbalance.
from sklearn.model_selection import StratifiedKFold, cross_val_score
from sklearn.linear_model import LogisticRegression
from sklearn.datasets import load_breast_cancer
X, y = load_breast_cancer(return_X_y=True)
skf = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)
scores = cross_val_score(LogisticRegression(max_iter=5000), X, y, cv=skf)
print('Stratified CV:', scores.round(3))
print('Mean:', scores.mean().round(3))Cross-validated hyperparameter search
Searches over several max_depth values using cross-validation scores only, finding the optimal depth without ever exposing the test set.
from sklearn.model_selection import cross_val_score
from sklearn.tree import DecisionTreeClassifier
from sklearn.datasets import load_digits
import numpy as np
X, y = load_digits(return_X_y=True)
best_depth, best_score = None, 0
for depth in [2, 4, 6, 8, 10, None]:
clf = DecisionTreeClassifier(max_depth=depth, random_state=0)
cv = cross_val_score(clf, X, y, cv=5).mean()
print(f'max_depth={depth}: {cv:.3f}')
if cv > best_score:
best_score, best_depth = cv, depth
print('Best depth:', best_depth)
Discussion