Random Forests
Combine many decision trees into one strong, robust model.
Syntax
RandomForestClassifier(n_estimators=100)A single tree is shaky. A random forest trains many trees on random slices of the data and averages their votes — an ensemble.
Why it works
Individual trees make different mistakes; averaging cancels them out. Forests are accurate, resist overfitting, and rank feature importance for free.
Example
from sklearn.ensemble import RandomForestClassifier
import numpy as np
X = np.array([[1,1], [2,1], [3,2], [6,5], [7,6], [8,7]])
y = np.array([0, 0, 0, 1, 1, 1])
model = RandomForestClassifier(n_estimators=100, random_state=0)
model.fit(X, y)
print(model.predict([[5, 4]])) # [1]
print(model.feature_importances_) # importance per featureWhen to use it
- An insurance company uses a Random Forest to predict claim amounts because it handles non-linear relationships and outliers better than linear regression.
- A data scientist uses Random Forest feature importances to shortlist the 10 most predictive variables from a 200-feature dataset before building a simpler model.
- A manufacturing plant deploys a Random Forest for real-time quality control because it is robust to noisy sensor readings and rarely overfits.
More examples
Train a Random Forest classifier
Trains a 100-tree Random Forest on cancer data and reports test accuracy, typically outperforming a single decision tree.
from sklearn.ensemble import RandomForestClassifier
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
X, y = load_breast_cancer(return_X_y=True)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.2, random_state=1)
rf = RandomForestClassifier(n_estimators=100, random_state=1).fit(X_tr, y_tr)
print('Accuracy:', rf.score(X_te, y_te).round(3))Feature importance ranking
Extracts and sorts feature importances to find the ten most predictive features in the Breast Cancer dataset.
from sklearn.ensemble import RandomForestClassifier
from sklearn.datasets import load_breast_cancer
import pandas as pd
data = load_breast_cancer()
rf = RandomForestClassifier(n_estimators=100, random_state=0).fit(data.data, data.target)
imp = pd.Series(rf.feature_importances_, index=data.feature_names)
top10 = imp.nlargest(10)
print(top10)Tune n_estimators and max_depth
Compares cross-validated accuracy for different numbers of trees to find the sweet spot between performance and compute cost.
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import cross_val_score
from sklearn.datasets import load_digits
import numpy as np
X, y = load_digits(return_X_y=True)
for n_trees in [10, 50, 100, 200]:
rf = RandomForestClassifier(n_estimators=n_trees, random_state=0)
cv = cross_val_score(rf, X, y, cv=3).mean()
print(f'n_estimators={n_trees}: CV accuracy={cv:.3f}')
Discussion