Logistic Regression
Despite the name, it is a classifier that outputs class probabilities.
Syntax
LogisticRegression().fit(X, y)Logistic regression predicts a category, not a number. It outputs a probability between 0 and 1, then picks a class.
Regression vs logistic
Linear regression draws a line through points; logistic regression draws a decision boundary separating classes, using an S-shaped (sigmoid) curve.
Example
from sklearn.linear_model import LogisticRegression
import numpy as np
# Hours studied -> passed exam (0/1)
X = np.array([[1], [2], [3], [5], [6], [7]])
y = np.array([0, 0, 0, 1, 1, 1])
model = LogisticRegression()
model.fit(X, y)
print(model.predict([[4]])) # [1] likely pass
print(model.predict_proba([[4]])) # [[0.38 0.62]]When to use it
- A credit-scoring team trains logistic regression on customer features to output the probability that a loan applicant will default.
- A medical researcher uses logistic regression with predict_proba to rank patients by disease risk and flag the top 10% for further screening.
- An email provider trains a binary logistic regression classifier on message features to separate spam from legitimate emails.
More examples
Binary logistic regression
Trains a binary logistic regression on the Breast Cancer dataset and reports test accuracy as a percentage of correctly classified tumours.
from sklearn.linear_model import LogisticRegression
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
X, y = load_breast_cancer(return_X_y=True)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.2, random_state=42)
model = LogisticRegression(max_iter=5000).fit(X_tr, y_tr)
print('Accuracy:', model.score(X_te, y_te).round(3))Probability scores for ranking
Uses predict_proba to get the probability of the positive class (malignant) and ranks patients by descending risk score.
from sklearn.linear_model import LogisticRegression
from sklearn.datasets import load_breast_cancer
import numpy as np
X, y = load_breast_cancer(return_X_y=True)
model = LogisticRegression(max_iter=5000).fit(X, y)
probs = model.predict_proba(X[:10])[:, 1] # P(malignant)
ranked = np.argsort(probs)[::-1]
print('Top 3 highest-risk samples:', ranked[:3])Multiclass logistic regression
Extends logistic regression to three classes using multinomial softmax, classifying Iris flowers into species.
from sklearn.linear_model import LogisticRegression
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
X, y = load_iris(return_X_y=True) # 3 classes
X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.25)
model = LogisticRegression(multi_class='multinomial', max_iter=200).fit(X_tr, y_tr)
print('Test accuracy:', model.score(X_te, y_te).round(3))
print('Classes:', model.classes_)
Discussion