Broadcasting
NumPy stretches smaller arrays to match larger ones so shapes line up automatically.
Broadcasting is the rule that lets NumPy combine arrays of different shapes. A scalar or a smaller array is virtually 'stretched' to fit.
The rule
Compare shapes from the right. Dimensions are compatible when they are equal, or one of them is 1. The size-1 axis is repeated.
This is how you add a per-column offset to every row of a matrix in one line.
Example
import numpy as np
matrix = np.array([[1, 2, 3],
[4, 5, 6]]) # shape (2, 3)
col_offset = np.array([10, 20, 30]) # shape (3,)
print(matrix + col_offset)
# [[11 22 33]
# [14 25 36]]
print(matrix * 2) # scalar broadcasts to every elementWhen to use it
- A data scientist subtracts the per-feature mean vector (shape 5,) from a 1000x5 feature matrix in one line without tiling or looping.
- An image-processing pipeline normalises an RGB image array of shape (H, W, 3) by dividing by a (3,) channel-mean vector using broadcasting.
- A researcher adds a bias vector of shape (1, units) to a batch output matrix of shape (batch_size, units) in a forward-pass computation.
More examples
Scalar broadcast over array
Adds a scalar to a 2D array: numpy broadcasts the scalar to match the array's shape automatically.
import numpy as np
X = np.array([[1, 2, 3],
[4, 5, 6]])
print(X + 10) # 10 broadcast to every elementRow vector broadcast over matrix
Subtracts a (3,) mean vector from every row of a (3, 3) matrix using broadcasting, a core step in feature normalisation.
import numpy as np
X = np.array([[10, 20, 30],
[40, 50, 60],
[70, 80, 90]])
mean = np.array([10, 20, 30]) # shape (3,)
norm = X - mean # broadcast: subtract from each row
print(norm)Column vector broadcast
Multiplies each row of a matrix by its own weight using a column vector of shape (3, 1), demonstrating broadcasting along the column axis.
import numpy as np
scores = np.array([[80, 90], [70, 85], [95, 60]])
weights = np.array([[0.4], [0.6], [0.5]]) # shape (3, 1)
weighted = scores * weights # broadcast along columns
print(weighted)
Discussion