Tensors
Tensors are the n-dimensional arrays that flow through a neural network.
A tensor is just an n-dimensional array — the deep-learning generalisation of NumPy's ndarray, able to run on a GPU.
| Rank | Name | Example |
|---|---|---|
| 0 | scalar | a single number |
| 1 | vector | one row of features |
| 2 | matrix | a batch of samples |
| 3+ | tensor | images (H×W×channels) |
An image batch is a 4D tensor: (batch, height, width, channels).
Example
import numpy as np
scalar = np.array(5) # rank 0, shape ()
vector = np.array([1, 2, 3]) # rank 1, shape (3,)
matrix = np.array([[1, 2], [3, 4]]) # rank 2, shape (2, 2)
images = np.zeros((32, 28, 28, 3)) # 32 RGB images
print(scalar.ndim, vector.ndim, matrix.ndim, images.ndim)
# 0 1 2 4When to use it
- A deep learning engineer converts a numpy batch of images to a PyTorch tensor to pass it through a CNN without manual memory management.
- A researcher moves a tensor to GPU with .to('cuda') to accelerate matrix multiplications during training of a large language model.
- A data pipeline uses torch.stack to batch a list of individual sample tensors into a single 4D batch tensor before feeding the DataLoader.
More examples
Create tensors from data
Creates PyTorch tensors from a numpy array and using built-in constructors, showing the basic shapes and conversion from numpy.
import torch
import numpy as np
arr = np.array([1.0, 2.0, 3.0])
t_from_np = torch.tensor(arr)
t_zeros = torch.zeros(2, 3)
t_rand = torch.randn(3, 4)
print(t_from_np)
print(t_zeros.shape)
print(t_rand.shape)Tensor operations and gradients
Demonstrates automatic differentiation: requires_grad=True tells PyTorch to track operations so .backward() can compute gradients used in learning.
import torch
x = torch.tensor([2.0, 3.0], requires_grad=True)
y = (x ** 2).sum() # y = x0^2 + x1^2
y.backward()
print('Gradient:', x.grad) # dy/dx = [2x0, 2x1] = [4, 6]Move tensor to GPU if available
Detects GPU availability and moves both the batch tensor and the model to the same device, enabling GPU-accelerated training with no code changes.
import torch
device = 'cuda' if torch.cuda.is_available() else 'cpu'
print('Using device:', device)
batch = torch.randn(32, 784).to(device)
model = torch.nn.Linear(784, 10).to(device)
output = model(batch)
print(output.shape, output.device)
Discussion