Taints & Tolerations

Taints repel Pods from nodes; tolerations let specific Pods ignore matching taints.

Taints and tolerations work as a pair to keep Pods off the wrong nodes — the opposite direction from nodeSelector.

Taints on nodes

A taint marks a node as "do not schedule here unless you tolerate this." Each taint has an effect:

  • NoSchedule — new Pods without a toleration are rejected.
  • PreferNoSchedule — the scheduler avoids it if possible.
  • NoExecute — also evicts running Pods that don't tolerate it.

Tolerations on Pods

A toleration on a Pod says "I can handle that taint," allowing it to be scheduled there. This is how you reserve nodes for special workloads — for example dedicating GPU nodes to ML jobs.

Example

Example · yaml
# Taint a node from the CLI:
# kubectl taint nodes gpu-1 dedicated=gpu:NoSchedule

apiVersion: v1
kind: Pod
metadata:
  name: gpu-job
spec:
  tolerations:
    - key: "dedicated"
      operator: "Equal"
      value: "gpu"
      effect: "NoSchedule"
  containers:
    - name: trainer
      image: ml-trainer:1.0

When to use it

  • An ops team taints dedicated database nodes with role=database:NoSchedule so that only database pods with a matching toleration can run on them.
  • A team uses a NoExecute taint to drain a node for maintenance: existing pods without tolerations are evicted within the grace period.
  • A platform team uses PreferNoSchedule taints on spot instances so workloads are steered away unless no other nodes are available.

More examples

Taint a node

Adds a NoSchedule taint to worker-3 that prevents non-tolerating pods from being placed on it, then removes it.

Example · bash
# Add a NoSchedule taint
kubectl taint node worker-3 role=database:NoSchedule

# Verify the taint
kubectl describe node worker-3 | grep Taints
# Taints: role=database:NoSchedule

# Remove the taint
kubectl taint node worker-3 role=database:NoSchedule-

Toleration in pod spec

Adds a toleration matching the node taint so this database pod can be scheduled on the tainted dedicated node.

Example · yaml
spec:
  tolerations:
    - key: role
      operator: Equal
      value: database
      effect: NoSchedule
  containers:
    - name: postgres
      image: postgres:15

NoExecute taint for node drain

Cordons, drains (evicts all pods via NoExecute taint), then uncordons a node for safe maintenance.

Example · bash
# Cordon node so no new pods are placed
kubectl cordon worker-3

# Drain existing pods (adds NoExecute taint)
kubectl drain worker-3 --ignore-daemonsets --delete-emptydir-data

# After maintenance, uncordon
kubectl uncordon worker-3

Discussion

  • Be the first to comment on this lesson.