Upgrading Clusters Safely

Read the deprecations, upgrade the control plane first, bump managed add-ons, then roll nodes one minor version at a time.

EKS lets you skip nothing: you upgrade one minor version at a time, and the order matters. A calm upgrade is a boring upgrade.

The safe sequence

  1. Read the release notes for removed APIs. Run kubectl with a tool like pluto or kubent to catch manifests using APIs that vanish in the target version — this is the number-one cause of broken upgrades.
  2. Upgrade the control plane first (aws eks update-cluster-version). The API server can run one minor version ahead of the kubelets, so this is safe.
  3. Update the core add-ons — VPC CNI, CoreDNS, kube-proxy — to versions compatible with the new control plane, before touching nodes.
  4. Roll the nodes. Managed node groups do a graceful rolling replace: they cordon, drain honoring PodDisruptionBudgets, and launch new-version nodes.

Protect workloads during the drain

  • Set a PodDisruptionBudget on every important Deployment so the drain never takes all replicas at once.
  • Make sure replicas are spread across nodes/AZs (topology spread) so replacing one node doesn't take a whole service down.
  • Watch out for extended support: staying on an old version past its window silently raises your control-plane bill.

Example

Example · bash
# 1. Scan for APIs removed in the target version BEFORE upgrading
pluto detect-all-in-cluster --target-versions k8s=v1.31.0

# 2. Upgrade the control plane one minor at a time
aws eks update-cluster-version --name demo --kubernetes-version 1.31

# 3. Bring core add-ons to a compatible version
aws eks update-addon --cluster-name demo --addon-name vpc-cni \
  --addon-version v1.18.5-eksbuild.1 --resolve-conflicts PRESERVE

# 4. Roll the managed node group (graceful cordon + drain + replace)
aws eks update-nodegroup-version --cluster-name demo --nodegroup-name ng-app

When to use it

  • A platform team reads the Kubernetes 1.30 migration guide, finds that PodSecurityPolicy is removed, and migrates all clusters to Pod Security Admission before upgrading.
  • An SRE team upgrades the control plane on a Friday afternoon, then waits until Monday to roll node groups, giving the weekend to catch any control plane regressions.
  • A DevOps engineer uses eksctl upgrade nodegroup with --force-upgrade=false to ensure nodes are only replaced when there are enough healthy nodes to run all workloads.

More examples

Check deprecated APIs before upgrade

Scans all live cluster resources for APIs deprecated or removed in the target version, surfacing manifest changes needed before the upgrade.

Example · bash
# Install pluto for API deprecation scanning
curl -Lo pluto.tar.gz https://github.com/FairwindsOps/pluto/releases/latest/download/pluto_linux_amd64.tar.gz
tar -xzf pluto.tar.gz
./pluto detect-all-in-cluster --target-versions k8s=v1.30

Upgrade control plane then add-ons

Shows the recommended upgrade sequence: control plane first, then add-ons updated to versions compatible with the new control plane.

Example · bash
# Step 1: upgrade control plane
eksctl upgrade cluster --name prod --version 1.30 --approve

# Step 2: upgrade managed add-ons
aws eks update-addon --cluster-name prod --addon-name vpc-cni \
  --addon-version v1.18.0-eksbuild.1
aws eks update-addon --cluster-name prod --addon-name coredns \
  --addon-version v1.11.1-eksbuild.4
aws eks update-addon --cluster-name prod --addon-name kube-proxy \
  --addon-version v1.30.0-eksbuild.3

Roll node groups after control plane

Rolls the node group to the 1.30 AMI after the control plane is healthy, then verifies all pods returned to Running state.

Example · bash
eksctl upgrade nodegroup \
  --cluster prod \
  --name workers \
  --kubernetes-version 1.30
kubectl get nodes -o wide
kubectl get pods -A | grep -v Running | grep -v Completed

Related

eks addonseks managed node groups

Discussion

  • Be the first to comment on this lesson.