Troubleshooting Failed and Stuck Releases

When a release is stuck pending-upgrade or an upgrade keeps failing, know how to read the real error, break the lock, and recover without nuking everything.

Sooner or later you meet a release stuck in pending-upgrade, or an upgrade that fails with a cryptic message. Panic-uninstalling loses history and can delete live resources. Here is the calm, senior recovery path.

Read the actual error first

The helm upgrade output truncates. Go to the source: helm status <release> shows the current phase, and kubectl describe / kubectl logs on the failing Pod or Job almost always reveal the real cause — an ImagePullBackOff, a failed migration hook, a missing Secret. Helm is usually the messenger, not the culprit.

The classic: stuck in pending-upgrade

If a helm upgrade is killed midway (CI runner dies, laptop closes), the release can be left in pending-upgrade and every retry refuses with "another operation is in progress." There is no operation — just a stale lock. Fix it by rolling back to the last deployed revision, which resets the status:

  • helm rollback <release> <last-good-revision> clears the stuck state and returns you to something known-good.
  • If even that refuses, the helm mapkubeapis plugin repairs releases broken by removed Kubernetes APIs, and in the worst case you can delete the specific sh.helm.release.v1.<name>.v<n> Secret for the bad revision.

Other frequent offenders

  • "field is immutable" — you changed a Deployment/StatefulSet selector or a Service's clusterIP. These cannot be patched; the resource must be recreated.
  • Ownership conflicts — "invalid ownership metadata" means a resource already exists but was not created by this release. Label/annotate it for adoption, or remove it.
Prevention beats cure: --atomic from the previous lesson means most failed upgrades roll themselves back and never leave a stuck release for you to untangle.

Example

Example · bash
# 1) Get the real story -- Helm status plus the failing Pod's events/logs
helm status my-web -n web
kubectl get pods -n web
kubectl describe pod -n web -l app.kubernetes.io/instance=my-web
kubectl logs -n web -l app.kubernetes.io/instance=my-web --tail=50

# 2) Release stuck in 'pending-upgrade'? Roll back to the last good revision
helm history my-web -n web            # find the last DEPLOYED revision
helm rollback my-web 4 -n web --wait  # clears the stale lock

# 3) Broken by removed Kubernetes APIs after a cluster upgrade
helm plugin install https://github.com/helm/helm-mapkubeapis
helm mapkubeapis my-web -n web

When to use it

  • An SRE encounters a release stuck in pending-upgrade and uses helm rollback to break the stuck state before attempting the upgrade again with a corrected values file.
  • A developer investigates a failed install by running helm get manifest to see which resources were applied and kubectl describe to read the events on the failing Pod.
  • A platform engineer uses helm history and kubectl get events to correlate a bad upgrade with a specific chart version and Kubernetes event timestamp, then rolls back.

More examples

Unstick a pending-upgrade release

Rolls back a release stuck in pending-upgrade, which resets its state to deployed and allows subsequent upgrade attempts.

Example · bash
# Release is stuck in pending-upgrade state
helm list --all -n production | grep pending

# Roll back to break the stuck state
helm rollback my-app -n production

# Verify release is now deployed
helm list -n production

Diagnose a failed install

Uses three commands to diagnose a failed install: Helm status for high-level info, get manifest to see what was applied, and events for root-cause signals.

Example · bash
# Check Helm release status
helm status my-app -n staging

# See the rendered manifests that were applied
helm get manifest my-app -n staging

# Check Kubernetes events for the namespace
kubectl get events -n staging --sort-by='.lastTimestamp' | tail -20

Force replace a stuck resource

Uses --force to delete and re-create resources that are stuck in a terminal state, or manually deletes a specific stuck resource so Helm can recreate it.

Example · bash
# When a resource is stuck and normal upgrade cannot proceed:
helm upgrade my-app ./myapp \
  -f values-prod.yaml \
  --force \
  -n production

# Or delete the specific stuck resource and let Helm re-create it:
kubectl delete pod my-app-migration-xyz -n production

Related

helm pro safe upgradeshelm pro debug templateshelm rollback

Discussion

  • Be the first to comment on this lesson.