Liveness, Readiness & Startup Probes Done Right

The three probes answer three different questions. Confusing them is the single most common cause of self-inflicted outages.

Kubernetes gives you three probes, and each answers a distinct question. Wiring the wrong one to the wrong endpoint is how teams accidentally take themselves down during a traffic spike.

One probe, one question

readiness — "Should this Pod receive traffic right now?" Failing it removes the Pod from Service endpoints but does not restart it.
liveness — "Is this container wedged and beyond recovery?" Failing it restarts the container.
startup — "Has this slow-booting app finished starting?" It gates the other two so a slow JVM boot is not mistaken for a hang.

The classic self-inflicted outage

Point a liveness probe at an endpoint that checks the database. The database has a brief hiccup, every Pod's liveness probe fails at once, Kubernetes restarts all of them simultaneously, and now you have a full outage instead of a blip. Rule: liveness must only test whether this process is healthy — never a downstream dependency. Downstream health belongs in readiness, which just pulls the Pod out of rotation until the dependency recovers.

Use a startup probe for slow boots

Instead of padding initialDelaySeconds on the liveness probe (which delays real crash detection forever), add a startup probe. Liveness and readiness stay dormant until it passes, then they run at their normal snappy intervals.

Example

apiVersion: v1
kind: Pod
metadata:
  name: probed-app
spec:
  containers:
    - name: app
      image: myorg/app:1.9.0
      ports:
        - containerPort: 8080
      # Gates the other probes until the app has booted (up to 30 x 2s = 60s)
      startupProbe:
        httpGet: { path: /healthz, port: 8080 }
        failureThreshold: 30
        periodSeconds: 2
      # 'Am I alive?' — process-local only, no DB call
      livenessProbe:
        httpGet: { path: /healthz, port: 8080 }
        periodSeconds: 10
        failureThreshold: 3
      # 'Can I serve traffic?' — MAY check dependencies
      readinessProbe:
        httpGet: { path: /ready, port: 8080 }
        periodSeconds: 5
        failureThreshold: 2

When to use it

A Spring Boot service uses a startupProbe allowing 5 minutes for JVM initialization before the liveness probe activates, eliminating false-positive restarts during boot.
A team separates the readiness probe path (/ready) from the liveness probe path (/healthz) so traffic is cut before the pod is restarted, giving the app time to drain.
A gRPC service uses a TCP socket probe since it does not expose HTTP endpoints, checking that the gRPC port accepts connections.

More examples

Startup then liveness separation

Uses startupProbe to allow a generous 5-minute boot window before liveness and readiness probes activate, preventing premature restarts.

containers:
  - name: app
    image: myapp:1.0
    startupProbe:
      httpGet:
        path: /ready
        port: 8080
      failureThreshold: 30
      periodSeconds: 10   # 5 minutes max startup
    livenessProbe:
      httpGet:
        path: /healthz
        port: 8080
      periodSeconds: 15
    readinessProbe:
      httpGet:
        path: /ready
        port: 8080
      periodSeconds: 5

TCP socket probe for gRPC

Uses TCP socket probes to check a gRPC server's port connectivity when no HTTP health endpoint is available.

livenessProbe:
  tcpSocket:
    port: 9090
  initialDelaySeconds: 10
  periodSeconds: 15
readinessProbe:
  tcpSocket:
    port: 9090
  initialDelaySeconds: 5
  periodSeconds: 10

Tune probe thresholds for production

Sets a high failureThreshold (5) so transient network blips or GC pauses do not immediately trigger a pod restart.

livenessProbe:
  httpGet:
    path: /healthz
    port: 8080
  initialDelaySeconds: 20
  periodSeconds: 10
  timeoutSeconds: 3
  successThreshold: 1
  failureThreshold: 5  # 50s of failures before restart

Discussion

Be the first to comment on this lesson.