Probes, Requests and Limits
Liveness restarts a stuck container, readiness takes a pod out of rotation, and requests and limits decide its CPU and memory.
livenessProbe: { httpGet: { path: /healthz, port: http }, periodSeconds: 10, failureThreshold: 3 }A probe is a check the kubelet (the Kubernetes agent on each machine) runs against your container on a schedule. There are three, and they do different jobs.
The three probes
A liveness probe decides whether to restart the container. A readiness probe decides whether the pod receives traffic: when it fails, the pod's address is taken out of the Service and put back when it passes again. A startup probe runs only while the container is starting, and liveness and readiness are held back until it succeeds.
The defaults are worth knowing: periodSeconds 10, timeoutSeconds 1, failureThreshold 3, initialDelaySeconds 0, successThreshold 1. An HTTP probe connects to the pod's own IP address, so a server bound to 127.0.0.1 fails every probe.
Keep dependencies out of liveness. A liveness probe that queries the database restarts every replica at once the next time the database is slow, which is the opposite of what you want. That check belongs in readiness.
Requests and limits
A request is the amount reserved for the container, and the scheduler uses it to pick a machine. A limit is the ceiling. CPU is counted in millicpu, where 100m is a tenth of a CPU, and a CPU limit is enforced by throttling: the kernel stops handing the container CPU time once it reaches the limit, and requests queue behind it. A memory limit is enforced by killing the container, which appears as OOMKilled with exit code 137.
Example
When to use it
- Stopping a deploy from sending traffic to a pod whose connection pool has not opened yet.
- Letting a slow-starting service boot without weakening the liveness check.
- Diagnosing pods that restart every few minutes under load but pass every check when idle.
- Choosing CPU and memory numbers that get a pod scheduled without it being throttled or OOM-killed.
More examples
All three probes, plus resources
The startup probe is what lets liveness stay aggressive. Without it, a service that takes two minutes to run migrations needs a long initialDelaySeconds, and that delay then also applies to a pod that wedges after an hour. Leaving the CPU limit off is a deliberate choice: the request reserves CPU and the pod can use spare capacity instead of being throttled during a traffic spike.
containers:
- name: api
image: registry.example.com/api:1.4.2
ports:
- name: http
containerPort: 8080
startupProbe: # 30 * 10s = up to 5 minutes to boot
httpGet:
path: /healthz
port: http
periodSeconds: 10
failureThreshold: 30
livenessProbe: # process-only check, no dependencies
httpGet:
path: /healthz
port: http
periodSeconds: 10
timeoutSeconds: 2
failureThreshold: 3
readinessProbe: # may check the database and the cache
httpGet:
path: /readyz
port: http
periodSeconds: 5
failureThreshold: 2
resources:
requests:
cpu: 250m
memory: 256Mi
limits:
memory: 512Mi # no cpu limit: the request still guarantees CPUSizing the V8 heap against the memory limit
Reach for this when pods show OOMKilled and exit code 137 under load but no heap snapshot shows a leak. The symptom of getting it wrong in the other direction is different and more obvious: a JavaScript heap out of memory crash, which Node prints, rather than a silent kill with no log line at all.
# A Node process uses more memory than its JavaScript heap: buffers, the
# binary, native allocations. If the heap alone is allowed to grow to the
# container's limit, the kernel kills the container before V8 ever decides
# to collect garbage harder.
#
# With limits.memory: 512Mi, leave headroom:
node --max-old-space-size=384 server.js
# Node.js 24 also documents --max-old-space-size-percentage, which sets the
# same ceiling as a percentage of available memory:
node --max-old-space-size-percentage=70 server.jsA readiness endpoint with a timeout of its own
The internal deadline matters as much as the query. A readiness check that waits on a hung connection pool blocks until the probe times out, and under load those checks pile up. Returning 503 quickly is the useful outcome: traffic goes to the other replicas while this one recovers.
// /readyz may check dependencies, but it must answer quickly: the probe's
// own timeoutSeconds is 1 by default.
async function readyz(req, res) {
const ac = new AbortController();
const timer = setTimeout(() => ac.abort(), 500);
try {
// A trivial query. Not a count, not a join.
await pool.query({ text: 'select 1', signal: ac.signal });
res.writeHead(200).end('ready');
} catch {
// 503 removes this pod from the Service. The process keeps running and
// the pod rejoins by itself when the check passes again.
res.writeHead(503).end('not ready');
} finally {
clearTimeout(timer);
}
}
Discussion