Tuning the Horizontal Pod Autoscaler
A default HPA reacts late and flaps. Good CPU targets, stabilization windows, and custom metrics turn it into a calm, predictable autoscaler.
The Horizontal Pod Autoscaler adds and removes replicas to hit a target metric. Left at defaults it tends to flap — scaling up and down repeatedly and thrashing your rollout. Tuning three things fixes almost every case.
1. Requests must be honest
The HPA computes CPU utilization as usage divided by the request. If your CPU request is far too low, 40% real usage shows as 300% and the HPA over-scales wildly. Right-size requests first — the autoscaler is only as good as them.
2. Pick a target with headroom
A CPU target of 80% leaves almost no room to absorb a spike while new Pods start (which takes tens of seconds). 50–60% is a common sweet spot for user-facing services.
3. Stabilization windows stop the flapping
The behavior block lets you scale up fast but scale down slowly. A long scaleDown.stabilizationWindowSeconds (say 300s) means the HPA waits five minutes of sustained low load before removing Pods — no more sawtooth.
Beyond CPU
CPU is a proxy. Real workloads often scale better on custom or external metrics — requests per second, queue depth, or p95 latency — via the Pods or External metric types. Scaling a worker pool on Kafka lag beats scaling it on CPU every time.
Example
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: web-hpa
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: web
minReplicas: 3
maxReplicas: 20
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 55 # headroom to absorb spikes
behavior:
scaleUp:
stabilizationWindowSeconds: 0 # react immediately to load
policies:
- type: Percent
value: 100 # can double the Pods in one step
periodSeconds: 30
scaleDown:
stabilizationWindowSeconds: 300 # wait 5 min before shrinkingWhen to use it
- A team uses custom metrics-based HPA to scale a consumer pod based on the number of unprocessed messages in a RabbitMQ queue rather than CPU usage.
- A platform team sets HPA scale-down stabilization to 5 minutes so a temporary traffic drop does not immediately remove pods that will be needed again.
- A SaaS product uses KEDA (Kubernetes-based Event Driven Autoscaler) to scale a worker deployment to zero during off-hours, cutting compute costs significantly.
More examples
HPA with custom metric (queue depth)
Scales workers based on RabbitMQ queue depth: one worker per 50 messages, up to 50 replicas, down to 1 when idle.
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: worker-hpa
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: queue-worker
minReplicas: 1
maxReplicas: 50
metrics:
- type: External
external:
metric:
name: rabbitmq_queue_messages
selector:
matchLabels:
queue: orders
target:
type: AverageValue
averageValue: "50"Scale-down stabilization window
Configures slow scale-down (max 2 pods/minute with a 5-min window) and fast scale-up (double replicas every 30 seconds) for asymmetric scaling.
spec:
behavior:
scaleDown:
stabilizationWindowSeconds: 300
policies:
- type: Pods
value: 2
periodSeconds: 60
scaleUp:
stabilizationWindowSeconds: 0
policies:
- type: Percent
value: 100
periodSeconds: 30KEDA ScaledObject for zero-scale
Uses KEDA to scale a worker deployment to zero when the RabbitMQ queue is empty and back up when messages arrive.
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
name: worker-scaledobject
spec:
scaleTargetRef:
name: queue-worker
minReplicaCount: 0 # Scale to zero!
maxReplicaCount: 20
triggers:
- type: rabbitmq
metadata:
queueName: orders
queueLength: "10"
Discussion