Tuning Target-Tracking Autoscaling
Pick the right target metric and cooldowns so your service scales smoothly instead of thrashing.
Target-tracking autoscaling is the right default for almost every service, but the out-of-the-box settings are rarely the ones you actually want. The knobs are simple; the judgement is in choosing them.
Pick a metric that reflects real load
Average CPU is the usual choice, but it lies for I/O-bound services that sit at 20% CPU while requests pile up. For a web tier fronted by an ALB, ALBRequestCountPerTarget is often a far better signal — it scales on the thing users actually feel.
Set the target where headroom lives
A target of 60–70% is the sweet spot for most services. Aim too high (say 90%) and you have no buffer for the time it takes new tasks to boot and pass health checks; aim too low and you pay for idle capacity.
Respect the two directions differently
- Scale-out should be eager — a short cooldown so you add capacity before users hurt.
- Scale-in should be patient — a longer cooldown (300s+) so a brief dip does not rip away tasks you will need again in five minutes.
Scale out fast, scale in slow. The cost of one extra task for ten minutes is trivial; the cost of a thrashing service is an outage.
Example
{
"TargetValue": 1000.0,
"PredefinedMetricSpecification": {
"PredefinedMetricType": "ALBRequestCountPerTarget",
"ResourceLabel": "app/web-alb/50dc6c495c0c9188/targetgroup/web-tg/6d0ecf831eec9f09"
},
"ScaleOutCooldown": 60,
"ScaleInCooldown": 300
}When to use it
- An SRE changes the scale-in cooldown from 60 to 300 seconds to prevent the service from yo-yoing between 5 and 10 tasks during a sustained medium-load period.
- A developer lowers the scale-out cooldown from 300 to 60 seconds so the service reacts within a minute when a traffic spike hits, rather than waiting five minutes.
- A team switches their autoscaling metric from CPU to ALB RequestCountPerTarget so scaling responds to actual HTTP request rate rather than CPU, which lags behind.
More examples
Target-Tracking on Request Count per Target
Sets a target of 500 requests per task, scaling out quickly (60 s) and scaling in conservatively (300 s) to avoid thrashing.
aws application-autoscaling put-scaling-policy \
--service-namespace ecs \
--resource-id service/prod/api \
--scalable-dimension ecs:service:DesiredCount \
--policy-name rps-tracking \
--policy-type TargetTrackingScaling \
--target-tracking-scaling-policy-configuration \
'TargetValue=500,CustomizedMetricSpecification={MetricName=RequestCountPerTarget,Namespace=AWS/ApplicationELB,Dimensions=[{Name=TargetGroup,Value=targetgroup/api-tg/abc}],Statistic=Sum},ScaleOutCooldown=60,ScaleInCooldown=300'Tune Scale-In Cooldown to Prevent Thrashing
Keeps 60% CPU as the target but uses a 10-minute scale-in cooldown to prevent the service from removing tasks too aggressively after a spike.
aws application-autoscaling put-scaling-policy \
--service-namespace ecs \
--resource-id service/prod/api \
--scalable-dimension ecs:service:DesiredCount \
--policy-name cpu-tuned \
--policy-type TargetTrackingScaling \
--target-tracking-scaling-policy-configuration \
'TargetValue=60,PredefinedMetricSpecification={PredefinedMetricType=ECSServiceAverageCPUUtilization},ScaleOutCooldown=60,ScaleInCooldown=600'Set Minimum and Maximum Task Counts
Sets absolute bounds on scaling — always at least 3 tasks for HA across AZs, and capped at 50 to prevent runaway cost if a metric goes haywire.
aws application-autoscaling register-scalable-target \
--service-namespace ecs \
--resource-id service/prod/api \
--scalable-dimension ecs:service:DesiredCount \
--min-capacity 3 \
--max-capacity 50
Discussion