Health Checks & Rollbacks
Automated health checks decide if a deploy is good and trigger rollback when it is not.
Zero-downtime deploys depend on knowing whether the new version is actually healthy. Health checks answer that; rollbacks undo a bad release automatically.
Layers of health checking
- Load balancer health check — polls a path like
/healthzand only routes to passing tasks. - Deployment circuit breaker — ECS aborts and rolls back if new tasks fail to stabilize.
- CloudWatch alarms — CodeDeploy watches error/latency alarms and rolls back if they trip.
Make rollback boring
Because you deploy immutable artifacts (an image tag, a Lambda version), rolling back is just re-pointing to the previous known-good version. No frantic hotfix required.
Example
# Enable automatic rollback on failed ECS deployments
aws ecs update-service \
--cluster myapp \
--service web \
--deployment-configuration \
'deploymentCircuitBreaker={enable=true,rollback=true},minimumHealthyPercent=100,maximumPercent=200'When to use it
- An ECS service fails its ALB health check after a deploy, and CodeDeploy automatically rolls back to the previous task definition within 5 minutes.
- A team configures a CloudWatch alarm on 5xx error rate as the rollback trigger so a surge in errors causes the canary to abort automatically.
- An ops engineer sets ECS deployment circuit breaker so if more than 50% of replacement tasks fail to start, the service reverts to the last stable version.
More examples
Enable ECS deployment circuit breaker
Enables the ECS circuit breaker so the service automatically rolls back if too many replacement tasks fail health checks.
aws ecs update-service \
--cluster prod \
--service myapp \
--deployment-configuration \
'deploymentCircuitBreaker={enable=true,rollback=true},minimumHealthyPercent=50,maximumPercent=200'CloudWatch alarm for deploy monitoring
Creates a CloudWatch alarm that fires when 5xx errors exceed 10 per minute for two consecutive minutes, signalling a bad deploy.
aws cloudwatch put-metric-alarm \
--alarm-name myapp-5xx-high \
--metric-name HTTPCode_Target_5XX_Count \
--namespace AWS/ApplicationELB \
--statistic Sum \
--period 60 \
--threshold 10 \
--comparison-operator GreaterThanThreshold \
--evaluation-periods 2 \
--alarm-actions arn:aws:sns:us-east-1:123456789012:ops-alertsConfigure CodeDeploy automatic rollback
Configures CodeDeploy to automatically roll back if the deployment fails or the 5xx CloudWatch alarm fires.
aws deploy update-deployment-group \
--application-name myapp \
--deployment-group-name myapp-prod \
--auto-rollback-configuration \
'enabled=true,events=[DEPLOYMENT_FAILURE,DEPLOYMENT_STOP_ON_ALARM]' \
--alarm-configuration \
'alarms=[{name=myapp-5xx-high}],enabled=true'
Discussion