Retries, Backoff and Rate Limits

Retry only what can succeed next time, back off exponentially with jitter, and obey Retry-After on a 429.

SyntaxRetry(total=5, backoff_factor=0.5, status_forcelist=[429, 500, 502, 503, 504])

A retry is a bet that the same request will work later. That bet pays off for a dropped connection, a timeout, a 502 from a restarting pod or a 429 from a rate limiter. It never pays off for 400, 401, 403, 404 or 422 — those describe your request, and your request has not changed, so a retry is noise in someone else's logs.

Backoff, and why jitter is not optional

Exponential backoff doubles the wait each attempt: base * 2 ** attempt, capped so it cannot grow for ever. Full jitter then picks a random point inside that window — random.uniform(0, window) — and the randomness is the important half. Without it, every client that failed during the same incident has the same schedule, so they all come back at the same instant and knock the recovering service over again. A tight loop with no backoff at all does this continuously: one blip becomes an outage that you are causing.

Honour what the server tells you

A 429, and often a 503, carries Retry-After. It is either a number of seconds or an HTTP-date; email.utils.parsedate_to_datetime(date) parses the date form and raises ValueError on anything malformed. When the header is there, use it rather than your own arithmetic.

Bounds

Cap the attempts and cap the total time, measured with time.monotonic() so a clock change cannot confuse it. And be careful with POST: a retried create can create twice. urllib3.util.Retry is deliberately conservative here — its allowed_methods default is the idempotent set, and POST is not in it.

Example

Try it yourself
Loading editor…
Press Run to execute the code.

When to use it

  • Making a nightly sync survive a thirty-second upstream deploy.
  • Staying inside a vendor's rate limit without hard-coding a sleep.
  • Giving a queue consumer a total time budget so a stuck call cannot hold a message for ever.
  • Working out why a provider blocked you after an incident.

More examples

Retries you do not have to write

The version to use in production, because it retries inside the connection pool and already understands Retry-After. Note that total is the retry budget, not the attempt count, and that the timeout you pass applies per attempt, so five retries at ten seconds can still block for a minute.

Example · python
import requests
from requests.adapters import HTTPAdapter
from urllib3.util import Retry

retry = Retry(
    total=5,
    backoff_factor=0.5,          # sleeps backoff_factor * 2 ** (retries so far)
    backoff_jitter=0.3,          # urllib3 2.0+; without it, clients sync up
    backoff_max=30,              # the cap, default 120
    status_forcelist=[429, 500, 502, 503, 504],
    respect_retry_after_header=True,   # the default, and worth keeping
)

session = requests.Session()
adapter = HTTPAdapter(max_retries=retry)
session.mount("https://", adapter)
session.mount("http://", adapter)

r = session.get("https://api.example.com/v1/orders", timeout=10)
print(r.status_code)

A loop with a deadline instead of a count

Use this when the caller has a deadline of its own — an HTTP handler, a queue consumer with a visibility timeout. Counting attempts alone cannot promise an upper bound on elapsed time, and a budget can.

Example · python
import time
import httpx

def get_with_deadline(client, url, budget_seconds=30.0, base=0.5):
    deadline = time.monotonic() + budget_seconds   # monotonic: immune to clock changes
    attempt = 0
    while True:
        try:
            r = client.get(url, timeout=5.0)
            if r.status_code < 500 and r.status_code != 429:
                return r                            # success, or a 4xx worth surfacing
        except httpx.TransportError:                # connect/read/timeout failures
            pass
        remaining = deadline - time.monotonic()
        if remaining <= 0 or attempt >= 5:
            raise TimeoutError(f"{url} did not succeed within {budget_seconds}s")
        time.sleep(min(remaining, base * 2 ** attempt))
        attempt += 1

Reading Retry-After in both of its forms

Needed whenever you handle 429s yourself. The date form is rare but legal, and parsedate_to_datetime returns a naive datetime for a -0000 offset, which is the detail that breaks the subtraction if you do not handle it.

Example · python
from datetime import datetime, timezone
from email.utils import parsedate_to_datetime

def retry_after_seconds(value, default=1.0):
    if value is None:
        return default
    try:
        return max(0.0, float(value))        # the common form: delta-seconds
    except ValueError:
        pass
    try:
        when = parsedate_to_datetime(value)  # the other legal form: an HTTP-date
    except (TypeError, ValueError):
        return default                       # a malformed header is not a reason to crash
    if when.tzinfo is None:                  # "-0000" parses as naive
        when = when.replace(tzinfo=timezone.utc)
    return max(0.0, (when - datetime.now(timezone.utc)).total_seconds())

print(retry_after_seconds("120"))
print(retry_after_seconds("Wed, 21 Oct 2026 07:28:00 GMT"))

Related

py web clientpy web httppy web errorspy async timeoutspy ai retries

Discussion

  • Be the first to comment on this lesson.