Timeouts and Budgets

Set a timeout at every layer and give each hop less time than the one before, so one slow backend cannot stall the chain.

Syntaxctx, cancel := context.WithTimeout(r.Context(), 2*time.Second); defer cancel()

A timeout is a limit on one step: dialing the connection, waiting for the reply headers, reading the body. A budget is the time the whole request has, shared out between its steps. At a proxy the rule is that each hop — each forward to the next service — gets less time than the hop that called it. If the caller allows two seconds, the backend must get less than two, because the proxy needs some of it to write a reply.

Where the limits live

Four places, and you want all four. On http.Server: ReadHeaderTimeout, ReadTimeout, WriteTimeout, IdleTimeout. On the request: a deadline from context.WithTimeout, carried by every call made while handling it. On http.Transport: the dialer's timeout, TLSHandshakeTimeout, ResponseHeaderTimeout. On http.Client: Timeout, a ceiling over everything including reading the body.

Passing the budget on

A context deadline does not travel over the network. The next service cannot see it. You send the time left as a header, in milliseconds, and the receiver uses the smaller of that number and its own limit. gRPC does this for you with a header of its own; over plain HTTP you choose the name.

What trips people up

Retries multiply. Three attempts of two seconds each is a six-second wait, which breaks a three-second budget however carefully it was set. Check the time left before each retry and give up when it is less than one attempt needs.

Example

package main

import (
	"context"
	"errors"
	"fmt"
	"time"
)

// budget reports how long the next hop may take: the time left on ctx, less
// a margin kept back to write our own reply. now is a parameter rather than
// a clock reading, so the answer is the same on every run.
func budget(ctx context.Context, now time.Time, margin time.Duration) (time.Duration, error) {
	deadline, ok := ctx.Deadline()
	if !ok {
		return 0, errors.New("request has no deadline")
	}
	left := deadline.Sub(now) - margin
	if left <= 0 {
		return 0, context.DeadlineExceeded
	}
	return left, nil
}

func main() {
	start := time.Date(2026, time.January, 1, 12, 0, 0, 0, time.UTC)
	// The whole request gets two seconds, counted from start.
	ctx, cancel := context.WithDeadline(context.Background(), start.Add(2*time.Second))
	defer cancel()

	// 300ms has already gone, and we keep 50ms for ourselves.
	left, err := budget(ctx, start.Add(300*time.Millisecond), 50*time.Millisecond)
	fmt.Println("give the backend:", left, err)

	// 2.5s has gone: there is nothing left to give away.
	_, err = budget(ctx, start.Add(2500*time.Millisecond), 50*time.Millisecond)
	fmt.Println("expired:", errors.Is(err, context.DeadlineExceeded))
}

// Output:
// give the backend: 1.65s <nil>
// expired: true

When to use it

Stop a slow database query from holding an HTTP connection open for a minute.
Make sure a two-second page budget is not spent by three two-second backend calls.
Return 504 to the caller instead of waiting on a service that has stopped replying.
Tell a downstream service how much of the budget is left.

More examples

Every timeout a proxy needs

Copy this into the setup of any service that both answers HTTP and calls HTTP. The defaults are the problem: a zero value in any of the timeout fields means no limit, so an unconfigured server will hold a connection open for as long as a slow or malicious client likes. MaxIdleConnsPerHost is the exception: left at zero it does not mean unlimited but DefaultMaxIdleConnsPerHost, which is 2, so a busy proxy keeps dialing fresh connections until you raise it. Set the Transport on the proxy with proxy.Transport = transport so forwarded calls use these limits too.

srv := &http.Server{
	Addr:              ":8080",
	Handler:           mux,
	ReadHeaderTimeout: 5 * time.Second,  // a client that opens and says nothing
	ReadTimeout:       15 * time.Second, // headers plus body
	WriteTimeout:      30 * time.Second, // writing the reply out
	IdleTimeout:       60 * time.Second, // a kept-alive connection between requests
}

transport := &http.Transport{
	DialContext:           (&net.Dialer{Timeout: 2 * time.Second}).DialContext,
	TLSHandshakeTimeout:   2 * time.Second,
	ResponseHeaderTimeout: 3 * time.Second, // backend accepted, then went quiet
	IdleConnTimeout:       90 * time.Second,
	MaxIdleConnsPerHost:   64, // reuse connections instead of dialing per request
}
// Timeout covers everything, body included. Leave it out if a route streams.
client := &http.Client{Transport: transport, Timeout: 10 * time.Second}

Spending and passing on the budget

These two halves are what makes a budget hold across a chain of services: every service narrows the deadline it was given, and every outbound call announces what is left. Deploy both in each service and a request cannot outlive the time the edge allowed it, however many hops deep it goes. Taking the smaller of the two values is the part people leave out, and without it a caller can hand you a larger budget than you meant to offer.

const budgetHeader = "X-Timeout-Ms"

// withBudget gives every request a deadline: the smaller of our own limit
// and whatever the caller said it was still willing to wait.
func withBudget(limit time.Duration, next http.Handler) http.Handler {
	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		d := limit
		if ms, err := strconv.Atoi(r.Header.Get(budgetHeader)); err == nil && ms > 0 {
			if caller := time.Duration(ms) * time.Millisecond; caller < d {
				d = caller
			}
		}
		ctx, cancel := context.WithTimeout(r.Context(), d)
		defer cancel()
		next.ServeHTTP(w, r.WithContext(ctx))
	})
}

// send tells the next hop how long it has, keeping 50ms back for ourselves.
func send(ctx context.Context, client *http.Client, url string) (*http.Response, error) {
	req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
	if err != nil {
		return nil, err
	}
	if deadline, ok := ctx.Deadline(); ok {
		left := time.Until(deadline) - 50*time.Millisecond
		if left <= 0 {
			return nil, context.DeadlineExceeded
		}
		req.Header.Set(budgetHeader, strconv.FormatInt(left.Milliseconds(), 10))
	}
	return client.Do(req)
}

http.TimeoutHandler, and where not to put it

TimeoutHandler is the quickest way to put a ceiling on a whole group of routes, and it is a safety net rather than a replacement for per-call deadlines: it stops the client waiting, but a handler that ignores its context keeps running. Reach for it when you are adding limits to a service you did not write, where threading a context through everything is not yet realistic.

func routes(api *http.ServeMux, stream http.Handler) http.Handler {
	top := http.NewServeMux()
	// Replies 503 with "timed out" if api is still working after two seconds,
	// and cancels the request context, so a handler watching ctx.Done() stops.
	top.Handle("/api/", http.TimeoutHandler(api, 2*time.Second, "timed out"))
	// TimeoutHandler buffers the response and implements neither Hijacker nor
	// Flusher, so it breaks WebSockets and server-sent events. Streaming
	// routes go outside it, with deadlines of their own.
	top.Handle("/events", stream)
	return top
}

Discussion

Be the first to comment on this lesson.

Where the limits live

Passing the budget on

What trips people up

Example

When to use it

More examples

Every timeout a proxy needs

Spending and passing on the budget

http.TimeoutHandler, and where not to put it

Related

Discussion