Timeouts and Deadlines

Bound every outbound call, carry one deadline through the whole request, and stop an unbounded wait from taking the service down.

Syntaxusing CancellationTokenSource deadline = new(TimeSpan.FromSeconds(2));

A timeout is the longest you are willing to wait for an answer before giving up. Every call that leaves your process needs one. Without a timeout, the slowest dependency decides how long your service holds a request open, and a dependency that never answers holds it forever.

HttpClient has a backstop, not a policy

HttpClient.Timeout defaults to 100,000 milliseconds, which is 100 seconds. That is far longer than any user will wait, and it applies to every request made through that instance. Per-call limits come from a CancellationTokenSource instead. When both are set, whichever is shorter takes effect.

A deadline, not one timeout per call

A deadline is a single point in time by which the whole incoming request must be finished. You create it once when the request arrives and pass its token into every call below. Three calls with a two-second timeout each can still take six seconds; one two-second deadline cannot. CancellationTokenSource.CreateLinkedTokenSource combines your deadline with HttpContext.RequestAborted, so the work also stops when the caller hangs up.

Two numbers, not one

The Microsoft.Extensions.Http.Resilience package names both. TotalRequestTimeout, 30 seconds by default, bounds the whole call including every retry. AttemptTimeout, 10 seconds by default, bounds one attempt. Only the per-attempt number lets a retry happen at all; only the total number bounds what the user experiences. This package replaced the deprecated Microsoft.Extensions.Http.Polly package, which you will still see in older projects.

Example

// One deadline for the whole request: 500 milliseconds, however many
// steps the request turns out to need.
using CancellationTokenSource deadline = new(TimeSpan.FromMilliseconds(500));

// Longhand for the line above. The two are the same thing:
//   CancellationTokenSource deadline = new CancellationTokenSource();
//   deadline.CancelAfter(TimeSpan.FromMilliseconds(500));

try
{
    // Step one is quick and finishes well inside the deadline.
    await FetchAsync(TimeSpan.FromMilliseconds(50), deadline.Token);
    Console.WriteLine("profile: ok");

    // Step two is slow. The deadline fires while it is still waiting,
    // because the budget is shared, not reset per call.
    await FetchAsync(TimeSpan.FromSeconds(5), deadline.Token);
    Console.WriteLine("recommendations: ok");
}
catch (OperationCanceledException)
{
    Console.WriteLine("recommendations: deadline exceeded");
}

// Stands in for a call to another service. The token goes all the way down,
// so the wait ends the moment the deadline ends.
static Task FetchAsync(TimeSpan work, CancellationToken token)
{
    return Task.Delay(work, token);
}

// Output:
// profile: ok
// recommendations: deadline exceeded

When to use it

Giving a user-facing endpoint a hard 500ms budget shared across three outbound calls.
Fixing a service whose requests pile up for 100 seconds when one dependency stops responding.
Separating real dependency timeouts from client disconnects in a latency dashboard.
Choosing total and per-attempt timeouts for a client that retries twice.

More examples

A deadline linked to the caller's own cancellation

Run this and send a request with curl to http://localhost:5000/price/MSFT. If the rates service answers within 800 milliseconds you get its JSON body back with a 200. If it is slower, the deadline fires and the response is 504 Gateway Timeout with an empty body, after 800 milliseconds rather than after the client's 30-second backstop. If the caller aborts the connection first, the linked token cancels the outbound call too, so no work continues for a response nobody will read. The exception filter checks which of the two tokens fired, so a disconnect does not get reported as a dependency timeout.

WebApplicationBuilder builder = WebApplication.CreateBuilder(args);

builder.Services.AddHttpClient("rates", static client =>
{
    client.BaseAddress = new Uri("https://rates.internal");
    // A backstop only. The real bound is the deadline in the handler.
    client.Timeout = TimeSpan.FromSeconds(30);
});

WebApplication app = builder.Build();

app.MapGet("/price/{symbol}", async (
    string symbol,
    IHttpClientFactory factory,
    HttpContext http) =>
{
    // One budget for this request.
    using CancellationTokenSource deadline = new(TimeSpan.FromMilliseconds(800));

    // Stop on the deadline OR when the caller disconnects, whichever first.
    using CancellationTokenSource linked =
        CancellationTokenSource.CreateLinkedTokenSource(deadline.Token, http.RequestAborted);

    HttpClient client = factory.CreateClient("rates");

    try
    {
        HttpResponseMessage response = await client.GetAsync($"/rates/{symbol}", linked.Token);
        string body = await response.Content.ReadAsStringAsync(linked.Token);
        return Results.Text(body, "application/json");
    }
    catch (OperationCanceledException) when (deadline.IsCancellationRequested)
    {
        return Results.StatusCode(StatusCodes.Status504GatewayTimeout);
    }
});

app.Run();

The standard handler's two timeouts, and the same thing written out

Reach for the shorthand first; the defaults are deliberate and they are better than no resilience at all. Override the numbers when the call is on a path a user is waiting for, because 30 seconds of total budget is a page that looks broken. DisableForUnsafeHttpMethods turns retries off for POST, PATCH, PUT, DELETE and CONNECT, which is the right default for anything that writes. The option values are validated when the host starts, so an inconsistent set of numbers fails at startup rather than in the middle of an incident.

using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Http.Resilience;

ServiceCollection services = new();

// Shorthand. The standard handler chains five strategies: a rate limiter,
// a total timeout of 30s, retry (3 attempts, exponential, jittered),
// a circuit breaker, and a 10s timeout per attempt.
services.AddHttpClient("defaults")
    .AddStandardResilienceHandler();

// The same handler with the numbers chosen for a user-facing call. These
// are the same five strategies; only the values differ.
services.AddHttpClient("interactive")
    .AddStandardResilienceHandler(options =>
    {
        options.TotalRequestTimeout.Timeout = TimeSpan.FromSeconds(3);
        options.AttemptTimeout.Timeout = TimeSpan.FromSeconds(1);
        options.Retry.MaxRetryAttempts = 2;

        // Retrying a POST can insert the same record twice.
        options.Retry.DisableForUnsafeHttpMethods();
    });

Telling a timeout apart from a caller who went away

This is the single most common piece of wrong resilience code in a .NET service. TaskCanceledException derives from OperationCanceledException, so one catch block swallows both cases and the service reports an upstream outage as healthy client disconnects. Write the two filters out whenever you log or count outbound failures, and put the InnerException check first, because the second filter would otherwise match a timeout as well.

// Both arrive as OperationCanceledException, so the shape of the exception
// is the only way to tell them apart.
try
{
    using HttpResponseMessage response = await client.GetAsync("/rates/GBP", token);
    response.EnsureSuccessStatusCode();
}
catch (OperationCanceledException ex) when (ex.InnerException is TimeoutException)
{
    // HttpClient.Timeout elapsed. Since .NET 5 the inner exception is a
    // TimeoutException, and that is what separates this case from the next.
    // This one is the dependency's problem and belongs on a dashboard.
    logger.LogWarning(ex, "Rates call timed out");
}
catch (OperationCanceledException) when (token.IsCancellationRequested)
{
    // Our deadline fired, or the caller disconnected. Not an error to page
    // anyone about, so it is logged at a lower level.
    logger.LogInformation("Rates call abandoned");
}

Discussion

Be the first to comment on this lesson.