Rate Limiting

The built-in rate limiter: four algorithms, partitioning per user or IP, and answering with 429 plus Retry-After.

Syntaxbuilder.Services.AddRateLimiter(options => options.AddFixedWindowLimiter("per-minute", limiter => { ... }));

A rate limiter caps how many requests one caller may make in a period. It protects a service from a client stuck in a retry loop, a scraper, and a script guessing passwords. ASP.NET Core has one built in, in Microsoft.AspNetCore.RateLimiting, added in .NET 7. Register policies with AddRateLimiter, turn the middleware on with app.UseRateLimiter(), and attach a policy to an endpoint with .RequireRateLimiting("name").

Four algorithms

Limiter	Caps	Fits
Fixed window	N per fixed period	Simple public API quotas
Sliding window	N per rolling period	Stopping the burst at a window boundary
Token bucket	N refilled steadily	Bursty clients with a steady average
Concurrency	In-flight requests	Slow work: reports, uploads

The first three count requests over time. Concurrency counts only requests happening right now, and releases a permit when one finishes.

Partitioning

One shared counter lets a single noisy client exhaust everyone's budget. A partition gives each key its own counter. Build the key from the signed-in user's name, an API key, or the remote IP address.

Rejecting politely

The default rejection status is 503, which clients read as "the server is broken". Set RejectionStatusCode to 429. The limiter also knows when a permit frees up: read MetadataName.RetryAfter from the lease in OnRejected and send it as the Retry-After header, in seconds.

Example

using System.Globalization;
using System.Threading.RateLimiting;
using Microsoft.AspNetCore.RateLimiting;

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddRateLimiter(options =>
{
    // Without this the middleware answers 503, which reads as an outage.
    options.RejectionStatusCode = StatusCodes.Status429TooManyRequests;

    options.OnRejected = async (context, cancellationToken) =>
    {
        // The lease knows when a permit frees up. Pass that on.
        if (context.Lease.TryGetMetadata(MetadataName.RetryAfter, out var retryAfter))
        {
            context.HttpContext.Response.Headers.RetryAfter =
                ((int)retryAfter.TotalSeconds).ToString(NumberFormatInfo.InvariantInfo);
        }

        await context.HttpContext.Response.WriteAsync(
            "Too many requests.", cancellationToken);
    };

    // 100 per minute, with 10 more allowed to wait for the next window.
    options.AddFixedWindowLimiter("per-minute", limiter =>
    {
        limiter.PermitLimit = 100;
        limiter.Window = TimeSpan.FromMinutes(1);
        limiter.QueueProcessingOrder = QueueProcessingOrder.OldestFirst;
        limiter.QueueLimit = 10;
    });
});

var app = builder.Build();

app.UseRateLimiter();

app.MapGet("/search", () => "ok").RequireRateLimiting("per-minute");

app.Run();

When to use it

Cap a public API at a fixed number of requests per minute for each API key.
Slow down password-guessing against a sign-in endpoint without locking real users out.
Stop two concurrent report exports from saturating the database connection pool.

More examples

A counter per caller, applied everywhere

A named policy has to be attached endpoint by endpoint; GlobalLimiter covers the whole app at once, which is what you want for a blanket per-caller quota. Reach for it as a floor under everything, then add stricter named policies on the few expensive routes.

using System.Threading.RateLimiting;

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddRateLimiter(options =>
{
    options.RejectionStatusCode = StatusCodes.Status429TooManyRequests;

    // GlobalLimiter applies to every endpoint with no RequireRateLimiting call.
    options.GlobalLimiter = PartitionedRateLimiter.Create<HttpContext, string>(httpContext =>
        RateLimitPartition.GetFixedWindowLimiter(
            // Signed-in users get their own bucket. Everyone else shares one
            // per IP address, which is a bounded set of keys.
            partitionKey: httpContext.User.Identity?.Name
                ?? httpContext.Connection.RemoteIpAddress?.ToString()
                ?? "unknown",
            factory: _ => new FixedWindowRateLimiterOptions
            {
                PermitLimit = 60,
                Window = TimeSpan.FromMinutes(1)
            }));
});

var app = builder.Build();
app.UseRateLimiter();

app.MapGet("/orders", () => new[] { "order-1001" });

app.Run();

Token bucket for bursty clients, concurrency for slow work

The two limiters answer different questions. Token bucket caps a rate while tolerating a spike, which suits a dashboard that loads twenty tiles at once. Concurrency caps how much expensive work runs at the same time, which is what protects a database from three simultaneous full-table reports.

using System.Threading.RateLimiting;
using Microsoft.AspNetCore.RateLimiting;

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddRateLimiter(options =>
{
    options.RejectionStatusCode = StatusCodes.Status429TooManyRequests;

    // 100 tokens in the bucket, 10 more every second. A client may spend the
    // whole bucket at once, then settles to 10 requests per second.
    options.AddTokenBucketLimiter("bursty", limiter =>
    {
        limiter.TokenLimit = 100;
        limiter.TokensPerPeriod = 10;
        limiter.ReplenishmentPeriod = TimeSpan.FromSeconds(1);
        limiter.AutoReplenishment = true;
        limiter.QueueLimit = 0;
    });

    // Two reports at a time, whatever the rate. A third waits, a fourth is
    // rejected. A permit returns as soon as a report finishes.
    options.AddConcurrencyLimiter("reports", limiter =>
    {
        limiter.PermitLimit = 2;
        limiter.QueueProcessingOrder = QueueProcessingOrder.OldestFirst;
        limiter.QueueLimit = 1;
    });
});

var app = builder.Build();
app.UseRateLimiter();

app.MapGet("/prices", () => new { Symbol = "ACME", Price = 12.5m })
    .RequireRateLimiting("bursty");

app.MapPost("/reports", () => Results.Accepted("/reports/1"))
    .RequireRateLimiting("reports");

app.Run();

Checking the response a client actually gets

Rate limits are easy to get wrong in a direction nobody notices until real traffic arrives, so measure them. The exact counts depend on your PermitLimit and Window. Reach for a loop like this before deploying a new limit, and again after, because the limit that protects your database can also block your own health check.

# Send more requests than the window allows and show only the status line.
for i in $(seq 1 12); do
  curl -s -o /dev/null -w "%{http_code} " https://localhost:8443/search
done
echo
# 200 200 200 200 429 429 429 429 429 429 429 429

# Look at the headers on a rejected request.
curl -i -s https://localhost:8443/search | head -n 5
# HTTP/1.1 429 Too Many Requests
# Retry-After: 8

Discussion

Be the first to comment on this lesson.

Four algorithms

Partitioning

Rejecting politely

Example

When to use it

More examples

A counter per caller, applied everywhere

Token bucket for bursty clients, concurrency for slow work

Checking the response a client actually gets

Related

Discussion