Scaling SignalR

Run SignalR on several instances with sticky sessions or skipped negotiation, a Redis backplane or Azure SignalR Service, and known connection limits.

Syntax

builder.Services.AddSignalR().AddStackExchangeRedis(connectionString, o =>
    o.Configuration.ChannelPrefix = RedisChannel.Literal("orders-api"));

A single SignalR server keeps everything in memory: the connection, its transport and its groups. Adding a second replica breaks two separate assumptions, and each has its own fix.

Problem 1: a connection is several HTTP requests

The client negotiates with one request and connects with a second, carrying the connection token from the first. If the load balancer routes the second request to another instance, that instance has never seen the token and answers 404. Server-Sent Events and long polling are worse: every client-to-server message is a separate POST for the whole life of the connection. Two fixes:

Sticky sessions: cookie affinity on the ingress or load balancer. Required whenever fallback transports are allowed.
WebSockets only, negotiation skipped: the client sets skipNegotiation and opens the WebSocket directly, one long request that any instance can accept. Restrict the hub to HttpTransportType.WebSockets so nobody falls back.

Problem 2: each server only knows its own connections

Clients.Group("order-2042") on instance A reaches only connections held by A. A backplane fixes that: every instance publishes each message to a shared bus and every instance delivers it to its own matching connections.

Option	How	Trade-off
Redis backplane	`AddSignalR().AddStackExchangeRedis(...)`	You run Redis; every message reaches every server; nothing is buffered while Redis is down
Azure SignalR Service	`AddSignalR().AddAzureSignalR()`; clients connect to the service	No sticky sessions or backplane to run; billed per unit of 1,000 connections; Azure only
One bigger instance	Scale up, not out	Simplest; no redundancy, every deploy drops everyone

The Redis backplane uses pub/sub, which stores nothing. While Redis is unreachable, sends fail and those messages are lost. Every server receives every message, so a backplane raises the number of connections you can hold, not total message throughput. Set a distinct ChannelPrefix when apps or environments share a Redis, or they deliver each other's messages, and keep Redis in the same region as the servers.

Connection limits

Each connection holds a socket and buffers on every hop. Check the file descriptor limit, the proxy's connection caps and Kestrel's MaxConcurrentUpgradedConnections (unlimited by default).
Load balancers close idle connections: 60 seconds on AWS ALB and in nginx (proxy_read_timeout), 4 minutes on Azure Load Balancer. The 15-second server ping keeps connections active; never raise KeepAliveInterval past the shortest idle timeout.
A rolling deploy cuts every connection on the old pod at once. Default client reconnects fire after 0, 2, 10 and 30 seconds, so thousands of clients arrive together. Add jitter with a custom IRetryPolicy.
A reconnected client lands on any instance with a new connection id and no groups; whatever it missed is gone unless you stored it.

Example

Loading editor…

Press Run to execute the code.

When to use it

Run the orders API on four pods and still tell a customer their order shipped when the event is handled on a different pod
Share one Redis between the orders and invoices APIs' backplanes by giving each its own ChannelPrefix
Survive a rolling deploy with 20,000 connected dashboards by adding jitter to the client reconnect policy

More examples

Redis backplane with a channel prefix

The package is Microsoft.AspNetCore.SignalR.StackExchangeRedis. Every server subscribes to Redis channels for all-clients, groups, users and connections, and group membership is also coordinated through Redis, so <code>Clients.Group</code> on any instance reaches members on all of them. The connection string comes from configuration (a Kubernetes Secret in production). Watch the Redis connection in your health checks: a server whose subscription dropped keeps serving its own clients but silently misses everyone else's messages.

using Microsoft.AspNetCore.Http.Connections;
using Microsoft.AspNetCore.SignalR;
using StackExchange.Redis;

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddSignalR()
    .AddStackExchangeRedis(builder.Configuration.GetConnectionString("SignalRBackplane")!, options =>
    {
        // Apps and environments sharing this Redis must not hear each other's messages.
        options.Configuration.ChannelPrefix = RedisChannel.Literal($"orders-api:{builder.Environment.EnvironmentName}");
        // Keep retrying in the background instead of failing start-up when Redis is briefly away.
        options.Configuration.AbortOnConnectFail = false;
    });

var app = builder.Build();

// WebSockets only: with skipNegotiation on the client, no sticky sessions are needed.
app.MapHub<OrdersHub>("/hubs/orders", options => options.Transports = HttpTransportType.WebSockets);
app.Run();

public sealed class OrdersHub : Hub
{
    public Task WatchOrder(int orderId) => Groups.AddToGroupAsync(Context.ConnectionId, $"order-{orderId}");
}

Cookie affinity on the ingress

These ingress-nginx annotations pin a browser to one pod with a cookie, so negotiate, connect and every long-polling request reach the same instance. Other controllers have equivalents (Traefik sticky cookies on the Service, target group stickiness on AWS ALB). Affinity does not survive a pod being replaced: the client reconnects, gets a new cookie and a new pod. The raised read timeout gives headroom over the 15-second ping; it does not replace it.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: orders-api
  namespace: shop
  annotations:
    nginx.ingress.kubernetes.io/affinity: "cookie"
    nginx.ingress.kubernetes.io/session-cookie-name: "orders-affinity"
    nginx.ingress.kubernetes.io/session-cookie-max-age: "3600"
    nginx.ingress.kubernetes.io/proxy-read-timeout: "120"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "120"
spec:
  ingressClassName: nginx
  rules:
    - host: orders.example.com
      http:
        paths:
          - path: /hubs
            pathType: Prefix
            backend:
              service:
                name: orders-api
                port:
                  number: 8080

Client reconnects with jitter

With <code>SkipNegotiation</code> the client goes straight to the WebSocket, which only works when <code>Transports</code> is exactly WebSockets. The retry policy returns a random delay below an exponentially growing ceiling, so 20,000 clients cut by a deploy make their first attempt spread over 10 seconds, about 2,000 a second, rather than all within one second, and later attempts spread over up to a minute. Returning <code>null</code> stops reconnecting and raises <code>Closed</code>, where the app can show an offline state. Browser clients pass the same kind of policy object to <code>withAutomaticReconnect</code>.

using Microsoft.AspNetCore.Http.Connections;
using Microsoft.AspNetCore.SignalR.Client;

await using var connection = new HubConnectionBuilder()
    .WithUrl("https://orders.example.com/hubs/orders", options =>
    {
        options.SkipNegotiation = true;
        options.Transports = HttpTransportType.WebSockets;
    })
    .WithAutomaticReconnect(new JitteredRetryPolicy())
    .Build();

await connection.StartAsync();
Console.WriteLine($"Connected as {connection.ConnectionId}");
await Task.Delay(TimeSpan.FromMinutes(5));

public sealed class JitteredRetryPolicy : IRetryPolicy
{
    private static readonly TimeSpan MaxDelay = TimeSpan.FromSeconds(60);

    public TimeSpan? NextRetryDelay(RetryContext retryContext)
    {
        if (retryContext.ElapsedTime > TimeSpan.FromMinutes(10)) return null; // give up, raise Closed

        // The first attempt already spreads over 10 s; a 1 s window would still be a storm.
        var ceiling = Math.Min(MaxDelay.TotalMilliseconds, 10_000 * Math.Pow(2, retryContext.PreviousRetryCount));
        // Full jitter: spread a reconnect storm over the whole window instead of hitting new pods together.
        return TimeSpan.FromMilliseconds(Random.Shared.NextDouble() * ceiling);
    }
}

Discussion

Be the first to comment on this lesson.