Databases in Production
Choose the right database health check, retry only transient failures on idempotent work, and route reads between primary and replicas safely.
builder.Services.AddHealthChecks()
.AddCheck("self", () => HealthCheckResult.Healthy(), tags: ["live"])
.AddNpgSql(connectionString, tags: ["ready"], timeout: TimeSpan.FromSeconds(2));Database health checks
Probe design (what goes in liveness and readiness, probe timeouts, status codes) is covered in the Health Checks lesson; this section is only about checking a database. Two checks exist, and they test different things:
| Check | Package | What it proves |
|---|---|---|
AddNpgSql() | AspNetCore.HealthChecks.NpgSql | A connection opens and SELECT 1 runs |
AddDbContextCheck<ShopDb>() | Microsoft.Extensions.Diagnostics.HealthChecks.EntityFrameworkCore | CanConnectAsync succeeds, so the context is also configured and resolvable |
Build one NpgsqlDataSource, register it, and pass it to EF Core with UseNpgsql(dataSource). AddNpgSql() with no connection string resolves that data source from DI, so the check rents from the app's own pool instead of opening a second pool with its own budget. Give the check a timeout: a host that drops packets never answers, and without one the probe hangs until the caller gives up. Checks run on every probe, which is fine for SELECT 1, not for a query that scans a table. The database is shared by every replica, so whether it belongs in readiness at all is a deliberate trade-off, not the default: when it fails, every pod leaves the Service together. Reporting it as Degraded keeps pods serving while still showing the problem.
Retries on transient failures
Connections dropped by a failover or restart (an IOException inside the NpgsqlException), a server still starting (57P03), serialization failures (40001), deadlocks (40P01) and too many connections (53300) succeed on a later attempt. In EF Core, EnableRetryOnFailure() retries up to 6 times with exponential backoff capped at 30 seconds. For Dapper and ADO.NET, Npgsql marks these errors with NpgsqlException.IsTransient, which a Polly pipeline can test. Retry only what is safe to repeat: a timeout after an INSERT was sent may mean it committed, and the retry creates a second order. Protect writes with idempotency keys or a unique constraint, and keep total retry time under the request timeout.
Read replicas
A streaming replica takes reports and lists off the primary, but it lags: usually milliseconds, minutes under heavy writes or a long query on the replica. A client that creates an order and reads it back from a replica can get 404. Send reads that must see the caller's own writes to the primary and the rest to the replica, through a second data source (a cloud reader endpoint) or Npgsql multi-host with WithTargetSession(TargetSessionAttributes.PreferStandby). Each target has its own pool, so budget both.
Migrations
Ordering migrations against a rollout is covered in the Database Migrations in Deployments lesson. One database-side tip belongs here: set lock_timeout (for example SET lock_timeout = '5s') in migration scripts. An ALTER TABLE waiting for its lock blocks every query queued behind it, so failing fast and retrying beats an outage.
| Concern | Default to choose |
|---|---|
| Health check | AddNpgSql() on the shared data source, with a timeout |
| Retries | Transient errors, idempotent work only |
| Read-your-own-writes | Primary |
| Reports and lists | Replica, PreferStandby |
| Migration scripts | lock_timeout set |
Example
When to use it
- Keep pods running during a 30 second PostgreSQL failover by checking the database only in /health/ready, not /health/live
- Retry a stock lookup that hit a deadlock or failover with jittered backoff, while leaving order inserts to idempotency keys
- Set lock_timeout in a migration that adds an index to orders so a blocked ALTER TABLE fails in 5 seconds instead of stalling checkout
More examples
Real readiness checks for PostgreSQL and the DbContext
<code>AddNpgSql</code> with no connection string resolves the <code>NpgsqlDataSource</code> from DI and runs <code>SELECT 1</code>; <code>AddDbContextCheck</code> calls <code>CanConnectAsync</code>, which also proves the context is configured. EF Core and the check share one data source, so there is one pool to budget. No port split is needed: the default response writer prints only the overall status, and the exception and description go to logs and health check publishers, never into the response body. A detailed JSON writer belongs on a separate endpoint gated by <code>Connection.LocalPort</code> and authorization, as the Health Checks lesson shows.
using Microsoft.AspNetCore.Diagnostics.HealthChecks;
using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.Diagnostics.HealthChecks;
using Npgsql;
var builder = WebApplication.CreateBuilder(args);
var connectionString = builder.Configuration.GetConnectionString("Shop")
?? throw new InvalidOperationException("ConnectionStrings:Shop is not configured.");
// One data source for EF Core and the health check: one pool, one connection budget.
var dataSource = NpgsqlDataSource.Create(connectionString);
builder.Services.AddSingleton(dataSource);
builder.Services.AddDbContext<ShopDb>(o => o.UseNpgsql(dataSource));
builder.Services.AddHealthChecks()
.AddCheck("self", () => HealthCheckResult.Healthy(), tags: ["live"])
// Resolves the registered NpgsqlDataSource, so the check shares the app's pool.
.AddNpgSql(name: "postgres", tags: ["ready"], timeout: TimeSpan.FromSeconds(2))
.AddDbContextCheck<ShopDb>("shopdb-context", tags: ["ready"]);
var app = builder.Build();
app.MapHealthChecks("/health/live", new HealthCheckOptions { Predicate = c => c.Tags.Contains("live") });
// The default writer returns only "Healthy" or "Unhealthy", so failure details never reach the body.
app.MapHealthChecks("/health/ready", new HealthCheckOptions { Predicate = c => c.Tags.Contains("ready") });
app.Run();
public sealed class ShopDb(DbContextOptions<ShopDb> options) : DbContext(options);Retrying transient errors on idempotent Dapper reads
The predicate retries only errors Npgsql classifies as transient; a syntax error or a unique-constraint violation fails at once. Order matters in Polly: the first strategy added is the outermost. The 5 second timeout added before <code>AddRetry</code> bounds all attempts and delays together so a request never outlives its own deadline; the 2 second timeout added after it limits each attempt, and the retry handles its <code>TimeoutRejectedException</code>. This pipeline is for reads: wrapping an INSERT the same way can create duplicates when the first attempt committed but the reply was lost.
using Dapper;
using Npgsql;
using Polly;
using Polly.Registry;
using Polly.Retry;
using Polly.Timeout;
var builder = WebApplication.CreateBuilder(args);
builder.Services.AddSingleton(NpgsqlDataSource.Create(builder.Configuration.GetConnectionString("Shop")
?? throw new InvalidOperationException("ConnectionStrings:Shop is not configured.")));
// The first strategy added is the outermost: 5 s for the whole call, 2 s per attempt.
builder.Services.AddResiliencePipeline("db-read", pipeline => pipeline
.AddTimeout(TimeSpan.FromSeconds(5))
.AddRetry(new RetryStrategyOptions
{
// Covers dropped connections during failover, deadlock (40P01), serialization failure (40001),
// and an attempt cut off by the inner timeout.
ShouldHandle = new PredicateBuilder()
.Handle<NpgsqlException>(e => e.IsTransient)
.Handle<TimeoutRejectedException>(),
MaxRetryAttempts = 3,
Delay = TimeSpan.FromMilliseconds(200),
BackoffType = DelayBackoffType.Exponential,
UseJitter = true,
})
.AddTimeout(TimeSpan.FromSeconds(2)));
var app = builder.Build();
app.MapGet("/products/{sku}/stock", async (string sku, NpgsqlDataSource db,
ResiliencePipelineProvider<string> pipelines, CancellationToken ct) =>
{
var pipeline = pipelines.GetPipeline("db-read");
var onHand = await pipeline.ExecuteAsync(async token =>
{
// A new connection per attempt: a broken one is never reused for the retry.
await using var connection = await db.OpenConnectionAsync(token);
return await connection.ExecuteScalarAsync<int?>(new CommandDefinition(
"SELECT on_hand FROM inventory WHERE sku = @sku", new { sku }, cancellationToken: token));
}, ct);
return onHand is null ? Results.NotFound() : Results.Ok(new { sku, onHand });
});
app.Run();Primary and replica data sources with Npgsql multi-host
One multi-host data source discovers which host is the primary and which are standbys; <code>WithTargetSession</code> returns views over it that open connections only to the matching kind. <code>PreferStandby</code> falls back to the primary when no standby is up, so reports keep working during a replica outage. The routing rule is per endpoint: anything that must read its own writes goes to the primary.
using Dapper;
using Microsoft.Extensions.DependencyInjection;
using Npgsql;
var builder = WebApplication.CreateBuilder(args);
// e.g. Host=pg-1,pg-2,pg-3;Database=shop;Username=orders_api;Password=<from-secret-store>;Load Balance Hosts=true
var cluster = new NpgsqlDataSourceBuilder(builder.Configuration.GetConnectionString("ShopCluster")
?? throw new InvalidOperationException("ConnectionStrings:ShopCluster is not configured.")).BuildMultiHost();
builder.Services.AddSingleton(cluster);
builder.Services.AddKeyedSingleton<NpgsqlDataSource>("primary", cluster.WithTargetSession(TargetSessionAttributes.Primary));
builder.Services.AddKeyedSingleton<NpgsqlDataSource>("replica", cluster.WithTargetSession(TargetSessionAttributes.PreferStandby));
var app = builder.Build();
// Reports tolerate seconds of lag: replica.
app.MapGet("/reports/top-products", async ([FromKeyedServices("replica")] NpgsqlDataSource db, CancellationToken ct) =>
{
await using var connection = await db.OpenConnectionAsync(ct);
return Results.Ok(await connection.QueryAsync<(string Sku, long Units)>(new CommandDefinition(
"SELECT sku, sum(quantity) FROM order_lines GROUP BY sku ORDER BY 2 DESC LIMIT 10", cancellationToken: ct)));
});
// A client reading its own order right after POST /orders must see it: primary.
app.MapGet("/orders/{id:long}/status", async (long id, [FromKeyedServices("primary")] NpgsqlDataSource db, CancellationToken ct) =>
{
await using var connection = await db.OpenConnectionAsync(ct);
var status = await connection.ExecuteScalarAsync<string?>(new CommandDefinition(
"SELECT status FROM orders WHERE id = @id", new { id }, cancellationToken: ct));
return status is null ? Results.NotFound() : Results.Ok(new { id, status });
});
app.Run();
Discussion