Health Checks
Expose separate liveness and readiness endpoints with tagged health checks, check the database safely and keep details private.
builder.Services.AddHealthChecks()
.AddCheck<DbCheck>("db", tags: ["ready"]);
app.MapHealthChecks("/health/live", new() { Predicate = _ => false });
app.MapHealthChecks("/health/ready", new() { Predicate = c => c.Tags.Contains("ready") });builder.Services.AddHealthChecks() registers HealthCheckService, and .AddCheck<T>("name", tags: [...]) adds checks. app.MapHealthChecks("/health/ready", options) maps an endpoint that runs every matching check concurrently, takes the worst status, and writes it. Defaults: Healthy and Degraded return 200, Unhealthy returns 503, and the body is the plain-text status and nothing else. A check that throws is reported with its failureStatus (Unhealthy unless you set it). Each run gets its own DI scope, so a check can depend on a scoped DbContext.
Liveness, readiness, startup
| Probe | Question | Checks | On failure |
|---|---|---|---|
| Liveness | Is the process stuck? | Nothing external | Kubelet restarts the container |
| Readiness | Can it serve traffic now? | Database, cache, required config | Pod removed from the Service endpoints; no restart |
| Startup | Has it finished starting? | In-process warm-up only, never a dependency | Liveness and readiness wait; past failureThreshold the kubelet restarts the container |
The rule that prevents outages: liveness never checks a dependency. If the database is down and liveness includes it, Kubernetes restarts every pod, they come back still unable to reach the database, and a dependency incident becomes a crash loop with cold caches and a connection storm on recovery. The same goes for the startup probe, which restarts the container once it exceeds its failure threshold. Split them with tags and a Predicate: liveness runs a self check or none (Predicate = _ => false returns 200 whenever the app can answer), readiness runs checks tagged ready.
Database and dependency checks
AspNetCore.HealthChecks.NpgSql's AddNpgSql opens a connection and runs SELECT 1;. Give every dependency check a timeout of a few seconds, below the probe's timeoutSeconds, so a hung dependency is reported instead of making the probe itself time out. Probes multiply: 30 pods probed every 10 seconds is 3 queries a second, which is fine for SELECT 1 and not for a real query or a paid API. Register optional dependencies (a recommendations service) with failureStatus: HealthStatus.Degraded: visible on dashboards, still 200, pod stays in rotation.
Never expose details publicly
A JSON writer listing every check with exception messages is useful to operators and to attackers: host names, database names, versions, internal error text. Keep public endpoints on the default plain-text writer and always put the details endpoint behind RequireAuthorization, ideally also on a management port with no ingress route. RequireHost("*:8081") does not do that: it matches the client-supplied Host header, so a request to the public port with Host: x:8081 passes. Check the port the connection arrived on, HttpContext.Connection.LocalPort, in an endpoint filter or middleware. If the app has a fallback authorization policy, probe endpoints need AllowAnonymous() because the kubelet sends no token, and rate limiting and output caching must not apply to them.
Failure modes
- Requests refused during shutdown: a terminating pod leaves the Service endpoints because it is terminating, not because readiness fails, and kube-proxy and the ingress take seconds to notice while Kestrel stops accepting connections on SIGTERM. Add a
preStopsleep of 5-10 s, and setHostOptions.ShutdownTimeoutandterminationGracePeriodSecondsso in-flight requests finish (see net-zero-downtime). - A shared dependency in readiness: when the one database every replica uses is down, all pods leave rotation together and clients get the ingress's 503 instead of your ProblemDetails. Put per-instance dependencies in readiness; for a shared one, accept that trade-off deliberately or report it as Degraded and let requests fail with a proper error.
Example
When to use it
- Take one pod out of the load balancer while its local cache sidecar is unreachable, without Kubernetes restarting it or touching the other replicas.
- Report a slow recommendations service as Degraded on dashboards while orders keep being accepted.
- Give operators a JSON report of every check on a management port that the public ingress never routes to.
More examples
PostgreSQL and an optional dependency
<code>AddNpgSql</code> comes from the AspNetCore.HealthChecks.NpgSql package and runs <code>SELECT 1;</code>. With the connection-string overload it uses its own data source and pool, so it proves the database answers but not that the app's pool has free connections; resolve the app's <code>NpgsqlDataSource</code> from DI if you want the check to share that pool. The recommendations check is registered as Degraded: when that service fails, readiness still returns 200 and the pod keeps taking orders, while dashboards show the degradation. Returning <code>context.Registration.FailureStatus</code> keeps the decision in the registration, not hard-coded in the check.
using Microsoft.Extensions.Diagnostics.HealthChecks;
var builder = WebApplication.CreateBuilder(args);
var ordersDb = builder.Configuration.GetConnectionString("Orders")
?? throw new InvalidOperationException("ConnectionStrings:Orders is not configured");
builder.Services.AddHttpClient("recommendations", c => c.BaseAddress = new Uri("http://recommendations:8080"));
builder.Services.AddHealthChecks()
.AddNpgSql(ordersDb, name: "orders-db", tags: ["ready"], timeout: TimeSpan.FromSeconds(3))
.AddCheck<RecommendationsCheck>("recommendations",
failureStatus: HealthStatus.Degraded, tags: ["ready"], timeout: TimeSpan.FromSeconds(2));
var app = builder.Build();
app.MapHealthChecks("/health/live", new() { Predicate = _ => false });
app.MapHealthChecks("/health/ready", new() { Predicate = c => c.Tags.Contains("ready") });
app.Run();
public sealed class RecommendationsCheck(IHttpClientFactory clients) : IHealthCheck
{
public async Task<HealthCheckResult> CheckHealthAsync(HealthCheckContext context, CancellationToken ct)
{
using var response = await clients.CreateClient("recommendations").GetAsync("/health/live", ct);
return response.IsSuccessStatusCode
? HealthCheckResult.Healthy()
: new HealthCheckResult(context.Registration.FailureStatus, $"status {(int)response.StatusCode}");
}
}
Kubernetes probes wired to the endpoints
The startup probe allows up to 60 seconds (30 x 2 s) before liveness starts counting, so a slow start is not mistaken for a hang. It hits <code>/health/live</code>, not readiness: past its threshold the kubelet restarts the container, so a database outage during a rollout would otherwise crash-loop every new pod. Readiness <code>timeoutSeconds: 5</code> is longer than the 3-second check timeout in code, so a slow database is reported as Unhealthy rather than as a probe timeout. Liveness only hits <code>/health/live</code>, which checks nothing external.
apiVersion: apps/v1
kind: Deployment
metadata:
name: orders-api
spec:
replicas: 3
selector:
matchLabels:
app: orders-api
template:
metadata:
labels:
app: orders-api
spec:
containers:
- name: orders-api
image: registry.example.com/shop/orders-api:1.4.2
ports:
- containerPort: 8080
startupProbe:
httpGet:
path: /health/live
port: 8080
periodSeconds: 2
failureThreshold: 30
readinessProbe:
httpGet:
path: /health/ready
port: 8080
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 3
livenessProbe:
httpGet:
path: /health/live
port: 8080
periodSeconds: 10
timeoutSeconds: 3
failureThreshold: 3
Health endpoints next to a fallback policy
With a fallback policy every endpoint without metadata requires a user, so without <code>AllowAnonymous()</code> the probes get 401, Kubernetes marks every pod unready and the service disappears right after deploy. The details endpoint checks <code>Connection.LocalPort</code>, the port the connection really arrived on; <code>RequireHost</code> would match only the <code>Host</code> header, which any client can spoof. The operator role requirement is the real protection, in case the port is reachable from inside the cluster.
using Microsoft.AspNetCore.Authorization;
var builder = WebApplication.CreateBuilder(args);
builder.WebHost.UseUrls("http://+:8080", "http://+:8081"); // 8081: management, no ingress route
builder.Services.AddAuthentication().AddJwtBearer(); // the only scheme, so it is the default challenge
builder.Services.AddAuthorization(options =>
{
// Every endpoint requires an authenticated user unless it opts out.
options.FallbackPolicy = new AuthorizationPolicyBuilder().RequireAuthenticatedUser().Build();
});
builder.Services.AddHealthChecks();
var app = builder.Build();
app.UseAuthentication();
app.UseAuthorization();
// The kubelet sends no token, so probes must be anonymous.
app.MapHealthChecks("/health/live", new() { Predicate = _ => false }).AllowAnonymous();
app.MapHealthChecks("/health/ready").AllowAnonymous();
// Full report only on the management port and only for operators.
app.MapHealthChecks("/health/details")
.AddEndpointFilter(async (ctx, next) =>
ctx.HttpContext.Connection.LocalPort == 8081 ? await next(ctx) : Results.NotFound())
.RequireAuthorization(policy => policy.RequireRole("operator"));
app.MapGet("/orders/{id:int}", (int id) => Results.Ok(new { id, status = "shipped" }));
app.Run();
Discussion