The Runtime: JIT, Tiered Compilation and GC
Explain how IL becomes machine code, why the first request is slow, and how to choose and size the garbage collector in a container.
DOTNET_gcServer=1
DOTNET_GCHeapHardLimitPercent=0x46 # hexadecimal: 70 percent
<ServerGarbageCollection>true</ServerGarbageCollection>
<PublishReadyToRun>true</PublishReadyToRun>Your assemblies contain IL. The JIT compiles each method to machine code the first time it is called. Framework assemblies ship with ReadyToRun (precompiled) code so start-up does not JIT the whole base class library, but your own code, generic instantiations over your types and anything ReadyToRun could not cover are compiled at run time.
Tiered compilation and Dynamic PGO
To start fast, the JIT first produces tier-0 code: compiled quickly, barely optimised. A method called 30 times is queued for tier 1, fully optimised and compiled on a background thread, and the call site switches over. Loops in tier-0 methods can jump to optimised code mid-execution (on-stack replacement), so a long-running loop is not stuck on slow code. Dynamic PGO, on by default since .NET 8, inserts an instrumented tier that records which types and branches actually occur, then tier 1 uses that profile to devirtualise interface calls and inline the hot path. Throughput therefore keeps improving for the first seconds to minutes under load, and any benchmark that does not warm up measures tier-0 code.
Why the first request is slower
- JIT of routing, the endpoint, model binding and your handler.
System.Text.Jsonbuilding serialization metadata for each type on first use.- Singletons constructed lazily on first resolve; connection pools, TLS handshakes and an EF Core model created on first use.
Expect tens to hundreds of milliseconds for the first call against well under a millisecond once warm. Gate the readiness probe on a warm-up step, publish with PublishReadyToRun, use source-generated JSON, or go to Native AOT.
Workstation or Server GC
| Workstation | Server | |
|---|---|---|
| Heaps | One | One per logical CPU (adaptive with DATAS) |
| Collections run on | The allocating thread | Dedicated high-priority GC threads |
| Default for | Console and worker projects | Projects on Microsoft.NET.Sdk.Web |
| Trade-off | Lower memory, lower throughput | Higher throughput, larger memory footprint |
Background (concurrent) GC is on in both modes. Server GC now runs with DATAS (dynamic adaptation to application sizes) by default: it starts with few heaps and adds them under load, so idle replicas use far less memory than classic Server GC. DOTNET_GCDynamicAdaptationMode=0 turns it off when a throughput benchmark shows a regression.
What a container changes
The runtime reads cgroup limits. Environment.ProcessorCount reports the CPU limit rounded up, and with a memory limit the GC heap hard limit defaults to 75 percent of it. Server GC limited to one CPU behaves like workstation. Without a memory limit the GC sizes itself against the node's memory and the kernel OOM-kills the pod before the GC feels pressure, so always set one. The knobs are environment variables (DOTNET_gcServer, DOTNET_GCHeapHardLimitPercent, DOTNET_GCHeapCount, DOTNET_GCConserveMemory, DOTNET_TieredPGO, DOTNET_ReadyToRun) or the matching runtimeconfig.json settings. GC environment values are hexadecimal.
Example
When to use it
- Explain why p99 latency spikes for the first minute after each deploy, and gate readiness on a warm-up step.
- Stop pods with a 512Mi limit being OOM-killed by giving the GC a memory limit it can see and a heap hard limit percent.
- Decide whether a low-traffic internal API should run Workstation GC to cut its memory footprint per replica.
More examples
Kubernetes resources the GC can see
With a 2-CPU limit the runtime reports <code>ProcessorCount</code> as 2 and Server GC creates at most two heaps. The memory limit becomes the GC's budget: by default 75 percent of it, here lowered to 70 percent to leave room for native memory such as TLS buffers and thread stacks. Omit <code>limits.memory</code> and the GC plans against the node's RAM, which is how pods get OOM-killed while the heap still looks small.
apiVersion: apps/v1
kind: Deployment
metadata:
name: orders-api
spec:
replicas: 3
selector:
matchLabels:
app: orders-api
template:
metadata:
labels:
app: orders-api
spec:
containers:
- name: orders-api
image: registry.example.com/shop/orders-api:1.14.2
resources:
requests:
cpu: "500m"
memory: "384Mi"
limits:
cpu: "2"
memory: "512Mi"
env:
# Hexadecimal: 0x46 = 70 percent of the 512Mi limit for the managed heap
- name: DOTNET_GCHeapHardLimitPercent
value: "0x46"runtimeconfig.template.json with decimal values
Settings here ship inside the build output, so every environment gets them without remembering an environment variable, and the values are decimal. Environment variables still override them at deploy time, which is useful for an experiment on one replica. For the GC mode alone, <code><ServerGarbageCollection></code> in the project file is the more discoverable option.
// runtimeconfig.template.json next to Orders.Api.csproj; merged into the generated runtimeconfig.json
{
"configProperties": {
"System.GC.Server": true,
"System.GC.Concurrent": true,
"System.GC.HeapHardLimitPercent": 70,
"System.GC.ConserveMemory": 3,
"System.Runtime.TieredPGO": true
}
}Warming up before reporting ready
The readiness endpoint stays unhealthy until the warm-up has resolved expensive singletons and built JSON metadata, so the load balancer sends no customer traffic to a cold replica. Liveness ignores the check, so a slow warm-up is never mistaken for a hung process. Warming <code>JsonSerializerOptions.Web</code> instead would do nothing for requests, because ASP.NET Core serializes with its own options instance.
using System.Diagnostics;
using Microsoft.AspNetCore.Diagnostics.HealthChecks;
using Microsoft.Extensions.Diagnostics.HealthChecks;
using Microsoft.Extensions.Options;
using System.Text.Json;
using HttpJsonOptions = Microsoft.AspNetCore.Http.Json.JsonOptions;
var builder = WebApplication.CreateBuilder(args);
builder.Services.AddSingleton<WarmupState>();
builder.Services.AddSingleton<PriceList>();
builder.Services.AddHostedService<WarmupService>();
builder.Services.AddHealthChecks().AddCheck<WarmupHealthCheck>("warmup", tags: ["ready"]);
var app = builder.Build();
app.MapHealthChecks("/health/live", new HealthCheckOptions { Predicate = _ => false });
app.MapHealthChecks("/health/ready", new HealthCheckOptions { Predicate = c => c.Tags.Contains("ready") });
app.MapGet("/orders/{id:int}", (int id, PriceList prices) =>
TypedResults.Ok(new OrderSummary(id, "CUST-0042", prices.PriceOf("SKU-KETTLE"), "Paid")));
app.Run();
sealed class WarmupService(WarmupState state, PriceList prices, IOptions<HttpJsonOptions> json,
ILogger<WarmupService> logger) : BackgroundService
{
protected override Task ExecuteAsync(CancellationToken stoppingToken)
{
var sw = Stopwatch.StartNew();
// Metadata is cached per JsonSerializerOptions instance, so warm the one ASP.NET Core uses.
JsonSerializer.Serialize(new OrderSummary(0, "CUST-0000", prices.PriceOf("SKU-MUG"), "Paid"), json.Value.SerializerOptions);
state.MarkReady();
logger.LogInformation("Warm-up finished in {ElapsedMs} ms", sw.ElapsedMilliseconds);
return Task.CompletedTask;
}
}
sealed class WarmupState
{
private volatile bool _ready;
public bool IsReady => _ready;
public void MarkReady() => _ready = true;
}
sealed class WarmupHealthCheck(WarmupState state) : IHealthCheck
{
public Task<HealthCheckResult> CheckHealthAsync(HealthCheckContext context, CancellationToken cancellationToken = default) =>
Task.FromResult(state.IsReady ? HealthCheckResult.Healthy() : HealthCheckResult.Unhealthy("warming up"));
}
sealed class PriceList
{
private readonly Dictionary<string, decimal> _prices = new() { ["SKU-KETTLE"] = 39.90m, ["SKU-MUG"] = 7.50m };
public decimal PriceOf(string sku) => _prices[sku];
}
record OrderSummary(int Id, string CustomerId, decimal Total, string Status);
Discussion