IMemoryCache

Cache hot reads inside one process with IMemoryCache: GetOrCreateAsync, absolute and sliding expiry, size limits and the per-pod caveats.

Syntax

builder.Services.AddMemoryCache(o => o.SizeLimit = 10_000);
var p = await cache.GetOrCreateAsync(("product", id), async e => {
    e.AbsoluteExpirationRelativeToNow = TimeSpan.FromMinutes(5);
    e.Size = 1;
    return await repo.LoadAsync(id); });

IMemoryCache is a thread-safe dictionary of object references with expiry, living in the heap of one process. builder.Services.AddMemoryCache() registers a singleton MemoryCache (it uses TryAdd, so calling it twice is harmless). A hit costs a dictionary lookup and no serialisation, which makes it the fastest cache you have and also the least shared: every pod has its own copy, and a restart empties it.

GetOrCreateAsync and what it does not do

cache.GetOrCreateAsync(key, async entry => ...) looks the key up and, on a miss, runs the factory, applies whatever you set on entry and stores the result. Three details matter. A null result is cached like any other value, which is useful for negative caching with a short TTL. An exception from the factory caches nothing. And there is no stampede protection: ten concurrent misses for the same key run the factory ten times and the last write wins. For an expensive query on a busy key, that is ten identical database queries at the worst moment. HybridCache coalesces them; plain IMemoryCache does not.

Keys are compared with Equals, so a tuple like ("product", id) is a good key. A bare int is not: the orders feature and the products feature will collide on 42.

Expiry

Setting	Meaning	Use it for
`AbsoluteExpirationRelativeToNow`	Gone after a fixed time, however often it is read	The staleness bound you promise the business
`SlidingExpiration`	Gone after a period with no reads	Dropping cold keys early; always pair with an absolute limit
`Priority`	Order of removal under memory pressure; `NeverRemove` pins	Reference data you must not lose

Sliding expiry alone means a key read every few seconds never expires and never sees a price change. Expired entries are removed lazily: on access, and in a scan that runs at most once per ExpirationScanFrequency (default one minute), triggered by cache activity rather than a timer.

Size limits

By default SizeLimit is null and the cache grows until the pod is OOM-killed. Set SizeLimit and every entry must set Size in units you choose (one per entry, or approximate bytes); an entry without a size throws InvalidOperationException. When an insert would exceed the limit the new entry is silently not stored, and a background compaction removes CompactionPercentage (default 5%) of entries: expired first, then low priority, then least recently used. The caller still gets its value, so the only symptom is a falling hit rate. Turn on TrackStatistics and export GetCurrentStatistics() to see it.

Production failure modes

Cached objects are shared references. A handler that mutates a cached Order changes it for every later request. Cache immutable records.
With three replicas, a write evicts only the pod that handled it. The other two serve the old value until the TTL. Keep TTLs short, or broadcast invalidations.
Caching per-user data under a key without the user id leaks data between users.

Example

Loading editor…

Press Run to execute the code.

When to use it

Keep the 2,000 most-read product records in each pod for 5 minutes so catalogue pages stop querying PostgreSQL on every request.
Cache the result of an exchange-rate lookup for 60 seconds per currency pair so checkout does not call the rates service per line item.
Cache a 'customer not found' result for 10 seconds to absorb a client retrying GET /customers/{id} in a tight loop.

More examples

What a size limit really does when the cache is full

With <code>SizeLimit</code> set, an entry without <code>Size</code> throws instead of being stored. An insert that would overflow the limit is not stored at all and does not throw; it triggers a compaction that removes a percentage of entries on another thread. In an API that shows up only as more cache misses and more database load, so watch the hit rate from <code>GetCurrentStatistics()</code> rather than waiting for an exception.

using Microsoft.Extensions.Caching.Memory;

using var cache = new MemoryCache(new MemoryCacheOptions
{
    SizeLimit = 3,
    CompactionPercentage = 0.34,
    TrackStatistics = true,
});

try
{
    cache.Set("rates:EUR", 1.08m); // no Size while SizeLimit is set
}
catch (InvalidOperationException ex)
{
    Console.WriteLine($"rejected: {ex.Message}");
}

var sized = new MemoryCacheEntryOptions { Size = 1 };
cache.Set("rates:EUR", 1.08m, sized);
cache.Set("rates:GBP", 1.27m, sized);
cache.Set("rates:JPY", 0.0067m, sized);

// The cache is full: this insert is dropped silently and a compaction is scheduled.
cache.Set("rates:CHF", 1.12m, sized);
Console.WriteLine($"CHF cached: {cache.TryGetValue("rates:CHF", out _)}");

await Task.Delay(100); // compaction runs on the thread pool
var stats = cache.GetCurrentStatistics()!;
Console.WriteLine($"entries now: {stats.CurrentEntryCount}, size: {stats.CurrentEstimatedSize}");

Pitfall: mutating a cached object, and the fix

<code>IMemoryCache</code> stores the reference you give it and hands the same reference to every caller, so a write to a cached object is a write to shared state, often from many threads at once. Distributed caches hide this bug because each read deserialises a fresh copy, which is why it tends to appear only after someone swaps Redis for an in-memory cache. Cache records or other immutable types and never modify what you get back.

using Microsoft.Extensions.Caching.Memory;

using var cache = new MemoryCache(new MemoryCacheOptions());

// Broken: a mutable class is cached by reference.
var retail = cache.GetOrCreate("price-list:retail", _ => new MutablePriceList { Discount = 0m })!;
retail.Discount = 0.25m; // one request applies a VIP discount "locally"
var next = cache.Get<MutablePriceList>("price-list:retail")!;
Console.WriteLine($"next request sees discount {next.Discount}"); // 0.25: leaked

// Fixed: cache an immutable record and derive new values with `with`.
var list = cache.GetOrCreate("price-list:wholesale", _ => new PriceList("wholesale", 0.10m))!;
var vip = list with { Discount = 0.25m };
Console.WriteLine($"cached {cache.Get<PriceList>("price-list:wholesale")!.Discount}, local {vip.Discount}");

public sealed class MutablePriceList
{
    public decimal Discount { get; set; }
}

public sealed record PriceList(string Channel, decimal Discount);

A separate cache per feature with its own limit

One shared <code>SizeLimit</code> means a feature with many large entries pushes out everyone else's. A keyed singleton gives a feature its own <code>MemoryCache</code> and budget, and its statistics can be exported separately. The post-eviction callback runs on a thread pool thread after removal and receives an <code>EvictionReason</code> such as <code>Expired</code>, <code>Capacity</code> or <code>Removed</code>; a lot of <code>Capacity</code> evictions means the limit is too small.

using Microsoft.Extensions.Caching.Memory;

var builder = WebApplication.CreateBuilder(args);

// The shared IMemoryCache stays for small lookups.
builder.Services.AddMemoryCache(o => o.SizeLimit = 5_000);

// Tax rules are large; give them their own budget so they cannot evict everything else.
builder.Services.AddKeyedSingleton<IMemoryCache>("tax-rules", (_, _) =>
    new MemoryCache(new MemoryCacheOptions { SizeLimit = 200, TrackStatistics = true }));

builder.Services.AddSingleton<TaxRuleProvider>();
var app = builder.Build();

app.MapGet("/tax-rules/{country}", async (string country, TaxRuleProvider rules, CancellationToken ct) =>
    TypedResults.Ok(await rules.GetAsync(country.ToUpperInvariant(), ct)));

app.Run();

public sealed record TaxRule(string Country, decimal StandardRate, decimal ReducedRate);

public sealed class TaxRuleProvider([FromKeyedServices("tax-rules")] IMemoryCache cache, ILogger<TaxRuleProvider> logger)
{
    public async Task<TaxRule> GetAsync(string country, CancellationToken ct) =>
        (await cache.GetOrCreateAsync(country, async entry =>
        {
            entry.Size = 1;
            entry.AbsoluteExpirationRelativeToNow = TimeSpan.FromHours(6);
            entry.RegisterPostEvictionCallback((key, _, reason, _) =>
                logger.LogDebug("Tax rule {Country} left the cache: {Reason}", key, reason));
            await Task.Delay(20, ct); // stands in for the tax service call
            return new TaxRule(country, 0.20m, 0.05m);
        }))!;
}

Discussion

Be the first to comment on this lesson.

GetOrCreateAsync and what it does not do

Expiry

Size limits

Production failure modes

Example

When to use it

More examples

What a size limit really does when the cache is full

Pitfall: mutating a cached object, and the fix

A separate cache per feature with its own limit

Related

Discussion