In-Memory Caching

IMemoryCache keeps values in one server's memory: absolute and sliding expiry, size limits, and the stampede it does not prevent.

SyntaxTItem? GetOrCreate<TItem>(this IMemoryCache cache, object key, Func<ICacheEntry, TItem> factory, MemoryCacheEntryOptions? createOptions)

Some answers are expensive to produce. A database query, a call to another service, a long calculation. If the answer is the same for the next hundred callers, producing it a hundred times is waste. A cache is a copy of that answer kept somewhere cheap to read.

IMemoryCache is a cache that lives inside the running program, in the memory of one server. It behaves like a dictionary from a key to a value, with two extra rules: an entry can be given a lifetime, and the cache may throw entries out when it needs room. Throwing an entry out is called eviction. In a web app, call builder.Services.AddMemoryCache() and then ask for an IMemoryCache as a constructor parameter. Outside ASP.NET Core, add the Microsoft.Extensions.Caching.Memory package.

Two kinds of expiry

AbsoluteExpirationRelativeToNow gives an entry a fixed lifetime from the moment it was stored. SlidingExpiration measures from the last read, and every read restarts the clock. Sliding on its own is a trap: an entry that is read often never expires. Set both and the entry goes when either one runs out.

Size limits

The cache does not measure memory. If you set SizeLimit on MemoryCacheOptions, every entry must then set Size, in whatever unit you picked: one per entry, or the length of a string. With no limit, the cache grows until the process runs out of memory.

The stampede

Nothing locks a key. If an entry expires while fifty requests want it, all fifty miss and all fifty run the expensive work at once. GetOrCreate does not prevent that. HybridCache does.

Example

Example · csharp
using Microsoft.Extensions.Caching.Memory;

// A cache limited to 100 units of whatever "size" means here.
// Because the limit is set, every entry below must set Size.
using MemoryCache cache = new MemoryCache(new MemoryCacheOptions
{
    SizeLimit = 100
});

int loadCount = 0;

// Stands in for a database read. It runs only on a cache miss.
string LoadPrice(string sku)
{
    loadCount++;
    return sku == "A-1" ? "4.50" : "9.00";
}

string GetPrice(string sku)
{
    // GetOrCreate returns TItem?, and the factory never returns null here.
    return cache.GetOrCreate($"price:{sku}", entry =>
    {
        entry.Size = 1;
        entry.AbsoluteExpirationRelativeToNow = TimeSpan.FromMinutes(5);
        return LoadPrice(sku);
    })!;
}

Console.WriteLine(GetPrice("A-1"));
Console.WriteLine(GetPrice("A-1"));   // served from the cache
Console.WriteLine(GetPrice("B-2"));
Console.WriteLine($"loads: {loadCount}");

// Output:
// 4.50
// 4.50
// 9.00
// loads: 2

When to use it

  • Keep tax rates or country lists in memory instead of querying them on every request.
  • Hold an expensive report for 30 seconds so a dashboard refresh does not rebuild it.
  • Cache the signing keys fetched from an identity provider, with an absolute expiry.
  • Store per-tenant settings in a single-server app that has no shared cache yet.

More examples

The longhand that GetOrCreate replaces

TryGetValue plus Set, and GetOrCreate, are the same operation; GetOrCreate is the shorthand. Learn to read both, because the longhand is everywhere in existing code and is still the form you need when the miss path does more than build one value. Note that TryGetValue has a generic form, so the out variable can be typed string? rather than object.

Example · csharp
using Microsoft.Extensions.Caching.Memory;

using MemoryCache cache = new MemoryCache(new MemoryCacheOptions());

// Longhand: look, miss, build, store. Four steps written out.
if (!cache.TryGetValue("greeting", out string? greeting))
{
    greeting = "hello";

    MemoryCacheEntryOptions options = new MemoryCacheEntryOptions
    {
        SlidingExpiration = TimeSpan.FromMinutes(2),
        AbsoluteExpirationRelativeToNow = TimeSpan.FromMinutes(10)
    };

    cache.Set("greeting", greeting, options);
}

// Shorthand: the same four steps in one call. The lambda is the factory,
// and it runs only when the key is missing.
string? sameThing = cache.GetOrCreate("greeting", entry =>
{
    entry.SlidingExpiration = TimeSpan.FromMinutes(2);
    entry.AbsoluteExpirationRelativeToNow = TimeSpan.FromMinutes(10);
    return "hello";
});

Console.WriteLine(greeting);
Console.WriteLine(sameThing);

// Output:
// hello
// hello

Guarding one key against a stampede

Reach for this when one expensive key is hit hard and a miss would otherwise fire dozens of identical queries. The second look inside the gate is the part people leave out, and without it the first waiter reloads anyway. One gate for the whole cache also serialises unrelated keys, which is why HybridCache, which coordinates per key, is the better answer once you are allowed to add the package.

Example · csharp
using Microsoft.Extensions.Caching.Memory;

public static class PriceCache
{
    private static readonly MemoryCache _cache = new(new MemoryCacheOptions());

    // A SemaphoreSlim with one slot lets exactly one caller through at a time.
    private static readonly SemaphoreSlim _gate = new(1, 1);

    public static async Task<string> GetAsync(string sku, CancellationToken token)
    {
        if (_cache.TryGetValue(sku, out string? cached))
        {
            return cached!;
        }

        await _gate.WaitAsync(token);

        try
        {
            // Look again. Another caller may have filled the entry while we waited.
            if (_cache.TryGetValue(sku, out cached))
            {
                return cached!;
            }

            string loaded = await LoadFromDatabaseAsync(sku, token);
            _cache.Set(sku, loaded, TimeSpan.FromMinutes(1));
            return loaded;
        }
        finally
        {
            _gate.Release();
        }
    }

    private static Task<string> LoadFromDatabaseAsync(string sku, CancellationToken token)
    {
        return Task.FromResult("4.50");
    }
}

A cached lookup table with explicit invalidation

The shape to copy for small reference data that changes rarely: one entry holding the whole table, an absolute expiry as a safety net, and a Remove call on the write path so a change is visible at once. One warning the code cannot show: every caller receives the same dictionary object, not a copy of it, so adding to it or editing it changes what everyone else sees. Treat anything you put in a cache as read-only.

Example · csharp
using Microsoft.Extensions.Caching.Memory;

// Registered with builder.Services.AddMemoryCache();
// and builder.Services.AddSingleton<TaxRates>();
public sealed class TaxRates
{
    private const string CacheKey = "tax-rates";

    private readonly IMemoryCache _cache;

    public TaxRates(IMemoryCache cache)
    {
        _cache = cache;
    }

    public decimal GetRate(string region)
    {
        Dictionary<string, decimal> rates = _cache.GetOrCreate(
            CacheKey,
            entry =>
            {
                entry.AbsoluteExpirationRelativeToNow = TimeSpan.FromHours(1);
                return LoadRates();
            })!;

        return rates[region];
    }

    // Call this after writing new rates, so the next read reloads them.
    public void Invalidate() => _cache.Remove(CacheKey);

    private static Dictionary<string, decimal> LoadRates()
    {
        return new Dictionary<string, decimal>
        {
            ["uk"] = 0.20m,
            ["ie"] = 0.23m
        };
    }
}

Related

dotnet hybrid cachedotnet distributed cachedotnet di lifetimesdotnet dictionarydotnet background service

Discussion

  • Be the first to comment on this lesson.