Sizing a Cache
Working out the hot set, the hit rate the backend needs, and the RAM that implies - including when the whole dataset just fits.
RAM ≈ hot_entries * (key_bytes + value_bytes + overhead_bytes) ; backend_QPS = (1 - hit_rate) * QPSSizing a cache is three numbers: how many distinct entries are hot, how many bytes each costs including overhead, and what hit rate the backend needs. Guessing the first and ignoring the third is how a cache ends up too small to help and too expensive to keep.
The hit rate is the point
Backend load after a cache is (1 - hit rate) x QPS. A 90% hit rate takes the origin to a tenth; 99% takes it to a hundredth. That reads like an argument for chasing the last few points, but access distributions are long-tailed: the first 90% of requests may come from a few thousand keys and the next 9% from millions. Each extra point of hit rate costs more RAM than the one before it.
Overhead is not a rounding error
A Redis key is a full object, tracking encoding, reference count, idle time and expiry. Redis's own documentation measures 100,000 small values stored as plain keys at 11 MB, and the same data packed into hashes of about 100 fields each at 1.7 MB, on 64-bit Redis 2.2. Instagram reported the same trick taking one million entries from 70 MB to 16 MB. The compact encoding only applies under the configured thresholds: hash-max-ziplist-entries up to Redis 6.2, renamed hash-max-listpack-entries in Redis 7.0, default 512 either way.
When the whole thing fits
If the dataset is tens of gigabytes, it fits in one machine's RAM and the design simplifies sharply: no eviction policy to argue about, no cold-start stampede, and invalidation is the only hard problem left.
Example
// Three inputs decide a cache. Two of them you have to measure.
function sizeCache({ hotEntries, keyBytes, valueBytes, overheadBytes }) {
const bytesPerEntry = keyBytes + valueBytes + overheadBytes;
const bytes = hotEntries * bytesPerEntry;
return {
gb: bytes / 1e9,
// Redis never hands freed memory back to the OS, and allocators fragment.
// Size the box for the high-water mark, not the average.
provisionGb: (bytes / 1e9) * 1.3,
};
}
// overheadBytes is NOT a constant you can look up - it depends on the value
// type, the encoding and the Redis version. Measure it on your own data with
// MEMORY USAGE <key> (available since Redis 4.0) and use what you observe.
sizeCache({
hotEntries: 2e6, // 2 million hot sessions
keyBytes: 40, // "session:" + a UUID
valueBytes: 600, // serialised session blob
overheadBytes: 90, // measured, not guessed
});
// { gb: 1.46, provisionGb: 1.898 } -> a 4 GB node, comfortably
// What the hit rate buys the origin:
const backendQps = (qps, hitRate) => qps * (1 - hitRate);
backendQps(50_000, 0.80); // 10_000 -> a fifth
backendQps(50_000, 0.90); // 5_000 -> a tenth
backendQps(50_000, 0.99); // 500 -> a hundredth
// Then ask the question that decides the size: how many keys do I have to hold
// to get from 90% to 99%? If the answer is "a hundred times more", the honest
// design change is to make the origin cheaper, not the cache bigger.When to use it
- Choosing a Redis instance size before a launch instead of after the first out-of-memory incident.
- Deciding whether a 99% hit rate target is worth the RAM it would take over 95%.
- Arguing that a 40 GB dataset should live entirely in memory and skip the eviction design.
- Diagnosing why a cache holding "only a few million small keys" is using far more memory than the data does.
More examples
The hit-rate ladder, and where it stops paying
Converts hit rate into the only unit that settles the argument - origin hardware. Use it to put a number on what another point of hit rate is actually worth before anyone orders a bigger cache node.
// 50,000 QPS at the front door. Each origin read costs ~4 ms of database time.
const QPS = 50_000;
const ladder = [0.50, 0.80, 0.90, 0.95, 0.99, 0.999].map((hitRate) => ({
hitRate,
originQps: QPS * (1 - hitRate),
// 4 ms of DB work per miss, expressed as cores needed to keep up
originCores: (QPS * (1 - hitRate) * 4) / 1000,
}));
// hitRate 0.50 -> 25,000 qps, 100 cores
// hitRate 0.80 -> 10,000 qps, 40 cores
// hitRate 0.90 -> 5,000 qps, 20 cores
// hitRate 0.95 -> 2,500 qps, 10 cores
// hitRate 0.99 -> 500 qps, 2 cores
// hitRate 0.999 -> 50 qps, 0.2 cores
//
// Note where the savings are. Going 50% -> 90% removes 80 cores. Going
// 99% -> 99.9% removes 1.8. The first is a design decision; the second is
// a rounding error you will pay for in RAM.Packing many small values into fewer keys
Shows the long form (one key per object) next to the packed form, and names both spellings of the threshold directive so you recognise either in a config file. Reach for it when a cache of many tiny values is memory-bound rather than CPU-bound - and note the cost on older servers: a hash field had no expiry of its own until Redis 7.4, which added per-field TTLs (HEXPIRE, HPEXPIRE, HEXPIREAT, HTTL, HPERSIST). Before 7.4 the whole hash expires or nothing does, so a packed cache of entries with individual lifetimes has to evict them itself.
// Redis's documented measurement: 100,000 short values, 64-bit Redis 2.2.
// stored as plain keys ............ 11 MB
// packed into hashes of ~100 ...... 1.7 MB
// Instagram reported the same shape: 1M entries, 70 MB -> 16 MB.
// LONGHAND: one Redis key per object.
// SET object:1234 "somevalue"
await redis.set('object:1234', 'somevalue');
// SHORTHAND the memory wants: split the id so ~100 objects share one hash.
// All but the last two digits become the key; the last two become the field.
function hashSlot(id) {
const s = String(id);
return s.length > 2
? { key: `object:${s.slice(0, -2)}`, field: s.slice(-2) }
: { key: 'object:', field: s };
}
const { key, field } = hashSlot(1234); // { key: 'object:12', field: '34' }
await redis.hset(key, field, 'somevalue'); // HSET object:12 34 somevalue
// This only pays while each hash stays under the compact-encoding thresholds.
// The directive was renamed in Redis 7.0; both spellings default to 512:
// Redis <= 6.2 hash-max-ziplist-entries 512 / hash-max-ziplist-value 64
// Redis >= 7.0 hash-max-listpack-entries 512 / hash-max-listpack-value 64
// Exceed either and Redis converts to a real hash table, and the saving is gone.When the dataset fits and the design collapses
The check worth doing before any eviction or sharding discussion. When the whole working set fits in one machine's RAM, most of the hard parts of cache design are not your problem - so find out early.
// Catalogue: 2 million products, ~3 KB of rendered JSON each.
const products = 2e6;
const bytesEach = 3_000 + 40 + 90; // value + key + measured overhead
const gb = (products * bytesEach) / 1e9; // ~6.3 GB
const provisionGb = gb * 1.3; // ~8.2 GB -> one 16 GB node
// Everything fits, so several design problems simply disappear:
// - eviction policy: irrelevant, nothing is ever evicted under pressure
// - cold start: a single warm-up job, not a stampede on every deploy
// - tail latency: no miss path, so p99 is cache latency, not origin latency
// - hot keys: still a problem, but a read-concurrency one, not a capacity one
//
// What remains is invalidation: when a product changes, something has to
// update or delete this entry, and getting that wrong now serves stale data
// forever rather than for the length of a TTL.
//
// Always run this calculation first. "It all fits" is the cheapest answer
// available, and it is more often true than people expect.
Discussion