Numbers Every Engineer Should Know

A dozen latency figures, each tier roughly a thousand times slower than the one above it, that decide where data has to live.

Syntaxmemory 100 ns · SSD 4 KB random read 150 us · same-DC round trip 500 us · cross-continent round trip 150 ms

Most system design decisions are settled by orders of magnitude rather than by benchmarks, and the reference list for those magnitudes is Jeff Dean's "Numbers Everyone Should Know", built on an earlier table by Peter Norvig. The widely circulated version was last revised around 2012.

The gaps that matter

OperationTime
L1 cache reference0.5 ns
Main memory reference100 ns
Read 4 KB randomly from SSD150 us
Round trip within the same datacentre500 us
Disk seek10 ms
Packet California to Netherlands and back150 ms

Memory to a same-datacentre round trip is a factor of 5,000. That round trip to a cross-continent one is another 300. Nothing inside a process competes with a network hop, and nothing inside one datacentre competes with crossing an ocean.

What this decides

A cache belongs on the shortest hop that still gives an acceptable hit rate: in-process is 100 ns, a Redis call in the same datacentre is 500 us, a cross-region call is tens of milliseconds and is not really a cache. The same arithmetic says one user-facing request cannot afford more than a handful of sequential cross-service calls inside a 200 ms budget.

Where the list has aged

The ratios held; some absolutes did not. The 150 us random SSD read and the 10 ms disk seek describe a SATA SSD and a rotating platter, and both are pessimistic for current NVMe hardware. Use the list for ratios and measure for absolutes.

Example

Example · javascript
// Jeff Dean's "Numbers Everyone Should Know", normalised to nanoseconds.
// Keep one unit throughout; mixing ns, us and ms is where the mistakes happen.
// 1_000 and 1000 are the same literal - numeric separators are ES2021 syntax,
// purely for readability, and desugar to the plain form.
const NS = 1;
const US = 1_000;        // longhand: 1000
const MS = 1_000_000;    // longhand: 1000000

const LATENCY = {
  l1Cache:            0.5 * NS,
  branchMispredict:     5 * NS,
  l2Cache:              7 * NS,
  mutexLockUnlock:     25 * NS,
  mainMemory:         100 * NS,
  compress1KB:          3 * US,   // Zippy/Snappy, per the original list
  send1KBOver1Gbps:    10 * US,
  ssdRandomRead4KB:   150 * US,
  read1MBFromMemory:  250 * US,
  sameDatacentreRTT:  500 * US,
  read1MBFromSSD:       1 * MS,
  diskSeek:            10 * MS,   // rotating platter, NOT an SSD
  read1MBFromDisk:     20 * MS,
  crossContinentRTT:  150 * MS,   // California -> Netherlands -> California
};

// The only thing you should take from this table is ratios.
const ratio = (a, b) => LATENCY[a] / LATENCY[b];

ratio('sameDatacentreRTT', 'mainMemory');         // 5000
ratio('crossContinentRTT', 'sameDatacentreRTT');  // 300
ratio('read1MBFromDisk', 'read1MBFromMemory');    // 80

// Budget check: how many sequential same-DC hops fit in a 200 ms p99?
const hopsIn = (budgetMs) => Math.floor((budgetMs * MS) / LATENCY.sameDatacentreRTT);
hopsIn(200);  // 400 in theory - but service time, not RTT, is what eats the budget

When to use it

  • Deciding whether a value belongs in an in-process cache, in Redis, or recomputed on each request.
  • Arguing against a design that puts a synchronous cross-region call on the read path.
  • Setting a per-hop latency budget before splitting a monolith into services.
  • Explaining why batching 1,000 random reads into one sequential scan is not a micro-optimisation.

More examples

Cache the result, or recompute it?

Shows that a cache usually wins on throughput rather than on latency: the 4x latency gain is small, but moving 10 cores of work off the fleet is not. Reach for this when someone defends recomputing on the grounds that it is "fast enough" per request.

Example · javascript
// A derived value costs ~2 ms of CPU to recompute. Caching it in Redis in the
// same datacentre costs one round trip, ~500 us, plus serialisation.
const RECOMPUTE_US = 2_000;   // 2 ms of CPU
const REDIS_RTT_US =   500;   // same-datacentre round trip

// Latency says cache: 4x faster. But latency is not the reason that matters.
RECOMPUTE_US / REDIS_RTT_US;  // 4

// The reason that matters is CPU. At 5,000 QPS, recomputing burns
// 5000 * 2ms = 10 seconds of CPU per second - ten cores doing nothing else.
// The Redis path burns almost no CPU and spends wall-clock time instead.
const coresBurned = (qps, cpuMs) => (qps * cpuMs) / 1000;
coresBurned(5_000, 2);  // 10

Sequential versus random, costed

Quantifies why access pattern beats raw device speed. Use it when choosing a primary key or deciding whether to denormalise a read path into a single contiguous fetch.

Example · javascript
// From the list, in microseconds:
const SSD_RANDOM_4KB = 150;      // one random read
const SSD_1MB_SEQ    = 1_000;    // 1 MB streamed

// Fetching 256 scattered 4 KB rows (1 MB of payload, 256 seeks):
256 * SSD_RANDOM_4KB;            // 38,400 us = 38 ms

// Fetching the same 1 MB as one contiguous range:
SSD_1MB_SEQ;                     // 1,000 us = 1 ms

// ~38x. This is the whole argument for clustered indexes, for storing a
// timeline as one blob per user, and for columnar layouts in analytics.

A latency budget that does not close

Shows how to spot the one hop that makes a budget impossible before optimising anything else. Reach for this when a design keeps its database in one region and its traffic in another.

Example · javascript
// Proposed read path for a 200 ms p99 target, times in milliseconds.
const path = [
  { hop: 'TLS + CDN to origin',        ms: 20 },
  { hop: 'API gateway to app tier',    ms: 1 },
  { hop: 'auth service (same DC)',     ms: 5 },
  { hop: 'profile service (same DC)',  ms: 8 },
  { hop: 'primary DB in eu-west-1',    ms: 90 },  // app is in us-east-1
  { hop: 'render + response',          ms: 15 },
];

const total = path.reduce((sum, h) => sum + h.ms, 0);  // 139 ms

// 139 ms of mean looks fine against 200 ms. It is not: these are sequential,
// and p99 of a chain is far worse than the sum of the means. us-east-1 to
// eu-west-1 measures roughly 70-80 ms of round trip before the query even
// runs, so that hop has a hard floor you cannot tune and it carries the whole
// tail. (The 150 ms in the table is California to Netherlands and back, a
// longer path than Virginia to Ireland - do not reuse it for any region pair.)
// The fix is a read replica in-region, not shaving the 8 ms hop.

Related

sd qps and storagesd cache layerssd percentilessd when estimates decidesd bandwidth and cost

Discussion

  • Be the first to comment on this lesson.