Traffic and Storage Maths
Turning daily actives into requests per second and bytes per record into terabytes per year, in round numbers, without a calculator.
avg QPS = DAU * actions_per_user_per_day / 1e5 ; peak QPS = 2..3 * avg QPS ; bytes_per_year = records_per_day * bytes_per_record * 365Every capacity question starts from two conversions: users per day to requests per second, and bytes per record to bytes per year. Both are done in round numbers out loud, because every input is an estimate and precision you did not have going in is not precision coming out.
Daily actives to QPS
A day is 86,400 seconds. Round it to 100,000, or 1e5. Average QPS is then daily actions divided by 1e5: ten million daily actives doing five actions each is 50 million actions a day, so 500 requests per second. The rounding makes the answer about 16% low, which is nothing next to the error in "five actions each".
Average is not what you provision for. Consumer traffic typically peaks at two to three times its daily average; a product with a cron job, a daily digest or a single timezone concentrated behind it can be much spikier. State the factor you used.
Bytes to terabytes
Multiply bytes per record by records per day, then by 365. Count the record once per copy that exists: three replicas is three times the bytes, and secondary indexes are not free. Vendors bill in decimal units, so divide by 1e9 for GB rather than by 2^30, and say which you meant.
What trips people up
A request is not a query. One API call that reads a cache, writes a primary and calls two services is four or more backend operations, so the number at the front door and the number at the database differ by a factor you have to state.
Example
// A day is 86,400 seconds. We use 1e5. The three spellings below are the same
// literal: 1e5 is exponent notation, 100_000 uses ES2021 numeric separators,
// 100000 is the long form. Pick one and stay with it.
const SECONDS_PER_DAY = 1e5; // === 100_000 === 100000
function estimate({ dau, actionsPerUserPerDay, bytesPerRecord, peakFactor = 3, replicas = 3 }) {
const recordsPerDay = dau * actionsPerUserPerDay;
const avgQps = recordsPerDay / SECONDS_PER_DAY;
const peakQps = avgQps * peakFactor;
const bytesPerDay = recordsPerDay * bytesPerRecord;
const bytesPerYear = bytesPerDay * 365;
return {
avgQps,
peakQps,
// Decimal units, because that is how storage is billed and quoted.
perDayGB: bytesPerDay / 1e9,
perYearTB: bytesPerYear / 1e12,
// Every copy is a copy you pay for.
storedTB: (bytesPerYear * replicas) / 1e12,
};
}
estimate({ dau: 10e6, actionsPerUserPerDay: 5, bytesPerRecord: 1_000 });
// avgQps 500, peakQps 1500, perDayGB 50, perYearTB 18.25, storedTB 54.75
//
// Read it back out loud: "ten million actives, five writes each, so fifty
// million writes a day, five hundred a second, fifteen hundred at peak.
// A kilobyte each is fifty gigabytes a day, eighteen terabytes a year,
// fifty-five with three replicas."When to use it
- Opening a design interview by converting the stated user count into read and write QPS before drawing anything.
- Deciding whether a year of data fits on one database node or needs sharding.
- Checking whether a proposed retention policy triples your storage bill.
- Separating front-door request rate from backend query rate when sizing a connection pool.
More examples
Sanity-check your method against a published figure
Grounds the peak factor and the read/write ratio in a published number instead of a guess. Use it to justify the multipliers you apply out loud, and to decide which path deserves the whiteboard.
// Twitter's load, as reported in Designing Data-Intensive Applications
// (figures from November 2012):
const postTweetAvgQps = 4_600;
const postTweetPeakQps = 12_000;
const timelineReadQps = 300_000;
postTweetPeakQps / postTweetAvgQps; // 2.6 <- the peak factor, measured
timelineReadQps / postTweetAvgQps; // ~65 <- the read:write ratio
// Two things to take away. The 2-3x peak rule of thumb is not folklore; it is
// what a real consumer product measured. And a 65:1 read:write ratio is why
// the interesting part of this design is the read path, not the write path.Storage, counted per copy
Shows the three multipliers that turn a comfortable raw figure into an uncomfortable real one. Reach for it before claiming a dataset fits on a single node.
// Chat product: 1 KB per message, 50 million messages a day.
const bytesPerMessage = 1_000;
const messagesPerDay = 50e6;
const rawPerYearTB = (messagesPerDay * bytesPerMessage * 365) / 1e12; // 18.25 TB
// Now count what actually lands on disk:
const copies = {
replicas: 3, // quorum across three nodes
secondaryIndexes: 0.3, // index on (user_id, created_at), roughly 30%
compressionRatio: 0.4, // text compresses well; 2.5:1 is conservative
};
const storedTB =
rawPerYearTB * copies.replicas * (1 + copies.secondaryIndexes) * copies.compressionRatio;
// ~28.5 TB per year on disk
// 18 TB sounds like one node. 28.5 TB per year, growing, does not - and that
// is the number the sharding conversation has to start from.The powers you should recognise on sight
The arithmetic you need to do in your head, plus the shorthand and longhand spellings of exponentiation so you recognise either in someone else's code. Memorise the three second-counts; you will use them in every sizing conversation.
// Binary and decimal, side by side. 2^10 is 1024, which we call 1e3.
// The error compounds: by 2^40 the two differ by about 10%.
2 ** 10; // 1024 ~ 1e3 (thousand, KB)
2 ** 20; // 1_048_576 ~ 1e6 (million, MB)
2 ** 30; // ~1.07e9 ~ 1e9 (billion, GB)
2 ** 40; // ~1.10e12 ~ 1e12 (trillion, TB)
// ** is the exponentiation operator, added in ES2016. It is shorthand for the
// longhand call, and the two are interchangeable:
2 ** 30; // shorthand
Math.pow(2, 30); // longhand - identical result
// Useful constants worth memorising rather than deriving:
const SECONDS_PER_DAY = 86_400; // round to 1e5
const SECONDS_PER_MONTH = 2_592_000; // round to 2.6e6
const SECONDS_PER_YEAR = 31_536_000; // round to 3e7
Discussion