Back-of-the-envelope capacity estimation for system design

"It needs to handle millions of users." That sentence is where most system design conversations start, and it contains nothing a design can act on. A million users who click once a week is a hobby project; a million who poll every second is a serious engineering problem. Until the sentence becomes requests per second, bytes per second, and gigabytes per year, every box on the diagram is a guess about a guess.

Back-of-the-envelope estimation turns the sentence into those numbers in ten minutes, with arithmetic simple enough for your head and honest enough to be wrong by a factor of two rather than a thousand. The point is not precision. It is finding out which component saturates first, because that is the one the design has to be about.

Capacity ladder

One multiplier per rung, from users to bytes

link shortener, worked example

1. 01 daily active users 50,000,000
2. ↓ x 2 redirects per user
   02 requests per day 100,000,000
3. ↓ / 100,000 s (a rounded day)
   03 average requests / s 1,000
4. ↓ x 3 peak multiplier
   04 peak requests / s 3,000
5. ↓ x 800 bytes per request
   05 peak bandwidth 2.4 MB / s
6. ↓ writes x row bytes x 365
   06 storage per year 91 GB links, 3.65 TB clicks

saturates first

the click-event write path: 3,000 inserts / s at peak and 10 GB a day, on the same database the redirects read from

An estimate is not a forecast. It is a way of finding out which component saturates first, so the design can be about that one.

From users to requests per second

One example carries through the whole article: a link shortener. Users create short links and the world clicks them. Disagree with an assumption? Change it and re-run the arithmetic.

daily active users (DAU)          50,000,000
redirects per user per day        2
new links per user per day        0.01   (1 in 100)

redirects per day                 50,000,000 x 2    = 100,000,000
creates per day                   50,000,000 x 0.01 =     500,000

A day has 86,400 seconds. For mental math, call it 100,000; the result comes in about 14 percent low, well inside the error of every other assumption on the page.

average redirects per second      100,000,000 / 100,000 = 1,000
average creates per second            500,000 / 100,000 =     5

peak multiplier (3x)              reads 3,000 / s    writes 15 / s

Traffic is never flat. A consumer product peaks at two to three times its daily average; one with a synchronized moment, a push notification or a scheduled drop, peaks at five times or more. Pick one, write it down, design for the peak. The average is what the bill sees; the peak is what users see.

Bandwidth: the numbers you get to ignore

A redirect is a small request and a smaller response, a 301 with a Location header. Call it 300 bytes in and 500 bytes out.

peak bandwidth                    3,000 / s x 800 bytes = 2.4 MB / s
                                                        = about 19 Mbit / s

One network interface laughs at nineteen megabits per second. That is a real result: half the value of an estimate is the list of things it tells you not to build for. Bandwidth is off the list.

Storage per year, with retention

A link row holds a short code, the long URL, an owner, timestamps, and an expiry; call it 500 bytes with index overhead.

links per year                    500,000 x 365 = 182,500,000
storage per year                  182,500,000 x 500 bytes = 91 GB
five-year retention               about 455 GB

That is one PostgreSQL instance with a comfortable disk, but it is only the table everyone remembers. If the product records every click, a 100-byte click row at 100 million clicks per day is 10 GB per day, or 3.65 TB per year. The side table nobody put on the whiteboard is forty times bigger than the main one, and it decides the storage design.

The hot set and the cache

Reads outnumber writes 200 to 1, so the read path is where the design lives, and it is a cache question. The 80/20 rule is the standard estimate: 20 percent of links get 80 percent of clicks. Assume the day's 100 million redirects touch 20 million distinct links.

hot links                         20,000,000 x 20% = 4,000,000
cache entry (code + URL + ttl)    about 300 bytes
cache size                        4,000,000 x 300 bytes = 1.2 GB

database reads at peak, 80% hit   3,000 x 20% = 600 / s
database reads at peak, cold      3,000 / s

A 1.2 GB hot set fits on one cache node, and at an 80 percent hit rate the database sees 600 reads per second at peak, or 3,000 on the morning the cache restarts empty. The caching strategies guide covers what a cold start does to a database sized for the warm one.

Concurrency via Little's law

The estimate so far counts requests per second. Pools and worker counts are sized in requests in flight, and Little's law converts one to the other: L = lambda x W, the number in the system equals the arrival rate times the time each spends there.

database at peak, warm cache        600 / s x 5 ms   =   3 connections busy
database at peak, cold cache      3,000 / s x 5 ms   =  15 connections busy
same, if queries slow to 50 ms    3,000 / s x 50 ms  = 150 connections busy

web tier at peak                  3,000 / s x 10 ms  =  30 requests in flight
click events, batched             3,000 / s in batches of 500 per 100 ms = 1 worker

Three busy connections warm, fifteen cold, and 150 if the database slows to 50 milliseconds under the cold-cache load, which it will, because the load is what slows it. A pool of 100 across the fleet is fine on every row except the last, and the last row is the incident. W is not a constant; it moves when the system is in trouble, and the pool sized for the happy W is the pool that empties. The connection pooling article shows what exhaustion looks like from the request that gets blamed for it.

The same estimator in code, using the exact 86,400 rather than the rounded 100,000:

var shortener = new CapacityEstimate(
    DailyActiveUsers: 50_000_000,
    RequestsPerUserPerDay: 2,
    ReadRatio: 0.995,
    PayloadBytes: 800);

Console.WriteLine($"peak reads/s: {shortener.PeakReadQps:N0}");

public sealed record CapacityEstimate(
    long DailyActiveUsers,
    double RequestsPerUserPerDay,
    double ReadRatio,
    int PayloadBytes,
    double PeakMultiplier = 3)
{
    private const double SecondsPerDay = 86_400;

    public double RequestsPerDay => DailyActiveUsers * RequestsPerUserPerDay;
    public double AverageQps => RequestsPerDay / SecondsPerDay;
    public double PeakQps => AverageQps * PeakMultiplier;
    public double PeakReadQps => PeakQps * ReadRatio;
    public double PeakBandwidthBytesPerSecond => PeakQps * PayloadBytes;

    public double StorageBytesPerYear(int bytesPerWrite) =>
        RequestsPerDay * (1 - ReadRatio) * bytesPerWrite * 365;
}

An example run reports 1,157 average QPS, 3,472 peak QPS (3,455 of them reads), 2.8 MB/s of peak bandwidth, and 91 GB of link storage per year. The mental-math version said 1,000 and 3,000. Both point at the same component, which is the only sense in which an estimate is right.

The latency ladder

Counting how many is half of capacity. The other half is how long, which needs the ladder known as the latency numbers every programmer should know. Jeff Dean circulated the original figures in his "Numbers Everyone Should Know" slides, Peter Norvig published a version in Teach Yourself Programming in Ten Years, and later maintainers added the SSD row as hardware changed:

L1 cache reference                       0.5 ns
main memory reference                  100 ns
SSD random read                    150,000 ns   (150 us)
round trip within one datacenter   500,000 ns   (0.5 ms)
round trip across an ocean     150,000,000 ns   (150 ms)

The exact values drift with every hardware generation; the ratios do not, and ratios are what an estimate needs: memory is 200 times slower than L1, an SSD 1,500 times slower than memory, a hop inside the datacenter about three times slower than the SSD, and an ocean crossing 300 times slower than that. A redirect that makes one cache lookup pays about a millisecond; one that calls a service on another continent pays 150 milliseconds before doing any work. That is why the complexity of a request is measured in round trips, not CPU instructions.

Which component saturates first

Line the numbers up and the answer falls out. Bandwidth: irrelevant. Web tier: 30 requests in flight at peak, three small instances. Link storage: 91 GB a year, one database. Redirect reads: 600 per second warm, which a single primary handles without noticing, and 3,000 cold, which it handles only if the pool and the query time hold.

What saturates first was not on the whiteboard: the click-event write path, 3,000 inserts per second at peak and 3.65 TB a year, landing on the same database the redirects read from unless it is moved out. So the design is a cache on the redirect path and a queue with batching workers for clicks, leveling the write load into a separate store with a retention policy. Twenty minutes of arithmetic replaced "millions of users" with two decisions and a list of things to ignore.

The limits deserve honesty. The peak multiplier is a guess, the 80/20 split is a guess, and everything after the first line is proportional to a DAU figure somebody made up in a planning meeting. An estimate cannot tell you the p99 of a query on a cold cache; only a measurement can. What it can do is put a rough number on every component before you build any of them, so the first measurement you take is of the one that matters. Wrong by a factor of two is fine. The estimate exists to prevent wrong by a factor of a thousand.

Practicing the arithmetic

The habit forms by doing it against something that answers back. Katabench's System Design Studio runs a deterministic capacity simulation over every canvas you build: under a versioned traffic scenario it reports p99 latency, throughput, error rate, a modeled monthly cost, and the component that saturates first. Do the envelope first, build the topology, then compare your prediction with the simulation's answer. When they disagree, an assumption was wrong, and finding which one is the whole exercise. It is a reproducible model built for learning, not a provisioned load test. At code level, the performance guide covers how hidden large inputs and time budgets turn a complexity guess into a measured result, and the URL shortener walkthrough carries this estimate into a topology.