System design fundamentals: nine building blocks

The architecture diagram for a system serving a hundred million requests a day has thirty boxes, and the first time you see one it looks designed all at once by someone who already knew the answer. It was not. Nearly every box answers one number that got too big: reads the database stopped keeping up with, a deploy that took the only instance down, a write burst faster than a disk could absorb. Take the boxes away one pressure at a time and you are left with a client, a server, and a database.

That is the useful way to learn system design fundamentals: not as a catalog of components but as a short list of moves. Each move answers a specific pressure, buys a specific property, and charges for it with a new way to fail. Nine moves cover most of what you will meet in production. Here they are, pressure first.

Nine building blocks

Pressure → pattern → the failure mode it introduces

one pressure at a time

1. Clients hold the connection string

   01

   Trusted boundary

   charges: one process; a restart stops everything

2. One process, one deploy, one outage

   02

   Stateless instances + load balancer

   charges: traffic routed to an instance that is not ready

3. The same rows read thousands of times a minute

   03

   Cache-aside

   charges: a cold cache floods the database

4. The database is a single point of failure

   04

   Replication + promotable standby

   charges: replica lag; a lossy failover

5. Static bytes cross an ocean per request

   05

   CDN / edge cache

   charges: stale or private content served from cache

6. One client discovers the capacity for everyone

   06

   Rate limiting at the gateway

   charges: the shared counter store slows every request

7. LIKE '%term%' against fifty million rows

   07

   Derived search index

   charges: the index silently diverges from the data

8. Writes arrive faster than the disk commits

   08

   Durable queue + worker pool

   charges: a backlog nobody is watching

9. One database has no room left to grow

   09

   Sharding by key

   charges: a hot shard; queries that touch every shard

pressure pattern it buys failure mode it charges

Every box in a large architecture is the answer to one number that got too big. Each block buys one property and charges a new failure mode.

The request path: blocks one to three

1. A trusted boundary. The pressure is the first client that connects to the database directly: a mobile app shipping a connection string, a reporting tool with a read-only login. Now the schema is a public API, every table is reachable by anyone who can read the binary, and the database holds a connection per client rather than per server. The pattern is an application tier that owns the data: clients talk to an API, the API talks to the database, and nothing else does. It costs a network hop and a service to operate. The new failure mode is that the tier is a single process, and when it restarts, everything stops.

2. Stateless instances behind a load balancer. The pressure is that one process, pinned at full CPU during the lunch peak and absent during every deploy. The pattern is several identical instances that keep no per-user state, behind a load balancer that spreads requests across them (the algorithm it uses is its own decision). Any instance can serve any request, so the fleet grows by adding instances and survives losing one. The cost is that sessions, in-memory caches, and local files are no longer yours to keep. The new failure mode is the balancer's blind spot: it keeps sending traffic to an instance that is up but not ready, unless the instance says otherwise.

builder.Services.AddHealthChecks()
    .AddNpgSql(connectionString, name: "postgres", tags: ["ready"])
    .AddRedis(redisConnection, name: "redis", tags: ["ready"]);

app.MapHealthChecks("/health/live", new HealthCheckOptions
{
    Predicate = _ => false // the process is up; report nothing else
});

app.MapHealthChecks("/health/ready", new HealthCheckOptions
{
    Predicate = check => check.Tags.Contains("ready")
});

Liveness says "restart me if this fails". Readiness says "do not route to me until this passes". Conflate them and a slow database becomes a fleet-wide restart loop.

3. Cache-aside in front of the database. The pressure is read traffic: the same product page fetched thousands of times a minute and changed once an hour. The pattern is a cache the application checks first and fills on a miss, with a time-to-live. The arithmetic is the whole argument. At 10,000 reads per second and a 95 percent hit rate, the database sees 500 reads per second, a twentyfold reduction. The cost is staleness for the length of the TTL, plus the invalidation and stampede problems that follow. The new failure mode is the same arithmetic in reverse: the morning the cache restarts empty, the database receives twenty times its usual load in one second, and a database sized for 500 reads per second does not survive 10,000.

The same read, before and after the first three blocks:

before
  client -> database              SELECT ... WHERE product_id = 42   (schema is the API)

after
  client -> load balancer -> api-2 -> cache hit                       ~1 ms
  client -> load balancer -> api-1 -> cache miss -> database          ~12 ms
                                   -> cache set, ttl 60 s

Surviving and shielding: blocks four to six

4. Replication and a promotable standby. The pressure is the database as a single point of failure. The pattern is a primary that accepts writes and streams its log to replicas that serve reads and can be promoted when the primary dies. PostgreSQL's streaming replication is the reference shape. The cost is write amplification: with two replicas, every row write is performed three times across the fleet and shipped over the network twice, so 1,000 writes per second become 3,000 disk writes per second, and replication adds nothing to write capacity. The new failure mode is lag. A replica 200 milliseconds behind tells a user that the comment they just posted does not exist, and a failover that promotes a lagging replica loses the writes it never received.

5. A CDN, or an edge, for cacheable content. The pressure is distance and bandwidth. Images, scripts, and stylesheets are most of the bytes, they never change, and a round trip across an ocean costs about 150 milliseconds whatever your servers do. The pattern is a content delivery network that caches static and cacheable responses close to the client. The cost is that cache-control headers now matter, and every deploy needs a purge step. The new failure mode is serving something stale, or something private, to the wrong person because a header said it was cacheable.

6. Rate limiting at the gateway. The pressure is one client, usually not malicious, that discovers your capacity on behalf of everyone: a for loop with no sleep in it. The pattern is a limit enforced at the gateway before a request costs anything, with a token bucket as the default algorithm. Across many gateway instances the counters have to live in a shared store, so the limiter is itself a network call on every request. The new failure mode is that store: if it is slow, every request is slow, and if it is down, you choose between failing open with no limits and failing closed with no traffic.

When the write side grows: blocks seven to nine

7. A derived search index fed from committed changes. The pressure is WHERE title ILIKE '%term%' against a table of fifty million rows. The pattern is a search engine holding a derived copy of the data, fed by changes after they commit, whether through an outbox or a change feed. The cost is a second system with its own schema, and the index is by definition behind the database. The new failure mode is divergence: an indexer that crashed mid-batch, a mapping change that quietly dropped a field, and no signal until a customer searches for something that exists.

8. A durable queue and a worker pool. The pressure is a write spike the database cannot absorb at the rate it arrives. Say 5,000 orders per second for one minute against a database that commits 1,000 per second. Without a buffer, 4,000 per second fail. With a queue in front of a worker pool, the request tier acknowledges all 300,000, the workers drain at 1,000 per second, and the 240,000 left over after that minute clear in four more. The cost is that the request returns before the work is done, so every caller now needs a way to learn what happened. The new failure mode is the backlog itself: a queue is latency you have not paid yet, and a consumer that dies quietly leaves it growing.

9. Sharding once one database is not enough. The pressure is that vertical scaling has run out: the working set no longer fits one machine's memory, or the write rate exceeds one primary's disk. The pattern is splitting rows across databases by a key, usually a tenant or a user, so each shard carries a fraction of the data and the writes. The cost is the highest of the nine. Any query that omits the shard key touches every shard, cross-shard transactions become application logic, and resharding is a migration project. The new failure mode is the hot shard: one tenant that grows to a quarter of the traffic while sitting on a database sized for a twentieth of it.

One pressure at a time

Read the nine back and a pattern in the patterns appears: each block exists because a specific number crossed a line, and each introduces the number the next block has to answer. The cache creates the cold-start flood that replicas absorb. Replication creates the lag that a derived index inherits. The queue creates the backlog that sharding is eventually asked to drain. A diagram with all nine boxes is not complicated. It is nine simple decisions that happen to be stacked.

That is also how to practice: not by memorizing the finished diagram, but by starting with a client, a server, and a database, applying one pressure, adding the block that answers it, and checking what it cost. Do the capacity arithmetic before each step, because the pressure is a number and the block is justified only when the number says so.

Katabench's System Design Studio is built on exactly that sequence. Its nine fundamentals challenges are these nine blocks, one each: Route Through the App, Scale the Web Tier, Cache the Hot Path, Survive a Database Failure, Serve from the Edge, Rate-Limit at the Gateway, Derive a Search Index, Buffer the Write Spike, and Shard the Database. You build each on a canvas from provider-neutral components, deterministic rules check required paths and forbidden bypasses, and a capacity simulation reports p99 latency, throughput, error rate, and the component that saturates first. It is a reproducible model for learning, not a load test, and there is no single correct diagram. The how it works page explains the grading; the studio is in beta, and six of its challenges are free with an account.