From numbers to components

Part of the free System Design course on LogicWiz, in the chapter: Back-of-the-envelope estimation. The interactive whiteboard exercises and simulations run in the app.

Week 2, Day 4: The sale that melted the database

"We launched a 90 percent off phone at noon. By 12:01 the product page was a spinner."

Kabir, telling an old SwiftKart story

One database is not infinite

Estimates only matter if they change what you draw. Here is the bridge. Every component has a rough ceiling, and your job is to compare your peak number with that ceiling.

Here are some rules of thumb. Treat them only as rough guides: real limits depend on hardware, query shape and data size.

A single relational database serves roughly 1,000 to 10,000 simple lookups per second. Simple means a lookup by key. A heavy query that joins several tables or scans a big range manages only a few hundred a second, or a few dozen. In the worked examples below we treat the main database (the primary, the one that takes the writes) as comfortable at about 2,000 simple queries per second.
A cache held in memory answers tens of thousands to a hundred thousand lookups per second on one node. It skips the disk and the query planner (the database step that works out how to run a query).
One application server instance handles very roughly hundreds to a couple of thousand requests per second, depending on how much work each request does.
A single database server stores comfortably up to a few terabytes. Beyond that you start thinking about splitting the data.

Treat these as orders of magnitude. If your peak is 500 per second, you are far below any of them. If it is 50,000, you are above the database's ceiling and must act. If it is 5,000, you are in the grey zone where the query shape decides. Say which assumption you made. (SwiftKart's tracking page from the last lesson, at about 1,000 reads per second, sits comfortably inside the range.)

Meera meets the sale

SwiftKart's product page for a flash sale is the opposite of last week's tracking page. About 1 million shoppers open the page in the first minute. Say each looks at it about 5 times as the countdown ends: 5,000,000 reads in 60 seconds, which is about 83,000 reads per second. Meanwhile, roughly 20 percent of them, 200,000 people, try to buy in that minute, about 3,300 write attempts per second.

Compare with the rules of thumb. 83,000 reads per second is far above what one database can serve, even at the generous end. 3,300 write attempts a second is inside the raw range. But most of them hit the same stock row, so they queue behind each other. Stock needs its own design, such as a counter or a queue. Drawing "client, service, database" is no longer a design. It is a prediction of what Kabir saw at 12:01.

Now notice something helpful. The product page is the same for everyone: name, price, photos, stock badge. Eighty-three thousand requests a second for the same few rows is the easiest kind of read to serve. The answer can be stored once and reused.

Add a cache when the reads repeat

A cache is a faster copy of data you already have, kept close to the service, usually in memory. When a request arrives, you look in the cache first. If the answer is there (a hit), you skip the database. If not (a miss), you read the database and put the answer in the cache for next time.

Meera's arithmetic: if 99 percent of reads are hits, only 1 percent reach the database. 83,000 × 1% = 830 reads per second, comfortably inside the 2,000 a second one primary handles. The cache turned an impossible number into an ordinary one. One cache node serving 83,000 lookups a second is around the upper end of a single node, so she plans two or three.

💡 Tip: Quote the hit rate as an assumption: "I'll assume 99 percent hits for a page identical for everyone. A page unique to each user would cache far worse." That one sentence shows you understand why the cache works.

flowchart LR
  C["Shoppers"] --> S["Service"]
  S --> K["Cache"]
  K -->|"1% miss"| D["Database"]
  D --> R["Replica"]

Add replicas for survival and for reads

The database still has two jobs. The remaining 830 or so reads a second are fine. But the 3,300 writes go to one machine, and if that machine dies the sale stops. A replica is a second database that continuously copies the first. Reads can go to the replica. (It can be a little behind the primary; Week 5 returns to this.) If the main one fails, the replica can take over. That solves availability and a good deal of read load.

A note on the whiteboard: there, "replicas" on a component means total instances. Setting it to 2 means a primary plus one copy.

Replicas do not multiply write capacity: every write still lands on the primary. Suppose Meera's write estimate were 50,000 per second. Replicas would not save her. She would need to split the data across machines, which a later module covers. Her number is 3,300, so she notes the risk and moves on.

When one machine is not enough

The same check applies to application servers. Peak requests ÷ per-server capacity gives a count. Say each instance handles about 1,000 requests per second. The sale needs 83 instances at exactly the peak, so call about 100 the bare floor. Compare 100 against a first guess of "two servers" and you see why the estimate mattered. Run several instances behind a load balancer (Module 3) and add more when the numbers grow.

Always leave headroom. Plan for roughly twice your computed peak. Machines are not usable at 100 percent, and your peak factor is a guess. With that rule the sale wants roughly 160 to 170 instances. That is why most of this traffic should be absorbed earlier, by a cache or a CDN (a cache placed close to users, Module 3).

A cache has to earn its place

The cache is the most popular box in interviews, and the most overused. Candidates often say, "I'll add a cache everywhere, to be safe." A cache is justified by a number: repeated reads that outrun the database. The tracking page from last lesson peaks near 1,000 reads per second, which one database handles easily. Adding a cache there gives you a second system to run, a staleness bug to chase and a new failure mode. It adds nothing in return.

Ask three questions before drawing it: Is the read rate above what the database can carry? Do many requests ask for the same data? Can the data be a little stale? If the answers are no, skip it and say why.

Open the calculator and load the flash-sale preset. It assumes 5 million shoppers and a peak 2,000 times the average, far harsher than our 1-million-shopper minute. Reads come out in the millions per second, and writes near a hundred thousand. Both sets of numbers say the same thing: no single machine survives.

Now draw the fix.

📌 Summary: Compare your peak with rough ceilings: one database handles about 1,000 to 10,000 simple queries a second. A cache absorbs repeated reads. Replicas add survival and read capacity, never write capacity.

What Meera learns next

With the sale page cached, requests still have to reach the right machines. Next week is the entry layer: DNS, CDNs, load balancers and rate limiting, the front door the whole design walks through.

Check your understanding

The order-tracking page peaks at about 1,000 simple reads a second, and the primary is comfortable at about 2,000. A teammate wants to add a cache "to be safe". What is the best response?
The flash-sale page peaks at about 83,000 reads a second and the cache hit ratio is 99 percent. Roughly how many reads a second reach the database?
Suppose the sale's write estimate were 50,000 a second instead of 3,300. A teammate suggests adding three more database replicas. Why does that not solve it?
Peak traffic is 20,000 requests a second and one app instance handles about 1,000. How many instances should you plan for?