Ask, size and specify the shortener

Part of the free System Design course on LogicWiz, in the chapter: Design a URL shortener. The interactive whiteboard exercises and simulations run in the app.

Week 10: Ask first, size second

"I drew a sharded database in the first minute. Then the interviewer asked how many redirects a second, and I had no number."

Meera, after her first mock interview with Kabir

What a URL shortener is, and who needs one

Anita from SwiftKart's marketing team is about to send a festival SMS to 3 million customers. A text message has little room, and the product link is a long URL full of tracking tags. She wants sk.in/diwali-sale instead.

A URL shortener turns a long web address into a short code. When anyone opens the short link, it sends them on to the long address. That second step is called a redirect: the server answers "go to this other address", and the browser follows.

Meera's first thought was "it is a map from code to URL, ten minutes of work". The map is the easy part. The hard parts are the numbers: thousands of redirects a second, codes that never collide, and links that must not break. Interviewers ask this question because it has three real problems hidden in a small box.

Kabir's rule is simple. The interview has five phases: requirements, estimation, API and data model, the design, and the deep dives. The first five minutes belong to the first three phases, not to boxes. This lesson covers those three.

Ask before you draw

A candidate who starts drawing has not asked who the users are. Ask these questions, and say your assumed answer out loud if the interviewer lets you choose.

  • What can a user do? Create a short link, and open it. Assume also a custom alias (sk.in/diwali-sale), an optional expiry date, and click counts per link.
  • How many new links? Assume 100 million a month.
  • How many redirects? Assume about 10,000 a second at peak.
  • How fast must a redirect be? Assume p99 under 50 ms at the server. That means 99 out of 100 redirects finish within 50 milliseconds.
  • How available? Assume 99.99% for redirects, because a dead link breaks every campaign that printed it. Creating links can be 99.9%.
  • May a code change later? No. A code, once issued, never points anywhere else.

Then write what is out of scope, so nobody expects it: user accounts and billing, scanning target URLs for phishing (mention it, do not design it), and editing where a code points.

The numbers that drive the design

Estimating is arithmetic with round numbers. Show every step. Here it is for the shortener.

  1. Writes. 100M links a month, and a month has about 30 days. 100M / 30 = 3.3M a day. A day has 86,400 seconds, so 3.3M / 86,400 = about 40 creates a second. A peak of 3 times that is about 120.
  2. Reads. For every new link, people click about 100 times. 40 * 100 = 4,000 redirects a second on average. The prompt says about 10,000 at peak, because an SMS to millions lands all at once.
  3. Storage. One row holds the code, the long URL, the owner and two dates: roughly 500 bytes. 100M * 500 B = 50 GB a month, so 600 GB a year and 3 TB in 5 years. Keep 3 copies: about 9 TB.
  4. Code length. In 5 years: 100M * 12 * 5 = 6 billion links. A code uses base62: the 62 characters 0 to 9, a to z and A to Z. Six characters give 62^6 = about 57 billion codes. Seven give 62^7 = about 3.5 trillion.
  5. Cache size. Say 20 million distinct links are clicked on a busy day. Assume the top 20% take about 95% of the clicks. 20% of 20 million is 4 million links, and 4M * 500 B = about 2 GB. That fits in the memory of one small machine.
  6. Database load after the cache. The 95% above is the hit ratio, the share of redirects answered from the cache. The database sees the other 5%: 10,000 * 5% = 500 reads a second.
  7. Bandwidth. 10,000 * 500 B = 5 MB a second. Not a concern, so say so and move on.

Now try it yourself. The calculator starts from 30 million daily users, 0.11 new links each and 100 reads per write. Change the peak factor and the read ratio, and see which numbers move. At a peak factor of 3 it lands near 11,000 redirects a second, the same size as the 10,000 you say out loud.

Three conclusions come out of the sums. Reads dominate, so the redirect path needs a cache. Nine terabytes is small, so size is not the problem. And 40 writes a second is tiny, so we can afford to be careful when we make codes, instead of fast.

Why seven characters when six hold 57 billion codes? Because 6 billion links in a space of 57 billion means about 1 code in 10 is a real link. A person guessing codes would hit one every ten tries. With seven characters the chance falls to about 1 in 580. That slows a scraper but does not stop one, so we also rate-limit later.

The API and the data model

Keep the API small. Each line below is a verb, a path, and what comes back.

POST /v1/links {long_url, alias?, expires_at?} -> 201 {code, short_url}   (409 if the alias is taken)
GET /{code} -> 302 Location: long_url   (404 if unknown, 410 if expired)
GET /v1/links/{code}/stats?from=&to= -> 200 {clicks_by_day[], top_countries[]}
DELETE /v1/links/{code} -> 204   (the code is retired, never reused)

The redirect has no /v1 prefix, because the short link itself is the address people share. A 302 is a temporary redirect. We pick it for a reason the deep dives explain.

The data model follows the access pattern. The redirect looks up one row by code, and nothing else.

  • links(code PK, long_url, owner_id, created_at, expires_at, is_custom): a lookup by code only, so a replicated key-value store fits.
  • click_events: a stream of {code, ts, country, referrer}, one event per redirect, written but never read on the redirect path.
  • link_stats_daily(code, day, clicks): totals the stats API reads.

Write the estimate and the API into notes now, the way you would on a whiteboard.

📌 Summary: Ask for scale, latency and scope first. The numbers are about 40 creates and 10,000 redirects a second, 9 TB of data and a 2 GB hot set, so the redirect needs a cache.

What Meera learns next

With the numbers on the board, Meera can finally draw. Next lesson she starts from one server and one database, and adds each box only when a number demands it.

Check your understanding

  1. The interviewer says "design a URL shortener" and the candidate immediately draws a sharded database. What should the first five minutes have been?
  2. A shortener gets 100 million new links a month, and people click each link about 100 times. About how many creates and redirects is that a second, on average?
  3. A colleague says "6 base62 characters hold 57 billion codes and we need only 6 billion, so 6 is enough". What does the estimate say?
  4. A visitor opens a short link whose expiry date passed last week. Which answer fits the API we wrote?