Design it: cheap reads, careful writes

Part of the free System Design course on LogicWiz, in the chapter: Design ticket booking (BookMyShow). The interactive whiteboard exercises and simulations run in the app.

Week 17: Build it from one box

"We cached the seat map to survive Friday night. Then the cache told customers that seat H7 was free for ten minutes after it was sold."

Kabir, on a cache that held the wrong thing

Start with the simplest version

Meera draws one box for the app, one for the booking service and one for the database. Browsing is a query, and a hold is an update on the seats table. At 50 bookings a second this is fine.

Now she asks what breaks, using the numbers from Day 1. Browsing is 3,000 reads a second at the evening peak. Each page joins cities, theatres, shows and seats. A heavy query like that costs a database a few hundred a second, not thousands. So browsing would crowd out the bookings that share its database. The fix is to split the two paths, because they want opposite things.

Path one: browsing, read-heavy and cacheable

The catalogue is cities, theatres, films, posters and show times. It changes rarely: a theatre adds a show a day or more ahead. Everything here can be a copy, and three layers hold the copies.

  • CDN (content delivery network, copies of files in many cities): posters and static files.
  • Cache (a fast in-memory store): the list of shows for a city and film, kept for 5 minutes. A new show appears up to 5 minutes late, which is acceptable.
  • Seat map cache: the map of one show, kept for only 3 seconds.

Check the arithmetic. At a 95% hit ratio, 5% of 3,000 reads reach the database: about 150 a second. For a seat map with a 3-second life, the database serves one show about once every 3 seconds, however many people are watching it, as long as one request refills the cache while the others wait for it. A hot show with 50,000 viewers then costs about 0.3 reads a second.

Meera asked the natural question: "If the cached map says H7 is free, can we skip the check and book it?" No. The cached map is a hint: a fast answer that may be a few seconds old. If it is wrong, the customer taps a taken seat, the hold step answers 409, and the app redraws the map. That costs one failed tap. Trusting the hint would cost a double booking.

Path two: the seat state machine

Every seat in a show is in exactly one of three states: FREE, HELD or BOOKED.

flowchart LR
  F["FREE"] -->|"customer taps seats"| H["HELD, with held_until"]
  H -->|"payment confirmed"| B["BOOKED"]
  H -->|"8 minutes pass, or customer leaves"| F

Every move is a guarded update: a write that carries its own condition, so the database checks and changes in one step. FREE becomes HELD only if the seat is still FREE. HELD becomes BOOKED only if it is still held by this customer's hold. BOOKED never moves again, because cancelling a ticket is out of scope.

One more rule matters. A HELD seat whose held_until is in the past counts as FREE, even if no cleaner has touched it yet. Every read and every guarded update checks the time. So a late cleaner never blocks a seat.

Add the booking path, one box at a time

Each box answers a number or a rule. Nothing is drawn for decoration.

  • Booking service, several copies. It keeps no state, so we add copies as the rush grows.
  • Seat database, SQL, with copies. A booking of H7 and H8 must be all or nothing, and the database must refuse duplicates. SQL gives transactions and unique keys. One primary takes the writes, and one of the two copies takes over if it fails. It is only 8.5 GB, so we split by show later.
  • Payment gateway. The customer pays outside our system. We open a 7-minute payment session and wait for its webhook.
  • Expiry queue and sweeper. When a hold is created, a message is scheduled to fire at 8 minutes. A worker, the sweeper, frees the seats if the hold is still HELD.
  • Notification. The ticket goes out through a queue, so a slow SMS provider never delays a booking.

Each choice has a price, and you should say it. The cache makes reads cheap but allows a stale map. SQL makes booking safe, but one hot show can take only some hundreds of holds a second (roughly 500 to 1,000, so measure it). The expiry queue is one more thing to run and watch. Day 3 deals with the hot show.

Build it yourself, starting from the single box.

One booking, start to finish

sequenceDiagram
  participant C as Customer app
  participant B as Booking service
  participant D as Seat database
  participant P as Payment gateway
  participant X as Expiry queue
  C->>B: GET /shows/42/seats
  B-->>C: seat map from cache, 3 s old
  C->>B: POST /holds (H7, H8)
  B->>D: guarded update, FREE to HELD
  D-->>B: 2 rows changed
  B->>X: schedule expiry at 8 minutes
  B-->>C: hold, expires in 8 minutes
  C->>B: pay
  B->>P: open 7-minute session
  P-->>B: webhook, paid
  B->>D: HELD to BOOKED, only if still HELD
  B-->>C: ticket

Read the two paths. The seat map read touches only the cache, almost always. The hold touches the database once, in one transaction, and returns 409 if either seat changed rows. A retry of the same hold, with the same idempotencyKey, returns the hold that already exists instead of making a second one. If the customer walks away, the expiry message fires at 8 minutes and the sweeper frees H7 and H8.

💡 Tip: Say out loud which answers are hints and which are decisions. The seat map is a hint. The guarded update is the decision.

📌 Summary: Split browsing from booking. Cache browsing as hints, and let one guarded update on the seat database decide every hold. Seats move FREE, HELD, BOOKED, and an expiry queue frees unpaid holds.

What Meera learns next

The design stands, but it has not met a real crowd. Next: two taps on the same seat in the same millisecond, a payment that arrives after its hold expired, and the 10:00 rush.

Check your understanding

  1. The seat map is cached for 10 minutes. A customer sees H7 as free, taps it, but another customer booked it 4 minutes ago. In a correct design, what happens?
  2. Browsing is 3,000 reads a second at peak and the show list cache hits 95% of the time. About how many reads reach the database each second?
  3. Why does the seat database use SQL in this design?
  4. A hold on H7 ended 2 minutes ago, but the sweeper worker has been down for 10 minutes. A new customer taps H7. What should happen?