Understand it and size it

Part of the free System Design course on LogicWiz, in the chapter: Design a notification service. The interactive whiteboard exercises and simulations run in the app.

Week 13: Design a notification service

"Rohan got 'Your order is confirmed' four times in one minute. Three teams had written three senders, and a retry bug did the rest."

A SwiftKart support ticket, read out in the review

Why one shared service exists

SwiftKart's order team sends "order confirmed". The payments team sends "payment received". Marketing sends offers. Each team wrote its own code to call an SMS provider, a company that sends texts for you. Last month a retry bug in one of them sent Rohan the same confirmation four times in a minute. The same night, marketing texted him at 2 a.m., although he had opted out of offers.

Meera's first reaction: "Sending a message is one call to a provider. Where is the hard part?" Kabir showed her what goes wrong:

  • Providers fail or slow down several times a week.
  • The same event arrives twice, from a caller's retry or from a queue.
  • Providers allow only so many messages a second.
  • A sale blast and a login code compete for the same capacity.
  • Users have preferences, and every team checks them differently, or not at all.

One call is easy. Ten million a day, with these five problems, for every team, is a system. So the company builds one notification service. Internal services say "tell this user about this event". The service decides whether, how and when to send. The sending itself is done by external providers, because we do not run our own SMS gateways or mail servers.

Four channels, four different jobs

A channel is a route to the user. Candidates often treat them as one thing with four names. They differ, and the differences shape the design.

  • Push is a message on the phone's lock screen, sent through the phone platform's push service. Each app install has a device token, its address. Push costs very little and is fast. But it is best effort: the phone may be off, the user may have switched notifications off, and a token goes stale when the app is removed. A failed push often tells you to delete that token.
  • SMS is a text through a telecom provider. It reaches almost every phone, even without internet, which is why OTPs (one-time passwords, the short codes used to log in) travel by SMS. But each message costs money, the provider caps how many you send a second, and the delivery receipt can be late or missing. In many countries, India included, telecom rules require registered sender names and message templates.
  • Email is cheap, high in volume and allows long content. "Delivered" only means the receiving mail server accepted it, not that a person read it. Senders also have a reputation: if many people mark your mail as spam, all of it starts landing in junk. Marketing email needs an unsubscribe link.
  • In-app is a message in an inbox inside the app. There is no external provider: we write a row, and the app shows it when opened. It is cheap and reliable, but it only reaches users who open the app. The inbox screen is out of scope; the write is not.

So one event, "order confirmed", may go as a push for speed, an SMS as backup, and an email as the receipt. The service picks channels per message and per user.

Ask first, then write the requirements

Spend the first minutes on questions. Each one changes the design.

  • Who calls us? Only our own services. So callers are trusted and authenticated, and there is no public sign-up.
  • What kinds of message? Transactional messages answer something the user just did: an OTP, "order confirmed". They are small and urgent. Promotional messages are offers. They are huge in number and can wait.
  • How fast? Say an OTP must reach the provider within 2 seconds at p95 (95 out of 100 messages), and a promotion within 15 minutes.
  • What is worse, a duplicate or a miss? The brief says never twice. Ask if that also holds for an OTP, where a repeat is harmless and a miss is not.
  • Do users control what they get? Yes: opt-outs per category, and quiet hours, the hours when the user wants no promotions.

Then say the requirements aloud. Functional: accept a notification from a caller and choose channels from the user's preferences. Send through providers, with a second provider as backup per channel. Retry temporary failures, never send twice, and report status per channel. Non-functional: about 10 million notifications a day, peaks near 10,000 sends a second, 99.95% availability for accepting requests, and an accepted notification is never lost. Out of scope: the template editor, choosing who gets a campaign, the inbox screen, our own gateways.

Size it, step by step

Step 1, average. 10,000,000 a day divided by 86,400 seconds is about 115 a second. Each notification goes to about 1.5 channels, so about 175 provider sends a second (115 times 1.5 is 172).

Step 2, peak. A flash-sale blast to 5,000,000 users, one send each, spread over 10 minutes (600 seconds), is 5,000,000 divided by 600, about 8,300 a second. Plan for 10,000 sends a second. That is about 60 times the average of 175 sends a second, so a design for the average fails at the first sale.

Step 3, workers. A provider call takes about 200 ms, so one thread does 5 sends a second. 10,000 divided by 5 is 2,000 sends in flight: 40 worker processes with 50 threads each.

Step 4, writes. Each notification is one insert plus about 2 status updates (queued, sent, delivered): 3 writes. 10,000 times 3 is 30,000 writes a second. One relational primary is comfortable at about 2,000, so it is 15 times too small. We need a store partitioned by notification id.

Step 5, storage. About 1 KB per notification with its attempts: 10,000,000 times 1 KB is 10 GB a day. Keep 90 days: 900 GB. Three copies: 2.7 TB.

Step 6, the real limit. Suppose the SMS providers allow 1,000 a second in total, and the SMS part of the blast asks for 8,000 a second for 600 seconds. That is 4,800,000 messages. The backlog grows by 8,000 minus 1,000, so 7,000 a second. After 10 minutes, 600,000 are sent and 4,200,000 wait. Sending all 4,800,000 at 1,000 a second takes 4,800 seconds, which is 80 minutes. The 15-minute target is impossible by SMS. So big blasts go mostly by push and email, or start earlier and stagger. The number that shapes this design is not 10,000. It is 1,000.

The API and the data model

  • POST /v1/notifications {idempotency_key, user_id, template_id, params, category, priority} returns 202 {notification_id, status: "queued"}
  • GET /v1/notifications/{id} returns the status per channel, with attempts and last error
  • POST /v1/notifications/batch {idempotency_key, user_ids[], template_id, params, category} returns 202 {batch_id}
  • PUT /v1/users/{id}/preferences {channels, opt_outs[], quiet_hours}
  • POST /v1/providers/{provider}/callbacks {provider_msg_id, status} for delivery receipts

Three choices to explain. The reply is 202, "accepted for processing": we promise not to lose it, not that it is sent. The idempotency key is chosen by the caller, for example "order-4821-confirmed", so a repeated request is recognised. Callers send a template id and parameters, not finished text. We render the message later, per channel and language.

The data has four parts. A notifications table holds id, key, user, template, params, category, priority and status. A deliveries table has one row per channel attempt. preferences are kept per user and cached in Redis. Short-lived dedup keys also live in Redis. The deliveries rows answer "why did I not get my OTP?" without guessing.

Now write the sizing yourself.

📌 Summary: Channels differ in cost, speed and receipts. Size the peak (10,000 sends a second), the writes (30,000 a second) and the provider limit (1,000 SMS a second). Accept with 202 and an idempotency key.

What Meera learns next

The numbers say what we need: a queue, a write-friendly log and many workers. Next: draw the design from the simplest version, add each box for a reason, and follow one OTP through it.

Check your understanding

  1. A mail provider reports one of SwiftKart's emails as "delivered". What does that tell you?
  2. SwiftKart will send 20 million notifications a day, each to about 1.5 channels. Roughly how many provider sends a second is that on average?
  3. A calling service times out after our API accepted its request, so it retries with the same idempotency key. What should the API return?
  4. A promotion needs 1.8 million SMS. The provider allows 500 a second in total. Marketing says: add 10 times more workers. What is the earliest finish?