Chat app: ask, size it, shape the API

Part of the free System Design course on LogicWiz, in the chapter: Design a chat app (WhatsApp). The interactive whiteboard exercises and simulations run in the app.

Week 12: Design a chat app (WhatsApp)

"Our first prototype asked the server 'any new messages?' every three seconds. Nine answers in ten were 'no', and the server still fell over."

A tester, describing a chat build that Kabir had to rewrite

What a chat app really is

Kabir sets a mock interview for Meera: "Design a chat app like WhatsApp, for 50 million people a day." She reaches for the database at once. He stops her. "First say what the product does."

Here it is in plain words. Asha types "Reached Pune" to Ravi. Ravi's phone buzzes within a moment. Asha sees one tick when our server has the message, two ticks when Ravi's phone has it, and blue ticks when Ravi has read it. If Ravi's phone is off, the message waits and arrives when he is back. The same works for a group of friends.

Four words to fix now.

  • A conversation is one chat: two people, or a group.
  • A message is one piece of text sent into a conversation.
  • A receipt is a small note that says a message was delivered or read.
  • A connection is the open line between one phone and our servers.

Meera's first idea is the one most candidates have: the phone asks for new messages every few seconds. Do the sum. Say 10 million people are online at the evening peak, and each phone asks every 3 seconds. The servers see 10,000,000 / 3, about 3.3 million requests per second. Nearly every answer is "nothing new". And a message still waits up to 3 seconds. The fix is to keep the connection open, so the server can push a message the moment it arrives. That one idea shapes the whole design.

Ask first, then write the requirements

Kabir's rule: five questions before any box. Meera asks them out loud.

  1. "One-to-one only, or groups too?" Both. Groups of up to 100 people.
  2. "What does the sender see?" Three states: sent (our server stored it), delivered (the recipient's phone received it), read (the recipient opened the chat).
  3. "What if the other person is offline?" The message is kept and delivered on return. A push notification, a short alert shown by the phone's own system, tells them to come back.
  4. "How many users, and how fast?" 50 million a day. An online recipient gets the message in under 500 ms at p99. That means 99 messages out of 100 arrive within 500 milliseconds.
  5. "Photos, calls, encryption?" Out of scope. We say where they would fit.

Meera's first draft of the non-functional list said only "must be fast". Kabir circled it. Fast is the easy part. Users notice two other promises: a message is never lost once it shows the sent tick, and everyone in a chat sees the same order. Write those down with numbers.

  • Functional: send text to one person or a group of up to 100. Sent, delivered and read receipts. Offline delivery. History and new-device sync. An inbox sorted by latest activity, with unread counts.
  • Non-functional: 50 million daily users, 40 messages each. Under 500 ms at p99 to an online recipient. Never lost after the sent tick. Same order for every member. 99.95% availability, which allows about 4 hours of downtime a year.
  • Out of scope: calls, media upload (it goes to object storage, and the message carries a link), encryption key exchange, channels with thousands of members.

Size it, step by step

Every box you draw later must be earned by a number from here. Work the steps in order.

Messages a day:      50,000,000 users x 40 = 2,000,000,000
Average a second:    2,000,000,000 / 86,400 = about 23,000
Peak (3 to 4 times): about 70,000 to 90,000, plan for 80,000
Deliveries:          2 per message x 80,000 = 160,000 a second
Open connections:    20% of 50M online = 10,000,000
Gateway servers:     10,000,000 / 50,000 each = 200
Storage a day:       2,000,000,000 x 200 bytes = 400 GB
Storage a year:      400 GB x 365 = about 146 TB
3 years, 3 copies:   about 440 TB x 3 = about 1.3 PB
Session lookups:     about 160,000 a second at peak

Two lines need a word. A delivery is one copy of a message sent to one recipient. It is 2 per message on average. A one-to-one message has 1 recipient. Say 1 message in 100 goes to a full group of 100: 0.99 * 1 + 0.01 * 100 is about 2. A gateway is a server whose job is to hold phone connections open. We plan for about 50,000 on each, a round figure to check against real tests.

Try the numbers yourself. The calculator uses a peak factor of 4, so its peak reads a little higher, about 90,000. Both are the same size of problem, and that is what matters.

Meera looked at 1.3 PB and said storage must be the hard part. It is not. 1.3 PB spread over many machines is routine, and messages are written once and rarely changed. The hard parts are the 10 million open connections and keeping order. The numbers told you where to spend your time.

The API

Five calls cover the product. Read the lines, then the reasons.

WS  /v1/connect (auth token)  -> server pushes message.new, receipt.update
WS  message.send {conversation_id, client_msg_id, body} -> message.ack {message_id, seq, server_ts}
POST /v1/conversations/{id}/messages {client_msg_id, body} -> 201 {message_id, seq}
GET  /v1/conversations/{id}/messages?before_seq=&limit=50 -> 200 {messages[], next_cursor}
GET  /v1/sync?since={seq per conversation} -> 200 {missed messages and receipts}
POST /v1/conversations/{id}/receipts {delivered_seq?, read_seq?} -> 204

Sending and receiving use the open WebSocket (a connection that stays open, so either side can send at any time). The POST is a fallback for when the socket is down.

The client_msg_id is made by the phone, once per message, and reused on every retry. Phones lose signal. If Asha's app sends, hears nothing and sends again, the server sees the same client_msg_id and does not store a second copy. The seq (sequence number) in the reply is the message's place in that conversation: 4,811, then 4,812. History uses a cursor, before_seq, because new messages keep arriving and page numbers would shift. Sync takes the last seq the phone holds for each conversation and returns everything after it.

The data model

Choose keys from the questions you will ask most: "the latest 50 messages of this conversation" and "what did I miss".

messages(conversation_id PARTITION KEY, seq CLUSTERING KEY, message_id, sender_id, client_msg_id, body, created_at)
conversations(conversation_id PK, type, member_ids[], last_seq, created_at)
user_inbox(user_id PARTITION KEY, last_activity DESC, conversation_id, unread_count)
receipts(conversation_id, user_id, delivered_seq, read_seq)
sessions (in memory, TTL 60 s): user_id -> [{device_id, gateway_id, connected_at}]

A partition is a slice of the table kept together on one machine, and the clustering key sorts the rows inside it. Messages are keyed by conversation and ordered by seq, so one conversation lives together and the latest 50 are one cheap read. The store must spread writes over many machines (it is sharded), and this key spreads them by conversation. One very old chat would grow a huge partition, so real systems add a time bucket to the key, such as the month.

Meera's first receipts table had one row per message per member. For a read message in a group of 100, that is 100 rows. Kabir's version keeps one row per member holding a number: "Ravi has read up to seq 4,812". One update covers many messages, and write load stays small.

Now write the opening notes as you would on the whiteboard.

📌 Summary: Ask about groups, receipts, offline users and latency first. The sizing shows connections and ordering are hard, not storage. Key messages by conversation and seq, and give each send a client_msg_id.

What Meera learns next

The numbers say: 10 million open connections, 80,000 messages a second at peak, and nothing lost after the tick. Next lesson she builds the design box by box, starting from one server and watching it break.

Check your understanding

  1. A teammate proposes that each online phone asks the server for new messages every 5 seconds. With 10 million phones online, what is the main problem?
  2. SwiftKart-sized chat: 2 billion messages a day, each about 200 bytes. How much new data does the message store take in per day?
  3. A group of 100 members reads one message. Which receipt model keeps write load small?
  4. Asha's phone sends a message, loses signal before the reply, and sends again. How does the server avoid storing it twice?