Chat app

One-to-one and small-group chat for 50 million daily users, with delivery and read receipts. The interesting parts are persistent connections, keeping messages in order within each conversation, and getting messages to people who were offline.

Design a chat app for one-to-one and small-group messages, with delivery receipts, used by 50 million people a day.

Requirements

Functional

Send a text message to one person or to a group of up to 100 members.
Sent, delivered and read receipts per message (one tick, two ticks, blue ticks).
Messages to an offline user are stored and delivered when they come back. Meanwhile, the user gets a push notification.
Load conversation history, newest first, and sync a new device.
An inbox that lists conversations by latest activity, with unread counts.

Non-functional

Scale: 50 million daily users, about 40 messages each a day. That is about 23,000 messages a second on average and roughly 80,000 at the evening peak.
Latency: a message reaches an online recipient within India in under 500 ms at p99 (99 out of 100 messages).
Durability: once the sender sees the "sent" tick, the message is never lost.
Ordering: messages within one conversation appear in the same order for every member.
Availability: 99.95%. A client that loses its connection reconnects and catches up without gaps or duplicates.

Out of scope

Voice and video calls.
Media upload (photos, video). Mention that media goes to object storage, with a link in the message.
End-to-end encryption key exchange. Mention that it changes what the server can read, but do not design it.
Large broadcast channels with thousands of members.

Estimate

Messages: 50M users x 40 messages = 2B messages a day; 2B / 86,400 ≈ 23,000 messages a second on average.
Peak: evenings run about 3 to 4 times the average → roughly 70,000 to 90,000 a second; plan for 80,000.
Deliveries: most chats are one-to-one with some small groups, so about 2 deliveries per message → about 160,000 pushes a second at peak, plus receipts (which are small messages too).
Connections: if 20% of daily users are online at peak, that is 10M open WebSockets. At about 50,000 connections per gateway box, 10M / 50,000 = 200 gateway instances.
Storage: 2B x ~200 bytes = 400 GB a day → about 146 TB a year; 3 years ≈ 440 TB, x3 replication ≈ 1.3 PB. That is a sharded, write-optimised store.
Session lookups: every delivery asks "which gateway holds this user?" → about 160,000 reads a second at peak against an in-memory store.

API

WS /v1/connect (auth token) -> server pushes message.new, receipt.update, typing events
WS message.send {conversation_id, client_msg_id, body} -> message.ack {message_id, seq, server_ts}
POST /v1/conversations/{id}/messages {client_msg_id, body} -> 201 {message_id, seq} (fallback when the socket is down)
GET /v1/conversations/{id}/messages?before_seq=&limit=50 -> 200 {messages[], next_cursor}
GET /v1/sync?since={per-conversation seqs} -> 200 {missed messages and receipts}
POST /v1/conversations/{id}/receipts {delivered_seq?, read_seq?} -> 204

Data model

messages(conversation_id PARTITION KEY, seq CLUSTERING KEY, message_id, sender_id, client_msg_id, body, created_at): one partition per conversation, ordered by seq.
conversations(conversation_id PK, type, member_ids[], last_seq, created_at).
user_inbox(user_id PARTITION KEY, last_activity DESC, conversation_id, unread_count): the inbox list.
receipts(conversation_id, user_id, delivered_seq, read_seq): one row per member, so a read receipt is one number, not one row per message.
sessions (in-memory, TTL 60 s, refreshed by heartbeat): user_id -> [{device_id, gateway_id, connected_at}].

A request, step by step

Asha opens the app. The app opens a WebSocket (a connection that stays open, so the server can push messages) through the load balancer to some gateway.
The gateway records "Asha is on gateway 17" in the session store with a 60-second TTL, so the entry expires unless it is renewed. Heartbeats (small regular "I am still here" messages) keep renewing it.
Asha sends "Reached Pune" with a client_msg_id that her app generated. The gateway forwards it to the message service. If that client_msg_id was already seen, this is a retry and the service drops it. Otherwise it assigns the next seq (sequence number) for the conversation, say 4,812.
The service writes the message to the conversation's partition in Cassandra. Only then does it acknowledge Asha with {message_id, seq}, and she sees one tick. The message is durable before it is acknowledged.
The service publishes the message to Kafka, keyed by conversation_id. So every message of a conversation goes to one partition, in order.
A delivery worker looks up each member in the session store. Ravi is online on gateway 42, so the worker pushes to gateway 42, which writes to Ravi's socket. Ravi's app acknowledges. The delivered receipt (delivered_seq = 4,812) flows back the same way, and Asha sees two ticks.
If Ravi is offline, the worker sends a push notification instead. When Ravi reconnects, his app calls sync with the last seq it has for each conversation and pulls everything after it. Nothing is skipped, even if a live push was lost.

Deep dives

Ordering messages in a conversation

Weak: order by the sender's phone clock. Phone clocks disagree by seconds or minutes. In a group, people then see replies above the questions they answer.

Server timestamps are better but still unsafe. Two message-service instances can stamp messages in the same millisecond, and their clocks drift too.

Good: a sequence number per conversation, assigned on the server. One place hands out seq 4,811, 4,812, 4,813 for a conversation. It can be an atomic increment on conversations.last_seq, or a counter in Redis. Clients sort by seq.

Gaps are easy to detect. If Ravi has 4,810 and receives 4,812, he knows to fetch 4,811.

Great: a single writer per conversation. Route every message of a conversation to the same Kafka partition (keyed by conversation_id) and the same owner process. That process assigns seq and stores the message.

There is no lock for processes to fight over, and order is kept all the way through delivery. The seq also serves as the sync cursor and the read-receipt pointer ("read up to 4,812").

Trade-off: you give up a global order across conversations, which nobody needs. A very busy group is limited by the throughput of one partition. Ordering is per conversation, which is exactly what users notice. Retries are made safe by removing duplicates on client_msg_id, so a resent message never gets a second seq.

Offline users and never losing a message

Weak: deliver only over the live socket. If the recipient is offline, or the push to the gateway fails, the message never reaches them. Messages kept only in a gateway's memory also vanish when that server restarts.

Good: store first, then deliver, then push a notification. The message is in the database before the sender gets the "sent" tick. Delivery workers try the live socket. If the user is offline, they send a push notification, and the message waits in storage.

Great: make the pull path the source of truth. Every device remembers the last seq it has per conversation. On reconnect, it calls sync and pulls everything after that seq. Live pushes become a speed optimisation, not the guarantee.

Delivery is at-least-once: a message can arrive both by push and by sync. The client removes duplicates by message_id. Receipts work the same way: delivered_seq and read_seq per member, so one update covers many messages.

Trade-off: sync on reconnect causes a burst of reads when millions reconnect at once, for example after a network blip or a gateway deploy. So add jitter (a small random delay) to reconnects, and paginate sync. In return, a lost push, a crashed worker or a gateway restart can delay a message but never lose it.

Holding 10 million open connections

Weak: clients poll GET /messages every few seconds. If 10 million online users poll every 3 seconds, that is over 3 million requests a second. Almost all of them return nothing, and messages still arrive up to 3 seconds late.

Good: WebSockets on a gateway tier. Each device keeps one persistent connection, and the server pushes when there is something to send. At about 50,000 connections per server, 10 million connections need about 200 gateways. A session store maps each user to their gateway, so any worker can find the right server.

Great: keep gateways simple, with no message state. Gateways only hold sockets, check authentication and forward messages. All ordering, storage and fan-out (copying a message to every recipient) happen behind them. So a gateway can be drained and redeployed: its clients reconnect elsewhere, with jitter, and catch up with sync.

Heartbeats every 30 seconds keep NAT mappings alive and refresh presence with a TTL. (NAT mappings are the router entries that let traffic reach a phone.) A dead connection disappears from the session store within a minute, with no cleanup job.

Trade-off: long-lived sticky connections make deploys and load balancing uneven. A new server gets no traffic until clients reconnect. The session lookup also adds an extra hop to every delivery. Both costs are smaller than polling, and stateless gateways are what make failures survivable.

What breaks

A gateway crashes: its 50,000 clients drop and reconnect elsewhere with jittered backoff (growing waits plus a random delay). They sync from their last seq, so nothing is lost. You notice it as a drop in open connections and a spike in reconnects.
Kafka consumer lag grows, meaning delivery workers fall behind the queue. Messages are safe in storage but arrive late. Alert on lag and on send-to-deliver p99, then add workers or partitions.
The session store loses data: workers cannot find recipients, so they fall back to push notifications. Clients register again on their next heartbeat, within a minute.
Write latency on the message store spikes: the sender sees a spinner, and the app retries with the same client_msg_id. The retry cannot create a duplicate.
A mass reconnect after a regional network blip causes a sync storm against the message store. Degrade by rate-limiting sync per user, and serve the first page of each conversation first.

What each level is expected to cover

Mid level: A Mid candidate should estimate tens of thousands of messages a second, and choose WebSockets with a gateway tier behind a load balancer. They should store messages before acknowledging them, and put a queue between sending and delivering. They should handle offline users with storage plus push, and explain the ticks clearly.

Senior: A Senior candidate also gives per-conversation sequence numbers with a single writer. They make the pull-based sync the real guarantee, with at-least-once delivery and duplicate removal on the client. They model receipts as per-member seq pointers, size the connection tier, and handle reconnect storms and gateway deploys.

Common mistakes

Polling for new messages instead of holding a persistent connection.
Ordering by client timestamps, so group conversations show replies before questions.
Acknowledging the sender before the message is durably stored.
Storing undelivered messages only in the gateway's memory.
One read-receipt row per message per member, which multiplies writes by the group size.
Putting messages in a single relational table with no partitioning plan, at about 400 GB a day.

Follow-up questions

How would group chats of 10,000 members change the fan-out design?
A user has the app open on a phone and a laptop. How do both stay in sync, including read state?
How would end-to-end encryption change what the server stores and can do?
How do you show "online" and "last seen" without overloading the session store?
How would you deploy a new gateway version without disconnecting everyone at once?