Chat System
A real-time chat system lets users send and receive messages instantly. Designing one requires balancing low-latency delivery, persistent storage, presence awareness, and scalability across millions of concurrent connections. The canonical examples are WhatsApp, Messenger, Slack, and Discord.
The hardest challenges are connection management — keeping millions of WebSocket connections alive — and message synchronization — ensuring every device sees the right messages in the right order, even after going offline.
This case study designs a chat system supporting one-on-one chats, group chats, presence, read receipts, and media sharing. It covers requirements, capacity estimation, protocol choice, storage, and delivery architecture.
Start by defining what the system must do and the scale it must support. Chat systems have both real-time and persistent requirements, which create tension.
Functional Requirements
Non-Functional Requirements
info
Estimating scale helps choose the right storage, connection, and message routing architecture.
| 1 | Assumptions: |
| 2 | - 100M daily active users |
| 3 | - Each user sends 50 messages/day |
| 4 | - Total messages/day: 5B |
| 5 | - Average message size: 200 bytes |
| 6 | - 10M concurrent connections at peak |
| 7 | - Each connection: ~10 KB memory on server |
| 8 | |
| 9 | Messages per second: 5B / 86,400 ≈ 57,870 msg/sec |
| 10 | Peak message rate: 3× average ≈ 173,610 msg/sec |
| 11 | |
| 12 | Daily storage (text messages): |
| 13 | 5B × 200 bytes ≈ 931 GB/day |
| 14 | |
| 15 | Connection memory: |
| 16 | 10M × 10 KB ≈ 100 GB RAM across chat servers |
| 17 | |
| 18 | Media storage is orders of magnitude larger and |
| 19 | is offloaded to object storage with CDN delivery. |
note
Real-time chat needs a persistent, bidirectional connection. WebSocket is the standard for browser and mobile apps. For environments where WebSocket is blocked, long polling and Server-Sent Events can fall back.
| Protocol | Direction | Best For |
|---|---|---|
| WebSocket | Bidirectional | Primary chat transport |
| Long polling | Client pull | Fallback for restrictive networks |
| Server-Sent Events | Server to client | Live updates, one-way streams |
best practice
The chat system has three major components: a connection layer for WebSockets, a message routing layer for delivery, and a storage layer for persistence and history. A presence service tracks who is online.
| 1 | High-level architecture: |
| 2 | |
| 3 | Clients (mobile/web/desktop) |
| 4 | │ |
| 5 | ▼ |
| 6 | Load Balancer (L4/L7, sticky by connection) |
| 7 | │ |
| 8 | ▼ |
| 9 | Chat Servers (WebSocket gateways) |
| 10 | │ |
| 11 | ├─▶ Message Service (store and route) |
| 12 | │ │ |
| 13 | │ ├─▶ Message Database (sharded) |
| 14 | │ └─▶ Message Queue (Kafka) |
| 15 | │ |
| 16 | ├─▶ Presence Service (Redis) |
| 17 | │ |
| 18 | └─▶ Media Service (object storage + CDN) |
| 19 | |
| 20 | Users connect to a chat server. Messages are |
| 21 | routed through the message service to the |
| 22 | recipient's chat server or stored for later delivery. |
warning
The storage model depends heavily on access patterns. Chat history is read in time order, usually paginated. Writes are append-only per conversation. A database that supports efficient range scans on a composite key (conversation_id, timestamp) is ideal.
| 1 | Message table schema: |
| 2 | |
| 3 | CREATE TABLE messages ( |
| 4 | conversation_id BIGINT, |
| 5 | message_id BIGINT, |
| 6 | sender_id BIGINT, |
| 7 | content TEXT, |
| 8 | created_at TIMESTAMP, |
| 9 | PRIMARY KEY (conversation_id, message_id) |
| 10 | ); |
| 11 | |
| 12 | message_id can be a Snowflake ID that embeds |
| 13 | timestamp, allowing time-ordered retrieval. |
| 14 | |
| 15 | Conversation membership: |
| 16 | |
| 17 | CREATE TABLE participants ( |
| 18 | conversation_id BIGINT, |
| 19 | user_id BIGINT, |
| 20 | joined_at TIMESTAMP, |
| 21 | PRIMARY KEY (conversation_id, user_id) |
| 22 | ); |
| 23 | |
| 24 | CREATE INDEX idx_user_conversations |
| 25 | ON participants(user_id, joined_at); |
Sharding Strategy
Shard messages by conversation_id to keep all messages for a conversation on the same shard. This makes range queries efficient. For very large groups, consider separate sharding or splitting message metadata from content.
best practice
Presence tracks whether a user is online, offline, or recently active. Redis is a natural fit: store user_id → status with a TTL, and update on connect, disconnect, and heartbeat. Broadcast presence changes to friends or channel members.
| 1 | Presence model: |
| 2 | |
| 3 | Key: presence:user:{id} |
| 4 | Value: { status: "online", last_seen: 1234567890, device: "mobile" } |
| 5 | TTL: 60 seconds |
| 6 | |
| 7 | Heartbeat: |
| 8 | - Client sends heartbeat every 30 seconds |
| 9 | - Server extends TTL in Redis |
| 10 | |
| 11 | On disconnect: |
| 12 | - Mark status as "offline" or "last_seen" |
| 13 | - Notify subscribers (friends, group members) |
| 14 | |
| 15 | Fan-out: |
| 16 | - For 1:1 chats, notify the single recipient |
| 17 | - For groups, publish to a presence topic or pub/sub channel |
info
When a user sends a message, the system must store it, update conversation metadata, and deliver it to online recipients. Recipients who are offline retrieve messages when they reconnect via sync.
| 1 | Send message flow: |
| 2 | |
| 3 | 1. Client sends message to its chat server |
| 4 | 2. Chat server forwards to Message Service |
| 5 | 3. Message Service: |
| 6 | a. Generates message_id |
| 7 | b. Writes message to database |
| 8 | c. Updates conversation last_message_at |
| 9 | d. Publishes to message queue |
| 10 | 4. Message queue fans out to recipient chat servers |
| 11 | 5. Recipient chat servers push over WebSocket |
| 12 | 6. Acknowledgment flows back to sender (delivered receipt) |
| 13 | |
| 14 | Offline recipient: |
| 15 | - Message is stored; no immediate push |
| 16 | - On reconnect, client requests sync from last known message_id |
best practice
Read receipts track when a message has been seen by the recipient. They are small but numerous: every viewed message can generate a receipt. Aggregate read receipts per conversation rather than sending one per message.
| 1 | Read receipt strategy: |
| 2 | |
| 3 | Per-conversation watermark: |
| 4 | - Store max_message_id_read per user per conversation |
| 5 | - Sender sees: "Read up to message X" |
| 6 | - This reduces receipt volume from O(messages) to O(conversations) |
| 7 | |
| 8 | For group chats: |
| 9 | - Store read watermark per participant |
| 10 | - Display "Read by Alice, Bob" or aggregate count |
| 11 | |
| 12 | Delivery receipt: |
| 13 | - Sent when message reaches recipient device |
| 14 | - Separate from read receipt |
warning
Group chats amplify fan-out. A single message in a 1,000-person group must be delivered to 1,000 devices. The architecture must limit the work done synchronously and leverage efficient fan-out mechanisms.
| 1 | Group chat fan-out: |
| 2 | |
| 3 | Small group ({'<'} 1000 members): |
| 4 | - Write message once |
| 5 | - Fan out to each member's inbox / queue |
| 6 | - Each recipient pulls from their queue on reconnect |
| 7 | |
| 8 | Large group (10K+ members): |
| 9 | - Write message once |
| 10 | - Publish to a group channel (pub/sub) |
| 11 | - Online members receive push |
| 12 | - Offline members sync on reconnect |
| 13 | - Avoid storing per-member copies for all messages |
| Group Size | Strategy |
|---|---|
| < 100 | Direct fan-out, per-member inbox |
| 100 - 10,000 | Hybrid: fan-out + channel subscription |
| 10,000+ | Channel-based, pull-on-sync |
pro tip
Users expect messages, read receipts, and deletions to sync across phones, tablets, and desktops. Each device maintains a sync cursor — the last message_id or timestamp it has seen — and requests everything newer on reconnect.
| 1 | Sync flow: |
| 2 | |
| 3 | 1. Device connects and sends last_sync_message_id |
| 4 | 2. Server queries messages where message_id {'>'} last_sync_id |
| 5 | for all conversations the user participates in |
| 6 | 3. Server returns new messages, receipt updates, and metadata changes |
| 7 | 4. Device updates local state and acknowledges new cursor |
| 8 | |
| 9 | Optimizations: |
| 10 | - Paginate sync responses |
| 11 | - Exclude muted conversations unless opened |
| 12 | - Use push notifications to trigger sync on mobile |
note