System Design Case Study

How does a real-time chat system deliver one message to 50,000 online users in under 200ms · while sustaining 10B+ messages per day at peak 200K msg/sec?

Functional Requirements

What the system must do · the core user-facing behaviours

Non-Functional Requirements

The quality constraints that shape every architectural decision

Scale Estimation

Given numbers from the question ? derive infrastructure sizing. Show this reasoning in an interview.

APIs

Three surfaces: WebSocket (real-time delivery) · REST (send + history) · Internal gRPC (fan-out worker ? Gateway)

Architecture Overview

Write path (send) ? Kafka ? Fan-out workers ? Gateway ? WebSocket push to clients

Data Model

Hot path (Redis, sub-ms) for presence and routing · Cold path (Vitess/MySQL) for durable message history

-- -- COLD PATH: Vitess/MySQL (sharded by workspace_id) ----------------------

CREATE TABLE messages (
  workspace_id   BIGINT NOT NULL,    -- shard key: all workspace data on one shard
  channel_id     BIGINT NOT NULL,
  channel_seq    BIGINT NOT NULL,    -- monotonic per channel (gap detection + backfill)
  user_id        BIGINT NOT NULL,
  client_msg_id  VARCHAR(64),        -- client UUID for idempotent dedup (indexed)
  content        TEXT,
  ts             TIMESTAMP(6),
  PRIMARY KEY (workspace_id, channel_id, channel_seq),
  UNIQUE KEY uq_client_msg (workspace_id, user_id, client_msg_id)
  -- PK doubles as the range-scan index for catch-up reads
);

CREATE TABLE channel_members (
  workspace_id   BIGINT NOT NULL,
  channel_id     BIGINT NOT NULL,
  user_id        BIGINT NOT NULL,
  joined_at      TIMESTAMP,
  PRIMARY KEY (workspace_id, channel_id, user_id)
);

-- -- HOT PATH: Redis (sub-ms, ephemeral) -------------------------------------

channel:{id}:online    ? SET of user_ids currently connected (updated on WS connect/disconnect)
user:{id}:gateway      ? STRING "gateway-host-7"  (which Gateway holds this user's WS)
channel:{id}:members   ? SET of all user_ids       (cached from MySQL, TTL-refreshed)

-- Fan-out worker reads channel:{id}:online ? groups by gateway ? gRPC batch push
-- On disconnect: DEL user:{id}:gateway + SREM channel:{id}:online {user_id}

Fan-out Strategy · The Hard Part

A 50K-member channel with 200K msg/sec would require 10 billion writes/sec if you pushed to every member. The fix: push only to online members (~10% of total).

Presence Protocol

How does the system know who is online? Heartbeat-based presence with Redis TTL · no gossip protocol needed at this scale

Ordering & Consistency

All messages in #general must arrive in the same order for every user · achieved via Kafka partition-per-channel + monotonic sequence numbers

Sequence Number Generation

How does the system assign monotonic channel_seq without becoming a bottleneck? Single-writer-per-channel via Kafka partition ownership.

Resilience & Edge Cases

Five failure modes and how the architecture handles each without data loss or visible gaps

Backpressure & Flow Control

What happens when a Gateway is overwhelmed, Kafka consumers lag, or a viral message hits a 50K channel? Layered defense from client to storage.

Multi-Region Architecture

How does a message sent in US-East reach a user in EU-West within 250ms? GeoDNS + regional Kafka clusters + async cross-region replication

Tech Stack & Tradeoffs

Each choice made for a specific reason · and what was rejected

Related Deep Dives

Each problem below explores one aspect of this system at greater depth or different scale

Interview Cheat Sheet

The 8 things an interviewer wants to hear · say these and you've covered the design