Presence at 100M Concurrent Users

🎯 Design a presence system that tracks online/offline status for 100M+ concurrent users, delivering status changes to relevant contacts within seconds

Concepts Involved

Redis (TTL) Pub/Sub Consistent Hashing WebSocket Rate Limiting

Problem Statement

How does a presence system track online/offline status for 100M+ concurrent users in real-time, delivering status changes to relevant contacts within seconds without flooding the network with unnecessary updates?

Scope: Presence tracking and propagation only · not messaging, media, or call signaling. The hard question: how do you fan-out status changes to hundreds of contacts per user without creating a thundering herd of 27.5M updates/sec✗

100M

concurrent users

online at any moment

total registered users

~5% online at peak

<5s

status propagation

P99 delivery target

~500

contacts per user

avg fan-out factor

Cross-reference: Slack's Presence Protocol handles 10M users with centralized Redis · this problem is 10· that scale requiring distributed presence. A single Redis cluster cannot handle 27.5M presence updates/sec fan-out.

Functional Requirements

What the system must do · the core user-facing behaviours for presence

Must Have (Core)

1. Track user online/offline status based on active connection
2. Propagate status changes to relevant contacts within <5 seconds
3. Show "last seen" timestamp when a user goes offline
4. Support privacy controls · user can hide presence from specific contacts
5. On app open, fetch bulk presence for visible contacts (chat list)
6. Detect network disconnects vs intentional close (grace period before marking offline)

Out of Scope (this design)

✗ Typing indicators (separate ephemeral event system)
✗ Message delivery (separate messaging pipeline)
✗ Read receipts (separate aggregation problem)
✗ Group presence aggregation (e.g., "5 members online")
✗ Custom status messages ("In a meeting")
✗ Push notifications for presence changes

Non-Functional Requirements

The quality constraints that shape every architectural decision

Property	Target	Why It Matters / Design Impact
Latency	P99 <5s status propagation	Users expect near-instant presence. Drives push-based delivery over polling.
Throughput	55K status changes/sec ingress · 27.5M updates/sec fan-out	Requires sharded presence servers and subscription-based delivery to avoid broadcast storms.
Availability	99.9% (<8.7 hrs/year)	Presence is best-effort · brief inconsistency is acceptable. Availability over strict consistency.
Consistency	Eventual · converge within 10s	Stale presence for a few seconds is acceptable. No need for strong consistency.
Scalability	Linear scale to 100M+ concurrent	Shard by user_id. Each shard independently manages presence for its user subset.
Efficiency	Minimize unnecessary updates	Only notify online subscribers. Suppress flapping. Batch where possible.
Fault tolerance	Survive shard failures gracefully	Shard failure = users on that shard appear "unknown" briefly. No cascading failures.
Privacy	Respect per-user visibility settings	Privacy checks at fan-out time. Never leak presence to blocked contacts.

Key tension: Freshness vs. Efficiency. Pushing every heartbeat to 500 contacts would be fresh but wasteful. The system trades a few seconds of staleness for 100· reduction in network traffic via subscription-based delivery and flap suppression.

Scale Estimation

Given numbers → derive infrastructure sizing. Show this reasoning in an interview.

Given (from question): 100M concurrent users · 2B total users · <5s status propagation · ~500 contacts per user

Step	What to Derive	Calculation	Result	Design Decision
1	Status changes/sec	100M users · 1 change per 30min avg = 100M · 1800	~55K changes/sec	Ingress rate · manageable for sharded presence servers
2	Fan-out updates/sec	55K changes/sec · 500 contacts per user	27.5M updates/sec	Naive broadcast is impossible · must filter to online subscribers only
3	Online contacts (filtered)	500 contacts · 5% online rate = 25 online contacts avg	55K · 25 = 1.37M updates/sec	Subscription model reduces fan-out by 20· · now feasible
4	Presence server shards	100M users · 1M users/shard (memory + CPU budget)	~100 shards	Each shard holds 1M user states in-memory (~50MB RAM per shard)
5	Memory per shard	1M users · 50 bytes (user_id + status + timestamp)	~50MB per shard	Fits easily in RAM. Subscription lists add ~200MB more per shard.
6	Cross-shard routing	User A on shard-7, contact B on shard-42 → internal pub/sub	~80% cross-shard	Internal message bus (Kafka/NATS) for cross-shard subscription routing
7	Why centralized Redis breaks	27.5M ops/sec · ~300K ops/sec per Redis node	~92 Redis nodes just for fan-out	Redis cluster can't handle subscription routing · need purpose-built sharding
8	Network bandwidth	1.37M updates/sec · 64 bytes per update	~88 MB/s outbound	Distributed across 100 shards = ~880 KB/s per shard · trivial

Interview tip: Start with step 1·2 (the naive fan-out math) to show the problem is hard. Then step 3 (subscription filtering) is the key insight that makes it tractable. The 27.5M → 1.37M reduction is what drives the entire architecture.

Architecture Overview

Sharded presence servers with subscription-based delivery and hierarchical aggregation

Architecture principle: Each presence shard is a self-contained unit · it owns the state for its users AND the subscription list of who wants to know about those users. Cross-shard communication happens via internal pub/sub, not shared state.

Subscription Model

User subscribes to contacts' presence on connect. Server only pushes changes to online subscribers. Unsubscribe on disconnect.

Key optimization: Batch subscribe operations. When User B connects, send one message per shard containing all contacts on that shard · not 500 individual subscribe requests. This reduces 500 ops to ~100 ops (one per shard that has B's contacts).

Distributed Presence Sharding

Shard by user_id. Each shard owns presence state for a subset of users. Cross-shard subscription routing via internal pub/sub.

Shard rebalancing: Use consistent hashing with virtual nodes. Adding a shard moves only ~1% of users. During migration, old shard forwards requests to new shard for a brief window. Presence is ephemeral · worst case is a few seconds of stale data during rebalance.

Optimization Techniques

Batch updates, suppress flapping, lazy propagation, and presence priority to minimize network load

➔ Batch Updates

Collect presence changes over a 500ms window before fanning out.
If 10 users on the same shard change status within 500ms, send one batch notification to each gateway instead of 10 individual pushes.
Reduces gRPC calls by ~5· during peak hours.

? Flap Suppression

If user goes online?offline?online within 30s, suppress the offline notification entirely.
Common cause: network switch (WiFi?cellular), brief tunnel, app backgrounded momentarily.
Eliminates ~40% of status changes that would confuse contacts.

➔ Lazy "Last Seen" Propagation

"Last seen" timestamps are NOT pushed in real-time. They're stored on the shard and fetched on-demand when a contact opens a chat.
Only the online/offline transition is pushed. Timestamp is pulled lazily.
Saves ~50% of update payload size on the push path.

? Presence Priority

Not all contacts are equal. Close contacts (frequent chat partners) get presence updates first.
Under load shedding, deprioritize updates to contacts you haven't chatted with in 30+ days.
Graceful degradation · important contacts always get fresh presence.

Combined effect: Batching (5·) + flap suppression (40% reduction) + subscription filtering (20·) = the system handles ~100· less traffic than naive broadcast. This is what makes 100M scale feasible on ~100 presence servers.

Resilience & Edge Cases

What happens when things go wrong · and how the system recovers gracefully

Scenario	Impact	Mitigation
Presence shard crash	1M users' presence becomes unknown for ~10s	Standby replica takes over. Clients re-subscribe on reconnect. Presence rebuilds from gateway heartbeats within seconds.
Gateway crash	500K users disconnect simultaneously	30s grace period before marking offline (flap suppression). Users reconnect to another gateway and re-subscribe. Stale subscriber entries cleaned up lazily.
Network partition	Cross-shard pub/sub delayed	Presence updates queue in NATS/Kafka. On partition heal, batch-deliver queued updates. Users see slightly stale presence (seconds, not minutes).
Thundering herd (mass reconnect)	Millions subscribe simultaneously after outage	Rate-limit subscribe operations. Stagger reconnects with exponential backoff + jitter on client. Serve cached presence (possibly stale) during recovery.
Celebrity user (10K+ contacts online)	Single status change fans out to 10K+ subscribers	Cap subscriber list at 1000. Beyond that, use pull-based presence (contacts fetch on chat open). Tiered delivery: close contacts first.
Stale subscriber entries	Push to users who already disconnected	Gateway returns "not_found" for disconnected users. Presence shard removes stale entries. Periodic sweep every 60s as safety net.
Privacy setting change	User blocks contact · must stop presence immediately	Privacy change triggers immediate unsubscribe on the relevant shard. Blocked contact sees "last seen: unavailable" from that point.
Clock skew across shards	"Last seen" timestamps inconsistent	Use logical timestamps (Lamport clocks) for ordering. Physical timestamps only for display · minor skew (seconds) is acceptable for "last seen".

Design philosophy: Presence is best-effort, not mission-critical. A few seconds of stale data is acceptable. The system optimizes for availability and efficiency over strict consistency. No user action depends on presence being perfectly accurate.

Tech Stack

Purpose-built components optimized for presence at 100M scale

Component	Technology	Why This Choice
Presence Servers	Custom Erlang/Elixir or Rust	Millions of lightweight processes (Erlang) or zero-cost async (Rust). In-memory state with fast HashMap lookups. Erlang's fault tolerance via supervisors.
Gateway (WS termination)	Custom C++/Rust with epoll	500K+ connections per server. Minimal per-connection overhead. Protocol-aware routing to presence shards.
Internal Pub/Sub	NATS JetStream	Low-latency cross-shard messaging. Subject-based routing (presence.shard.{N}). At-most-once delivery is fine for presence.
Shard Discovery	ZooKeeper / etcd	Consistent hashing ring management. Shard assignment and leader election for replicas.
Persistent Storage	Cassandra / ScyllaDB	"Last seen" timestamps persisted for offline queries. Write-heavy, eventually consistent · perfect fit.
Privacy/Contact Store	MySQL/Vitess (sharded)	Contact lists and privacy settings. Read on connect, cached in presence shard memory.
Monitoring	Prometheus + Grafana	Per-shard metrics: subscriber count, fan-out latency, flap rate, cross-shard message lag.
Load Balancing	Envoy / HAProxy (L4)	TCP-level load balancing for WebSocket connections. Consistent hashing for presence shard routing.

Interview Cheat Sheet

5 key points to nail the presence system design interview

1. Start with the math · show the problem is hard.
100M users · status change every 30min = 55K changes/sec. Each change · 500 contacts = 27.5M updates/sec. This is why naive broadcast fails and why you need subscription-based delivery.

2. Subscription model is the key insight.
Only push to online subscribers (not all contacts). 500 contacts → ~25 online subscribers = 20· reduction. Subscribe on connect, unsubscribe on disconnect. This single decision makes the system feasible.

3. Shard by user_id, not by "channel" or "group".
Each shard owns presence state + subscriber lists for its users. Cross-shard routing via internal pub/sub. ~100 shards for 100M users. Each shard is an independent failure domain with its own replica.

4. Flap suppression + batching for efficiency.
30s grace period before marking offline (handles WiFi?cellular switches). Batch updates in 500ms windows. Combined with subscriptions, total traffic is ~100· less than naive approach.

5. Presence is best-effort · design for availability over consistency.
A few seconds of stale presence is acceptable. No user action depends on it being perfectly accurate. This relaxation allows eventual consistency, simpler failure handling, and graceful degradation under load.

System Design Case Study