System Design Case Study

Design notification aggregation — "5 people liked your post" instead of 5 separate notifications

🎯 Design notification aggregation — batch multiple events about the same entity into one notification ("5 people liked your post") with immediate first-event delivery and time-windowed subsequent batching
Concepts Involved
Time-Windowed Aggregation Redis Counters Stream Processing Collapse/Update Notifications Per-Entity Batching

Problem Statement

How do you aggregate multiple engagement events (likes, comments, follows) about the same entity into a single notification — delivering the first event immediately but batching subsequent events into updates like "5 people liked your post"?

Scope: Time-windowed aggregation per entity per user, first-event immediate delivery, subsequent events queued into batch window, dynamic count templates, collapse/update existing notifications. Not covering channel delivery, device management, or analytics.
1B/day
engagement events
likes, comments, shares, follows
15 min
max batch window
aggregate within this window
85%
notification reduction
10 likes → 1 aggregated notification
150M
aggregated notifications/day
down from 1B raw events

Functional Requirements

What the aggregation system must do · core batching behaviours

Must Have (Core)

1. First-event immediate delivery · first like on a post notifies immediately
2. Subsequent event batching · 2nd-Nth events within window aggregate into update
3. Per-entity aggregation key · group by (target_user, entity_type, entity_id)
4. Dynamic count template · "{actor} and {N} others liked your {entity}"
5. Collapse existing notification · update in-app feed entry instead of adding new row
6. Window expiry flush · after 15 min, flush batch regardless of count

Out of Scope (this design)

✗ Channel delivery mechanics (push/email/SMS workers)
✗ Preference management (opt-out, frequency caps)
✗ Content personalization (ML-driven message copy)
✗ Actor deduplication within display (showing top 3 names)
✗ Cross-entity aggregation ("activity on 3 of your posts")

Non-Functional Requirements

Quality constraints shaping the aggregation architecture

PropertyTargetWhy It Matters / Design Impact
First-Event Latency<3sFirst like on a post must notify immediately. No batching delay for the initial event.
Batch Window15 min maxAfter 15 min, flush whatever is in the batch. User shouldn't wait longer than 15 min for aggregated update.
Throughput1B events/day (11.5K/sec)Kafka partitioned by aggregation key for per-entity processing.
Notification Reduction85%+ reductionFrom 1B raw events down to ~150M aggregated notifications. Reduces user notification fatigue.
ConsistencyCount accuracy ±1Aggregated count may be slightly stale (±1) but must converge. Eventual consistency acceptable.
Collapse GuaranteeNo duplicate notificationsAn already-displayed "3 people liked" must update to "5 people liked", not show a separate notification.
ScalabilityLinear with eventsPartitioned by aggregation key. Adding Kafka partitions + consumers scales linearly.
Key tension: Immediacy vs. Aggregation. Users want to know about the first like instantly, but don't want 50 separate notifications. Solution: immediate delivery of first event, then switch to batching mode for that entity for the next 15 minutes.

Scale Estimation

Derive infrastructure sizing from given constraints. Show this reasoning in an interview.

Given: 1B engagement events/day · 15 min batch window · avg 6.7 events per aggregated notification
StepWhat to DeriveCalculationResultDesign Decision
1 Avg events/sec 1B / 86,400s ~11,574/sec Kafka with 64 partitions (key = aggregation_key hash)
2 Aggregated notifications/day 1B / 6.7 avg events per batch ~150M/day 85% reduction in downstream notification volume
3 Active aggregation windows 11,574 new entities/sec × 900s window ~10.4M active windows Redis hash per active window. ~10M keys at any time.
4 Redis memory for windows 10.4M windows × 256 bytes (key + count + actors + timer) ~2.7GB Single Redis cluster (8 shards) with room for peak
5 Window flush rate 10.4M windows / 900s avg lifetime ~11,500 flushes/sec Matches event rate. Each flush emits one aggregated notification.
6 Peak event burst 11.5K × 10x (viral post) ~115K/sec peak Auto-scale consumers. Hot partition detection for viral entities.
7 Consumer instances 64 partitions × 1 consumer per partition 64 consumers Each consumer processes ~180 events/sec avg. Headroom for burst.
8 Window timer storage Redis sorted set for timer expiry, 10.4M entries ~1.5GB ZRANGEBYSCORE polls every 1s for expired windows
Interview tip: The key insight is the 85% reduction ratio. For every 1B raw events, you only send ~150M actual notifications. This is the core value proposition of aggregation — massive reduction in user notification fatigue while preserving awareness of activity.

APIs

Aggregation service internal APIs · event ingestion and window management

POST /v1/events/engagement (Internal — from engagement service)
Ingest an engagement event for aggregation processing
{
  "event_id": "evt_001",
  "event_type": "like",                 // like | comment | share | follow | mention
  "actor_id": "u_actor_456",
  "actor_name": "Jane Smith",
  "target_user_id": "u_owner_123",      // notification recipient
  "entity_type": "post",                // post | comment | story | profile
  "entity_id": "post_789",
  "entity_preview": "My vacation photos...",
  "timestamp": "2024-03-15T10:30:00Z"
}
Response: 202 Accepted · { "aggregation_key": "like:post_789:u_owner_123", "window_status": "active" }
GET /v1/aggregation/window/{aggregation_key} (Internal)
Query current aggregation window state (for debugging/monitoring)
GET /v1/aggregation/window/like:post_789:u_owner_123

Response 200:
{
  "aggregation_key": "like:post_789:u_owner_123",
  "count": 5,
  "first_actor": "Jane Smith",
  "recent_actors": ["Jane Smith", "Bob Lee", "Alice Kim"],
  "window_opened_at": "2024-03-15T10:30:00Z",
  "window_expires_at": "2024-03-15T10:45:00Z",
  "first_event_delivered": true,
  "last_batch_sent_at": "2024-03-15T10:35:00Z"
}
POST /v1/aggregation/flush (Internal — timer-triggered)
Force-flush an aggregation window (called by timer or threshold trigger)
{
  "aggregation_key": "like:post_789:u_owner_123",
  "reason": "window_expired",          // window_expired | threshold_reached | manual
  "emit_notification": {
    "template_id": "engagement_aggregated",
    "target_user_id": "u_owner_123",
    "params": {
      "count": 5,
      "first_actor": "Jane Smith",
      "entity_type": "post",
      "entity_preview": "My vacation photos..."
    },
    "collapse_key": "like:post_789:u_owner_123"
  }
}

Data Model

Redis structures and persistent store for aggregation windows

aggregation_window (Redis Hash per active window)

// Key: agg_window:{aggregation_key}
// aggregation_key = {event_type}:{entity_id}:{target_user_id}
HSET agg_window:like:post_789:u_owner_123
  "count"           "5"
  "first_actor_id"  "u_actor_456"
  "first_actor_name" "Jane Smith"
  "recent_actors"   "[\"Jane Smith\",\"Bob Lee\",\"Alice Kim\"]"  // last 3
  "entity_type"     "post"
  "entity_preview"  "My vacation photos..."
  "window_opened"   "1710495000000"     // unix ms
  "first_delivered"  "1"                // boolean: first event already sent
  "last_batch_ts"   "1710495300000"     // last batch notification sent

EXPIRE agg_window:like:post_789:u_owner_123 960  // 16 min (window + buffer)

window_timers (Redis Sorted Set — for expiry polling)

// Score = window_expiry_time_ms (opened_at + 15 min)
ZADD window_timers {expiry_ms} "like:post_789:u_owner_123"

// Poller runs ZRANGEBYSCORE every 1s to find expired windows
ZRANGEBYSCORE window_timers 0 {now_ms} LIMIT 0 1000

notification_feed (update existing entry on collapse)

// In-app feed entry — updated in place when aggregation count changes
{
  "notification_id": "n_agg_001",
  "target_user_id":  "u_owner_123",
  "collapse_key":    "like:post_789:u_owner_123",   // used to find & update
  "template_id":     "engagement_aggregated",
  "params": {
    "count": 5,
    "first_actor": "Jane Smith",
    "others_count": 4,
    "entity_type": "post",
    "entity_preview": "My vacation photos..."
  },
  "rendered_text":   "Jane Smith and 4 others liked your post",
  "updated_at":      "2024-03-15T10:45:00Z",
  "read":            false
}

Kafka Topics

// Input: raw engagement events (partitioned by aggregation_key)
Topic: engagement-events
  Key: "like:post_789:u_owner_123"
  Value: { event_id, actor_id, actor_name, timestamp, ... }

// Output: aggregated notifications (to notification service)
Topic: aggregated-notifications
  Key: "u_owner_123"
  Value: { template_id, params, collapse_key, ... }

Architecture Overview

Event Stream → Aggregation Processor → Window Manager → Batch Flusher → Notification Service

EVENT INGESTION · Engagement events from social platform Like Service 1B likes/day Comment Service 200M comments/day Share Service 100M shares/day Follow Service 50M follows/day Kafka (64 partitions) Key = aggregation_key AGGREGATION PROCESSOR · First-event gate + window accumulation First-Event Gate Check: does window exist for this agg_key? NO → deliver immediately + open window Window Accumulator YES window exists → HINCRBY count Append actor to recent_actors list Threshold Checker If count hits threshold (10, 50, 100) → emit intermediate batch update WINDOW TIMER · Track expiry + flush on timeout Redis ZSET (window_timers) Score = expiry_time · 10.4M entries Timer Poller (every 1s) ZRANGEBYSCORE 0 {now} → expired windows Flush Dispatcher Read window state → emit aggregated notification NOTIFICATION EMISSION · Template + Collapse + Deliver Template Renderer "Jane and 4 others liked your post" Collapse Manager Find existing notif by collapse_key → update Push Updater (silent push to refresh badge) Update in-app feed + silent push for badge count NOTIFICATION SERVICE · Channel delivery (push / email / in-app) First Event: Immediate Push "Jane Smith liked your post" (instant) Batch: Collapsed Update "Jane and 4 others liked your post" (update) In-App Feed Update Overwrite existing row + bump timestamp KEY INSIGHT First event = immediate delivery Opens 15-min aggregation window. Subsequent events accumulate.

Key Design Decisions

Tradeoffs made in this architecture and their rationale

DecisionOptions ConsideredChosenWhy
First-Event Strategy Always batch (delay all), always immediate, hybrid (first immediate + batch rest) Hybrid: first immediate, rest batched Users want instant feedback on first interaction. But 50 separate "X liked your post" is spam. Hybrid gives both immediacy and aggregation.
Window Duration 5 min, 15 min, 30 min, adaptive 15 min fixed window Short enough that users get timely updates. Long enough to meaningfully aggregate. Adaptive adds complexity without proportional UX benefit.
Aggregation Key Per-entity only, per-entity-per-type, per-user-per-entity-per-type {event_type}:{entity_id}:{target_user_id} Separate likes from comments on same post. Per-user ensures personalized notification per recipient.
Collapse Behavior New notification each batch, update existing, replace existing Update existing notification in-place User sees one entry per entity in feed, count updates. No notification list pollution. Push notification updates silently.
Intermediate Updates Only on window expiry, on every event, on thresholds (10, 50, 100) Threshold-based intermediate updates For viral posts with 1000+ likes, update at thresholds (10, 50, 100, 500) so user sees activity without waiting 15 min.
Window State Store In-memory (Flink state), Redis, database Redis Hash per window Fast atomic HINCRBY for count. EXPIRE handles cleanup. Survives consumer restart (unlike in-memory Flink state).
Why first-event immediate matters: If someone likes your post and you only find out 15 minutes later in a batch, the platform feels unresponsive. The first event creates the "someone noticed me" dopamine hit. Subsequent events are less urgent — "5 people" vs "6 people" doesn't matter in real-time.
Collapse key prevents notification pollution: Without collapse, a viral post generates a new notification every batch window (every 15 min). With collapse_key, the existing notification updates in-place: "5 people liked" → "50 people liked" → "1,000 people liked". User sees ONE entry in their feed, count grows.
Hot entity protection: A celebrity post may get 10M likes in 1 hour. Without protection, this creates 10M events on a single Kafka partition (same aggregation_key). Solution: detect hot entities (>1K events/min), promote to dedicated processing with sampling — count accurately but process fewer events.
Actor list management: Showing "Jane, Bob, and 998 others" requires storing actor names. Don't store all 1000 — keep only the 3 most recent actors in the Redis hash. The count is accurate, but the displayed names are the latest. Template: "{actor1}, {actor2}, and {count-2} others".

Resilience & Edge Cases

How the aggregation system handles failures and edge cases

ScenarioProblemSolutionRecovery
Consumer Restart Aggregation consumer restarts mid-window. In-memory state lost. All window state is in Redis (not in-memory). Consumer reads Redis state on each event. Stateless consumer design. Instant recovery (Redis has state)
Window Timer Miss Timer poller misses a window expiry (poller crash/lag). Redis EXPIRE on window hash ensures data is cleaned even if timer missed. Catch-up: scan for windows older than 15 min + buffer. <30s catch-up on poller recovery
Unlike/Undo Event User unlikes a post after being counted in aggregation. Decrement count in window (HINCRBY -1). If count drops to 0, cancel the pending aggregated notification. Don't send "0 people liked". Real-time count correction
Viral Entity (10M events) Single entity gets millions of events, overwhelming one partition. Hot entity detection: >1K events/min triggers sampling mode. Count = sample_count × sample_rate. Accuracy ±5% is acceptable for "~10K people liked". Automatic sampling mode
Redis Failure Redis cluster unavailable, window state inaccessible. Fallback: treat every event as first-event (immediate delivery). Temporarily disable aggregation. Resume when Redis recovers. Brief notification burst acceptable. Degraded mode until Redis recovers
Duplicate Events Same like event processed twice due to Kafka redelivery. Event deduplication: check event_id in Redis SET before processing. TTL 1 hour. Prevents double-counting in aggregation window. Zero count inflation

Interview Cheat Sheet

5 key points to nail a notification aggregation interview

1. Hybrid Strategy: First Immediate + Rest Batched
The first event for an entity triggers immediate delivery ("Jane liked your post"). This opens a 15-minute aggregation window. Subsequent events increment a counter in Redis. On window expiry, emit one aggregated notification ("Jane and 4 others liked your post"). This gives users both immediacy and spam reduction.
2. Aggregation Key Design
Key = {event_type}:{entity_id}:{target_user_id}. This groups all likes on the same post for the same recipient. Different event types (like vs. comment) on the same post get separate aggregation windows. The key also serves as the Kafka partition key, ensuring all events for one window go to one consumer.
3. Collapse Key for In-Place Updates
When the batch fires, it doesn't create a new notification. It finds the existing notification (by collapse_key) and updates it in-place: count goes from 3 to 5, actor list refreshes. The user sees one clean feed entry that evolves over time, not a stream of "X and Y others liked" entries.
4. 85% Notification Volume Reduction
The core value proposition: 1B raw events → 150M actual notifications. Average 6.7 events per aggregation window. This dramatically reduces push notification fatigue, improves user engagement (fewer dismissals), and reduces infrastructure cost on downstream delivery systems.
5. Window State in Redis (Not In-Memory)
Storing window state in Redis (not in consumer memory) makes consumers stateless. On restart, no state is lost. HINCRBY atomically increments counts. EXPIRE auto-cleans dead windows. This design trades slightly higher latency (Redis round-trip) for operational simplicity and fault tolerance.