Edge/DDoS rate limiting at the CDN layer

➔? Design edge-layer rate limiting to absorb DDoS attacks at the proxy/CDN layer before traffic reaches origin servers

Concepts Involved

Rate Limiting CDN Load Balancer Security

Problem Statement

Design coarse rate limiting at the CDN/edge layer to absorb volumetric DDoS attacks before traffic reaches origin servers. Per-IP limiting at the proxy layer, where approximate counting is acceptable, with fail-open vs fail-closed trade-offs under active attack.

Core challenge: At the edge, you're processing millions of requests per second per node. You cannot afford Redis calls or complex algorithms. Rate limiting must be in-memory, per-worker, approximate — and that's fine. The goal isn't precision; it's absorbing 99% of attack volume so origin servers survive. False positives are acceptable during active attack.

10M+

req/sec attack volume

volumetric DDoS

Per-IP

coarse key

+ /24 subnet

Approximate

good enough

speed > precision

Fixed Window

simplest & fastest

NGINX-style

Functional Requirements

What the system must do · edge-layer volumetric attack mitigation

Must Have (Core)

1. Per-IP fixed window counter in-memory (no external deps)
2. Subnet-level blocking (/24 IPv4, /48 IPv6) for rotating IPs
3. Progressive escalation: 429 → TCP RST → IP blocklist → L3 null-route
4. Operator-controlled fail-open/fail-closed switch
5. Challenge page (JS challenge) for gray-area traffic during attacks

Out of Scope (this design)

✗ Application-layer attack analysis (L7 payload inspection)
✗ Bot fingerprinting ML (behavioral detection models)
✗ Origin server protection (separate concern, behind CDN)
✗ Legitimate traffic classification (user reputation systems)
✗ Geographic filtering (geo-blocking decisions)

Non-Functional Requirements

Quality constraints shaping the edge rate limiting design

Property	Target	Why It Matters / Design Impact
Latency	Nanoseconds (L1 cache hit)	No network I/O. In-memory hash map per worker. Counter access from CPU cache.
Throughput	Handle 10M+ req/sec per edge node	Each worker processes millions of requests. Fixed window O(1) check is essential.
Accuracy	Approximate (±5%)	Irrelevant at DDoS scale. If counter says 1050 when real is 1000, still an attacker. Speed > precision.
Memory	Bounded hash map with LRU eviction (max 1M entries/worker)	Must not exhaust worker memory. Evict stale entries. ~20MB per worker.
Mode Switch	Operator can toggle within seconds	Fail-open (normal) to fail-closed (under attack) via control plane broadcast.

Key constraint: Speed over precision. At 10M req/sec you cannot afford Redis calls, sorted sets, or weighted calculations. Fixed window counter with in-memory hash map is the only viable approach at this scale.

Scale Estimation

Given numbers from the question → derive edge infrastructure sizing

Given (from question): 10M req/sec attack · 500K req/sec legitimate · threshold 1000 req/sec per IP

Step	What to Derive	Calculation	Result	Design Decision
1	Botnet profile	100K IPs × 95 req/sec each	Below per-IP threshold individually	Need aggregate detection when per-IP is insufficient (challenge page trigger).
2	Memory per worker	1M entries × 20 bytes (IP + count + timestamp)	~20MB per worker	Fits in L2/L3 cache. Hot entries in L1. LRU eviction at boundary.
3	Total edge capacity	100 workers × 10M req/sec	1B req/sec before saturation	Edge fleet absorbs attack. Origin never sees volumetric flood.
4	False positive rate	Legitimate users rarely hit 1000 req/sec per IP	<0.1% during attack	Threshold is generous. Only corporate NATs risk hitting it (whitelist those).
5	Attack absorption	IPs exceeding threshold get dropped	>99% attack traffic filtered	Remaining 1% handled by challenge page for sophisticated botnets.

Interview tip: Emphasize the scale difference first — "at 10M req/sec, we need nanosecond decisions, not millisecond." Then show memory is bounded (20MB/worker) and false positives are negligible. The sophisticated botnet case (below threshold) is handled by challenge page, not per-IP counting.

APIs

Not user-facing · internal control plane for operator management of edge behavior

POST /edge/mode

Operator toggle: switch between fail-open (normal) and fail-closed (under attack) mode across all edge workers.

{ "mode": "fail-open" | "fail-closed" }

POST /edge/blocklist

Manual IP block with TTL. Broadcast to all edge workers via gossip or pub/sub.

{ "ips": ["203.0.113.0/24", "198.51.100.5"], "ttl": 3600 }

POST /edge/challenge

Enable JavaScript challenge for specific path prefixes. Humans pass, bots fail.

{ "path_prefix": "/api/*" }

GET /edge/stats

Returns current block rate, pass rate, and top offending IPs for operator visibility.

Data Model

In-memory per-worker counters · no persistent storage needed

In-Memory Per Worker · Fixed Window Counter

// Per-worker hash map (bounded at 1M entries with LRU eviction)
HashMap<IP/Subnet, { count: u32, window_ts: u32 }>

// Example entries:
203.0.113.5     → { count: 1842, window_ts: 1705312200 }  // OVER threshold
198.51.100.0/24 → { count: 95000, window_ts: 1705312200 } // subnet aggregate
10.0.0.1        → { count: 12, window_ts: 1705312200 }    // legitimate

// LRU eviction when entries > 1M
// Window resets: if current_ts != window_ts → reset count to 1

Shared Blocklist · Control Plane Broadcast

// Broadcast from control plane to all workers (gossip or pub/sub)
blocklist: Set<IP/Subnet> with TTL per entry
  Example: { "203.0.113.0/24": expires_at_1705315800 }

// Workers check blocklist BEFORE counter (O(1) set lookup)
// Blocklisted IPs get dropped immediately — no counter update needed

No Persistent Storage

// Ephemeral counters rebuilt each window (1-second buckets)
// Worker restart = counters reset = acceptable (brief false negatives)
// No database, no Redis, no disk I/O — purely in-memory
// This is by design: speed > durability for DDoS mitigation

Architecture · Fixed Window at the Edge (NGINX-style)

In-memory fixed-window counters per IP — fastest possible, O(1), no external dependencies

Decision	Choice	Why
Algorithm	Fixed window counter (1-second buckets)	Simplest. O(1) increment + check. No sorted sets, no weighted calculations. Speed is everything at the edge.
Storage	In-memory hash map per worker process	Zero network I/O. Counter access is nanoseconds. Each worker handles its own connections independently.
Key	Client IP (+ /24 subnet for IPv4, /48 for IPv6)	Coarse but effective for volumetric attacks. Subnet blocking catches rotating IPs within same block.
Threshold	1000 req/sec per IP (configurable per endpoint)	Generous enough for legitimate users (even aggressive scrapers). Catches bot floods instantly.
Action on exceed	Escalating: 429 → TCP RST → IP blocklist → null-route at L3	Progressive response. Start polite (429), escalate to connection-level drop for persistent attackers.
Fail mode	Fail-open in normal operation, switch to fail-closed under active attack (operator decision)	Normal: availability matters. Under DDoS: protection matters. Operator flips mode via control plane.

Why approximate is fine: At the edge, you're filtering volumetric attacks (10M+ req/sec). If your counter says an IP sent 1050 requests when the real number is 1000, that IP is still an attacker. ±5% accuracy doesn't matter when the threshold is 100x normal traffic. Speed > precision.

Fail-open vs fail-closed: Normal traffic: fail-open (if limiter crashes, allow traffic — don't self-DoS). Under active attack: operator activates fail-closed mode — unknown/unverified traffic is blocked by default. The switch is a manual/automated operator decision based on attack detection signals.

Why NOT sliding window at the edge: Sliding window log stores timestamps per request — memory explodes at 10M req/sec. Sliding window counter needs weighted calculation — unnecessary overhead for DDoS mitigation. Token bucket allows bursts — bad for DDoS (you want to catch any spike immediately). Fixed window is the right tool.

Real-world: NGINX · ngx_http_limit_req_module (leaky bucket variant). Cloudflare · per-IP fixed window at edge workers. AWS Shield · L3/L4 rate limiting before L7 processing. Akamai · edge rate controls with progressive blocking.

Resilience & Edge Cases

Failure	Impact	Recovery
Distributed attack (10K unique IPs)	No single IP exceeds per-IP limit	Aggregate rate limiting: total requests to origin > threshold = enable challenge page (CAPTCHA). Behavioral analysis: request patterns, header fingerprints.
Legitimate CDN/proxy IP (shared NAT)	Corporate office (1000 users, 1 IP) looks like attack	Whitelist known corporate IPs. Use X-Forwarded-For with trusted proxy chain. Rate limit by inner IP when possible.
Application-layer attack (slow POST)	Few requests but hold connections open	Connection timeout limits. Request body size limits. Concurrent connection limits per IP (separate from rate).
Flash crowd (legitimate viral event)	Legitimate traffic looks like DDoS	Challenge page (JavaScript challenge) to distinguish humans from bots. Passed challenges get allow-listed for session.
Memory exhaustion (too many unique IPs)	Counter hash map grows unbounded	LRU eviction of stale entries. Or probabilistic counting (Count-Min Sketch) for memory-bounded approximation.

Interview Cheat Sheet

Key points for edge/DDoS rate limiting

1. Fixed window, in-memory · fastest possible (O(1), nanoseconds), no external dependencies
2. Approximate is fine · at 10M req/sec, ±5% accuracy doesn't matter for attack mitigation
3. Per-IP + subnet · coarse keying catches volumetric floods, subnet for rotating IPs
4. Fail-open normally, fail-closed under attack · operator-controlled mode switch
5. Progressive escalation · 429 → connection drop → IP blocklist → L3 null-route
6. Challenge page for gray area · JavaScript challenge distinguishes humans from bots during flash crowds

System Design Case Study