System Design Concepts

No fluff — visual, concise, interview-ready

🔄 11 · DATA PIPELINES

Change Data Capture (CDC)

Stream DB changes from the transaction log to improve real-time updates (propagate changes), decoupling (no app code changes), and integration (sync downstream systems), with trade-offs in consistency (event lag), ordering (out-of-order events), and operational complexity (setup, monitoring).

CDC Pipeline — How It Works
Source DB WAL / Binlog App writes normally INSERT · UPDATE · DELETE all logged in txn log read log CDC Connector INSERT UPDATE DELETE Debezium / AWS DMS Kafka Connect plugin publish Kafka topic per table ordered per partition Kafka / Kinesis / Pulsar TARGET SYSTEMS Database (replica) Cache (Redis) Search (ES) Data Warehouse cross-region sync invalidate / update reindex documents analytics / reporting
Why CDC? The Dual-Write Problem

❌ Dual Write (Dangerous)

App 1. write DB 2. write cache DB ✓ Cache ✗ DB succeeds Cache fails → inconsistent! No atomicity across two systems

✓ CDC (Safe)

App write DB DB ✓ WAL CDC Kafka Cache Single write → WAL → CDC → all targets ✓ Eventually consistent, never lost
CDC Methods
MethodHowLatencyImpact on Source
Log-BasedRead WAL/binlog (Debezium)~msZero — reads log, not tables
Query-BasedPoll with WHERE updated_at > ?~sec-minLoad — queries hit DB
Trigger-BasedDB triggers write to shadow table~msHigh — triggers on every write
Guarantees: At-least-once delivery (consumers must be idempotent). Ordering per table. No dual-write problem — single source of truth is the DB, everything else derived from its log.
Real-world: LinkedIn — 100+ Kafka topics via CDC. Shopify — real-time inventory sync. Airbnb — Debezium for search index updates. Stripe — outbox + CDC for payment events.
CDC Tools: Debezium — open-source, log-based, Kafka Connect plugin (most popular). AWS DMS — managed, serverless, AWS-native migrations + replication. Fivetran — SaaS, 200+ connectors, zero-config ELT to warehouses. Airbyte — open-source Fivetran alternative, 300+ connectors, self-hosted or cloud. All log-based = zero impact on source DB.

ETL / ELT

ETL: Transform before loading (traditional). ELT: Load raw, transform in warehouse (modern).

Tools: Airflow (DAG orchestration). dbt (SQL transformations). Dagster/Prefect (modern alternatives). Lambda Architecture: Batch + Speed + Serving layers. Kappa: Everything streaming (simpler).

Stream Processing

Process data in real-time as it arrives — event by event, no waiting

Ride Events Click Stream IoT Sensors continuous evt evt evt evt Stream Processing process each event as it arrives ⚡ Surge pricing (Uber) 🚨 Fraud alert (Stripe) 📊 Live dashboard 🔔 Real-time alerts ms–sec latency
FrameworkModelReal-World Example
Apache FlinkTrue streaming, exactly-once, statefulUber — surge pricing: count ride requests per zone in 5-min sliding window → price multiplier in real-time
Kafka StreamsLibrary (no cluster), Kafka ecosystemLinkedIn — "Who viewed your profile" notifications: join view events with user profiles, emit within seconds
Spark StreamingMicro-batch (100ms–sec intervals)Netflix — real-time viewing metrics: aggregate play/pause/skip events per title for trending dashboard
Key Concepts: Windowing — tumbling (fixed, non-overlapping: "count per 5 min"), sliding (overlapping: "last 10 min, updated every 1 min"), session (gap-based: "user activity until 30 min idle"). Watermarks — handle late-arriving events ("allow 5 sec late data"). Exactly-once — checkpointing + idempotent sinks. State — RocksDB in Flink for keyed state (e.g., running count per user).

Batch Processing

Process large datasets in scheduled jobs — collect first, process later

App Logs Transactions User Activity store first Data Lake / DB scheduled Batch Engine Spark / MapReduce 📧 Daily email reports 🤖 ML model training 📊 Monthly analytics 🔄 Data warehouse ETL min–hrs latency

MapReduce → Spark

Map parallel transform Shuffle group by key Reduce aggregate Spark: in-memory DAG 100× faster

Real-World Batch Examples

Netflix — nightly Spark job processes PBs of viewing data → "Because you watched" recommendations
Spotify — daily batch: aggregate all listening history → generate "Discover Weekly" playlist
Uber — end-of-day batch: calculate driver earnings, trip summaries, tax reports
Banks — nightly batch: reconcile all transactions, generate statements, fraud reports
AspectStream ProcessingBatch Processing
WhenAs data arrives (continuous)Scheduled (hourly/daily/weekly)
Latencyms – secondsminutes – hours
DataUnbounded (infinite stream)Bounded (fixed dataset)
StateIn-memory (RocksDB, checkpoints)Disk (HDFS, S3)
Use CaseFraud detection, surge pricing, alertsReports, ML training, ETL, analytics
ToolsFlink, Kafka Streams, Spark StreamingSpark, MapReduce, Hive, Presto
Lambda Architecture: Run both — batch for accuracy (recompute everything), stream for speed (approximate, real-time). Merge results in serving layer. Kappa Architecture: Stream only — replay Kafka log for reprocessing instead of separate batch. Simpler but needs durable stream (Kafka retention).

Data Warehouse

OLAP — columnar storage, fast aggregations. BigQuery, Snowflake, ClickHouse

Guarantee: Columnar storage reads only needed columns — SUM(revenue) skips all other columns. 10:1 compression. OLTP — Runs the business (transactional). OLAP — Analyzes the business (analytical).

Data Lakes & Lakehouse

Raw files on S3/GCS. Lakehouse = lake (cheap) + warehouse (ACID). Delta Lake, Iceberg, Hudi add ACID + time travel.

Columnar Formats: Parquet (most popular), ORC. Column-oriented, compressed, splittable. 10x smaller than CSV.

Data Quality

Validate at ingestion — garbage in, garbage out

Source Quality gate Great Expectations / dbt tests Pass → land Fail → quarantine + alert
CheckExample
Schemacolumns + types match contract
Nullnessuser_id NOT NULL
Range / set0 ≤ score ≤ 100, currency ∈ USD…
Freshnesslast partition < 6 h old
Volumerow count within ±10% of 7-day median
Tools: Great Expectations, dbt tests, Soda, Monte Carlo (observability).

Schema Registry (Pipelines)

Single source of truth for event payloads

CompatibilityEffect
BackwardNew consumer reads old data — default for adding optional fields
ForwardOld consumer reads new data — safe to remove fields
FullBoth — strictest, safest in production
{
  "type": "record",
  "name": "Order",
  "fields": [
    {"name":"id",     "type":"string"},
    {"name":"amount", "type":"double"},
    {"name":"coupon", "type":["null","string"], "default":null}
  ]
}
Enforce in CI: reject schema PR if it breaks compatibility against the registry. Confluent / Apicurio / Glue.

Data Lineage

Where did this number come from?

orders.csv stage.orders dim_customer fact_orders Revenue dashboard
ToolNiche
OpenLineageOpen spec, plugs into Airflow / dbt / Spark
DataHubLinkedIn-born catalog + lineage
Apache AtlasHadoop ecosystem
MarquezReference impl of OpenLineage
Why it matters: GDPR "where is user X's data?", impact analysis before schema changes, root-cause for bad metrics.

Real-Time Analytics

Sub-second OLAP from streaming sources

App events clicks, txns Kafka Pinot / Druid columnar OLAP Dashboard
EngineSweet spot
Apache PinotUser-facing analytics (LinkedIn "who viewed")
Apache DruidTime-series OLAP, slicing & dicing
ClickHouseLightning-fast columnar SQL (Cloudflare)
Materialize / RisingWaveStreaming SQL with incremental views
Architecture: Kafka → stream engine (Flink/Spark) → OLAP store → BI / app dashboards. Sub-second freshness, query in ms.