System Design Fundamentals

System design interviews are structured conversations, not trivia. You have roughly 45 minutes to show you can turn a vague product idea into a production architecture. The framework below is the same skeleton used at Google, Meta, Amazon, and in courses like Hello Interview — but here you will learn what to say at each step with concrete numbers and diagrams.

Why it matters

Without a framework, candidates jump straight to "we need Kafka and Redis" and lose 15 minutes on the wrong problem. Interviewers grade process: clarifying ambiguity, estimating scale, drawing a sane high-level design, then diving deep on two components with trade-offs. A repeatable framework prevents panic and signals seniority — you drive the room instead of waiting for prompts.

Staff-level hires consistently: (1) state assumptions aloud, (2) time-box each phase, (3) check in — "Should I go deeper on the write path or caching?"

Where to use

Use this skeleton for every open-ended design question: URL shortener, Twitter feed, payment system, chat, rate limiter, job scheduler, video streaming. Even when the interviewer says "design X," they are testing whether you can decompose X using the same steps.

Real use cases

• Design a news feed (Meta-style): Clarify read vs write ratio (often 100:1), estimate 500M DAU → ~6K average QPS, peak 60K QPS → CDN + cache for reads, fan-out on write vs pull on read deep dive.
• Design payments: Clarify strong consistency, idempotency, PCI scope → skip CDN on money path → deep dive on DB transactions and saga/outbox.
• Design a file upload service: Clarify max file size (100 MB), 10K uploads/min → estimate bandwidth 17 GB/min → deep dive S3 multipart + metadata DB.

When NOT to use

Do not rigidly recite slides when the interviewer is clearly steering you ("let's skip estimates, go straight to sharding"). Flex the framework — but never skip requirements. Also avoid spending 20 minutes on requirements for a simple LRU cache question; compress clarify + estimate to 5–7 minutes total for smaller scopes.

Alternatives

Execution / data flow

Walk through the interview clock — say these time boxes out loud:

1. Clarify (5 min): functional requirements, non-functional (latency p99, availability %, consistency), explicit out-of-scope
2. Estimate (5 min): DAU, QPS (avg + 10× peak), storage, bandwidth — round numbers
3. High-level design (10 min): client, CDN, LB, services, cache, DB, async queues
4. Deep dive (15 min): pick 2 components — data model, hot path, sharding, failure handling
5. Trade-offs (5 min): SQL vs NoSQL, sync vs async, strong vs eventual consistency
6. Wrap-up (5 min): failure modes, monitoring, future 10× scale

Vague prompt → Clarify scope → Numbers on paper → Boxes & arrows → Zoom into 2 boxes → Compare options → Failures & metrics

Scalability means your system can handle more load (users, QPS, data) by adding resources — without rewriting the product. This section covers how you scale machines (vertical vs horizontal). NFR scalability targets (10× growth, elasticity) are covered in the NFR lesson.

Why it matters

Every successful product outgrows a single server. Instagram started on one Postgres; within years they needed sharded storage and thousands of app servers. Interviewers want to hear: stateless app tier scales horizontally; databases scale via replicas, partitioning, or different stores. Wrong scaling order (shard before cache) wastes months of engineering.

Concrete trigger numbers: single Postgres often tops out around 5K–10K write QPS and 50K–100K read QPS with replicas — beyond that you need partitioning or a write-optimized store.

Where to use

• App servers: horizontal scale behind a load balancer (stateless REST APIs).
• Read-heavy DB: read replicas + connection pooling before sharding.
• Write-heavy DB: partitioning/sharding, queue-based write absorption, or Cassandra/Dynamo-style scale-out.
• Early startup: vertical scale (bigger instance) is fine until ~$500–2K/month DB tier becomes painful.

Real use cases

• Netflix API tier: thousands of stateless instances; session and preferences in external stores — classic horizontal app scale.
• Shopify peak (Black Friday): auto-scale app + read replicas; cart writes still hit primary — vertical + horizontal mix.
• Discord voice: stateful (UDP rooms) — horizontal scale with consistent hashing so users land on the same gateway; not simple round-robin.
• Internal admin tool (50 users): one t3.medium — vertical scale is enough; horizontal scale adds ops cost with no benefit.

When NOT to use

• Vertical scale only when you need 99.99% availability — one big machine is a single point of failure.
• Horizontal app scale when the app keeps sessions in local memory — sticky sessions are a crutch; fix statelessness first.
• Sharding early at 200 QPS because "we might scale" — operational nightmare; cache and optimize first.

Alternatives

Execution / data flow

Standard scale path when traffic grows 10×:

Traffic spike → Profile & optimize (indexes, N+1) → Cache + CDN → Scale reads (replicas) → Scale writes (queue, shard) → Auto-scale app tier

Numbers example: API at 8K QPS, 50 ms p99 per instance handles ~200 QPS → need ~40 instances at peak (with 2× headroom → 80 instances). DB at 8K reads/s → 2 read replicas (4K each) + 32 GB cache for hot 20% of keys.

Back-of-envelope math turns "millions of users" into QPS, storage, and bandwidth so your architecture choices are justified. You are allowed to round aggressively — interviewers care about orders of magnitude, not exactness.

Why it matters

Without estimates you might propose a single MySQL for YouTube-scale video or 50 Kafka clusters for a todo app. Estimates drive: how many app servers, whether you need a CDN, cache size, and sharding strategy. A 2-minute calculation often eliminates half the wrong designs.

Where to use

Every system design interview after clarifying requirements — typically 5 minutes with pen and paper. Also used in RFCs, capacity planning, and cost forecasts ($/month for S3, RDS, egress).

Real use cases

• Twitter-like read-heavy: 300M DAU × 50 timeline reads/day = 15B reads/day ≈ 175K QPS average; 10× peak ≈ 1.7M QPS → CDN + aggressive cache mandatory.
• Photo storage: 500M users × 200 photos × 2 MB = 200 PB raw → S3 + lifecycle to Glacier; metadata in DB ~100 GB.
• Chat messages: 10M users × 100 messages/day × 500 B = 500 GB/day ≈ 6 MB/s write bandwidth → Kafka partition count ~50–100.

When NOT to use

Skip deep estimation when the interviewer says "assume infinite scale" or for tiny scoped problems (design a mutex API). Do not spend 15 minutes on bandwidth if functional design is still unclear.

Alternatives

Execution / data flow

Worked example — URL shortener:

• 100M DAU, each creates 1 short URL/day + 10 redirects/day
• Writes: 100M/day ≈ 1,200 write/s avg; peak 10× ≈ 12K write/s
• Reads: 1B redirects/day ≈ 12K read/s avg; peak ≈ 120K read/s
• Storage: 100M new URLs/day × 500 B × 365 days × 5 years ≈ 90 TB (+ indexes ~2×)
• Bandwidth: 120K RPS × 1 KB response ≈ 120 MB/s egress at peak

DAU → actions/day → divide by 86,400 → QPS avg → multiply 10× peak → storage = records × size × retention → bandwidth = QPS × payload

Reference numbers to memorize: 1 day = 86,400 s ≈ 100K for mental math; 1 MB = 1,000 KB; SSD ~1 GB/s; network cross-region ~50–100 ms; Redis GET ~1 ms; Postgres indexed PK ~1–5 ms; cross-service hop ~5–20 ms.

A load balancer (LB) distributes incoming traffic across multiple backend servers so no single machine melts and failed nodes are removed automatically. It is the front door of almost every horizontally scaled system.

Why it matters

Without an LB, one app server gets overloaded, others sit idle, and a crashed server still receives traffic. LBs provide high availability (health checks), SSL termination (offload TLS from apps), and routing (canary 5% to new version). At 50K QPS, a single nginx or AWS ALB can often handle the fan-out — apps scale behind it.

Where to use

• Between clients and stateless API servers (always).
• Between API tier and internal microservices (service mesh / sidecar LB).
• Database read replica routing (PgBouncer, ProxySQL, RDS reader endpoint).
• Global traffic: DNS + GeoDNS or anycast to regional LBs.

Real use cases

• AWS ALB (L7): Route /api/* to ECS, /static/* to S3; sticky sessions for legacy; weighted target groups for canary.
• NGINX (L4/L7): 100K+ concurrent WebSocket connections with least_conn and upstream keepalive.
• Consistent hashing (Memcached): key → server mapping minimizes reshuffle when one node added (still used in caches, not primary LB for HTTP).
• Global LB (Cloudflare / AWS Global Accelerator): user in Tokyo → Tokyo region LB → reduces RTT from 200 ms to 20 ms.

When NOT to use

• Single-server deployments (<500 QPS internal tool) — LB adds cost and complexity.
• Stateful protocols without sticky sessions or shared state — random routing breaks in-memory rooms.
• Using LB as database sharding router without understanding hot shards — use application-level sharding keys instead.

Alternatives

Execution / data flow

Client TLS → LB terminates SSL → health check OK backends only → pick algorithm → forward to app → app responds → LB returns to client

Health checks: HTTP GET /health every 10s; 2 failures → remove from pool; 30s to drain long requests. At 10K QPS across 20 servers, each instance sees ~500 QPS — if one fails, LB redistributes in <30s.

Active-active vs active-passive: Stateless APIs = active-active (all nodes serve traffic). Stateful leader (single writer DB) = active-passive failover with VIP or DNS flip — RTO 30s–5 min typical.

Caching stores frequently accessed data in fast memory (Redis, in-process) so you avoid slow disk/network on every request. A good cache can turn 50 ms DB reads into 1 ms and cut DB QPS by 80–95% for read-heavy workloads.

Why it matters

Database is the bottleneck in most systems. At 20K read QPS, Postgres without cache dies; with 90% hit rate, DB sees only 2K QPS. Caching is often the first scale lever after basic indexing — cheaper than sharding.

Numbers: Redis single node ~100K–200K ops/s; hit ratio 80–95% typical for hot keys; TTL 60s–24h depending on freshness needs.

Where to use

• Read-heavy APIs: user profiles, product catalog, feed pages.
• Session storage: JWT validation + server-side session in Redis.
• Rate limiting counters, leaderboard scores, feature flags.
• CDN edge cache for static assets and cacheable GET JSON.

Real use cases

• Facebook memcached layer: billions of keys; cache-aside; invalidation on profile update.
• Amazon product page: CDN + Redis for product metadata; inventory may skip cache or use short TTL (5s) for accuracy.
• Stack Overflow: heavy in-process + Redis; question pages cached; writes invalidate tags.
• Session store: 50M sessions × 2 KB = 100 GB Redis cluster with replication.

When NOT to use

• Strong consistency required on every read (bank balance) — cache adds staleness unless careful read-your-writes routing.
• Data accessed once ever (cold archival) — cache waste.
• Dataset entirely unique per request (random UUID lookups with no repeat) — 0% hit rate.
• When invalidation is harder than the problem (high churn inventory) — consider shorter TTL or no cache on that path.

Alternatives

Execution / data flow

Read → Check Redis → Hit: return (1 ms) | Miss → Query DB (10–50 ms) → Set cache with TTL → Return

Hot key problem: 1M QPS on celebrity tweet key — single Redis shard melts. Fixes: local in-process cache (1s TTL), replicate key across shards, pre-warm before event.

Cache stampede: TTL expires, 10K threads hit DB simultaneously. Fixes: singleflight (one thread repopulates), probabilistic early expiration, mutex per key.

The CAP theorem states that in a distributed system during a network partition, you must choose between Consistency (every read sees latest write) and Availability (every request gets a response). Partition tolerance is not optional in real networks — cables fail, regions go dark.

Why it matters

Wrong consistency choice causes lost payments, double bookings, or angry users seeing stale feeds. Interviewers use CAP to test whether you match storage and replication to business rules: money = CP; social likes = AP.

PACELC extension: Even without partition, you trade Latency vs Consistency — sync cross-region replication adds 50–150 ms per write.

Where to use

• Choosing database: Postgres (CP-ish single primary) vs Cassandra (AP tunable).
• Multi-region design: sync replication (CP, higher latency) vs async (AP, eventual).
• Leader election (ZooKeeper/etcd): CP for coordination.
• Shopping cart during partition: often AP — better available cart than error page.

Real use cases

• Bank transfer: CP — single leader, sync replication, reject write if quorum lost; p99 write latency 100–300 ms acceptable.
• Instagram like count: AP/eventual — count may lag 1–5s; available during partial outage.
• DynamoDB / Cassandra: AP with tunable consistency (QUORUM vs ONE).
• Google Spanner: CP with TrueTime — strong global consistency at higher cost/latency.

When NOT to use

• Single-node Postgres on one machine — CAP is about distributed trade-offs; don't over-quote CAP for monoliths.
• Claiming "we chose CAP" without naming concrete failure scenario — always tie to partition or replication lag.
• Strong consistency everywhere "because it's safer" — kills availability and latency at global scale.

Alternatives

Execution / data flow

Partition scenario: US-East and US-West lose network link. CP system: one side stops accepting writes (or entire cluster read-only) to prevent split-brain. AP system: both sides accept writes; merge conflicts later (vector clocks, last-write-wins — risky for money).

Write → Leader primary → Sync replicas (CP wait) OR Async replicate (AP fast) → Read from primary (strong) OR replica (eventual)

Numbers: Async replica lag under load: 100 ms–2 s. Sync quorum (3 nodes, write 2): add ~10–30 ms LAN, ~100 ms cross-region. Availability target 99.9% = 8.7 hr downtime/year — CP failover may violate unless multi-AZ.

No single database wins every workload. Polyglot persistence means using Postgres for transactions, Redis for cache, Elasticsearch for search, S3 for blobs — each optimized for its access pattern.

Why it matters

Picking MongoDB because "it scales" for heavy relational reporting fails. Picking Postgres for 500K write/s time-series fails. Interviewers want access-pattern reasoning: read/write ratio, join needs, consistency, query shape, retention.

Where to use

• OLTP transactions: SQL (Postgres, MySQL).
• High write throughput, wide rows: Cassandra, HBase.
• Flexible document model: MongoDB, DynamoDB.
• Sub-ms session/cache: Redis, Memcached.
• Full-text search / logs: Elasticsearch, OpenSearch.
• Graph traversals (friends-of-friends): Neo4j (or SQL with limits).
• Vector similarity (RAG): Pinecone, pgvector, Weaviate.

Real use cases

• Stripe: Postgres for money (ACID); Redis for idempotency keys; Kafka for events.
• Netflix: Cassandra for viewing history (high write); S3 for video; EV cache.
• Airbnb search: Elasticsearch for listings; MySQL for bookings source of truth.
• OpenAI RAG app: pgvector or Pinecone for embeddings; Postgres for user metadata.

When NOT to use

• Redis as sole source of truth for financial records — persistence optional, not ACID bank.
• Elasticsearch as primary transactional store — not designed for atomic multi-doc updates.
• Graph DB for simple user CRUD at 100M users — ops complexity without graph queries.
• Sharding Postgres before exhausting cache + read replicas — premature sharding pain.

Alternatives

Execution / data flow

API write → Validate → Postgres transaction → Publish event to Kafka → Worker indexes Elasticsearch → Cache invalidation in Redis

Sharding when: single table > 500 GB–1 TB, write QPS > 5K–10K on primary, or index maintenance blocks writes. Shard key: user_id (even spread) not country (hot shard).

A Content Delivery Network (CDN) caches static and cacheable content at edge servers (PoPs) worldwide so users download from a nearby city instead of your origin in Virginia. Latency drops from 200 ms to 20–50 ms; origin bandwidth can drop 80–95% for static assets.

Why it matters

At 1M users streaming a 2 MB JS bundle, origin serves 2 TB per full cache miss wave. CDN absorbs repeat traffic. For video (Netflix, YouTube), CDN/edge is non-negotiable — origin cannot serve 100 Gbps peaks.

Cost: egress from cloud origin ~$0.08–0.12/GB vs CDN ~$0.02–0.05/GB at scale.

Where to use

• Static assets: JS, CSS, fonts, images, WASM.
• Cacheable GET APIs: public product catalog, blog posts (with Cache-Control).
• Video/audio segments (HLS/DASH chunks).
• Software downloads, game patches, firmware.

Real use cases

• Cloudflare / Akamai in front of SaaS: global Anycast DNS → nearest PoP; DDoS scrubbing included.
• Spotify album art: S3 origin + CDN; immutable URLs with hash in path.
• API caching: GET /v1/products with Cache-Control: public, max-age=300 — 5 min stale OK.
• Live sports: CDN delivers 10 Tbps aggregate during events; origin only encodes streams.

When NOT to use

• Personalized responses (user-specific JSON with auth) — cache miss every time unless edge includes user segment key.
• POST/PUT/DELETE — not cacheable by standard CDN rules.
• Highly dynamic data (<1s freshness): stock ticker — CDN TTL too coarse; use dedicated streaming.
• Internal microservices east-west traffic inside VPC — CDN wrong tool.

Alternatives

Execution / data flow

Client DNS → CDN edge (hit: return 20 ms) | miss → origin fetch → cache at edge → return to client

1. Origin — S3 or app server (source of truth for content)
2. CDN edge PoPs — 200+ cities; cache hot objects
3. DNS / anycast — route client to nearest healthy PoP
4. Client — receives cached bytes; origin load drops 90%+ for static

Tracing one HTTP request from finger tap to database row shows how every fundamental piece connects — DNS, TLS, CDN, LB, app, cache, DB, async workers. Senior candidates narrate this path with a latency budget.

Why it matters

Optimizing the wrong layer wastes effort. If p99 is 800 ms because DB missing index, CDN won't help. Latency budget forces accountability: each hop gets a millisecond cap; sum must meet SLA (often p99 < 200–300 ms for APIs).

Where to use

Explain hot path in every interview after high-level diagram. Production debugging: distributed tracing (Jaeger) shows which span blew the budget.

Real use cases

• Read product page: CDN hit on image (15 ms) + API miss cache → Postgres (40 ms) + ES search sidebar (30 ms) — parallelize ES.
• Post tweet: Write primary (20 ms) + fan-out async to 10M followers impossible sync — queue + batch.
• Login: LB → API → Redis session check (2 ms) → optional Postgres user lookup — keep auth off critical CDN path.

When NOT to use

Don't enumerate every hop for batch jobs or offline ETL — different path (S3 → Spark → warehouse). Don't ignore async: user-facing path should not wait for email/analytics indexing.

Alternatives

Execution / data flow

Client → DNS (20 ms) → TLS CDN/LB (30 ms) → App (30 ms) → Redis (1 ms) → DB (10 ms) → JSON serialize (10 ms) → Response

Use this section the night before interviews and as a mental close in the last 5 minutes of every mock. Missing items correlate with "lean no" feedback: no estimates, no trade-offs, no failure modes.

Why it matters

Checklists convert broad topics into verifiable signals. Interviewers often score on a rubric — requirements, scale, diagram, depth, trade-offs, ops. Tick each box aloud in practice until automatic.

Where to use

• Final 5 minutes of live interview wrap-up.
• Mock interview self-review.
• RFC peer review ("did we estimate QPS?").

Real use cases

Candidate designs notification system, forgets idempotency — checklist catches "failure modes + retries." Another designs cache without TTL — checklist item "invalidation + stampede" triggers fix before interviewer asks.

When NOT to use

Don't read checklist robotically for 3 minutes while interviewer wants deep dive on sharding. Use as internal mental model; verbalize only gaps relevant to the problem.

Alternatives

Rubric apps (Hello Interview grader), peer mock sheets, or company-specific ladders (Amazon L6 bar). This checklist maps to universal FAANG-style loops.

Execution / data flow

Finish deep dive → Scan checklist mentally → Verbalize 2 gaps you didn't cover → Propose monitoring + scale plan → Ask interviewer for questions

• Opened with functional + non-functional requirements
• Calculated QPS (avg + peak), storage, bandwidth with stated assumptions
• Drew CDN → LB → App → Cache → DB
• Explained vertical vs horizontal scaling with thresholds
• Named caching strategy + TTL + invalidation
• Applied CAP / consistency choice per data type (money vs likes)
• Picked SQL vs NoSQL with access-pattern justification
• Mentioned CDN for static and cacheable GET
• Async queue for non-critical work off critical path
• Failure modes: DB down, cache stampede, hot keys, partition
• Monitoring: metrics, logs, traces, SLOs, alerting
• Future scale: 10× users — what breaks first?