System design roadmap
Every stage in order, from the terms used to discuss trade-offs, through the edge, load balancing, databases, caching, queues and failure handling, to worked interview problems.
Core and recommended topics. Optional ones stay hidden until you have some real work behind you.
- Core: needed for most jobs of this kind
- Recommended: worth learning early
- Optional: learn it when your work needs it
Stage 1
Trade-offs and terms
System design discussions are about trade-offs, and they rely on six terms: performance, scalability, latency, throughput, availability and consistency. Many candidates use them loosely. Learn to use each one precisely before drawing any architecture.
Core: Performance and scalability
Performance is how quickly one request is handled. Scalability is how well the system copes as load grows. They look alike at low load and separate once a bottleneck appears.
CodexPerformance and scalability Outside linkLethain: architecting systems for scale
Core: Latency and throughput
Latency is how long one operation takes; throughput is how many complete per second. Little's Law links them: concurrency equals throughput times latency. Improving one often costs some of the other.
CodexLatency and throughput SimulatorBig-O race Outside linkLittle's Law
Core: Availability and consistency
During a network partition a system must choose between staying available and staying consistent (CAP). PACELC adds that without a partition the choice is between latency and consistency. Many systems accept slightly stale reads with a limit on how stale.
CodexAvailability and consistency CodexCAP and PACELC SimulatorCAP theorem
Recommended: Consistency models
Weak, eventual, session, strong and linearisable consistency. PostgreSQL, DynamoDB, Spanner and Cassandra each offer different points on this range.
Recommended: Availability patterns
Cold, warm and hot failover, replication layouts, and how availability percentages such as "five nines" are calculated. More redundancy also means more parts that can fail, so it helps only up to a point.
CodexAvailability patterns CodexReplication CodexFailure detectors
Stage 2
The edge: DNS, CDNs and anycast
Most fast websites are not fast because of their origin servers. They are fast because much of each page is served from a cache a few milliseconds from the user. Understand what happens at the edge before discussing the application servers.
Core: DNS resolution
Names are turned into addresses through a chain of cached lookups before a connection is made. The TTL trades how fresh an answer is against how many queries are made.
GuideHow DNS works SimulatorDNS resolution CodexDNS Outside linkJulia Evans: implement DNS in a weekend
Core: Push and pull CDNs
A CDN is a network of caches near users. A pull CDN fetches from the origin on the first miss; a push CDN is filled by the origin in advance. Most sites use pull because it needs no extra setup.
GuideHow CDNs work SimulatorCDN cache CodexAnycast routing Outside linkCloudflare: what is a CDN
Recommended: Anycast and BGP
One IP address announced from many locations, with BGP sending each user to a nearby one. This is how CDNs and DNS providers serve users from the closest site.
GuideBGP & internet routing SimulatorAnycast routing CodexRouting protocols Outside linkCloudflare anycast explained
Stage 3
Load balancing and reverse proxies
After a request passes the edge, a load balancer chooses which application instance handles it. Layer 4 and layer 7 balancing, active-active and active-passive setups, and the algorithms (round robin, least connections, consistent hashing) come up in nearly every design discussion.
Core: Layer 4 and layer 7 load balancing
A layer 4 balancer picks a backend from TCP or UDP information and never looks at the request, which is fast. A layer 7 balancer reads the HTTP path and headers and can route on them. Many systems use both.
GuideLoad balancing CodexLoad balancing SimulatorLoad-balancer
Recommended: Active-active and active-passive
Active-active runs several load balancers that all take traffic. Active-passive keeps a standby that takes over when the primary fails. The trade is cost against how fast failover happens.
Core: Reverse proxy
Nginx, HAProxy and Envoy sit in front of application servers and handle TLS, compression, caching, rate limiting and routing. Often deployed with the load balancer, though they are a separate layer.
Recommended: Consistent hashing for sticky routing
When requests for the same key must reach the same backend (sessions, sharded caches, WebSocket connections), consistent hashing means adding or removing a node moves only about K/N of the keys.
Stage 4
The application tier
Most systems start as a monolith and split into services when one shared deploy starts to slow teams down. The useful questions are how services find each other, how they share schemas and how they handle partial failure, more than whether to use microservices at all.
Core: Monolith or microservices
A monolith deploys as one unit; microservices deploy separately. Microservices add network calls and operational work in exchange for independent deploys and clearer ownership. Team size usually decides it more than system size.
Core: Service discovery
How service A finds a healthy instance of service B. In client-side discovery (Consul, Eureka) the caller looks it up; in server-side discovery a load balancer does. Kubernetes supports both.
GuideService discovery CodexKubernetes networking Outside linkConsul docs
Recommended: API gateway
A single entry point that handles routing, authentication, rate limiting, request transformation and protocol translation between clients and internal services.
Recommended: Stateless services
If a service keeps no per-user state, any instance can handle any request and scaling out is easy. The state still has to go somewhere, usually the database or a separate session store.
Stage 5
Relational databases
A single relational database can support a system for a long time, and senior engineers know roughly how long before sharding or NoSQL is needed. Discussions of the data layer use ACID, replication, MVCC, isolation levels and the page cache.
Core: ACID and isolation levels
Read uncommitted, read committed, repeatable read, snapshot and serializable. Each level prevents more kinds of anomaly and costs more.
Core: MVCC
Multi-version concurrency control: readers see a consistent snapshot while writers create new versions of rows. It lets reads proceed without blocking in PostgreSQL, Oracle and many other databases.
Core: Write-ahead logging
Changes are appended to a sequential log before the data pages are modified. This is how a database recovers from a power cut without corruption.
Recommended: B-trees and LSM trees
The two main storage structures. B-trees suit read-heavy workloads with updates (PostgreSQL, MySQL). LSM trees suit write-heavy workloads and merge their files in the background (Cassandra, RocksDB).
Core: Primary-replica replication
One node accepts writes and replicas serve reads. Asynchronous replication is fast but replicas can return stale data; synchronous replication is consistent but waits on the slowest replica. Many systems replicate asynchronously and send a user's reads to the primary right after they write.
Recommended: Federation
Splitting one database into several by function, such as users in one and products in another. Often done before sharding, because each database stays simple.
Stage 6
NoSQL databases
NoSQL covers four main data models, each suited to different access patterns. To choose, write down the queries you need to serve, then pick the model that answers each in one round trip.
Core: Key-value stores
Redis, Memcached and DynamoDB look up values by exact key. The simplest model, with the highest throughput per node. Used for sessions, leaderboards and rate-limit counters.
GuideHow Redis works SimulatorRedis operations PlaybookKey-value store
Recommended: Document stores
MongoDB, Couchbase, and DynamoDB with secondary indexes store JSON documents by ID, with optional indexes on fields inside them. They suit catalogues, user profiles and content.
Recommended: Wide-column stores
Cassandra, ScyllaDB and Bigtable store rows under a partition key and a sort key, with columns that can vary per row. Built for very high write rates and time-series data.
Core: SQL or NoSQL?
Start with SQL. Move a workload to NoSQL when one access pattern needs throughput or scale SQL cannot provide and you can give up joins or strong consistency for it. Many systems use both.
Stage 7
Sharding and denormalisation
Most senior design interviews reach sharding at some point. The hard part is choosing a shard key that handles growth, avoids hot spots and allows re-sharding later. Denormalisation is a smaller step: copy data so that reads are fast.
Core: Sharding strategies
Hash partitioning spreads load evenly but makes range scans expensive. Range partitioning supports scans but can create hot shards. Partitioning by region or tenant works when access follows those lines.
Core: Consistent hashing
Keys and nodes are placed on a hash ring, so adding or removing a node moves about K/N keys instead of nearly all of them. Dynamo, Cassandra and many key-value stores use it.
Recommended: Hot keys and celebrity accounts
When one key gets a hundred times the average traffic, the shard holding it becomes the bottleneck. Options include a cache in front, extra replicas for that key, or fanning out writes in advance.
Recommended: Denormalisation
Copy fields into other tables so reads do not need a join. Writes do more work and the copies must be kept in sync, in exchange for faster reads.
Stage 8
Caching
Fast systems cache at four or five layers. The questions are where to cache, how to invalidate, and what happens when a cache is empty and a burst of requests reaches the database at once.
Core: Client-side cache
HTTP cache headers, service workers and the browser's memory and disk caches. A response served from the browser cache needs no request at all.
Core: CDN cache
A pull CDN caches responses by URL and the headers listed in
Vary.Cache-Control: max-age,stale-while-revalidateand the CDN's own TTL settings control it.Core: Application cache
Redis or Memcached between the service and the database. The three common patterns are cache-aside (the application fills the cache on a miss), write-through (writes go to both) and write-behind (the cache takes writes and passes them to the database in batches).
GuideCaching SimulatorDistributed cache SimulatorCache eviction SimulatorLRU cache
Recommended: Database caches
Materialised views, the buffer pool and the operating system page cache. Easy to forget because the database manages them for you.
Recommended: Eviction policies
Which entry to remove when the cache is full: LRU, LFU, FIFO or TinyLFU. LRU is the usual default and LFU suits a stable set of popular keys. Caffeine uses W-TinyLFU, and Redis 4 added an approximate LFU policy.
Recommended: Cache invalidation
Expiry by TTL is simple. Invalidating on each write through pub/sub is fresher but couples services together. Write-through avoids stale reads; cache-aside accepts brief staleness for simplicity.
Stage 9
Queues, workers and back-pressure
Work that does not need to finish before the response should go through a queue: email, image processing, indexing and analytics, for example. The interesting parts are delivery guarantees, ordering, and keeping the queue itself from collapsing under load.
Core: Message queues and task queues
A message queue such as RabbitMQ or SQS only moves messages; a task queue such as Celery or Sidekiq adds job semantics on top. Use the simpler one that meets your needs.
Core: Kafka
A partitioned, append-only log where each consumer tracks its own position. Unlike a queue, messages can be replayed, and each partition keeps messages with the same key in order. Many event-driven systems are built on it.
GuideHow Kafka works PlaybookEvent ingestion playbook Outside linkKafka docs
Core: Delivery semantics
At most once (messages may be lost), at least once (retried until acknowledged, so duplicates are possible) and exactly once (hard and expensive). In practice exactly once usually means at least once with idempotent consumers.
Recommended: Back-pressure
What to do when consumers cannot keep up. An ever-growing buffer only delays the failure. Slow the producers, drop work deliberately or shed load.
Recommended: Dead letter queues
After a message has failed a set number of times, move it to a separate queue for someone to inspect. Without one, a single bad message can be retried forever and block or crash consumers.
Stage 10
Communication protocols
Design discussions eventually ask which protocol services use to talk to each other. The answer depends on the shape of the call (single request, streaming or two-way), the latency budget, the size of the payload and the tooling available.
Core: TCP
Ordered, reliable bytes with congestion control, and the default for most traffic. The costs are the handshake before any data is sent and head-of-line blocking when a packet is lost.
CodexTCP SimulatorTCP handshake SimulatorTCP congestion GuideTCP guide
Recommended: UDP
Datagrams with no ordering, retransmission or congestion control. Used for DNS, games, live video and other cases where late data is worse than missing data.
Core: HTTP/1.1, HTTP/2 and HTTP/3
HTTP/1.1 handles one request at a time per connection. HTTP/2 multiplexes many streams over one TCP connection but still suffers TCP head-of-line blocking. HTTP/3 runs over QUIC on UDP, combines the transport and TLS handshakes, and avoids that blocking.
GuideHTTP SimulatorHTTP/2 streams SimulatorHTTP/3 + QUIC CodexQUIC
Core: REST and gRPC
REST over HTTP with JSON is readable and easy to debug. gRPC uses HTTP/2 and Protocol Buffers, defines the schema first and is more efficient, but harder to inspect on the wire. Many companies use REST for public APIs and gRPC between internal services.
Recommended: WebSockets and server-sent events
A WebSocket is a persistent two-way connection; server-sent events push data from server to client only. Used for chat, presence, live dashboards and anything the server needs to push.
GuideWebSockets CodexWebSockets & SSE GuideRealtime communication
Stage 11
Capacity estimates
Most design interviews ask for estimates: daily users to requests per second, requests to storage, storage to shards, shards to machines. The arithmetic is simple, and practice makes it quick.
Core: Powers of two
210 is about a thousand, 220 a million, 230 a billion and 240 a trillion. These make storage and memory estimates quick.
Core: Latency numbers
Roughly: an L1 cache hit 0.5 ns, main memory 100 ns, an SSD read 100 µs, a round trip within a data centre 500 µs, and across a continent around 100 ms. The large gaps between these explain where time goes in a request.
CodexPerformance methods Outside linkJeff Dean: numbers every programmer should know
Core: From daily users to requests per second
Daily active users times actions per user, divided by 86,400 seconds, gives average requests per second. Multiply by three to five for the peak. Ten million users doing five actions a day is about 580 requests per second on average.
Recommended: Estimating storage
Average record size times records per day times days kept, times the replication factor and the index overhead. Candidates often forget the last two.
Stage 12
Failure modes
Here the focus moves from being correct to staying correct when things fail. Networks drop packets, latency is never zero and machines die. Senior interviews almost always cover this stage.
Core: Idempotence
An operation is idempotent if doing it twice has the same effect as doing it once. Retries are only safe for idempotent operations.
Core: Retries with backoff and jitter
Immediate retries make an overloaded dependency worse. Exponential backoff with random jitter spreads retries out so the dependency can recover.
SimulatorRetry strategy CodexBack-pressure & retries Outside linkAWS: exponential backoff + jitter
Core: Circuit breakers
Three states: closed (calls pass through), open (calls fail immediately) and half-open (a few calls test whether the dependency has recovered). They stop a failing dependency from taking down its callers.
Recommended: Timeouts and bulkheads
Give every network call a timeout. Bulkheads give each dependency its own pool of threads or connections, so one slow dependency cannot use them all.
Recommended: Failure detectors and leases
How a cluster decides a node is down. Simple timeouts can lead to split brain; production systems use gossip with suspicion levels (SWIM, phi accrual) together with leases.
CodexFailure detectors SimulatorSplit-brain CodexLeases CodexGossip
Stage 13
Security in system design
Security questions in design interviews rarely go deep into cryptography. They ask how one service authenticates to another, where secrets are kept and what happens if a token leaks.
Core: What TLS does
TLS encrypts data in transit and verifies the server's identity. It does not encrypt data at rest. Mutual TLS also verifies the client and is common between internal services.
Core: OAuth 2.0 and OpenID Connect
OAuth handles authorization (what a client may do); OpenID Connect, built on top, handles authentication (who the user is). Browser apps use the authorization code flow with PKCE, and services use the client credentials flow.
Recommended: JWT pitfalls
A JWT is a signed token carrying claims. It cannot be revoked early without extra work, such as a deny list, and is easy to misconfigure (for example by accepting
alg: none). The usual pattern is short-lived access tokens with longer-lived refresh tokens.Recommended: Rate limiting
Token bucket, leaky bucket, fixed window and sliding window algorithms behave differently under the same load. Apply limits at the edge, at the gateway and in each service.
Stage 14
Observability
Logs, metrics and traces, plus two methods: USE for resources and RED for requests. Together they cover most of what an on-call engineer needs. The harder part is choosing what goes on a dashboard so the problem is obvious during an incident.
Core: Logs, metrics and traces
Logs record events, metrics are aggregated numbers, and traces connect one request to every call it made.
Core: RED method
Rate, errors and duration: the three numbers to track for every service that handles requests, and the minimum dashboard.
Recommended: USE method
Utilisation, saturation and errors, checked for every resource: machines, queues and connection pools. The counterpart to RED.
Recommended: SLOs, SLAs and error budgets
Set latency and availability targets, measure them, and let the remaining error budget decide how fast to release.
Stage 15
Worked problems
Knowing the components is different from designing with them. You improve by working through common problems out loud and defending each choice. Designing Data-Intensive Applications by Martin Kleppmann is the book to read first.
Core: A six-step approach
Scope, estimate, API, data model, high-level design, then a deeper look at one or two parts. With practice it fits comfortably in a 45-minute interview.
Core: Design Pastebin
Similar to a URL shortener, with text snippets, expiry and view counts. Tests caching and key-value storage.
Core: Design a URL shortener
A common first problem: generating short keys, a key-value store and caching. The follow-up questions usually cover hot keys or click analytics.
Core: Design a news feed
Fan-out on write, fan-out on read, or a hybrid. Accounts with millions of followers break the simple version of either. Tests sharding, caching and queues.
Core: Design a chat service
A fleet of WebSocket servers, message ordering, group fan-out and presence. Tests real-time communication and storage.
Core: Design object storage
Very high durability for very large amounts of data: erasure coding, separate metadata and data paths, multipart uploads and background repair.
Core: Design a rate limiter
A token bucket in Redis, distributed across servers and applied per user and per endpoint. Tests the algorithm, Redis and edge cases.
Recommended: Design autocomplete
Tries, prefix indexes and ranking. Tests the choice of data structure, caching and handling a high rate of updates.
Recommended: Design a web crawler
A breadth-first frontier, deduplication, robots.txt and a distributed work queue. Tests queueing, storage and crawling politely.
Recommended: Design ride matching
Geospatial indexes (geohash, quadtree, S2), dispatch and areas with heavy demand. Tests spatial data and matching.
Recommended: Design event ingestion
A pipeline that accepts millions of events per second and lets several consumers read at their own pace. Tests partitioning, back-pressure and retention.
Recommended: Design notifications
Preferences per user and per channel, delivery retries, deduplication and batching. Tests queueing, idempotence and state.
Core: Design Twitter
A read-heavy feed for hundreds of millions of daily users: timeline fan-out, celebrity accounts, ranking and search. The usual answer combines push and pull.
Recommended: Design Instagram
Photo uploads straight to object storage, a pipeline that encodes several sizes of each image, and a CDN that serves most of the traffic.
Recommended: Design Netflix
Video streaming worldwide: adaptive bitrate, the Open Connect CDN Netflix runs itself, per-shot encoding and recommendations.
Core: Scaling to millions of users on AWS
How a system grows in stages: one EC2 instance, then a load balancer and autoscaling group, then services and sharded data, then several regions. What breaks first at each step.
After this roadmap
- System design playbookDesign problems worked from start to finish, such as a chat service, a news feed and a URL shortener.
- SimulatorsModels of Raft, consistent hashing, caches, load balancers and rate limiters that run in the browser.
- How it worksGuides to DNS, load balancing, caching and the other components in the diagram above.