🏛️ System Design

🧭 What a System Design Interview Tests

💡 Break a vague, big problem into infra pieces. There's no single right answer. You're judged on 4 things, and Problem Navigation matters most.

• 🧠 Vague problem → infra pieces: services, load balancers, databases.
• 🛠️ Practical, closer to real work than a LeetCode round.
• 🎯 Many right answers. They judge how you navigate, weigh trade-offs, and communicate.
• ⚠️ Going deeper than an interview needs can be a yellow flag.

🗺️ 6 interview types

e.g. "Design Uber" (product) · "Design a rate limiter" (infra). Dashed = other guides.

📋 Interviewer rubric

What they check · what an interviewer writes

• 🎤 #1 cause of failure = no structure → follow the Delivery Framework.
• 🤔 They probe, doubt you, and ask "why not X?" to catch memorised answers → answer with trade-offs, don't defend.
• 🏋️ Practice beats just reading: people who only read stumble on the day.

🛤️ Delivery Framework

Most mid-level candidates don't fail for lack of knowledge. They fail because the clock runs out before they have a working system, and the feedback says "time management". The fix isn't going faster. It's a track to run on: 6 steps, in order. It's also your fallback when nerves hit.

⏱️ The track · tap a step

POST /v1/tweets      {text}
GET  /v1/tweets/{id}  → Tweet
POST /v1/follows     {followee_id}
GET  /v1/feed        → Tweet[]

That's the loop: the first five steps get you a simple system that works. The last step makes it fast, scalable and reliable. Never the other way round, unless the interviewer pulls you there first.

🌐 Layers and the Life of a Web Request

Every design you draw is boxes talking over a network. Networking is a huge field ("one of the heaviest textbooks in school, gross"), but interviews need only a few layers and one story: what happens when you load a page. Learn that once and every arrow between boxes makes sense.

💡 Know 3 layers: IP (L3) routes, TCP / UDP (L4) deliver, HTTP and friends (L7) carry your app's data. One "request" is really many packets, and each one costs latency.

🧱 7 layers, 3 that matter

Layers are abstractions. You don't need to know which voltage means 1 on the wire, only how to use the layer below. Like calling open instead of telling the disk how to read bytes.

🧠 The OSI model, top to bottom, 7 → 1.

OSI is the textbook's 7 layers. The real internet runs on TCP/IP, which squeezes them into 4. ➕ not in the article

One analogy for the whole stack: sending a letter ✉️. Each layer does one job, top to bottom on the sender, bottom to top on the receiver. ➕ Adi's notes, checked; not in the article

"The important ones": 3, 4, 7.

📦 Inside a packet: MAC, IP, port

Each layer has one job, and it does it by writing a label on the data. On the way down, every layer wraps the data in its own envelope; the receiver opens them in reverse. ➕ not in the article

📏 One packet carries about 1,500 bytes, and the TCP + IP labels take ~40 of them. A 1 MB image ≈ 700 packets, which is why TCP numbers them (seq #) to put them back in order.

• 🏠 A router joins two networks, so it has two IPs: private inside (192.168.1.1) and public outside (from your ISP). It swaps your private IP for its public one on the way out: NAT.
• 📣 To find the router's MAC, your phone shouts "who has 192.168.1.1?" and only the router answers: ARP.

So the IP says who it's finally for, the MAC says who gets it next, the port says which app. Now watch those packets move.

🎞️ One web request, packet by packet

Now watch the three layers work together. DNS finds the server, TCP builds a connection over IP, HTTP asks for the page, and TCP tears it all down. Tap the diagram to step.

🔭 3 things to take from it

What the animation just showed you:

📍 Layer 3: where IPs come from

IP routes to addresses. So who hands them out?

• 🎲 IPs are arbitrary. They only mean something because we tell people about them. On your own private network you can pick any.
• 🌍 For internet traffic to find you, you need public IPs: routable, allocated by an RIR. Internet routing is optimised to know where public IPs are.
• 🍎 Any address starting with 17 (e.g. 17.0.0.0) is Apple's. The backbone knows to send those packets to Apple's routers.

That's the base of every arrow you'll draw: DNS to find, TCP to connect, HTTP to talk. Next up: TCP isn't the only way to deliver.

🚚 TCP vs UDP

IP just throws packets at an address and hopes. Before an app can use them, someone has to decide: do we care if packets get lost or arrive out of order? That's the transport layer. In an interview it boils down to one choice: TCP or UDP.

💡 Default to TCP: reliable and ordered, and interviewers assume it. Pick UDP only when speed beats reliability, and say what browsers get instead.

The transport layer gives us guarantees "instead of handing us a jumbled mess of packets". Three protocols live here: TCP, UDP, QUIC. ⭐ The real choice is TCP vs UDP. Treat QUIC as "a better TCP" that isn't widely adopted yet.

🔫 vs 🐴 Machinegun vs workhorse

🎯 Why would anyone want UDP?

No setup sounds great, but losing and reordering packets "kinda suck". So UDP wins only where speed matters more than reliability, and the client can cope with a missing packet.

📞 VoIP drops the odd packet: a hiccup in the audio, but the call is still intelligible. That beats retransmitting and clogging the network with ACKs.

⚖️ When to choose each

Interviewers expect TCP by default, often without you even saying it. That's also the course's advice. Extra points come from making a solid case for UDP "and not bungling the details".

🔫 Choose UDP when:

So: choose TCP and say nothing, or choose UDP and say why, plus what browsers get. Next: what we actually send over the connection.

📨 HTTP and API Styles: REST, GraphQL, gRPC

TCP gives us a reliable pipe. What goes through it? Almost always HTTP, with an API style on top. The interview question isn't "which is coolest?" but "what's the safe default, and when do I leave it?"

💡 HTTP is stateless request → response. Default to REST (resources + verbs). GraphQL when the frontend needs flexibility. gRPC for fast internal calls. REST outside, gRPC inside.

📄 HTTP: the web's foundation

Clients send requests, servers send responses. HTTP is stateless: each request stands alone, and the server keeps nothing between them. ⭐ That's good. Keep the stateful part of your system as small as you can. Most simple HTTP servers are just a function of the request.

You can type this by hand: open a TCP connection with netcat and send it.

Methods and status codes are standard, like enums. Know the common ones. Interviewers rarely dig in unless you're designing a RESTful API.

Headers are the flexible part: free-form key/value pairs. That pragmatism is a lesson in itself.

🔒 HTTPS = HTTP + TLS/SSL encryption, against eavesdropping and man-in-the-middle attacks. A public website uses HTTPS, without exception.

🧩 REST: simple and flexible

HTTP alone builds websites. Designs are about services talking through APIs, and there are three main paradigms: REST, GraphQL, gRPC. ⭐ REST is the most common.

📦 Clients do simple operations on resources (think database tables, or files on a server). The challenge is modelling the resources. REST uses HTTP verbs plus path conventions, usually with JSON bodies (not required).

GET  /users/{id}        → User
PUT  /users/{id}        → User  (update)
POST /users             → User  (no id:
                          server assigns)
GET  /users/{id}/posts  → [Post] (nested =
                          relationship)

📍 Real example: Elasticsearch manages documents and indexes over REST. 🐢 REST isn't the fastest for very high throughput, and JSON is inefficient to (de)serialize. But most apps aren't bottlenecked there.

⭐ Like TCP, REST is your interview default. Reach for GraphQL, gRPC, SSE or WebSockets only for needs REST can't meet. 🔗 API Design lesson.

🔍 GraphQL: ask for exactly what you need

REST's weak spot shows when a frontend team (say, a mobile app) builds a new page on top of a backend API. Facebook open-sourced GraphQL around 2015 to fix exactly this.

Three options, and "none of these are particularly good":

query GetUsersWithProfilesAndGroups {
  users(limit: 10) {
    id username
    profile { fullName avatar }
    groups { name category { icon } }
    status { isActive lastActiveAt }
  }
  _metadata { totalCount hasNextPage }
}

Trimmed from the article's query: you just list the fields and nested objects you want.

🎤 In interviews its benefits are "murky": requirements are fixed, and the interviewer wants to see you optimise specific queries, where GraphQL "is frequently just in the way". ⭐ Bring it up when the problem is about flexibility (adapting apps fast to changing requirements) or the requirements are deliberately uncertain.

⚡ gRPC: efficient service-to-service calls

REST and GraphQL ship JSON text. When your own services call each other constantly, text gets expensive. gRPC is Google's (the "g") RPC framework: HTTP/2 + Protocol Buffers.

🧬 Protocol Buffers are "JSON with a more rigid schema", which makes them smaller and faster to parse:

message User {
  string id = 1;
  string name = 2;
}

0A 03 31 32 33 12 08 6A 6F 68 6E 20 64 6F 65

🔎 Nit: 6A is a lowercase "j", so these bytes spell "john doe", not "John Doe".

service UserService {
  rpc GetUser (GetUserRequest)
    returns (GetUserResponse);
}

• 🛠️ Definitions compile into client and server stubs for many languages.
• 🧰 Built for microservices at scale: streaming, deadlines, client-side load balancing.
• ⭐ The key fact: it's binary, faster and more efficient than JSON over HTTP. Strong typing catches errors at compile time. Some benchmarks show 10x throughput.
• 🔒 Use it for internal calls, when performance is critical or latency is dominated by the network, not the server's work.
• 🚫 Not for public APIs or clients you don't control: binary, and the tooling is less mature than JSON over HTTP.

⭐ REST for public-facing APIs, gRPC for internal service-to-service, especially with binary data or when performance is critical. In many interviews, REST for both is fine. Build from there as the problem and the interviewer push.

So: HTTP carries it, REST shapes it, GraphQL and gRPC are for specific needs. All of them are the client asking. Next: when the server has to speak first.

📡 Real-time: SSE, WebSockets, WebRTC

Everything so far is the client asking. But an auction, a chat or a video call needs the server, or another user, to speak first. gRPC can stream, but no browser supports it today. So the web has three tools, one for each shape of conversation.

💡 Server → client only: SSE. Both ways, high-frequency: WebSockets (justify them!). Audio/video calls: WebRTC, peer-to-peer over UDP.

📬 SSE: one response that never ends

Server-Sent Events is "a nice hack on top of HTTP": a spec that lets the server stream many messages, over time, in a single response.

{ "events": [
  { "id": 1, … },
  …
  { "id": 100, … } ] }

data: {"id": 1, …}
data: {"id": 2, …}
…
data: {"id": 100, …}

⚠️ "Tradeoffs!" Some nasty, misbehaving networks batch all the SSE chunks into one response, which brings back exactly what we were avoiding.

🎯 Use it when clients should get notifications or events as soon as they happen, e.g. keeping auction bidders up to date on the current price. 🔗 Realtime Updates covers the server side.

🔌 WebSockets: both sides talk, any time

SSE pushes one way only. For real-time bidirectional traffic, WebSockets give a persistent, TCP-style connection with broad support, browsers included. The server pushes without a request, and so does the client.

🎯 Use them for high-frequency, persistent, bi-directional communication: real-time apps, games, anything that must send and receive the moment it happens. Just request/response, or SSE pushes are enough? Then WebSockets are overkill.

🎥 WebRTC: browser to browser, no server in between

The third shape: two users talking to each other, not to you. WebRTC connects browsers peer-to-peer. It's built for video and audio calls, and it's the only app protocol here that uses UDP.

🚧 Peer-to-peer is harder: most clients don't allow inbound connections, and most users sit behind a NAT (network address translation) device. Two standard workarounds:

🎯 ⭐ Use it for audio/video calling and conferencing. Collaborative editors? Occasionally, especially at scale. But most don't need thousands of clients and need a central server anyway, which is why the course's Google Docs design uses WebSockets. The alternative is WebRTC + CRDTs, sending updates straight between clients.

So: SSE when only the server talks, WebSockets when both do, WebRTC when the users talk to each other. Each one holds connections open. Next: spreading all those connections across many servers.

⚖️ Load Balancing

We know how one client talks to one server. But one server won't hold, so we scale. The moment there are three servers, every client has the same question: which one do I talk to?

🍽️ Think of a busy restaurant. One chef can't cook for the whole street, so you hire more chefs. Now every diner walking in asks: which chef cooks for me? This whole card is that question. ➕ analogy not in the article

💡 Horizontal scaling needs load balancing. Internal services: client-side (built into gRPC). Everything else: a dedicated LB. ⭐ L4 for WebSockets, L7 for HTTP.

📈 Bigger servers or more servers

🧑‍🏫 The author scales vertically wherever possible: modern hardware is incredibly powerful, and the days of thousands of tiny servers are over (🔗 Numbers to Know). ⭐ But in interviews, horizontal scaling is the most common pattern.

🍽️ Vertical = give your one chef a bigger stove. Horizontal = hire more chefs. A bigger stove has a ceiling, and if that chef gets sick, nobody eats. ➕ analogy not in the article

Adding boxes to the whiteboard doesn't help if clients don't know which one to call.

Spread the load by deciding which server handles each request. Two ways: decide on the client side, or on the server side with a dedicated load balancer.

🧭 Client-side: the client picks

The client asks a service registry (directory) for the list of servers, then calls one directly. It polls, or gets pushed updates when servers are added or removed.

🍽️ Diners look at a board of free chefs by the door (the registry) and walk straight to one. No one stands in the way, so it's fast. But if the board is old, a diner may walk to a chef who just went home. ➕ analogy not in the article

⚡ Very fast: no extra network hop on every request, just a periodic sync of the server list. Two examples:

📖 DNS is client-side load balancing too. Each lookup of example.com returns a rotated list of IPs (a different order, or even a different set), so different clients hit different servers.

📖 DNS is a phone book that lists the restaurant's numbers in a different order for each caller, so calls spread out. The catch: people keep the old page for a while (the TTL). ➕ analogy not in the article

So when does client-side work? It depends on how many clients must hear about changes, and how fast.

⏳ Why "delays": update time grows with the number of clients to notify. DNS entries have a TTL (time to live), so far-flung DNS servers can cache them for their own clients, but your updates can never be faster than the TTL.

🎤 Great for internal microservices (built into gRPC). Interviewers rarely ask about the lines between your services, but if they do, mention client-side load balancing!

🚦 Dedicated load balancers

For everything else: clients you don't control, clients that must see changes quickly, or clients that shouldn't even know there are many servers. Put a server or hardware device in the middle.

🍽️ A host at the door. Every diner stops there first (the extra hop), and the host seats them. Diners never see the kitchen, and the host knows right away when a chef leaves or joins. ➕ analogy not in the article

"Load balancer decides which server to route to."

🧱 LBs work at different layers of the stack. Which one depends on what your app needs.

4️⃣ Layer 4: forwards connections, never reads

• 🔗 Keeps persistent TCP connections between client and server.
• ⚡ Fast and efficient: minimal packet inspection.
• 🙈 Can't route on application data.
• 🎯 Used when raw performance is the priority, and for WebSockets and anything that needs a persistent connection. "If you're using websockets in your interview, you probably want an L4 load balancer."

☎️ L4 is a phone switchboard: it connects your call to one line and steps away. It never listens to what you say, so it can't route on it, but it's fast. A WebSocket is a long phone call: connect once, then talk for hours. That's exactly what a switchboard is for. ➕ analogy not in the article

7️⃣ Layer 7: reads every request

• ✂️ Terminates incoming connections, creates new ones to the backends.
• 🔍 Routes on request content: URL, headers, cookies.
• 🔥 More CPU-intensive (it inspects packets).
• 🧰 More flexibility and features. Best for HTTP traffic: every protocol here except WebSockets.
• ⚠️ It hides TCP details, but for persistent connections to the same server you still can't ignore the connection.

📨 L7 is a receptionist who opens every letter: "billing? → desk 3, support? → desk 5". Smart routing by what's inside, but reading costs time (CPU). For a long phone call there's nothing to read, so the receptionist just holds two lines open for hours. ➕ analogy not in the article

🩺 Health checks: route around failures

Balancing is half the job. The LB also watches its servers, and that automatic failover is what makes LBs essential for high availability, with no human needed.

🍽️ The host peeks into each kitchen every few seconds. TCP check = "is the kitchen door open?" L7 check = "can you actually cook a dish?" Chef down → stop seating diners there until they're back. ➕ analogy not in the article

🎲 Algorithms

A dedicated LB also lets you choose how traffic is spread.

🃏 Round robin = dealing cards: one each, in turn. Least connections = joining the shortest checkout queue at the supermarket. IP hash = your regular waiter: same face every visit. ➕ analogy not in the article

• ⭐ Round robin or random suit stateless apps where no server is more popular. A new server starts getting traffic with no special config.
• ⭐ Persistent connections (SSE, WebSockets) → Least Connections, so one server doesn't slowly collect all the active connections.

🏭 In the real world

🚀 Enterprise hardware LBs handle 100s of millions of requests per second; software ones less. Scaling the LB itself is almost never asked in SWE interviews (except networking specialisations). If your LB throughput looks huge, mentioning hardware LBs "is a good way out".

So: client-side for services you control, a dedicated LB for everyone else; L4 to keep connections, L7 to read requests. With servers spread out, the next problems come from geography and failure.

🌍 Latency and Failures: Distance, Retries, Circuit Breakers

Protocols ✓, load balancing ✓, persistent connections ✓, high availability ✓. Two deep-dive favourites are left, and both come from the real world: physics makes distance slow, and networks break.

💡 Two real-world problems, and the tools for each. Every tool is explained further down 👇

🌐 Regionalization: physics wins

Global services put servers around the world: several data centers in one region (Amazon's "availability zones", so a broken pipe in one building doesn't take you down), repeated in cities across the globe. "A great victory for humanity", but now distance costs time.

The fix, everywhere in computing: data locality. Performance is best when the data sits as close as possible to the computation.

📚 Data locality = keep the books you need on your desk, not in a library across town. Every trip to the library costs time, however fast you read. ➕ analogy not in the article

• 📍 Keep the data a query needs (a) close together and (b) close to the user.
• 🐢 User data in Los Angeles, web servers in New York → every DB query pays tens of ms, before processing.
• 🤷 Some latency is unavoidable: if users are far apart, they're far apart. Two strategies optimise within physics 👇

📦 CDNs: the answer is up the road

• ⭐ The most common way to cut latency. Content Delivery Networks have servers in hundreds or thousands of cities: edge locations.
• 🧊 It only works because of caching: data that doesn't change much or often. Best for static content: images, videos, assets.
• 🎤 Use one when data is very cacheable and queried from across the globe. e.g. caching Facebook search results → lower latency and less load on the backend.

🏪 A corner shop stocks copies of the popular items from the big warehouse. Milk and bread: grab it up the road. Something rare or fresh off the line: still the long drive. ➕ analogy not in the article

🗺️ Regional partitioning: mirror the geography

Uber's key insight: a rider in Miami never wants a driver who's in New York. Millions of riders and drivers a day, but only a few thousand in any one city. So the architecture can mirror that.

Each region has its own database in that geography, with request servers co-located next to it. Booking a ride or checking status stays inside the region. "Nice and optimal!"

🏦 Like a bank with city branches: your account lives at your branch, so everyday visits never call head office in another country. ➕ analogy not in the article

💥 Failures: plan for them

The other frequent deep dive. Servers crash, solar flares flip bits, power gets cut. The network fails too: cables get cut, routers fail, packets get dropped.

🔁 Timeouts and retries with backoff

The most elementary hygiene: expect a call to take X, set a timeout, give up and try again. A temporarily slow server will likely answer the retry. But retries are double-edged: retrying requests that keep failing makes things worse.

⏳ Backoff: don't retry immediately. Wait a little, and longer after each failure, so the system can recover. 🎲 Jitter: add randomness, so clients don't all retry at the same moment "like a jackhammer".

📞 Calling a busy friend: no answer → wait 1 min, then 2, then 4 (backoff). If 1,000 people all redial on the exact same second, the line jams again; each waiting a slightly random time (jitter) spreads the calls out. ➕ analogy not in the article

🎤 ⭐ Interviewers often listen for the magic phrase "retry with exponential backoff". In more senior interviews, be ready to explain jitter. 📖 AWS: timeouts, retries and backoff with jitter.

🔑 Idempotency: retry without double-charging

Retries are cool, except when they have side effects. ⭐ This is why idempotent APIs are the key to safe retries: calling them many times gives the same result. GET is the classic one: the content may change, but fetching it doesn't change the system. For writes, add a key 👇

🔌 Circuit breakers: stop the cascade

Senior candidates often get "what happens when this service goes down?" Sometimes "fail and retry until it's back" is fine. But picture a database that went down cold and has to boot one instance at a time. A firehose of retries and angry users can pin each instance before it starts: a thundering herd. You can't get the first one up, so the whole DB never comes back. Stuck!

🛍️ Thundering herd = a Black Friday crowd at the doors: open one door and the crush knocks the first clerk over, so the shop never manages to open. ➕ analogy not in the article

What does a cascade look like in real life? Usually it starts with something slow, not something dead. 👇 ➕ example not in the article

🚗 A traffic jam: one lane is blocked, cars back up into the junction, and soon streets that never used that lane are stuck too. ➕ analogy not in the article

The first tool that stops it is the circuit breaker:

Inspired by electrical circuit breakers: it stops cascading failures and gives failing services time to recover.

🔌 The fuse in your house: a fault trips it, and power to that circuit stops at once (fail fast) instead of burning the wiring. After a while you try flipping it back (a few test requests); if it holds, power is back. ➕ analogy not in the article

🎤 Bring them up when the interviewer digs into reliability, failure modes or disaster recovery. That knowledge is usually "won at 3:00am battling a hardware failure when the system Just. Won't. Come. Back. Up." Put them on:

🛡️ The cascade toolkit

A breaker alone isn't the whole answer. Each tool fixes one step of the cascade above. ➕ bulkhead, shedding, queue: not in the article

• ⏱️ Timeout · fixes ① Threads wait forever
  Give up after ~2 s, not 30. Hang up instead of holding the phone all day.
• 🔁 Backoff + jitter, retry limit · fixes ④ Retries pile on
  Retry a few times, waiting longer and a little randomly. Never retry forever.
• 🔑 Idempotency key · fixes ④ Retry charges twice
  Same order number → the bank charges once.
• 🔌 Circuit breaker · fixes ② Still calling a broken service
  Stop calling after N failures, test again later. The house fuse.
• 🚢 Bulkhead · fixes ⑤ One dependency eats every thread
  A separate, capped pool per dependency. A ship's watertight rooms: one floods, the ship floats.
• 🚦 Load shedding / rate limit · fixes Too much traffic arriving
  Turn away extra requests early (fast 503) so the ones inside finish. A bouncer at a full club.
• 🪂 Fallback · fixes User gets nothing
  Give a lesser answer: "Order placed, payment confirming", cached results, a default list.
• 📬 Queue it · fixes Slow work blocks the user
  Accept the order now, charge from a queue. The bank's slowness no longer holds a user thread.

🏁 The whole lesson in four steps

Networking choices touch latency, throughput, reliability and security. The course gives sensible defaults, but most problems have no single right answer: justify each choice from the requirements, and show the interviewer how you weigh the trade-offs.

🧪 Try it yourself: capture your own traffic in Wireshark to see the whole stack in action. Then fake a nasty cell connection (latency, packet loss) with Mac's Network Link Conditioner (via Xcode) and watch apps break.