Walkthrough: Designing a CDN

The problem

You’re asked to design a CDN — the system Cloudflare, Akamai, and Fastly sell, and that every large product builds some private version of. The interviewer says something like: “Content lives on origin servers in a few datacenters. Users are everywhere. Serve them static assets — images, scripts, video segments — fast, and don’t let global traffic crush the origin.”

Sounds like “put caches near users.” It is — and that sentence hides all three real questions: how a user finds the right cache (routing is the part that touches DNS and BGP, where the interesting failures live), what happens on a miss (the cache hierarchy and origin shielding decide whether the origin survives), and how content changes (invalidation is where caching’s one hard problem actually bites). Candidates who only design the cache box answer a quarter of the question.

Below is how I’d walk through this, start to finish.

1. Requirements

First 3–5 minutes. Questions I’d ask to pin down what we’re building:

Functional requirements

• R1. Serve cacheable HTTP(S) content from locations near users, globally.
• R2. Route every request to a nearby, healthy edge — automatically, including during PoP failures.
• R3. Protect the origin: high offload, plus miss-traffic that arrives coalesced and shielded, not as a stampede.
• R4. Invalidation: customers can update content — both “this asset changed” (versioned) and “purge this now” (compliance, mistakes) within seconds globally.
• R5. Survive flash crowds — a product launch or breaking-news spike at one PoP must not melt the edge or the origin.

Scope

Explicitly out of scope:

• Live video streaming specifics (segment pipelines are the video-streaming walkthrough; this CDN serves its output)
• Edge compute / programmable workers (the Level 5 evolution; named, not designed)
• WAF and DDoS scrubbing depth (the edge is where they run; designing them is a security interview)
• Building physical PoPs (peering, power, real estate — real, not this hour)

Non-functional requirements

• Latency target. TTFB p99 globally? This is mostly a geography problem — the design’s job is shortening the wire.
• Offload target. What origin egress is acceptable? Sets the hit-ratio bar and the hierarchy depth.
• Purge propagation time. Seconds or minutes? Decides the control-plane design.
• Catalog shape. Zipf-like (a hot head and a long tail)? The tail decides cache sizing and admission policy.

Say the interviewer confirms: ~200 PoPs, 50M requests/sec aggregate peak, TTFB p99 ≤ 50 ms in covered geographies, ≥ 95% origin offload, purge visible globally in ≤ 5 seconds, catalog ~1 PB with classic Zipf access (top 1% of objects ≈ 90% of requests), HTTPS everywhere.

2. Capacity estimate

• Traffic: 50M req/sec × ~50 KB average object ≈ ~2.5 Tbps egress at the edge — spread over 200 PoPs, ~12 Gbps average each, with hot PoPs 10× that. Edge capacity is an egress-bandwidth fleet, not a compute fleet.
• Cache sizing: top 1% of a 1 PB catalog ≈ 10 TB serves ~90% of requests — comfortably inside one PoP’s SSD tier (hundreds of TB). The next nine percentage points of hit ratio are what the regional/shield tier exists for.
• Miss math (the one that matters): at 95% edge offload, 5% of 50M req/sec = 2.5M req/sec arriving at shields; if shields absorb 80% of that, the origin sees 500K req/sec — still enormous, which is why coalescing (§7) exists. Every point of hit ratio is ~500K req/sec of origin exposure.

I’d say out loud: “Two conclusions. Latency is geography — below ~50 ms the budget is round trips, so the design is about wire length and connection reuse, not server speed. And the miss stream is the real design object: 5% of 50M is 2.5M req/sec of stampede pointed at someone’s origin unless the hierarchy shapes it.”

3. High-level architecture

Three logical subsystems:

1. Edge PoPs — the cache fleet users hit: TLS termination, cache lookup, peer-fill within the PoP.
2. Routing — the machinery that gets each user to the right PoP (DNS + BGP anycast) and shifts traffic when a PoP degrades.
3. Control plane — distributes customer config and purges to 200 PoPs in seconds; collects logs/metrics back. Deliberately out-of-band from the data path.

flowchart TD
  User[User] -->|routing: DNS + anycast| Edge[Edge PoP]
  Edge --> Cache[(Edge cache)]
  Cache -->|miss| Shield[Regional shield PoP]
  Shield --> Origin[Customer origin]

  classDef store fill:#eef2f1,stroke:#2c5f5d,stroke-width:2px,color:#1f2937;
  class Cache store

Routing decides the system’s character: caches are interchangeable boxes; which box a user reaches, and how fast that answer changes under failure, is what separates CDN designs.

4. Architecture evolution

Each level keeps the previous one’s semantics — it’s HTTP caching (RFC 9111) all the way down, which is why browsers, edges, and shields can stack: they speak one caching protocol.

The design in this walkthrough is Level 4.

I’d say out loud: “I’m designing Level 4 — global edges with a shielded hierarchy. The discipline I’ll keep: every tier is just an HTTP cache honoring the same headers; the layers differ in where they sit, not what they are.”

Target architecture (Level 4)

Every section that follows explains one part of this diagram:

flowchart TD
  User[User] -->|DNS + anycast| Edge[Edge PoP]
  Edge --> Cache[(Edge cache · RAM + SSD)]
  Cache -->|miss, coalesced| Shield[Regional shield]
  Shield --> SCache[(Shield cache)]
  SCache -->|miss, coalesced| Origin[Customer origin]

  CP[Control plane] -.->|config + purges| Edge
  CP -.->|config + purges| Shield
  Edge -.->|logs, health| CP

  classDef store fill:#eef2f1,stroke:#2c5f5d,stroke-width:2px,color:#1f2937;
  class Cache,SCache store

5. Core design choice: how users reach the edge

Three options:

(a) GeoDNS steering. The CDN’s authoritative DNS returns a PoP’s address based on the resolver’s location and current load. Centralized, smart, per-customer tunable. Two structural flaws: DNS sees the resolver, not the user (a public resolver in one city “locates” users from a whole country — mitigated but not fixed by EDNS Client Subnet), and agility is bounded by TTLs that resolvers routinely ignore — draining a PoP takes minutes-to-hours for the stragglers.

(b) BGP anycast. Every PoP announces the same IP prefix; internet routing delivers each packet to the topologically nearest announcement. Failover is routing-table fast (withdraw the announcement, traffic shifts in seconds, no client cooperation needed), and granularity is per-packet-path. The flaws: “nearest” is BGP’s opinion (hop count and peering politics, not latency — sometimes badly wrong), per-PoP load control is crude (you can’t tell BGP “send 30% less”), and a mid-connection route flap moves TCP state to a PoP that doesn’t have it.

(c) Hybrid. Anycast for the data plane (one IP, instant failover, resolver problem irrelevant) with DNS as the coarse tuner (regional prefixes/maps to bias load), plus connection-level fixes for anycast’s flap problem (long-lived prefixes per region; modern stacks lean on QUIC connection IDs surviving path changes).

Choice: (c), leaning anycast-first — the Cloudflare-style posture — because the requirements weight failover agility (R2: automatic, during failures) and TLS-everywhere (anycast’s one-IP model also simplifies cert deployment) over per-PoP load precision, and the load-precision gap is patchable with regional DNS maps while DNS’s agility gap is not patchable from the server side.

Tradeoff matrix

Architecture decisions

Core data structures

6. Read path: a request at the edge [R1, R3]

flowchart TD
  User[User] -->|TLS · anycast IP| LB[PoP front: L4 balancer]
  LB -->|consistent hash on cache key| Node[Cache node]
  Node --> RAM[(RAM hot set)]
  RAM -->|miss| SSD[(SSD tier)]
  SSD -->|miss, coalesced| Shield[Regional shield]
  Shield -->|miss, coalesced| Origin[Origin]

  classDef store fill:#eef2f1,stroke:#2c5f5d,stroke-width:2px,color:#1f2937;
  class RAM,SSD store

Step by step:

1. The user’s TCP/QUIC connection lands on the nearest anycast PoP; the L4 balancer picks a front-end, TLS terminates with a resumed session (one round trip saved — at a 50 ms budget, that’s material).
2. The front-end computes the cache key — host + path + the normalized bits of negotiation (e.g., Accept-Encoding collapsed to a few classes) — and consistent-hashes it to the owning cache node in the PoP, so each object lives on ~one node and the PoP’s SSDs sum instead of duplicate. (The hashing math is the partitioning reference’s; the hot-head exception gets replicated to N nodes when request counters say so.)
3. RAM hit (the Zipf head): serve in microseconds. SSD hit: milliseconds. Either way the response carries correct Cache-Control/Age — the edge is a conforming HTTP cache, which is what lets browser caches stack on top for free.
4. Miss: the node coalesces — one upstream fetch per object per PoP, all concurrent requesters parked on it — and fetches from the regional shield, which repeats the same logic (its own cache, its own coalescing) before touching the origin.
5. The response streams through — first bytes to the user while the tail is still arriving — and is admitted (or not, §8) to SSD on the way.

7. Invalidation [R4]

The two schemes, and why the primary/escape-hatch split matters:

Versioned URLs (primary). The asset’s content hash rides in the URL (app.9f3ab2.js); the object is immutable, cached with Cache-Control: immutable, max-age=31536000, and “updating” it means deploying HTML that references the new hash. The cache-consistency problem doesn’t get solved — it gets deleted: nothing cached ever needs to change. This is the same move the file-storage design makes with content-addressed chunks, and it should carry ~all deploy-driven change.

Tagged purge (escape hatch). For the residue — mistakes, takedowns, mutable URLs you don’t control — the control plane fans a purge to every PoP. The 5-second global SLO shapes the design: purges propagate over persistent control-plane connections to each PoP (a pub/sub spine, with sequence numbers per PoP); a PoP that misses purge N discovers the gap at N+1 and hard-resyncs. Purging by surrogate tag (“everything tagged product-123”) beats by-URL because one logical change touches many cache keys (variants, encodings) — the tag index at each node makes that one message, not a key enumeration.

The trick that makes 5 seconds honest under partition: purge is implemented as epoch bump, not deletion — entries carry the config epoch they were cached under; a purge bumps the tag’s epoch; lookup treats older entries as stale (revalidate before serve). A node that was offline during the purge gets the epoch with its config resync before it takes traffic — correctness doesn’t depend on the deletion racing the reconnect.

8. Deep dives

Admission and eviction: protecting the cache from the tail [R1]

A Zipf tail means most objects are requested once per PoP per day — “one-hit wonders.” Admitting them on first touch evicts proven-hot bytes and burns SSD write endurance for zero future hits (the Facebook photo-caching study measured the effect that motivated this). The standard fix: admit on second hit — a small probabilistic filter (bloom-filter over recent misses) remembers first touches; only a re-request within the window earns SSD residency. Eviction stays boring (segmented LRU: RAM hot set above SSD bulk); admission is where the win is.

Large files (video segments, installers) cache in ranges: a 10 GB object stores as independent ~1 MB slices keyed by (object, range), so a user seeking into a video doesn’t force a full-object fill and partial popularity (everyone watches the first minute) caches proportionally.

Flash crowds: the launch-day path [R5]

A flash crowd is the cache’s best case wearing a scary mask — a million users for one object is one miss and 999,999 hits, if the system doesn’t trip over itself in the first second. The kit, in firing order:

• Coalescing (already in §6): concurrency on a cold object collapses to one origin fetch per tier.
• stale-while-revalidate (RFC 5861): expiry doesn’t stop the world — serve the stale copy, refresh in background; users never queue on a revalidation.
• stale-if-error: origin down ≠ content down — keep serving the last good copy within a generous grace window. For a CDN, origin failure is a caching policy, and this single header is most of “we survived the outage.”
• Hot-object replication: request counters promote the object from one-node-per-PoP to every-node-in-PoP, removing the intra-PoP hashing bottleneck.

The control plane is a second CDN [R4]

Config (origins, TTL rules, keys) and purges must reach 200 PoPs in seconds, survive partitions, and never block the data plane — which makes the control plane a small content-distribution system of its own: a versioned config store at the core, persistent streams to PoPs, per-PoP sequence acking, snapshot-resync for stragglers. The design rule that matters: the data plane runs on its last-known config — a PoP cut off from the control plane keeps serving (staleness over outage), and config application is atomic-per-PoP so no request sees half a customer’s rules. (Same hint/truth split as the file-storage notifications: the stream is the hint, the versioned snapshot is the truth.)

9. API design [R3, R4]

The CDN’s API is its customer control surface (the data path is just HTTP):

PUT  /v1/properties/{site}                      [R3 · config]
  body: { origins: [...], cache_rules: [{match, ttl, key_recipe}],
          shield_region, signing_keys }

POST /v1/purge                                  [R4]
  body: { tags: ["product-123"] } | { urls: [...] }
  returns: { purge_id, epoch }

GET  /v1/purge/{purge_id}                       [R4]
  returns: { acked_pops: 198, total: 200, eta_ms }

GET  /v1/analytics?site=...&window=...          [R3]
  returns: { hit_ratio, origin_rps, egress, p99_ttfb, by_pop: [...] }

Two non-obvious decisions:

Purge returns a trackable ID with per-PoP acks. A purge that “probably propagated” is useless for compliance takedowns; the API exposes the fan-out’s progress because the 5-second SLO is a customer promise, not an internal metric.

Cache rules include the cache-key recipe. Which headers/cookies/query params join the key is the single most consequential per-customer setting — too broad a key shreds the hit ratio (every ?utm_source a separate copy); too narrow leaks one user’s variant to another (§11’s poisoning). Making it explicit, validated config — not origin-header archaeology — is the API decision that prevents both.

The API hides PoP topology, tier counts, and routing entirely. Customers declare intent (cache this, this long, keyed by this); where bytes sit is the CDN’s problem — that abstraction is what lets the operator re-tier, drain, and rebuild PoPs invisibly.

10. Data model and storage

Requirements → components

Access frequency

Storage technology choices

Access pattern matrix

Schemas

Cache entry [R1, R4]

cache_entry (slab storage):
  key:         hash(host, path, normalized_vary)
  slices:      [(range, ssd_location), ...]        # large objects in ~1MB slices
  headers:     stored response headers (Cache-Control, ETag, ...)
  tags:        [surrogate tags]
  epoch:       config epoch at fill time            # purge = epoch compare
  freshness:   fetched_at, ttl, swr_window, sie_window
  heat:        request counter (admission/promotion)

Customer cache rule [R3]

cache_rule:
  match:       path pattern / content-type
  ttl:         edge_ttl, browser_ttl
  key_recipe:  { include_query: [allowlist], include_headers: [...], cookies: none }
  features:    swr, sie, signed_urls?

11. Failure modes

Grouped by what they threaten. The pattern: name what fails, name what degrades, name what doesn’t.

Routing failures

A PoP dies or degrades

Fails: one PoP’s capacity. Degrades: BGP withdrawal (automatic on health failure) shifts its users to neighboring PoPs in seconds — who now take the load cold: hit ratios dip while their caches absorb the new working set, and the shield tier eats the difference (this is the shield’s second job). Doesn’t degrade: correctness anywhere; availability, if neighbors carry N+1 headroom — which is why capacity planning is per-region, not per-PoP.

Anycast route flap

Fails: long-lived connections mid-stream when BGP re-converges and packets land on a PoP without their TCP state. Degrades: those transfers reset and retry (range requests resume large downloads — the slice design pays off here); QUIC connection IDs survive path moves and shrink the blast radius. Doesn’t degrade: new connections, which simply land wherever routing now points.

Origin failures

Origin down or overwhelmed

Fails: the fill path. Degrades — by policy, not accident: stale-if-error keeps serving every object within its grace window; only genuinely-uncached content 404s/504s. The CDN’s posture during an origin outage is “the last good internet, frozen” — and that posture is a configuration the customer chose, which is the right place for it. Doesn’t degrade: the hit path at all; most users never notice.

Synchronized expiry stampede

Fails: origin load when a popular object’s TTL lapses everywhere at once. Degrades: prevented three ways already in the design — per-tier coalescing (one fetch per PoP), shield collapsing PoP fetches to one, and stale-while-revalidate making expiry non-blocking. Add TTL jitter at fill time and the synchronization itself dissolves. This failure is the checklist’s test of the miss-path design: if it’s scary, §6 was built wrong.

Control-plane failures

Control plane unreachable from a PoP

Fails: config updates and purges to that PoP. Degrades: the PoP serves on last-known config (staleness over outage — the §8 rule); purges queue and apply on reconnect before the epoch resync admits it back; the purge API’s per-PoP acks (§9) make the lag visible to the customer instead of silent. Doesn’t degrade: the data plane — by construction it never waits on the control plane.

Bad config or purge-everything mistake

Fails: potentially everything — the control plane is the CDN’s own biggest risk. Degrades: staged rollout (config canaries on a PoP subset, then waves), validation at the API edge (a purge matching > X% of cache requires break-glass), and epoch-based purge being revertible (re-lower the epoch threshold) where deletion wouldn’t be. Doesn’t degrade — if staging works: more than the canary slice. The honest sentence: most global CDN outages in the news were control-plane changes, not traffic.

12. What I’d skip, and say I’m skipping

Time check — five minutes left. Things I’d explicitly defer:

• DDoS scrubbing and WAF. The edge is the natural enforcement point and anycast is the natural diffuser (attack traffic spreads across 200 PoPs); designing detection/rules is a security interview.
• Edge compute. Running customer code at the PoP is the Level 5 platform; it changes the business more than the cache.
• Peering and PoP buildout. Where to put PoPs and whom to peer with is the actual moat of real CDNs — and entirely out of scope in an hour.
• Video-specific delivery. Segment naming, manifest rewriting, midgress optimization — the video-streaming walkthrough owns the pipeline; this design serves its output as ranged immutable objects.
• Billing/metering. Per-byte counting at 50M req/sec is the ad-click aggregation problem wearing a finance hat; named, not redesigned.

Saying “I’d skip this, and here’s why” is a strong signal. It shows you know the full surface and are making deliberate scoping choices.

Interview flow summary

Architecture at a glance

HIT PATH (the product)                  MISS PATH (the design)

User                                    Edge miss
 ↓ anycast → nearest PoP                 ↓ coalesce (1 fetch/PoP)
TLS resume → cache node                 Regional shield
 ↓ (consistent hash in PoP)              ↓ shield cache? else coalesce again
RAM → SSD                               Origin (once, warm connection)
 ↓                                       ↓
serve + Age header                      stream through, admit on 2nd hit
                                         ↓
                                        stale-while-revalidate / stale-if-error
                                        (expiry and outages don't block serving)

The walkthrough order for your whiteboard

1. Requirements        — hit ratio is the product; purge SLO; Zipf catalog
2. Capacity            — latency = geography; the miss stream is the object
3. Evolution           — origin cache → global PoPs → shielded hierarchy
4. Core choice         — DNS vs anycast vs hybrid; commit with failover logic
5. Read path           — key → consistent hash → RAM/SSD → coalesced miss
6. Invalidation        — versioned-immutable primary, tag-epoch purge escape
7. Deep dives          — admission (one-hit wonders), flash crowds, control plane
8. API                 — trackable purges, explicit cache-key recipes
9. Data model          — slab entries, epoch purges, config replicas
10. Failure modes      — routing / origin / control-plane groups

If the interviewer pushes deeper

13. Wrap-up

One crisp sentence before the interviewer’s next question:

This design treats the edge hit ratio as the product and the miss stream as the engineering problem — anycast gets users to a nearby cache in one step, a shielded hierarchy turns two hundred potential stampedes into one warm origin connection, and versioned-immutable content deletes the invalidation problem everywhere it’s allowed to.

What separates levels on this question

• SDE II places cache servers near users, names TTLs and LRU, and draws edge→origin — a Level 3 sketch with the routing, stampede, and invalidation questions unasked.
• SDE III argues DNS vs anycast with failure modes and commits; builds the shield tier from stampede arithmetic; splits invalidation into versioned-primary and purge-escape-hatch; and knows why admission policy (not eviction) is where Zipf tails are fought.
• Staff/Principal prices everything in hit-ratio points and origin exposure; treats the control plane as the system’s biggest risk and designs its blast radius (canaries, break-glass purges, serve-on-stale config); names the cache-key/poisoning security surface; and knows what’s not in scope of architecture — peering and PoP placement — but is the real competitive line.

The difference isn’t knowledge. It’s designing the misses — anyone can describe a cache hit; the CDN is everything that happens, and doesn’t happen, when the byte isn’t there.

Further reading

• RFC 9111: HTTP Caching — the semantics every tier of this design must honor; reading it is what makes the layers stack.
• RFC 5861: stale-while-revalidate and stale-if-error — the two headers carrying most of the flash-crowd and origin-outage story.
• A Brief Primer on Anycast — Cloudflare. The routing model, from the operator that bet the whole data plane on it.
• An Analysis of Facebook Photo Caching — the measured case for admission policies and tiered hit ratios on a real Zipf workload.
• The Akamai Network: A Platform for High-Performance Internet Applications — the DNS-steered overlay design — the other pole of the §5 tradeoff, argued by its inventors.
• Designing Data-Intensive Applications — Kleppmann. Chapter 5’s replication-lag framing applies directly to thinking about edge caches as asynchronous replicas of the origin.
• System Design Interview Vol. 1 — Alex Xu. Chapter 1’s CDN section is the quick orientation; this walkthrough is what its boxes expand into.