Walkthrough: Designing a Video Streaming Platform

The problem

You’re asked to design a video streaming platform — YouTube-style: “Users upload videos. Other users watch them on phones, TVs, and laptops, anywhere in the world, at varying network speeds. Videos should play instantly, never buffer, and look good on whatever device the viewer has.”

Looks like a file-storage problem. It isn’t. The hidden trap is that the same source video must be served as 8+ different encoded variants to handle device and network diversity, and the read traffic is so skewed (the top 0.1% of videos generate >50% of bandwidth) that your storage architecture and your CDN strategy are inseparable. The question tests whether you understand the full upload→transcode→deliver pipeline as a coherent system, not as three loosely-coupled bullet points.

Important

The core insight: A video is not a file — it’s a manifest of immutable segments. This reframing unlocks everything: segments are write-once (no cache invalidation problem), each segment can be transcoded independently (massive parallelism), and the player — not the server — decides which quality to use segment-by-segment (ABR). The architecture follows from this mental model.

Below is how I’d walk through this.

1. Clarify before you design

First 4–6 minutes.

Scale. Hours of video uploaded per minute? Concurrent viewers globally at peak? (YouTube-scale: ~500 hours uploaded per minute, ~100M concurrent viewers at peak.)
Video lengths and formats. Average duration? Max upload size? What source formats do we accept?
Quality tiers. What resolutions do we serve? (360p, 480p, 720p, 1080p, 4K, plus mobile-optimized variants.)
Live vs on-demand. Live streaming is a different system — is it in scope? (Default: VOD only.)
Geographic distribution. Global, single-region, or multi-region with edge CDN?
Protected content. DRM (paid content), geo-restricted, or open?
Recommendations and search. In scope? (Usually a separate system — keep at the API boundary.)
Comments, likes, subscriptions. In scope? (Often deferred to focus on the streaming itself.)

Say the interviewer confirms: 500 hours/min uploaded, 100M concurrent viewers, average video 8 minutes, 5 quality tiers, VOD only, global with regional CDN, no DRM in scope, no recommendations or social features.

2. Capacity estimate

Numbers that drive design decisions:

Upload volume. 500h/min × 60 = 30,000 minutes of video per minute. At ~50MB per minute of source 1080p, that’s ~1.5 TB/min ingested = ~2 PB/day ingest before transcoding.
Storage after transcoding. Each video is stored in 5+ variants. The full set typically totals ~2–3× the source size. After 5 years of retention: roughly an exabyte for the active corpus. (YouTube has reportedly hundreds of EB; this matches the order of magnitude.)
Read traffic. 100M concurrent viewers × 3 Mbps average bitrate = 300 Tbps egress. The CDN absorbs nearly all of this.
Skew. ~50% of bandwidth comes from the top 0.1% of videos. ~95% comes from the top 5%. The long tail is real but bandwidth-cheap.
Metadata storage. ~10 KB per video metadata record × 10B videos = ~100 TB. Trivial relative to the video corpus.

The takeaway: “The architecture is dominated by two facts. One — write throughput is huge but contained: ingest + transcode + write once. Two — read throughput is enormous and skewed: 95% of egress is from the hot 5%. Storage tiering and CDN caching are not features; they’re the architecture.”

3. API design

POST /api/videos/upload
  body:    { user_id, title, description, tags[],
             multipart_chunk, idempotency_key }
  returns: { upload_id, chunk_ack }

POST /api/videos/{upload_id}/finalize
  body:    { total_chunks, checksum }
  returns: { video_id, status: "transcoding" }

GET /api/videos/{video_id}
  returns: { video_id, title, status, duration,
             manifest_url, poster_url, qualities[] }

GET /api/videos/{video_id}/playback
  returns: { manifest_url (HLS/DASH), license_url?,
             ttl_seconds }

GET /api/users/{user_id}/library
  returns: { videos[], next_cursor }

Two non-obvious decisions:

Upload is chunked + resumable. A 4 GB video upload from a spotty mobile connection cannot be a single POST. We use ~5 MB chunks with per-chunk acks; the client retries failed chunks; finalize is the explicit “I’m done” call.
Playback returns a manifest URL, not a video URL. The player fetches an HLS or MPEG-DASH manifest that lists all available bitrates and segments. The player itself decides which bitrate to use moment-to-moment (adaptive bitrate streaming, ABR). The API doesn’t pick the quality.

4. Core design choice: the upload-to-playable pipeline

This is the question. The problem isn’t “where do we store video.” It’s “what’s the sequence of work that makes a 4 GB phone video into something playable on 30 different devices at 8 different network speeds.”

Important

Name the pipeline as the architecture. The answer to “design YouTube” is not a storage system with some CDN in front — it’s a pipeline with at least 8 distinct stages, each with its own scaling story and failure mode. Candidates who describe a blob store + CDN + FFmpeg are missing the structure that makes the system work.

The pipeline:

1. Chunked upload      → object storage (raw)
2. Validation          → reject malformed, malicious, etc.
3. Probe + queue       → extract duration, codec, fps, etc.
4. Transcode (parallel) → 5+ output variants
5. Segment & manifest  → HLS/DASH chunks
6. Replicate to CDN    → push hot tiers; pull others
7. Mark playable       → update video record
8. Async: thumbnail, captions, content moderation

Three design choices to defend:

(a) Chunked upload with object-storage commit

Direct upload from client to object storage (S3 / GCS) via a pre-signed URL. The application server hands out the upload destination but doesn’t proxy the bytes. Multi-part upload supports resumable chunks natively.

This is the right call for this problem because: video bytes don’t need to traverse the application tier — bypassing it saves order-of-magnitude bandwidth and latency. The app server only sees lightweight metadata (auth, idempotency, signing).

(b) Transcoding as massively parallel, fan-out work

Note

Interview signal: Naming segment-level parallelism — not video-level parallelism — is the key insight. Parallelizing across videos is easy; parallelizing within a single video (split → fan-out by segment × quality → merge) is what collapses wall time from hours to minutes. Most candidates describe the former; the latter is what the question is actually testing.

Source video → split into 30-second segments → fan-out N parallel transcoders, each producing one (segment × quality) output → merge segments per quality → write manifest.

A 60-minute video produces 120 segments × 5 qualities = 600 transcode tasks, each runnable on a separate worker. Wall time collapses from “encode 60 minutes” (could be hours) to “encode 30 seconds” (typically 30–60 seconds with GPU acceleration).

Why this shape: transcoding is the slow part of the pipeline, and it’s embarrassingly parallel at the segment level. A serialized encode of a 1-hour video on one worker is the wrong default by 100×.

(c) Cold storage tiering

Videos older than 90 days with low view counts are migrated to cold object storage (S3 Glacier-style). On rare access, a quick-restore (~1 minute) brings them back to warm tier. Hot storage costs $/GB-month; cold is $/TB-month.

The tiering policy is dictated by the long-tail viewing pattern specific to YouTube: 95% of bandwidth comes from 5% of videos, so the inverse — 95% of storage is for the 5% rarely-viewed tail — is the candidate for cold storage.

I’d commit:

“Direct chunked upload to object storage, segment-level parallel transcoding to 5 quality variants, HLS/DASH manifests for adaptive playback, and cold-tier migration for the long tail. The pipeline is the system.”

5. Data model and storage

videos (RDBMS, sharded by video_id):
  video_id (PK), uploader_id, title, description, status,
  duration_s, source_codec, created_at, made_public_at,
  privacy, manifest_path, poster_path, view_count_estimate

video_segments (NoSQL, e.g., Cassandra, partitioned by video_id):
  video_id (PK partition), quality, segment_index (clustering),
  storage_path, size_bytes, duration_ms

uploads (RDBMS):
  upload_id (PK), uploader_id, video_id?, status,
  total_size, received_chunks, started_at

video_blob (object storage, e.g., S3):
  /raw/{video_id}/source.{ext}
  /processed/{video_id}/{quality}/segment_{i}.ts
  /processed/{video_id}/manifest.m3u8
  /processed/{video_id}/poster.jpg

view_counts (separate write-heavy store, e.g., counter service):
  per video_id, eventually consistent

Why these choices:

videos in RDBMS. Metadata is queried by many access patterns (by user, by status, by recency); a relational store is natural. ~100 TB total — well within scale.
video_segments in Cassandra (or similar wide-column store). Each video can have hundreds of segments × 5 qualities = 1000+ rows. Partition by video_id gives single-partition reads for playback. The Cassandra wide-partition pattern fits this exactly.
video_blob in object storage. The raw and transcoded bytes belong nowhere else. Object storage is purpose-built for this access pattern (large objects, infrequent overwrite, high read throughput).
view_counts separately. View tracking is a write-heavy counter problem with relaxed consistency — completely different from metadata storage. We’d use the top-k pattern here for the trending list and an approximate-counter for raw view counts.

Topic-specific reasoning: video storage is write-once per quality variant. The mutation pattern is “add a new variant” (rare), not “modify existing bytes” (never). That’s exactly object storage’s sweet spot, and exactly the wrong workload for a relational DB.

6. The upload + transcode pipeline in detail

The architecture, end-to-end:

flowchart LR
  Client[Client] --> API[Upload API]
  API --> Signer[Pre-signed URL signer]
  Client -->|chunks via signed URL| Raw[(Raw object store)]
  API --> UploadDB[(Upload state DB)]
  Raw -->|finalize event| Probe[Probe service]
  Probe --> Q[(Transcode queue)]
  Q --> T1[Transcoder 1]
  Q --> T2[Transcoder 2]
  Q --> Tn[Transcoder N]
  T1 --> Processed[(Processed object store)]
  T2 --> Processed
  Tn --> Processed
  Processed --> ManifestSvc[Manifest service]
  ManifestSvc --> VideosDB[(Videos DB)]
  ManifestSvc --> CDN[(CDN replication)]

Walking through the upload:

Client requests upload destination. API calls the signer, returns a pre-signed URL pointing at the raw object store. Client uploads chunks directly to storage.
Finalize. Client posts to /finalize. API verifies all chunks present, computes checksum, marks videos.status = "uploaded", emits a video_uploaded event.
Probe. A consumer of the event reads enough header bytes to extract codec, duration, fps, resolution. Validates the video is well-formed (rejects 0-byte, malformed, malicious). Writes back videos.duration_s, videos.source_codec.
Queue. Probe enqueues N transcode tasks — one per (segment, quality) — into the transcode queue. The queue is partitioned by video_id to keep all of one video’s tasks together for priority management.
Transcoders. Worker pool consumes the queue. Each worker: downloads the source segment, runs FFmpeg (often GPU-accelerated) to produce the output for one quality + one segment, writes to processed object store. Acks the queue task.
Manifest. A separate service watches for “all segments of video X for quality Y are done.” When the full video finishes for at least one quality, the manifest service writes manifest.m3u8 (HLS) or manifest.mpd (DASH) referencing all segments. Updates videos.status = "playable". The video can now be watched.
CDN replication. New manifests trigger CDN pre-population for popular uploaders / pre-flagged videos. Cold uploads rely on pull-on-first-view.

The TTL/invalidation behavior worth naming:

Raw video bytes are retained 30 days post-transcode for re-transcoding (codec upgrades, bug fixes). Then deleted.
Manifests are immutable per video version. CDN cache TTLs can be effectively infinite (cached until manually invalidated on a re-encode).
Segments are also immutable per video version. Same long TTL.

Topic-specific reasoning: the immutability of segments and manifests is what makes CDN caching trivial. There’s no “cache invalidation problem” because the content doesn’t change after publish. Compare to the URL shortener’s similar property — write-once, mostly-immutable assets are the cleanest cacheable case.

7. Adaptive bitrate (ABR) and the player

The player’s job: pick the right bitrate for each segment based on current network conditions. The system’s job: provide a manifest that lists all available bitrates and segment URLs.

The HLS manifest fragment:

#EXTM3U
#EXT-X-STREAM-INF:BANDWIDTH=400000,RESOLUTION=480x270
360p/playlist.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=1000000,RESOLUTION=854x480
480p/playlist.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=2500000,RESOLUTION=1280x720
720p/playlist.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=4500000,RESOLUTION=1920x1080
1080p/playlist.m3u8

Each variant playlist lists segment URLs:

#EXTM3U
#EXT-X-TARGETDURATION:6
#EXTINF:6.0
segment_0.ts
#EXTINF:6.0
segment_1.ts
...

Why segment-level ABR matters: the player can switch quality between segments (every ~6 seconds). On a flaky connection, the player drops from 1080p → 720p → 480p as it observes buffer depth. On a fast connection, it climbs back up. This is what “adaptive” means; without it, viewers buffer or watch unnecessarily-blurry video.

The system’s contribution to ABR is making segments small (6 seconds is standard) and the manifest accurate. The player implements the bitrate-selection logic itself; we don’t.

8. CDN strategy

This is the section that decides whether the system is viable at scale. CDN is not a feature; CDN is the entire read path.

The CDN architecture:

flowchart LR
  Client[Client] --> CDN[CDN edge POP]
  CDN -->|miss| RegionalEdge[Regional CDN tier]
  RegionalEdge -->|miss| Origin[(Origin object store)]

  Origin -.->|preload hot videos| RegionalEdge
  Origin -.->|preload viral videos| CDN

  classDef new fill:#eef2f1,stroke:#2c5f5d,stroke-width:2px,color:#1f2937;
  class CDN,RegionalEdge new

Three layers:

Edge POPs. Hundreds globally. Cache the hottest segments (top 5% of videos by recent view count). Hit rate: ~80%.
Regional tier. ~10 globally. Cache a deeper working set (top 20% of videos). Hit rate from edge misses: ~80% → so ~96% combined.
Origin. Object storage. Cold-tier-aware: hot bytes in warm storage, cold bytes in Glacier-class. Origin egress is small (~4% of total) but expensive per byte.

Tip

CDN is not a feature added at the end — it’s the architecture. 96% of read traffic is served from cache. The origin (object storage) sees only 4% of requests. Every architectural decision about storage tiering, segment size, and manifest immutability is ultimately in service of maximizing that cache hit rate.

Pre-population strategies:

Trending videos (top-K from the top-k walkthrough) are pushed to all edge POPs proactively.
Geographically-correlated content (a Korean drama trending in Korea) is pushed to nearby POPs aggressively; far POPs handle on-demand.
Newly-uploaded videos by high-subscriber creators are pre-warmed at upload time — the algorithm can’t predict which uploads will trend, but a creator’s expected audience is predictable.

Why this pattern works for video: cache hit rates are exceptionally high because the access pattern is highly power-law-distributed — the same few thousand videos generate most of global bandwidth at any moment. With static content (no per-user customization at the byte level), the cache is near-perfect.

Where it doesn’t work: personalized content (DRM-licensed, per-user watermarked, per-user inserted ads). For ad-supported video, ad insertion happens at the manifest level (server-side ad insertion, SSAI) so the underlying segments remain shared; ads come from separate URLs.

9. The async pipelines: thumbnails, captions, moderation

Three pipelines run alongside the main transcode, each with its own queue:

Thumbnail extraction. Sample 5 frames at 10%, 30%, 50%, 70%, 90%. Run through a “best frame” model (often as simple as edge density + face detection). Pick one as the default poster; expose all five in the API for the uploader to choose.
Caption generation. Run the audio track through speech- to-text. Produce a .vtt file. This is offline work and may take longer than transcoding.
Content moderation. Run video through a moderation model (computer vision + audio classifier). Flag for review or automatic takedown for clearly disallowed content. This is a gating pipeline for some content classes — the video may not be made playable until moderation completes.

These are all eventually-consistent enrichments. None block upload-to-playable for the typical case.

10. Failure modes

Name what fails, name what degrades gracefully, name what doesn’t.

Transcode worker crashes mid-task. Queue’s visibility timeout expires; another worker picks up the task. Same input → same output (transcoding is deterministic). The video has the segment a few minutes later than ideal.
Object storage region fails. Cross-region replication (active-passive for cold tier, active-active for hot tier) keeps reads served from the surviving region. Writes during the outage queue and replicate when the failed region returns.
CDN edge POP fails. Client routes to next-nearest POP automatically (CDN’s responsibility). No system-level work needed; viewer experiences slightly higher latency, no unavailability.
Origin overwhelmed by cache miss surge. Thundering herd — a viral video drives all CDN tiers to miss simultaneously. Mitigations: request coalescing at each cache tier (only one origin fetch per missing segment), origin admission control (rate-limit total egress), cache-warming when a video is flagged trending.

Caution

The thundering herd on viral content is the most dangerous failure mode for this system. A video that goes viral in minutes can simultaneously miss at every CDN POP globally, creating millions of concurrent origin fetches. Without request coalescing at each CDN tier and proactive cache warming when trending signals appear, the origin can collapse. Name this explicitly and name the mitigations — most candidates don’t.

Manifest service is slow on a popular upload. Players retry with backoff; the video will appear “transcoding” for longer than usual. Acceptable degradation.
Player cannot adapt down enough on slow connections. Lowest tier (e.g., 144p mobile) must be present in the manifest. (YouTube has historically maintained 144p specifically for this reason.)
Upload fails mid-way. Resumable upload with chunk acks means the client retries only the missing chunks. Without this, every flaky upload starts over — which is unacceptable at scale.

11. What I’d skip, and say I’m skipping

Time check: a few minutes left.

Live streaming. A separate, materially different system (HLS-based live still uses similar segments but the ingest side is real-time, the manifest is rolling, and there’s no pre-transcode). Worth a sentence.
Recommendation engine. Off the streaming critical path. Probably the largest system at the company by code volume, but the design is bounded.
Search. Standard inverted-index over metadata; see the post-search walkthrough.
DRM. Only relevant for paid content; modular addition (license server + manifest wrapping). The transcoding and CDN paths are unchanged.
Comments, likes, subscriptions. Each is its own system; none are streaming-architecture concerns.
Per-user ad insertion. SSAI is the right pattern, but building it out is its own walkthrough.
Copyright detection (Content ID-style). A separate fingerprinting pipeline that consumes the same source bytes but is not gating on most uploads.

12. Wrap-up

The architecture is a write-once-then-cache-forever system. Upload feeds a parallel transcode pipeline that produces immutable segments and manifests; CDN tiers absorb 96% of read traffic; cold-tier migration handles the long tail. The system’s correctness lives in the pipeline; the system’s scale lives in the cache hierarchy.

That’s the answer that lands the round.

What separates SDE II from SDE III on this question

SDE II picks object storage, names FFmpeg, mentions a CDN, and describes one-resolution playback.
SDE III treats segment-level parallelism as the core transcode insight, picks HLS/DASH for ABR with the manifest as the API boundary, designs CDN as a tiered hot-warm-cold hierarchy with explicit pre-population strategy, names the immutability of segments as the reason cache invalidation vanishes, and treats thumbnails / captions / moderation as parallel async pipelines, not sequential blocking work.
Staff/Principal asks the questions that shape the long-term system: What’s the re-transcoding story when a new codec (AV1, VVC) becomes dominant — do we keep source files forever, and at what storage cost? How do we handle content moderation as a gating pipeline without adding minutes to time-to-playable? They think about cost at scale: at exabyte storage, the tiering policy (warm vs. Glacier-class) is a $100M/year decision, not a configuration option. They surface missing requirements: “if we need per-user watermarking for piracy detection, the segments are no longer shared and the entire CDN model breaks — is that in scope?” They design for operational excellence: when the manifest service is wrong (pointing to a missing segment), how does on-call know which video is affected and how quickly?

The difference: seeing the video as a manifest of immutable segments and the system as a pipeline plus a CDN tree, not as “big files in storage.”