Walkthrough: Designing a URL Shortener

The problem

You’re asked to design a URL shortener — TinyURL, bit.ly, t.co. The interviewer says something like: “Design a service that takes a long URL and returns a shorter one. When someone visits the short URL, they’re redirected to the original.”

Sounds simple. It isn’t. This is the canonical “read-heavy at internet scale” problem, and the traps are in what you choose to defend — ID generation, caching strategy, and whether you treat analytics as part of the core system or an afterthought.

Important

The core tension: The redirect path is read-heavy at extreme scale (100:1+ read/write), but correctness lives in the write path (no two URLs can share a short code). The architecture is fundamentally a caching problem with an ID-generation correctness problem underneath it.

Below is how I’d walk through this, start to finish, roughly in the order I’d speak the words.

1. Requirements

Start by clarifying scope. First 3–5 minutes. Resist the urge to start drawing.

Questions I’d ask:

Scale. How many URLs created per day? How many reads per create? A typical answer is ~100M creates/day with a ~100:1 read-to-write ratio — the system design primer is a good anchor for these ballpark numbers.
Length. Is there a target length for the short URL? Seven characters of base62 gives ~3.5 trillion combinations — probably enough.
Custom aliases. Can users pick their own short code?
Expiry. Do URLs expire? After how long?
Analytics. Do we track clicks? At what granularity?
Read consistency. Is it okay if a freshly-created URL takes a second or two to resolve everywhere?

The analytics and expiry questions matter because they change the storage model. The consistency question matters because it unlocks a lot of caching.

Say the interviewer confirms: 100M/day creates, ~10B/day reads, custom aliases yes, 5-year default expiry, basic click analytics, eventually consistent reads acceptable.

Functional Requirements

Shorten a long URL to a short code (7-character base62 by default)
Redirect a short code to the original long URL
Support optional custom aliases chosen by the user
Support optional URL expiry (default: 5 years)
Track click analytics (timestamp, referer, country, user agent)

Non-Functional Requirements

Low read latency: p99 redirect < 10 ms
High availability: 99.99% uptime (the redirect path must survive partial failures)
Scale: ~100M URLs stored per day, ~180B total over a 5-year retention window
Read-to-write ratio: ~100:1 (10B reads/day, 100M writes/day)
URLs are immutable after creation — no editing, only deletion/expiry
Eventually consistent reads acceptable: a new short URL may take a few seconds to propagate

2. Capacity Estimate

Brief. Don’t over-engineer — the point is to check your scale intuition.

100M/day ≈ 1,200 writes/sec average, ~10K/sec peak
10B/day ≈ 115K reads/sec average, ~1M/sec peak
Storage: 100M/day × 365 × 5 years ≈ 180B URLs total. At ~500 bytes per record (URL + metadata), that’s ~90 TB over the full retention window.

I’d say out loud: “This tells me two things. One — the read path needs heavy caching. Two — I probably want a KV store, not a relational database, given the record count and access pattern.”

3. API Design

Two endpoints do the work:

POST /api/urls
  body:    { long_url, custom_alias?, expiry? }
  returns: { short_url, short_code }

GET /:short_code
  returns: 301/302 redirect to long_url

Small but important decision: 301 or 302? 301 is permanent and cacheable by browsers — which means analytics under-count. 302 is non-permanent, so every request hits your server. Most URL shorteners use 302 for this exact reason. Worth saying out loud.

Tip

If analytics matter (and they always do in production), 302 is the right default. A 301 is only appropriate for a shortener that never tracks clicks — which is rarely the product requirement. Naming this trade-off unprompted is a strong signal.

4. Data Schema

Two tables conceptually:

urls:
  short_code (PK)
  long_url
  created_at
  expires_at
  created_by

clicks:
  short_code
  timestamp
  referer
  country
  user_agent

Storage choices:

urls table — a KV store like DynamoDB or Cassandra is a natural fit. Point lookups by short_code, no joins, no cross-record transactions. The primary access pattern — “given short_code, return long_url” — is the one thing these stores do best. The foundational reference here is the Dynamo paper; the trade-offs it describes are exactly the ones you’re inheriting.
clicks table — write-heavy and append-only. Cassandra handles this well, or you stream to a warehouse via Kafka (more on this below).

I’d say explicitly: “I’m not using a relational database for the main table. I don’t need joins, I don’t need transactions across records, and this access pattern is the pathological best case for a KV store.”

5. High-Level Architecture

flowchart LR
  Client[Client] --> CDN[CDN edge]
  CDN --> App[App server]
  App --> Redis[(Redis cache)]
  Redis -->|miss| DB[(KV store)]
  App --> IDGen[ID allocator]
  App -.->|click event| Kafka[(Kafka)]
  Kafka --> Consumer[Click consumer]
  Consumer --> Analytics[(Analytics DB)]

The CDN handles the bulk of redirect traffic. The app server handles creates and cache misses. The ID allocator coordinates short-code generation. Kafka decouples analytics from the critical redirect path.

6. Detailed Workflows

6a. Write path: Shorten a URL, step by step

Client sends POST /api/urls to the CDN/load balancer with body { long_url, custom_alias?, expiry? }.
Load balancer routes the request to an available app server. (Write requests bypass the CDN cache.)
App server validates the request — checks long_url is a valid URL, custom_alias (if provided) passes the namespace rules (e.g., must be ≥8 chars to avoid collision with generated 7-char codes), and the user is authenticated.
App server requests a short code. For generated codes: it draws the next ID from its in-memory batch (pre-allocated from the ID allocator). For custom aliases: it checks the urls KV store to confirm the alias is not already taken.
App server writes the record to the KV store: { short_code, long_url, created_at, expires_at, created_by }. This write uses quorum consistency (majority of replicas confirm) to ensure durability.
App server writes the short code to the Redis cache proactively (write-through warm): short_code → long_url with TTL slightly shorter than expires_at. This eliminates a cold-miss on the first redirect.
App server returns { short_url, short_code } to the client. The short URL is live within seconds as the cache warms across regions via async replication.

Custom aliases follow the same flow except step 4 checks the KV store for existence and rejects with 409 Conflict if taken.

6b. Read path: Redirect, step by step

Client sends GET /abc1234 (the short code) — typically by clicking a link or pasting into a browser.
CDN edge receives the request and checks its own cache for the abc1234 redirect response. Cache hit (most popular links): CDN returns a 302 redirect immediately. The app server never sees the request. For viral links, this is >99% of traffic.
CDN miss: CDN forwards the request to an app server.
App server queries Redis for short_code → long_url. Cache hit: return 302 redirect immediately, updating the CDN cache for future requests.
Redis miss: app server queries the KV store (DynamoDB/Cassandra) by short_code. DB hit: write the result back to Redis (cache-aside), return 302 redirect with Location: long_url. DB miss: the code does not exist or has expired — return 404.
App server fires a click event to Kafka asynchronously (fire-and-forget): { short_code, timestamp, referer, country, user_agent }. This does not block the redirect response — the 302 is returned to the client before Kafka acknowledges the event.
Client browser follows the 302 redirect to the original long URL.

7. Deep Dive: ID Generation

This is the question. It’s where SDE II and SDE III answers diverge.

Important

The ID generation choice is the core correctness question. Any scheme that requires “check then write” under concurrent writes has a TOCTOU race: two servers can simultaneously check and find no collision, then both write — producing a duplicate short code. The SDE III answer eliminates this by construction, not by hoping retries resolve it.

Three common approaches:

(a) Hash the long URL. Take MD5 or SHA-1, base62-encode the first seven characters.

Pros: Stateless, no coordination needed.
Cons: Collisions. You have to check the DB before writing; on collision, probe (append a char, rehash, etc.). This becomes a correctness problem under concurrent writes.

(b) Random base62. Generate seven random base62 chars, check the DB for collision, retry on conflict.

Pros: Simple. Write path is one read + one write.
Cons: Collision rate rises as the keyspace fills. Manageable at low fill, painful later.

(c) Distributed counter. A monotonically increasing integer, base62-encoded.

Pros: No collisions by construction. Clean writes.
Cons: Need a coordinator. A single counter in a single DB is a bottleneck and a SPOF.

Note

Interview signal: Walking through all three approaches and dismissing the first two with specific failure modes is the move. Candidates who jump straight to “I’d use a counter” without naming why hash-based and random approaches fail under concurrency miss the point of the question.

The SDE III answer is: counter-based, with range allocation. Each application server requests a batch of 1,000 IDs from a central allocator (ZooKeeper, a dedicated service, or a DB with SELECT ... FOR UPDATE). The server burns through its batch locally with zero coordination, then asks for another. This is essentially Twitter’s Snowflake approach without the embedded timestamp.

Out-of-order IDs are fine here — short codes don’t need to be sequential, they need to be unique. Range allocation gives you 10K+ writes/sec per app server with trivial load on the coordinator.

For custom aliases: they’re a separate write path that checks the existing index, with a reserved-namespace convention to avoid collisions with the generated codes (for example: custom aliases must be ≥8 chars, or must start with a capital letter).

8. Deep Dive: Caching

At 1M reads/sec peak, no database serves this directly. Caching isn’t an optimization — it’s the architecture.

A few layers worth naming:

CDN at the edge. If you cache the 302 redirect response itself at the CDN, the app server never sees the request. For popular links (think: viral tweet), this is 99%+ of traffic.
Redis in front of the DB. Use the cache-aside pattern: on miss, read DB, write cache, return. TTL of a few hours.
DB as fallback. Only cold or recently-created URLs hit it.

Cache size estimate. Hot working set is roughly 1% of total URLs — call it 2B records. At ~200 bytes per cached entry (just short_code → long_url, no metadata), that’s ~400 GB of Redis. Multi-node cluster, consistent-hashed.

Cache invalidation. URLs are mostly immutable after creation. The only mutation is deletion or expiry. A TTL slightly shorter than the actual expiry handles this cleanly — no clever invalidation protocol needed. (One of the few times the cache-invalidation problem vanishes instead of dominating.)

Tip

Immutability is a superpower in distributed systems — it eliminates the cache invalidation problem entirely for the hot read path. The interviewer is testing whether you recognize this and exploit it, rather than over-engineering a cache-write-back protocol for a system that doesn’t need one.

9. Deep Dive: Sharding and Multi-Region

The urls table is sharded by short_code. With DynamoDB this is automatic; with Cassandra you partition on short_code. Hot-partition risk is low because short codes are effectively random.

The Redis cluster is sharded via consistent hashing on short_code — standard. Any decent Redis client library does this.

Multi-region. If the service is global, I’d add regional read replicas of the DB and per-region Redis clusters. Writes can go to a primary region (simpler) or be replicated async (more failure modes, but lower write latency). For a URL shortener where reads can be eventually consistent, the trade-off heavily favors async regional replication.

Extending the architecture:

flowchart LR
  Client[Client] --> CDN[CDN edge]
  CDN --> App[App server]
  App --> Redis[(Redis · us-east)]
  Redis -->|miss| DB[(KV store · us-east)]

  DB -.->|async replication| DB2[(KV store · us-west)]
  Redis2[(Redis · us-west)] -->|miss| DB2
  CDN --> App2[App server · us-west]
  App2 --> Redis2

  classDef new fill:#eef2f1,stroke:#2c5f5d,stroke-width:2px,color:#1f2937;
  class DB2,Redis2,App2 new

The CDN routes each client to its nearest region. Writes land in us-east (primary); replicas in us-west are eventually consistent — acceptable given the product already tolerates a short consistency delay on create.

10. Deep Dive: Analytics Pipeline

The click path is the second-largest system hidden inside this problem. An interviewer noticing that you treat it as a peer to the core redirect path — rather than an afterthought — is a strong SDE III signal.

Naive design: every click writes a row to the DB. At 1M writes/sec peak, this is catastrophic for the main system.

Better: every click emits a message to Kafka. A downstream consumer batches and writes to the analytics store (Cassandra, or a columnar warehouse). The redirect itself doesn’t wait on any of this.

Extending the architecture one more time:

flowchart LR
  Client[Client] --> CDN[CDN edge]
  CDN --> App[App server]
  App --> Redis[(Redis · us-east)]
  Redis -->|miss| DB[(KV store · us-east)]

  DB -.->|async replication| DB2[(KV store · us-west)]
  Redis2[(Redis · us-west)] -->|miss| DB2
  CDN --> App2[App server · us-west]
  App2 --> Redis2

  App -.->|click event| Kafka[(Kafka)]
  App2 -.->|click event| Kafka
  Kafka --> Consumer[Click consumer]
  Consumer --> Analytics[(Analytics DB)]

  classDef new fill:#eef2f1,stroke:#2c5f5d,stroke-width:2px,color:#1f2937;
  class Kafka,Consumer,Analytics new

The redirect response is returned to the client immediately — the click event is fire-and-forget from the app server’s perspective.

Two wins from this shape:

Redirect latency is decoupled from analytics ingestion.
You can reprocess the stream (e.g., for backfilling a new aggregation) by replaying from Kafka.

For querying: if the product shows “clicks in the last 24 hours” on the creator’s dashboard, maintain a precomputed aggregate (Redis counter or a ClickHouse rollup). Don’t scan the raw clicks table on every dashboard load.

11. Deep Dive: Rate Limiting

For the create endpoint: a token bucket per API key or per IP, Redis-backed, keyed on user_id. This belongs to an API gateway layer, not the URL service itself — worth saying out loud so the interviewer knows you understand separation of concerns.

For the redirect endpoint: usually not rate-limited per user, but globally protected at the CDN and edge (DDoS protection).

12. Failure Modes

I’d proactively walk through what breaks, because this is one of the clearest differentiation signals. Don’t wait for the interviewer to ask.

ID allocator is down. App servers keep serving from their in-memory batch (~1,000 IDs) until exhaustion. If the allocator stays down long enough, writes fail. Mitigations: multiple allocator instances, fallback to hash-based IDs with collision retry.
Redis is down. Thundering herd onto the DB. Mitigations: request coalescing (only one miss per key triggers a DB read at a time), circuit breakers — see Martin Fowler on the circuit breaker pattern — and gradual cache warming from a secondary tier.

Caution

A cache-stampede scenario (Redis down + 1M req/sec hitting the DB simultaneously) is the most likely severe failure for this system. Without request coalescing, one cache outage can cascade into a DB outage. Name this failure mode and the mitigation explicitly — most candidates don’t.

DB partition lost. If using DynamoDB or Cassandra with quorum replication, a minority-partition loss is survived automatically. Say this explicitly: it’s where the KV choice earns its keep.
Kafka is down. Redirects still work — we drop click events or buffer them locally and retry. The product degrades gracefully: losing some analytics is strictly better than losing redirects.

The pattern to notice: name what fails, name what degrades gracefully, name what doesn’t.

13. What I’d skip, and say I’m skipping

Time check: five minutes left. Things I’d explicitly defer:

Link previews and phishing detection. A real shortener needs this, but it’s a separate system — URL classification pipeline, probably ML-based. Worth a sentence, not a slide.
User management. Standard OAuth/JWT, not interesting here.
Billing and quotas beyond rate limiting. Same.
A/B redirect rules or geo-routing per link. Features, not infrastructure.

Saying “I’d skip this, and here’s why” is a strong SDE III signal. It shows you know the full surface and are making deliberate scoping choices, not just running out of things to talk about.

14. Wrap-up

One crisp sentence before the interviewer’s next question:

This design optimizes for read throughput and availability at the cost of a short consistency delay on create. That trade-off matches the product — nobody cares if their new short URL takes two seconds to become globally visible.

That’s the kind of articulation that lands.

What separates SDE II from SDE III on this question

SDE II usually lands the API, picks a reasonable ID scheme, names Redis and a KV store, and sketches the read path.
SDE III drives scoping in the first five minutes, picks ID allocation with a specific rationale, walks through the analytics pipeline as a peer system, names three or more failure modes with mitigations, and explicitly defers features that don’t belong in the core design.
Staff/Principal frames the design as a portfolio of trade-offs: immutability enabling aggressive caching vs. the correctness constraint on ID generation; analytics decoupling via Kafka vs. added operational surface. They ask about operational realities — what does cache stampede look like in production, what’s the runbook when Redis goes down at 2am, how does the on-call team know which ID range was corrupted? They identify missing requirements that would materially change the architecture (e.g., “if custom aliases need to be globally consistent within 100ms, the eventually-consistent replication model breaks and we need synchronous cross-region writes”). They reason about build vs. buy (ZooKeeper for ID allocation vs. a dedicated ID service vs. database sequences) with total cost of ownership, not just technical correctness.

The difference isn’t knowledge. It’s which levers you pull and how you justify pulling them.