What it is
A cache is a fast, small store that sits between a client and a slower source of truth, holding recently- or frequently-accessed data. Useful caching requires picking two things: an access pattern (how reads and writes flow through the cache) and an invalidation strategy (when and how stale entries are removed).
When you care
Caching shows up in almost every system design interview. The trap is naming “we’ll use Redis” and moving on. Interviewers probe the access pattern (what happens on a miss, what happens on a write) and the invalidation story (how stale data is prevented or tolerated) — those are the decisions that actually shape correctness.
Read-side access patterns
| Pattern | Read flow | Miss handling | Good for |
|---|---|---|---|
| Cache-aside (lazy) | App reads cache. On miss, app reads DB, writes cache, returns. | App manages both stores. | General-purpose reads; default choice. |
| Read-through | App reads cache. On miss, cache reads DB, fills itself, returns. | Cache library manages the DB hit. | Uniform access path; offloads miss logic from app. |
| Refresh-ahead | Cache proactively refreshes entries before they expire. | No miss if prediction is right. | Predictable hot keys; latency-critical reads. |
Cache-aside is the default. It’s explicit, easy to reason about, and survives a cache outage — the app falls back to the DB. The cost is duplicated logic at every call site.
Read-through collapses that duplication into the cache layer, which only helps if your cache library natively supports it. A cache outage here is harder to degrade gracefully around.
Refresh-ahead is a latency optimization layered on top of either, not a standalone pattern. Only valuable when you can predict hot keys and the DB read cost is high.
Write-side access patterns
| Pattern | Write flow | Consistency | Good for |
|---|---|---|---|
| Write-through | App writes cache, which writes DB synchronously. | Cache and DB always match. | Read-after-write consistency required. |
| Write-behind (write-back) | App writes cache; cache writes DB asynchronously. | Cache ahead of DB briefly; risk on crash. | Write-heavy workloads where latency matters more than durability. |
| Write-around | App writes DB directly; cache is populated on read miss. | Cache may lag DB on writes. | Write-once-read-rarely data; avoids caching cold data. |
| Cache invalidation on write | App writes DB, then deletes the cache key. | Cache re-populated on next read. | Most cache-aside systems. Simple and correct. |
Write-through is the safest choice when reads must see writes immediately. Pays latency on every write.
Write-behind is fast but trades durability for throughput — if the cache dies before flushing, writes are lost. Rarely the right call outside specific high-write systems (counters, metrics, session data).
Write-around is the default pairing for cache-aside reads. Writes go to the DB; the cache fills naturally on the next read. A small variant — delete on write — explicitly invalidates the cache key on write and is the standard pattern for keeping cache-aside correct.
Invalidation strategies
| Strategy | Mechanism | Tradeoff |
|---|---|---|
| TTL (time-to-live) | Entry expires after N seconds. | Simple; tolerates some staleness for a bounded window. |
| Explicit invalidation | App deletes the key on write. | Correct but requires code at every write site. |
| Write-through / write-around | Cache is updated or invalidated synchronously with the DB write. | Strongest consistency; highest write latency. |
| Pub/sub invalidation | DB or service publishes a change event; cache nodes subscribe and invalidate. | Works across cache clusters; adds infrastructure. |
| Version stamping | Entries keyed by (id, version); new writes produce new keys, old keys age out. | Avoids invalidation entirely; grows key space. |
Eviction policies (when the cache is full)
| Policy | Removes | Good for |
|---|---|---|
| LRU (least recently used) | The entry unused for the longest. | General-purpose; the default in most cache libraries. |
| LFU (least frequently used) | The entry accessed the fewest times. | Workloads with stable hot sets. |
| FIFO | The oldest entry by insertion time. | Rarely the right choice; included for completeness. |
| TTL-only | Whichever entry expired first. | Time-bounded data (sessions, tokens). |
Eviction and invalidation are different. Eviction runs when memory fills; invalidation runs when data changes. A cache can use both.
Failure modes and how to avoid them
Under load, a cache can amplify failure instead of absorbing it — a burst of misses stampedes the database it was meant to protect. Four named scenarios show up constantly in interviews; know the mechanism and the mitigation for each.
| Scenario | What triggers it | Blast radius | Primary mitigation |
|---|---|---|---|
| Cache penetration (穿透) | Reads for keys that don’t exist — miss cache and DB every time | Every bad request hits the DB | Cache the negative result; bloom filter in front |
| Cache breakdown (击穿) | One hot key expires; concurrent readers all miss at once | One key’s traffic stampedes the DB | Mutex/single-flight on rebuild; logical (never-hard) expiry |
| Cache avalanche (雪崩) | Many keys expire together, or the cache tier goes down | Whole read load falls onto the DB | Jittered/staggered TTLs; layered cache + circuit breaker |
| Cache stampede (thundering herd) | A popular miss triggers N identical rebuilds simultaneously | N× redundant DB reads for one key | Request coalescing (single-flight); early recompute |
Cache penetration (穿透)
Requests ask for keys that don’t exist — a scraper hitting random user IDs, or a malicious client probing. Each request misses the cache, misses the DB, and returns nothing, so the cache never fills and every request reaches the DB.
- Cache the miss. Store a short-TTL negative marker (
nullsentinel) so the same nonexistent key doesn’t re-hit the DB. Keep the TTL short so a later-created key isn’t masked for long. - Bloom filter in front. A bloom filter of all valid keys rejects impossible lookups before they touch the cache or DB. Trades a small false-positive rate for cheap rejection of the definitely-absent.
Cache breakdown (击穿)
A single hot key (a celebrity profile, a flash-sale item) expires. In the instant before it’s repopulated, every concurrent reader misses and rushes the DB for the same key.
- Single-flight rebuild. Let only the first misser acquire a per-key mutex and recompute; the rest wait for the result. One DB read instead of thousands.
- Logical expiry. Never hard-expire hot keys. Store an explicit
expires_atinside the value, serve the stale value past it, and refresh asynchronously — the key is never physically absent.
Tip
Why: Breakdown (击穿) is one key; avalanche (雪崩) is many keys at once. The fixes differ accordingly — breakdown is solved per key (mutex, logical expiry), avalanche is solved across keys (TTL jitter, tiered fallback). Naming which one you mean is the interview signal.
Cache avalanche (雪崩)
Either a large set of keys share the same TTL and expire simultaneously, or the cache tier itself fails — and the entire read load lands on the DB at once, often taking it down.
- Jitter the TTLs. Add a random spread (e.g.,
TTL = base ± rand(0, 10%)) so keys expire over a window, not on the same tick. This alone prevents the synchronized-expiry variant. - Layered cache. A local/in-process L1 in front of the shared L2 (Redis) means an L2 outage degrades to L1 hits, not a full DB stampede.
- Circuit breaker + degradation. When DB latency spikes, trip a breaker and shed load — serve stale data or a graceful error rather than pile more reads onto a struggling DB.
- Warm-up. After a cold start or cache flush, pre-load hot keys before taking full traffic instead of letting live requests fault them all in.
Cache stampede (thundering herd)
The general form of breakdown: any popular key that misses triggers many identical, concurrent rebuilds. The fix is the same primitive as breakdown, applied broadly.
- Request coalescing / single-flight. Deduplicate in-flight rebuilds for the same key so one backend read serves all waiters.
- Early recompute. Probabilistically refresh a key before it expires (proportional to how close it is to expiry), so the value is renewed off the hot path rather than during a synchronized miss.
When to pick what
- Default for general reads: cache-aside + delete-on-write + TTL backstop + LRU eviction.
- Read-after-write required: write-through, or cache-aside with synchronous delete-on-write.
- Write-heavy, durability-tolerant: write-behind.
- Multi-node cache coherence: pub/sub invalidation on DB writes.
- Immutable or append-only data (URL shortener codes, event logs): TTL alone is enough; the cache-invalidation problem doesn’t apply.
Cache invalidation is one of the two hard problems in computer science for a reason — the correct strategy depends on the read/write ratio, the consistency requirement, and whether stale data is tolerable. Name the choice and the tradeoff; don’t just name Redis.
Related
- Walkthrough: Designing a URL Shortener — the canonical read-heavy cache-aside example; cache invalidation is trivial because data is immutable.
- Walkthrough: Designing an Ad Click Aggregation System — a system where the cache is a precomputed aggregate, not a read-through layer.
- Walkthrough: Designing a RAG System — discusses cache-aside patterns at the vector index tier.