Reference: Common System Design Decisions

What it is

A lookup catalog of the architectural decisions that recur across nearly every system design question. Each entry gives a default choice and the situations that should make you switch — the fast-review companion to the reasoning process in Thinking About Tradeoffs in System Design.

When you care

Use this the night before a loop, or when you’ve named a decision point and want the standard answer at a glance. The framework article teaches how to reason to a choice; this one is what the common choices are, so you can recognize the switch fast.

The trap this guards against: reciting a default without the trigger. An interviewer doesn’t want “use Redis” — they want “cache-aside by default, but because writes dominate here, write-behind.” Read each entry as default → when the situation changes, switch to → why, and say the switch out loud.

Storage engine

How data is modeled and physically stored.

SQL vs NoSQL

• Default: relational (Postgres/MySQL). You need transactions, joins, or the access patterns aren’t fixed yet. Start here unless a specific pressure pushes you off it.
• Known single-key access at scale → key-value / wide-column (DynamoDB, Cassandra). When every query is “by this key” and you’ll never join, the relational overhead buys nothing.
• Write-heavy ingest (time-series, events, logs) → LSM-tree store (Cassandra, Scylla, RocksDB). Sequential-append writes beat B-tree random writes.
• Read-heavy with predictable joins → relational + read replicas + a cache. Scale reads horizontally before reaching for a different data model.
• Common mistake: naming a NoSQL store for “scale” when you still need multi-row transactions — you’ll rebuild them in the app layer badly.

Full detail: SQL vs NoSQL Schema Design, Database Indexing.

Storage architecture

How stored data is distributed, replicated, and kept consistent.

Partitioning strategy

• Default: hash partitioning. Even spread across shards, no hot spots — the safe choice when you only query by key.
• You need range scans (time ranges, alphabetical) → range partitioning. Keeps neighbors together so a scan hits few shards. The cost: risk of a hot range.
• You’ll re-partition as you grow → consistent hashing. Adding or removing a node moves only 1/N of keys instead of reshuffling everything. This is the answer to “how do you add capacity without downtime?”
• Common mistake: one key dominates traffic (hot partition) — salt the key or split it. See NoSQL Hot Key Mitigation.

Full detail: Partitioning Strategies.

Replication strategy

• Default: single-leader (primary-replica). Simple, no write conflicts, easy reasoning. Reads scale via replicas; writes bound by the leader.
• Write throughput exceeds one leader, or you need multi-region writes → multi-leader / leaderless. Buys write scale and locality; costs conflict resolution.
• Availability over freshness → leaderless quorums (Dynamo-style). Tune R + W > N for the read/write balance you want.
• Common mistake: promising strong reads from async replicas — a follower can always lag the leader.

Full detail: Replication Strategies.

Consistency model

• Default for user-facing data: eventual / read-your-writes. Availability and latency matter more than every replica agreeing instantly. A social feed can be seconds stale.
• Money, inventory, uniqueness → strong consistency (linearizable). Anything where a wrong-but-fast answer is worse than a slow one. Pay the coordination latency.
• Multi-region, partition-tolerant → pick per the CAP/PACELC situation. Under a partition you choose availability or consistency; even without one, you choose latency or consistency.
• Common mistake: defaulting to strong consistency everywhere — you pay coordination latency on data that never needed it.

Full detail: Consistency Models.

Performance

How to hit latency and throughput targets without changing the core architecture.

Caching strategy

• Default: cache-aside + TTL. Reads are normal, brief staleness is fine. The app reads cache, falls back to DB on miss, backfills. A short TTL bounds how stale a value can get.
• Read-after-write required → write-through. The write updates cache and DB together, so a reader never sees a value older than their own write. Costs latency on every write.
• Write-heavy, loss-tolerant → write-behind. Buffer writes in cache, flush to DB asynchronously. Fast, but a crash loses the un-flushed window — only for counters, metrics, session data.
• Common mistake: a hot single key (celebrity, flash-sale item) expiring at once — add single-flight or logical expiry to prevent the cache-breakdown stampede.

Full detail: Cache Access and Invalidation Patterns.

Async processing

• Default: synchronous. The caller needs the result now, or the work is cheap. Simplest to reason about and debug.
• Slow, retryable, or fan-out work → async via a queue. Fan-out (notifications, feed writes), anything the user shouldn’t wait on. Decouples caller latency from the work.
• The exactly-once question → at-least-once delivery + idempotent consumers. True exactly-once is expensive; at-least-once plus an idempotency key is the standard answer.
• Common mistake: going async for work the caller actually needs synchronously — you’ve added a queue and eventual consistency for nothing.

Full detail: Queue Delivery Semantics.

Rate limiting

• Default: token bucket. Allows bursts up to the bucket size, then steady refill — matches how APIs actually want to behave.
• Smooth, no bursts → sliding-window counter. Stricter, more even.
• Distributed across nodes → centralized counter in Redis (with Lua for atomicity). Local-only counters let clients exceed the global limit.
• Common mistake: per-node local counters — N nodes means N× the intended global limit.

Full detail: Rate Limiting Algorithms.

Architecture

How services communicate, coordinate, and stay correct across process boundaries.

Distributed transaction pattern

• Default across services: saga (compensations). Each step has an undo; on failure you run the compensations backward. Scales because it holds no cross-service locks.
• You control all participants and can hold locks briefly → 2PC. Stronger atomicity, but a blocked coordinator stalls everyone — only viable in tight, controlled boundaries.
• Single database → a plain ACID transaction. Don’t reach for a distributed protocol when one DB’s transaction already gives you atomicity.
• Common mistake: reaching for 2PC across microservices — the blocked-coordinator failure mode is worse than the saga’s eventual consistency.

Full detail: Distributed Transaction Patterns.

Communication pattern

• Default: synchronous request/response (REST/gRPC). Direct, easy to trace, right when the caller needs an answer to proceed.
• Producer shouldn’t wait on the consumer → asynchronous messaging (queue/log). Decouples services and absorbs bursts; costs eventual consistency and harder tracing.
• One event, many interested services → publish/subscribe. Fan a single event out to independent consumers instead of point-to-point calls.
• Common mistake: synchronous call chains across many services — one slow dependency stalls the whole request and couples their uptimes.

Full detail: API Design Patterns.

Real-time delivery

• Default: WebSocket for true bidirectional (chat, collaboration).
• Server→client only → SSE. Live feed, notifications — simpler than WebSocket when the client doesn’t push.
• Legacy/compatibility → long-poll. The fallback when WebSocket/SSE aren’t available.
• Common mistake: high fan-out (millions of subscribers) pushed directly — put a pub/sub tier between producers and connections instead.

Full detail: Real-Time Connection Patterns.

Reliability

How the system survives partial failure — the decisions almost every design eventually needs.

Retry & backoff

• Default: retry with exponential backoff + jitter. Transient failures (blips, brief overload) recover on their own; backoff avoids hammering a struggling dependency, jitter avoids synchronized retry storms.
• Only retry idempotent operations → otherwise pair with an idempotency key. Retrying a non-idempotent write double-charges, double-ships.
• Common mistake: fixed-interval retries with no cap — they turn one dependency blip into a self-inflicted retry storm.

Idempotency

• Default: idempotency key per client-initiated mutation. The server dedupes on the key, so a retried request applies once. This is what makes at-least-once delivery safe.
• Natural idempotency exists → use it. SET x = v or an upsert keyed on a unique field needs no extra key.
• Common mistake: assuming exactly-once delivery instead — build for at-least-once + idempotency; true exactly-once is a networking myth at the transport layer.

Timeout & circuit breaker

• Default: a timeout on every remote call. No timeout means one hung dependency exhausts your thread/connection pool and cascades.
• Repeated failures to one dependency → circuit breaker. Trip open after a failure threshold, fail fast, probe periodically — stops a dead dependency from dragging you down and gives it room to recover.
• Common mistake: omitting timeouts on internal calls because “it’s the same datacenter” — internal calls hang too, and those are the cascades that take down whole systems.

Dead-letter queue

• Default: a dead-letter queue for messages that fail repeatedly. After N failed processing attempts, move the message aside so one bad (“poison”) message doesn’t block the whole partition.
• Need to recover them → make the DLQ replayable. Fix the bug, then re-drive from the DLQ.
• Common mistake: infinite in-place retries on a poison message — it stalls every message behind it in the partition.

At-a-glance: question → the decision it hinges on

A recall grid for review. Each row is a common question and the one decision it’s really testing — the place to spend your reasoning. Decisions marked with the sections above are cataloged here; the rest (ID generation, windowing, spatial indexing, content addressing) are covered in their walkthroughs and are candidates for future entries.

Related

• Framework: Thinking About Tradeoffs in System Design — the reasoning process that produces these choices: axes → context → cost. Read it first; use this catalog as the lookup.
• Reference: Consistency Models — the deep version of the consistency row above.
• Reference: Cache Access and Invalidation Patterns — the deep version of the caching and reliability rows, including the retry-storm and stampede failure modes.
• Reference: Queue Delivery Semantics — at-least-once, idempotency, and dead-letter handling behind the Reliability section.