System Design Interview Cheatsheet

A ten-minute review before the interview: back-of-the-envelope numbers and formulas, the building blocks interviewers expect, frameworks for the classic design questions, database selection paths and a minute-by-minute interview template. Each entry gives the problem, the key points, API / schema / code sketches, real systems and trade-offs.

Updated: 2026-10-08

22 entries

Quick tables

Reference and comparison tables for quick scanning.

Latency numbers worth memorizing

Jeff Dean / Peter Norvig numbers (the classic ~2010 list), rounded. Colin Scott's interactive "Latency Numbers Every Programmer Should Know" extrapolates newer hardware (for 2020 it shows, for example, SSD random read ≈ 16 µs and disk seek ≈ 2 ms). All are order of magnitude: scale by hardware, and remember the ratios matter more than the absolutes.

Latency numbers worth memorizing
OperationLatencyRule of thumb
L1 cache reference~0.5-1 nsEffectively free; the baseline for everything else
L2 cache reference~4-7 nsNorvig lists 7 ns; ~4 ns is a common modern figure. About 10× L1
Mutex lock/unlock~25 nsUncontended; contention costs far more (microseconds)
Main memory reference~100 nsAbout 100× L1: why cache locality dominates CPU-bound code
Compress 1 KB~µs scale (Norvig: 3 µs with Zippy)Compression is cheap relative to a network hop
Read 1 MB sequentially from RAM~250 µsRoughly 4 GB/s in the original figures; modern RAM is faster
SSD random read (4 KB)~100 µs (NVMe ~10-100 µs; Norvig: 150 µs)~1,000× slower than RAM, ~100× faster than a disk seek
Round trip within same datacenter~0.5 msA remote call costs thousands of memory reads: limit fan-out per request
Read 1 MB sequentially from SSD~1 msAbout 1 GB/s in the original figures; NVMe is several times faster
Disk seek / random read (HDD)~10 ms~100 random IOPS per spindle; avoid random access on spinning disks
Read 1 MB sequentially from HDD~20 msSequential is worth ~100× random: why logs and LSM trees append
Round trip California to Netherlands~150 msSpeed-of-light bound; cross-continent calls need regional replicas or CDNs

Source and choice: the classic Dean / Norvig list for the 2010-era values (L1 0.5 ns, L2 7 ns, mutex 25 ns, memory 100 ns, SSD 150 µs, datacenter RTT 500 µs, disk seek 10 ms, CA to NL 150 ms); where modern figures differ, a range is shown. Use for ratios, not benchmarks.

Size and throughput cheat values

Decimal, rounded values for quick arithmetic; state your assumption when the real figure varies (photos, video).

Size and throughput cheat values
ItemTypical sizeNote
1 ASCII character1 BUTF-8 encodes ASCII in one byte
1 CJK character (UTF-8)3 BMost CJK characters use 3 bytes in UTF-8
int64 / timestamp8 BUnix time in ms fits in 8 B
UUID16 B binary (36 chars as text)Store as binary where possible: 2× smaller
Tweet / short text post~300 B-1 KBText up to ~280 chars (~280 B ASCII, ~840 B CJK) plus metadata
Row with a few columns~100 B-1 KBPlus index overhead; 1 KB is a safe default
Photo~300 KB (thumbnail-ish) to 2-5 MB (original)Assumption: ~300 KB for a compressed web photo; pick one and say it
1 min of 720p video~30-60 MB~5 Mbps ÷ 8 × 60 s ≈ 37 MB/min; use ~40 MB or the higher figure for a safer upper bound
1 min of MP3 at 128 kbps~1 MB128,000 ÷ 8 × 60 = 0.96 MB
Log line~100-200 BStructured JSON lines are often larger, 300 B-1 KB
1 M users × 1 KB1 GB10^6 × 10^3 = 10^9 B
100 M users × 1 KB100 GB10^8 × 10^3 = 10^11 B; fits on one disk
1 B users × 1 KB1 TB10^9 × 10^3 = 10^12 B; replicate ×3 for 3 TB

Video size depends on codec, resolution and bitrate; 720p is commonly encoded anywhere from ~1.5 to ~5 Mbps, so treat 30-60 MB/min as the upper range and state your bitrate.

Capacity formulas

Capacity formulas
QuantityFormulaExample
Average QPSrequests per day ÷ 86,40010^8 / day ÷ 86,400 ≈ 1.2 K QPS
Peak QPSaverage QPS × peak factor (2-5, state it)1.2 K × 3 ≈ 3.5 K QPS
Read QPS from write QPSread QPS = write QPS × read:write ratio100 writes/s at 100:1 → 10 K reads/s
Storage per yearrecords/day × bytes × 365 (× replicas)10 M × 1 KB × 365 ≈ 3.7 TB; ×3 ≈ 11 TB
Replication overheadraw storage × replication factor (usually 3)3.7 TB × 3 ≈ 11 TB
Bandwidth (in / out)QPS × payload bytes × 8 for bits/s3.5 K QPS × 100 KB = 350 MB/s ≈ 2.8 Gbps egress
Cache memory (80/20)daily read working set × 0.2100 M reads/day × 1 KB = 100 GB; 20% ≈ 20 GB
Servers neededpeak QPS ÷ per-node QPS ÷ target utilization (0.5-0.7)3.5 K ÷ 1 K ÷ 0.6 ≈ 6 nodes
DB shardsmax(storage ÷ per-node capacity, peak QPS ÷ per-node QPS)11 TB ÷ 2 TB = 6; 3.5 K ÷ 5 K = 1 → 6 shards

Per-node QPS and capacity (1 K QPS, 5 K QPS, 2 TB above) are placeholder assumptions: replace them with the numbers you state for the workload. Running at 50-70% utilization leaves headroom for failures and bursts.

Availability and downtime

Allowed downtime for a given availability, using a 365-day year and a 30-day month.

Availability and downtime
AvailabilityDowntime / yearDowntime / monthTypical meaning
99%3.65 days (87.6 h)7.2 hInternal tools, batch systems; one bad afternoon a month is acceptable
99.9% (three nines)8.76 h43.2 minTypical SLA for ordinary web services; needs monitoring and quick rollback
99.95%4.38 h21.6 minCommon cloud-service SLA tier; multi-AZ with automated failover
99.99% (four nines)52.6 min4.32 minCore user-facing and payment paths; multi-AZ, no manual steps in failover
99.999% (five nines)5.26 min25.9 sTelecom-grade; usually multi-region and very expensive. Often not promised end to end

Serial dependencies multiply: three 99.9% services in a chain give 0.999³ ≈ 99.7% (~26 h/year). Independent parallel replicas give 1 - (1 - a)^N: two 99% replicas ≈ 99.99%. This assumes independent failures and working failover.

Rate limiting algorithms compared

Rate limiting algorithms compared
AlgorithmBurst behaviourMemoryAccuracyUse when
Token bucketAllows bursts up to bucket size, then the refill rateO(1) per key: tokens + last refill timeGood; exact on average rateDefault for public APIs where short bursts are fine
Leaky bucketNo bursts: output is smoothed to a constant rate (queue absorbs spikes)O(1) per key, plus a queue if requests are bufferedGood; strict output rateProtecting a backend that needs a steady rate
Fixed windowUp to 2x the limit across a window boundaryO(1) per key: one counterPoor at boundariesCheap coarse quotas (per day, per minute) where edge errors are fine
Sliding window logNo boundary burstO(N) per key: one timestamp per requestExactLow limits where exactness matters and N is small
Sliding window counterSmooth; approximates the sliding windowO(1) per key: two countersApproximate (assumes even spread in the previous window)High-volume traffic that needs smooth limits at low cost

In an interview: name token bucket as the default, say why (bursts allowed, O(1) state), and mention the sliding window counter as the cheap smooth alternative.

Kafka vs RabbitMQ vs SQS

Kafka vs RabbitMQ vs SQS
DimensionKafkaRabbitMQSQS
ModelPartitioned append-only log; consumers track offsetsBroker with exchanges, bindings and queues; messages are removed on ackManaged queue; messages are deleted after processing
OrderingPer partition (choose the key)Per queue, weakened by multiple consumers and requeuesStandard: best effort; FIFO: per message group
Retention / replayTime or size based (or compacted); replay by resetting offsetsClassic and quorum queues drop on ack; streams support replayNo replay; retention 1 minute to 14 days (default 4 days)
ThroughputVery high: scales with partitions, sequential disk I/OModerate: tens of thousands of msg/s per node is a common rangeStandard: nearly unlimited; FIFO: 300 API calls/s per action by default, 3,000 msg/s with batching, more in high-throughput mode
Delivery semanticsAt-least-once by default; exactly-once within Kafka via transactionsAt-least-once with acks; at-most-once withoutStandard: at-least-once; FIFO: deduplication within 5 minutes
RoutingTopic and partition only; filtering is the consumer's jobRich: direct, topic, fanout and headers exchanges, priorities, TTLSimple: one queue per consumer group; fan-out via SNS or EventBridge
Ops burdenHigh: brokers, partitions, rebalancing (managed offerings exist)Medium: clustering and queue mirroring/quorum tuningNone: fully managed, pay per request
Typical useEvent streaming, log pipelines, CDC, stream processing, replayable historyTask queues with routing, RPC-style workflows, low-to-medium volumeSimple work queues and buffering on AWS with no servers to run

SQS limits checked against AWS docs: max message size 1 MiB, visibility timeout 0 s to 12 h (default 30 s). Quotas change, so re-check before quoting exact numbers.

Cache write strategies

Cache write strategies
StrategyRead pathWrite pathConsistencyRiskUse when
Cache-asideApp checks cache; on miss reads DB and fills cacheApp writes DB, then deletes (or updates) the cache keyEventual; stale window between write and invalidateStampede on hot misses; races can leave stale dataGeneral default for read-heavy data
Read-throughCache itself loads from DB on a missUsually paired with another write strategySame as cache-asideCache library must know how to load; couplingYou want loading logic in one place
Write-throughReads hit a cache that is always freshWrite goes to cache and DB synchronouslyStrong between cache and DB for written keysHigher write latency; caches data that may never be readRead-after-write matters and writes are modest
Write-behindReads hit cacheWrite goes to cache; DB is updated asynchronously in batchesEventual; DB lags behindData loss if the cache dies before flushWrite-heavy, loss-tolerant or coalescable updates (counters)
Write-aroundMiss on first read, then loaded into cacheWrite goes straight to DB, bypassing cacheCache may be stale until TTL or invalidationRecently written data is a guaranteed missWritten data is rarely read soon after (logs, uploads)

Saga vs 2PC vs TCC

Saga vs 2PC vs TCC
DimensionSaga2PCTCC
IsolationNone: intermediate states are visibleLocks held until commit, so strong isolationResources are reserved in Try, giving business-level isolation
ConsistencyEventual, via compensationsAtomic across participantsEventual, via Confirm or Cancel
BlockingNo global locksYes: participants block if the coordinator fails after prepareNo DB locks across the call, but reserved resources are held
ComplexityMedium: design compensations and idempotencyLow in code, high in operationsHigh: three methods per service plus anti-hanging logic
RollbackCompensating actions (not a true undo)Automatic abort by the resource managersExplicit Cancel releases the reservation
LatencyLow per step, long end to endTwo round trips plus lock waitTwo phases, but each is short
InvasivenessMedium: each step needs a compensationLow if every participant supports XAHigh: business code is split into Try/Confirm/Cancel
Typical useLong business flows across services (orders, bookings)Few homogeneous databases inside one trust domainMoney or inventory that must be reserved (payments)

Which database for what

Conservative one-line positioning. Most products run fine on the first row until a measured need says otherwise.

Which database for what
DatabaseModelBest forAvoid whenScalingConsistency
PostgreSQLRelational (rich types, JSONB)Default OLTP: transactions, joins, constraints, mixed workloadsSustained write volume beyond one primary with no sharding appetite; huge scans for analyticsVertical, read replicas, partitioning; sharding via extensions or app layerACID; single primary; replicas asynchronous by default
MySQLRelational (InnoDB)OLTP with simple, predictable queries; mature replication ecosystemComplex analytical SQL; heavy reliance on advanced types or constraintsVertical, read replicas, application or proxy-based shardingACID; single primary; asynchronous or semi-synchronous replicas
MongoDBDocument (BSON)Aggregate-shaped data with evolving schema, nested attributes, catalogsMany cross-document joins and invariants; data that is naturally tabularReplica sets plus native sharding by a shard keyTunable: write concern and read concern; multi-document transactions exist but cost more
RedisIn-memory data structuresCache, sessions, counters, leaderboards, rate limits, queuesPrimary store for data that must be durable and larger than RAMReplicas for reads; Cluster shards by hash slotAsynchronous replication; acknowledged writes can be lost on failover
ClickHouseColumnar OLAPAggregations over billions of append-only rows: events, logs, metricsFrequent single-row updates or deletes; point lookups; transactional workloadsShards and replicas; add nodes for scan throughputEventual across replicas; no multi-row ACID transactions
CassandraWide column, LSM-basedVery high write rate, key-based access, multi-datacenter availabilityAd hoc queries, joins, secondary-index heavy access, strong consistencyHorizontal by partition key over a ring; add nodes onlineTunable per query (`ONE`, `QUORUM`, `ALL`); eventual by default
DynamoDBManaged key-value / documentPredictable key access at any scale with little operations workUnknown access patterns; analytics; cost-sensitive heavy scansAutomatic partitioning by partition key; capacity modesEventually consistent reads by default; strongly consistent reads optional per read
Elasticsearch / OpenSearchInverted-index search engineFull-text search, relevance ranking, faceting, log searchSystem of record; strong transactional guaranteesShards and replicas per index; scale by shard countNear-real-time visibility; eventual relative to the source of truth

Positioning, not benchmarks: every row has deployments that stretch it. Say the default, then the measured trigger that would make you leave it.

If ... then ...: decision paths

If ... then ...: decision paths
If your situation isStart withBecauseRevisit when
Orders, payments, inventory with multi-row invariantsPostgreSQL (or MySQL)Transactions, constraints and joins enforce the invariants for youOne primary can no longer absorb writes; then shard by tenant or user
Session store, counters, leaderboards, rate limitsRedisIn-memory structures (hashes, sorted sets, `INCR`) give sub-millisecond accessThe data must survive loss or exceeds RAM; keep the source of truth elsewhere
Append-only event log analytics over billions of rowsClickHouseColumnar storage and vectorised scans make aggregations cheapYou need row-level updates or low-latency point reads
Write-heavy time series with predictable key accessCassandra (or a time-series store)LSM writes are cheap and partitioning by series key spreads loadQueries become ad hoc, or the volume is small enough for PostgreSQL with partitioning
Full-text search over documentsElasticsearch / OpenSearchInverted index, tokenisation and relevance scoring are built inSearch is simple enough for PostgreSQL full-text; never use the index as the source of truth
Flexible catalog with nested, varying attributesMongoDB (or PostgreSQL with JSONB)The document is the unit of read and write; the schema changes oftenCross-document invariants or heavy joins appear
Global low-latency writes in several regionsDynamoDB global tables or CassandraMulti-region replication with local writes, trading away strict consistencyConflicts matter to the business; define merge rules or pick one write region
Simple key lookup at huge scale, minimal opsDynamoDBManaged partitioning and predictable latency per keyAccess patterns multiply or cost grows with scans
Small team, early product, unclear requirementsPostgreSQLOne system that does relational, JSON, text search and queues well enoughA measured bottleneck points at a specific workload
Relationship traversal is the query (friends of friends, paths)PostgreSQL with recursive queries first; a graph database if depth is largeMost graph needs are shallow and fit a join tableTraversals deeper than a few hops dominate latency
Dashboards need fresh numbers and the OLTP database is slowReplica or CDC feed into ClickHouseIsolates analytical scans from the transactional primaryStaleness requirements tighten to seconds on the primary data
Hot reads dominate and the data changes rarelyCache (Redis) in front of the existing databaseCheapest way to cut read load without changing the storeInvalidation bugs or hit ratio make the cache a liability

Estimation

The seven-step estimate for any design question: DAU to QPS, peak, read:write, storage, bandwidth, cache memory, then a sanity check against one machine.

The problem

The interviewer says "design X for 100 M users" and wants to see whether you can turn that into numbers that drive decisions: one box or a cluster, SQL or sharded store, cache or no cache. The goal is not a precise answer but a defensible order of magnitude, reached by explicit assumptions and round numbers.

How it works

  • Step 1, traffic: state users, then DAU (often 10-30% of MAU unless told), then actions per user per day. Total requests/day = DAU × actions.
  • Step 2, QPS: average QPS = requests per day ÷ 86,400 (round to 10^5). Peak QPS = average × a factor you state out loud, typically 2-5 (×3 is a fine default; use more for spiky events like sales or live matches).
  • Step 3, read:write ratio: social feeds and URL redirects are read-heavy (10:1 to 100:1 or more); logging and metrics ingestion are write-heavy. Derive read QPS and write QPS separately, because they scale differently.
  • Step 4, storage: records/day × bytes per record × retention days × replication factor (usually 3), plus 20-50% for indexes and growth. Do blobs (images, video) and metadata separately.
  • Step 5, bandwidth: QPS × payload size, for ingress and egress separately. Egress is usually reads × response size and dominates; convert bytes to bits (×8) before comparing with NIC speeds.
  • Step 6, cache memory: working set per day × hot fraction (the 80/20 rule: cache the hottest ~20% to serve most reads). Check it fits in RAM of a handful of nodes.
  • Step 7, sanity check: compare peak QPS with what one node can do (thousands to tens of thousands of simple QPS for a DB or app server, order of magnitude). Nodes ≈ peak QPS ÷ per-node QPS ÷ target utilization (0.5-0.7). Take the larger of the QPS-driven and storage-driven shard counts.
  • Worked example, traffic: DAU 10 M × 10 requests/day = 10^8/day; 10^8 ÷ 86,400 ≈ 1,160 QPS average (≈1.2K); ×3 peak ≈ 3,500 QPS (≈3.6K if you round to 1.2K first). One decent node handles this, so you shard for storage or availability, not QPS.
  • Worked example, storage: 100 M users × 1 KB profile = 100 GB, fits on one disk; with 3 replicas 300 GB. 10 M new 1 KB posts/day × 365 × 3 replicas ≈ 11 TB/year, which is when sharding starts to matter.

Examples

Go
import "fmt"

const secondsPerDay = 86_400

// Assumptions are explicit fields: change one, re-run, and the answer moves.
type Assumptions struct {
    DAU              float64 // daily active users
    ActionsPerUser   float64 // requests per user per day
    PeakFactor       float64 // peak QPS / average QPS, usually 2-5
    WritesPerAction  float64 // records stored per action
    RecordBytes      float64 // bytes per record
    RetentionDays    float64 // how long data is kept
    ReplicationCount float64 // physical copies, typically 3
}

type Estimate struct {
    AvgQPS       float64
    PeakQPS      float64
    StorageBytes float64
}

func avgQPS(perDay float64) float64 {
    return perDay / secondsPerDay
}

func peakQPS(avg, factor float64) float64 {
    return avg * factor
}

func storageBytes(recordsPerDay, recordBytes, days, replicas float64) float64 {
    return recordsPerDay * recordBytes * days * replicas
}

func estimate(a Assumptions) Estimate {
    perDay := a.DAU * a.ActionsPerUser
    avg := avgQPS(perDay)
    return Estimate{
        AvgQPS:       avg,
        PeakQPS:      peakQPS(avg, a.PeakFactor),
        StorageBytes: storageBytes(perDay*a.WritesPerAction, a.RecordBytes, a.RetentionDays, a.ReplicationCount),
    }
}

var examples = []struct {
    name string
    in   Assumptions
}{
    {"qps-example", Assumptions{DAU: 10e6, ActionsPerUser: 10, PeakFactor: 3}},
    {"posts-1y", Assumptions{DAU: 10e6, ActionsPerUser: 1, PeakFactor: 3, WritesPerAction: 1, RecordBytes: 1000, RetentionDays: 365, ReplicationCount: 3}},
}

func report() []string {
    var out []string
    for _, ex := range examples {
        e := estimate(ex.in)
        out = append(out, fmt.Sprintf("%s: avg=%.0f peak=%.0f storage=%.1fTB", ex.name, e.AvgQPS, e.PeakQPS, e.StorageBytes/1e12))
    }
    return out
}

The worked examples as integer shell arithmetic (no bc needed).

bash
#!/usr/bin/env bash
set -eu

# Example 1: DAU 10 M, 10 requests/user/day
dau=10000000
per_user=10
per_day=$((dau * per_user))        # 100,000,000 requests/day
avg=$((per_day / 86400))           # 1157 QPS, say ~1.2K
peak=$((avg * 3))                  # 3471 QPS, say ~3.5K (x3 assumed)
echo "avg=${avg} QPS peak=${peak} QPS"

# Example 2: 100 M users x 1 KB each
users=100000000
bytes_each=1000
gb=$((users * bytes_each / 1000000000))
echo "profile storage=${gb} GB"

# Example 3: 10 M posts/day x 1 KB x 365 days x 3 replicas
gb=$((10000000 * 1000 * 365 * 3 / 1000000000))   # 10950 GB
echo "posts for one year, replicated: ${gb} GB (~11 TB)"

In practice

  • Interview tip: say assumptions out loud — Write each assumption on the board ("DAU 10 M, 10 requests each, peak ×3") so the interviewer can correct it early; a wrong assumption stated is cheap, a silent one is fatal.
  • Interview tip: round aggressively — Use 86,400 ≈ 10^5, 1 KB = 1,000 B, 1 year ≈ 3×10^7 s. Being within 2× is enough, and clean numbers make the arithmetic visible and fast.
  • Interview tip: think in powers of ten — Compare to thresholds, not exact values: ~10^3 QPS is one node, ~10^5 needs a cluster or cache, ~10^2 GB fits in RAM, ~10^1 TB needs sharding. Stop estimating as soon as the decision is clear.
  • Interview tip: tie numbers to a decision — End each estimate with "so we need ..." (a cache, 10 shards, a CDN). Numbers that change nothing in the design are noise; skip them.

Trade-offs

Pros

  • Turns a vague prompt into numbers that justify each component
  • Cheap: five minutes of arithmetic avoids over- or under-engineering
  • Explicit assumptions make the discussion collaborative

Cons

  • Garbage in, garbage out: the DAU, ratio and payload guesses dominate the result
  • Averages hide hot keys, bursts and fan-out, so peak and skew still need a design answer
  • Over-precise arithmetic wastes interview time

Numbers to have in your head: 86,400 s/day, 1 M/day is about 12 QPS, the 2^10 ladder, availability nines, cache miss math and single-node ballparks.

The problem

Estimation goes fast only if a dozen constants are already in your head. Without them you stall on unit conversions; with them you can turn "1 B requests/day" into "about 12 K QPS" in seconds and judge whether one node, a cache or a cluster is the right answer.

How it works

  • Time: 1 day = 86,400 s ≈ 10^5 s, 1 year ≈ 3×10^7 s. So 1 M requests/day ≈ 12 QPS, 100 M/day ≈ 1.2 K QPS, 1 B/day ≈ 12 K QPS.
  • Size ladder (2^10 ≈ 10^3): KB = 10^3 B, MB = 10^6, GB = 10^9, TB = 10^12, PB = 10^15. 1 M × 1 KB = 1 GB; 1 B × 1 KB = 1 TB. Treating 1 KB as 1,000 B is fine in an interview.
  • Availability nines: 99.9% ≈ 8.8 h/year, 99.99% ≈ 53 min/year, 99.999% ≈ 5.3 min/year. Each extra nine divides the downtime budget by 10 and usually costs far more than 10× in redundancy and operations.
  • Composition: components in series multiply (0.999 × 0.999 × 0.999 ≈ 99.7%), so long call chains lose nines. N independent replicas in parallel give 1 - (1 - a)^N (two 99% replicas ≈ 99.99%), but only if failures are independent and failover works.
  • Cache effect: load on the backing store = QPS × miss rate. 50 K QPS at 90% hit ratio leaves 5 K QPS on the DB; 99% leaves 500. Going from 90% to 99% cuts DB load 10×, which is why hit ratio matters more than cache speed.
  • Read/write skew: read-heavy (e.g. 100:1) means replicas, caches and CDNs; write-heavy means partitioning, batching, append-only logs and queues. State the ratio before choosing storage.
  • Replication factor 3 is the default for durable stores (HDFS, Kafka and Cassandra are commonly run with 3): multiply raw storage by 3, and writes by 3 for network traffic.
  • Connection and thread limits: a thread-per-connection server is bounded by threads and memory (thousands), an event-loop or goroutine server by file descriptors and memory (tens to hundreds of thousands). Databases need pooling: a few hundred direct connections is where many RDBMSs start to suffer.
  • Single-node ballparks, order of magnitude only and workload-dependent: well-tuned Redis tens of thousands to ~100 K simple ops/s; PostgreSQL/MySQL primary thousands to low tens of thousands of simple QPS; a Kafka broker hundreds of MB/s sequential throughput with good disks and network. Use them to decide shard counts, never as promises.

Examples

Three one-liners to internalize: per-day to QPS, cache miss load, downtime per year.

bash
#!/usr/bin/env bash
set -eu

# Requests per day -> average QPS (86400 ~ 10^5 s/day)
for per_day in 1000000 100000000 1000000000; do
    echo "${per_day}/day = $((per_day / 86400)) QPS"
done

# Origin load behind a cache: load = QPS x (1 - hit ratio)
qps=50000
for hit in 80 90 99; do
    echo "hit ${hit}% -> origin $((qps * (100 - hit) / 100)) QPS"
done

# Downtime budget per year for N nines, in minutes
awk 'BEGIN { split("99 99.9 99.99 99.999", a, " "); for (i = 1; i <= 4; i++) printf "%s%% -> %.1f min/year\n", a[i], (100 - a[i]) / 100 * 365 * 24 * 60 }'

In practice

  • Interview tip: derive, do not recite — Interviewers prefer "86,400 ≈ 10^5, so 1 M/day ≈ 10 QPS" over a memorized table. Showing the derivation lets them trust the rest of your numbers.
  • Latency numbers (Dean / Norvig) — The classic list (memory ~100 ns, datacenter round trip ~0.5 ms, cross-continent ~150 ms) dates from around 2010 and should be scaled by hardware. Use the ratios, not the absolute values; see the latency table above.
  • Redis, PostgreSQL, Kafka — Vendor docs and benchmarks show wide ranges depending on payload, pipelining, hardware and durability settings. Quote ranges and say "order of magnitude" rather than a single figure.

Trade-offs

Pros

  • A small set of constants covers most interview estimates
  • Ratios (hit rate, nines, replication) transfer across systems

Cons

  • Single-node ballparks vary by 10× with workload, hardware and tuning
  • Parallel availability math assumes independent failures, which shared dependencies break
  • Memorized numbers go stale as hardware changes

Design patterns

Pick token bucket, leaky bucket or a window counter, decide where to enforce and what to key by, and say what happens when the limiter itself fails.

The problem

The interviewer says "design a rate limiter" or "protect this API from abuse". They want a decision, not a survey: which algorithm, enforced where, keyed by what, how it counts across many nodes, what the client sees, and what happens when the limiter itself is down.

How it works

  • Default pick: token bucket. A bucket of `capacity` tokens refills at `rate`/s; a request spends one. It allows bursts up to `capacity`, keeps O(1) state per key (tokens + last refill time) and is easy to explain. Leaky bucket is the same shape but smooths output to a constant rate.
  • Windows: fixed window is one counter but can let through 2x the limit across a boundary; sliding window log is exact but stores one timestamp per request; sliding window counter weights the previous and current window counters, O(1) and approximate.
  • Where to enforce: at the API gateway or edge for coarse per-client limits, per service for protecting its own capacity, and optionally client side (backoff) to be a good citizen. Say which one you choose and why.
  • Key by user id or API key for fairness, by IP for anonymous traffic (beware NAT and proxies sharing one IP), and by route or tenant plan for different limits. Layer them: a per-IP limit in front of a per-user limit.
  • Distributed counting: keep state in Redis. Fixed window is `INCR key` plus `EXPIRE` on the first hit; token bucket or sliding window needs a read-modify-write, so run it as a Lua script (atomic on the single-threaded server) rather than GET then SET from the app.
  • Cost: one Redis round trip per request. Reduce it with local token pre-fetch (take 100 tokens at once) or per-node limits of limit/N, accepting some inaccuracy. Shard keys across Redis nodes with consistent hashing.
  • Response: HTTP `429 Too Many Requests` with `Retry-After`, and informational `X-RateLimit-Limit`, `X-RateLimit-Remaining`, `X-RateLimit-Reset` headers (a de-facto convention, not a strict standard). Clients should back off with jitter.
  • Failure mode: if Redis is unreachable, fail open (allow traffic, alert loudly) for ordinary APIs so a limiter outage is not an outage; fail closed only for expensive or abuse-prone endpoints such as login or SMS sending.
  • Also say: rate limiting rejects excess, load shedding drops work when the server is overloaded, and a concurrency limit caps in-flight requests. They are complementary.

Go sketch

Go
import (
    "sync"
    "time"
)

// TokenBucket is a single-process limiter. now is injectable for tests.
type TokenBucket struct {
    mu       sync.Mutex
    capacity float64
    rate     float64
    tokens   float64
    last     time.Time
    now      func() time.Time
}

// NewTokenBucket starts full; rate is tokens added per second.
func NewTokenBucket(capacity, rate float64, now func() time.Time) *TokenBucket {
    if now == nil {
        now = time.Now
    }
    return &TokenBucket{
        capacity: capacity,
        rate:     rate,
        tokens:   capacity,
        last:     now(),
        now:      now,
    }
}

// Allow spends one token if available.
func (b *TokenBucket) Allow() bool {
    return b.AllowN(1)
}

// AllowN refills by elapsed time, then spends n tokens if possible.
func (b *TokenBucket) AllowN(n float64) bool {
    b.mu.Lock()
    defer b.mu.Unlock()
    t := b.now()
    if elapsed := t.Sub(b.last).Seconds(); elapsed > 0 {
        b.tokens = min(b.capacity, b.tokens+elapsed*b.rate)
        b.last = t
    }
    if b.tokens < n {
        return false
    }
    b.tokens -= n
    return true
}

In practice

  • Stripe — Its engineering blog describes layered limiters built on a token bucket kept in Redis: a request rate limiter, a concurrent-requests limiter, and load shedders for fleet and worker utilisation.
  • NGINX — `limit_req` is documented as a leaky-bucket implementation (`rate`, `burst`, `nodelay`), a handy example of enforcing at the edge.
  • Public REST APIs — Large API providers commonly return 429 with `Retry-After` and rate-limit headers; quoting this convention shows you think about the client experience.
  • Interview tip — Always state the failure mode (fail open vs closed) and the key you limit by; most candidates stop at the algorithm.

Trade-offs

Pros

  • Protects shared capacity and gives fair use between tenants
  • Token bucket is O(1) per key and tolerates legitimate bursts
  • Centralised counters in Redis give one consistent limit across nodes

Cons

  • Adds a network hop and a new dependency on the hot path
  • Window algorithms trade accuracy for memory; fixed window leaks at boundaries
  • IP-based keys punish users behind NAT and are easy to rotate for attackers
  • Local approximations (limit/N per node) drift when load is uneven

Decide whether you need a lock at all, then build it safely with a token, a TTL and fencing, or use ZooKeeper/etcd when correctness matters.

The problem

"Only one worker should do X at a time" shows up as a cron job on many pods, a leader duty or an exclusive update. The interview question is whether you know when a lock is the wrong tool and what makes one unsafe: a paused process that wakes up still believing it owns the lock.

How it works

  • First classify the need. Efficiency lock: duplicates only waste work (a double email, a repeated cache rebuild), so a simple Redis lock is fine. Correctness lock: a double run corrupts data, so you need fencing or a consensus-backed system.
  • Prefer not locking: make the operation idempotent (unique key, upsert), or use optimistic concurrency, e.g. `UPDATE ... SET v = v + 1 WHERE id = ? AND v = ?` (version column) or compare-and-set. These survive crashes without any lease logic.
  • Redis acquire: `SET lock:key <random-token> NX PX 30000` is atomic, sets an owner token and a TTL so a crashed holder cannot block forever. Never use `SETNX` then `EXPIRE` as two steps.
  • Safe release: the token must match, so delete via a Lua script (`if GET == token then DEL`). A plain `DEL` can remove a lock that expired and was re-acquired by someone else.
  • TTL and renewal: choose TTL well above normal work time, and run a watchdog that extends it while the holder is alive. A TTL that is too short causes double ownership; too long delays recovery.
  • Why TTL locks are not enough: a GC pause, a VM stall or a slow network call can outlast the lease, so two clients both believe they hold it. Wall-clock jumps make it worse. The fix is a fencing token: a monotonically increasing number issued with each grant, and the storage rejects writes with an older token.
  • Redlock (acquire on a majority of 5 independent Redis nodes) is contested. Kleppmann argued it relies on timing assumptions and issues no fencing token, so it is unsafe for correctness; antirez replied that bounded clock drift and the validity-time check make it adequate. Neutral summary: fine for efficiency, use something else for correctness.
  • For correctness use ZooKeeper (ephemeral sequential znodes, the sequence number doubles as a fencing token) or etcd (leases with keepalive, revision as the token). Both are consensus-backed and detect holder death through session or lease expiry.
  • Say it out loud: a lock narrows the window, only the resource check (fencing, version column) closes it.

Examples

Acquire with SET NX PX and release only if the token still matches, atomically via EVAL.

bash
TOKEN=$(openssl rand -hex 16)

# acquire: returns OK only if the key does not exist; expires after 30 s
redis-cli SET lock:report:42 "$TOKEN" NX PX 30000

# release: compare-and-delete in one atomic Lua script
RELEASE='if redis.call("get", KEYS[1]) == ARGV[1] then return redis.call("del", KEYS[1]) else return 0 end'
redis-cli EVAL "$RELEASE" 1 lock:report:42 "$TOKEN"

In practice

  • Google Chubby — The paper positions Chubby as a coarse-grained lock service for leader election; it also notes that lock holders can be delayed and suggests sequencers, the same idea as fencing tokens.
  • ZooKeeper / Curator — The standard recipe uses ephemeral sequential nodes: the lowest sequence number holds the lock and each waiter watches its predecessor.
  • etcd — Locks are built on leases kept alive by the client; Kubernetes leader election uses lease-style objects in the same spirit.
  • Interview tip — Opening with "can I avoid the lock with idempotency or a version column?" signals seniority more than reciting Redlock.

Trade-offs

Pros

  • Redis `SET NX PX` is simple, fast and enough for efficiency locks
  • ZooKeeper/etcd locks are consensus-backed and detect dead holders reliably
  • Fencing tokens make the protected resource the final judge

Cons

  • TTL locks are unsafe under pauses and clock jumps without fencing
  • Single-node Redis loses the lock on failover (async replication)
  • Consensus systems add latency and an operational dependency
  • Locks serialise work and create contention hot spots

Generate unique, roughly time-sortable IDs without a coordinator: UUID v4/v7, ULID, Snowflake, ticket servers and segment allocation.

The problem

Many services need to mint IDs for orders, messages or tweets at high rate without a single bottleneck. State the requirements first (unique, roughly time-sortable, no coordination, 64 vs 128 bits), then pick from a short list.

How it works

  • Requirements to ask: uniqueness is mandatory; sortability by time helps B-tree locality and pagination; no central coordination per ID; 64-bit fits a `BIGINT` and is half the index size of 128-bit; not guessable if exposed publicly.
  • UUID v4: 122 random bits, no coordination, collisions negligible. Downside: 128 bits and random, so inserts scatter across a B-tree index and hurt cache locality.
  • UUID v7 and ULID: a millisecond timestamp prefix followed by random bits. Still 128 bits, but roughly sortable, so index inserts are mostly appends. Order within one millisecond is not guaranteed.
  • Snowflake (64 bits): 1 sign bit + 41-bit millisecond timestamp + 10-bit machine id + 12-bit sequence. 2^41 ms is about 69 years from a custom epoch; 2^12 = 4096 ids per millisecond per node.
  • Snowflake pitfalls: clock rollback (NTP step) can reissue old timestamps, so refuse to generate or wait until the clock catches up; machine ids must be unique (assign via ZooKeeper/etcd/config); same-millisecond overflow means wait for the next ms.
  • DB auto-increment: trivial but a single writer and shard-local. Ticket server (Flickr): a dedicated MySQL table with `REPLACE INTO` and auto-increment, two servers handing out odd and even numbers for availability.
  • Segment (range) allocation: a node leases a block of, say, 1000 IDs from a DB row in one update and serves them from memory, so the DB is hit once per block. Gaps after a crash are fine. Meituan's Leaf documents this design.
  • Redis `INCR` is fast and atomic but needs persistence/replication care; IDs can be lost or repeated after failover if configured loosely.
  • Decision rule: need 64-bit sortable and can manage worker ids, pick Snowflake; want zero coordination, accept 128 bits, pick UUID v7/ULID; already have a relational shard, use sequences or segments.

Go sketch

Go
import (
    "errors"
    "sync"
    "time"
)

const (
    epochMs   = 1704067200000
    nodeBits  = 10
    seqBits   = 12
    maxNode   = (1 << nodeBits) - 1
    maxSeq    = (1 << seqBits) - 1
    nodeShift = seqBits
    timeShift = seqBits + nodeBits
)

// ErrClockBackwards is returned when the clock moves behind the last id.
var ErrClockBackwards = errors.New("clock moved backwards")

// Snowflake packs 41-bit ms | 10-bit node | 12-bit sequence into an int64.
type Snowflake struct {
    mu     sync.Mutex
    node   int64
    lastMs int64
    seq    int64
    now    func() time.Time
}

func NewSnowflake(node int64, now func() time.Time) (*Snowflake, error) {
    if node < 0 || node > maxNode {
        return nil, errors.New("node id out of range")
    }
    if now == nil {
        now = time.Now
    }
    return &Snowflake{node: node, now: now}, nil
}

// Next returns a unique id or ErrClockBackwards.
func (s *Snowflake) Next() (int64, error) {
    s.mu.Lock()
    defer s.mu.Unlock()
    ms := s.now().UnixMilli() - epochMs
    if ms < s.lastMs {
        return 0, ErrClockBackwards
    }
    if ms == s.lastMs {
        s.seq = (s.seq + 1) & maxSeq
        if s.seq == 0 {
            for ms <= s.lastMs {
                ms = s.now().UnixMilli() - epochMs
            }
        }
    } else {
        s.seq = 0
    }
    s.lastMs = ms
    return ms<<timeShift | s.node<<nodeShift | s.seq, nil
}

In practice

  • Twitter Snowflake — The original design: 41 bits of milliseconds since a custom epoch, 10 bits of worker id and 12 bits of sequence in a 64-bit integer, with machine ids coordinated through ZooKeeper.
  • Instagram — Documented a variant generated inside PostgreSQL: 41 bits of time, 13 bits of logical shard id and 10 bits from a per-shard sequence.
  • Flickr ticket servers — Dedicated MySQL servers with auto-increment (odd/even pair) as a central, simple ID source.
  • Discord — Public API IDs are Snowflake-style 64-bit values that embed a creation time, so the timestamp can be extracted from any ID.

Trade-offs

Pros

  • Snowflake gives compact 64-bit, time-ordered IDs with no per-ID coordination
  • UUID v4 needs nothing at all; UUID v7/ULID add index-friendly ordering
  • Segment allocation amortises DB access to one write per block

Cons

  • Snowflake depends on sane clocks and unique machine ids
  • Random UUIDs bloat indexes and fragment B-trees
  • Time-ordered IDs leak creation time and rough volume
  • Central counters (auto-increment, Redis) are a bottleneck and a single point of failure

Kafka vs RabbitMQ vs SQS in one pass: log vs broker queue, replay, routing, throughput and ops burden, plus DLQ and idempotent consumers.

The problem

"Which message queue would you use?" is asked in nearly every design. Answer with the access pattern first: is this a work queue (each message handled once, then gone) or an event log (many consumers, replay, ordering by key)? Then map it to a product and name the guarantees.

How it works

  • Kafka is a partitioned, replicated, append-only log. Consumers track offsets, so messages are retained by time or size and can be replayed. Ordering is per partition; throughput scales with partitions.
  • RabbitMQ is a broker: producers publish to exchanges (direct, topic, fanout, headers), bindings route to queues, and messages are removed on ack. It offers per-message ack, TTL, priorities and dead-lettering; streams add a log-like mode.
  • SQS is a managed queue with no servers. Standard queues: very high throughput, at-least-once, best-effort ordering. FIFO queues: ordering per message group and 5-minute deduplication, with lower default throughput (300 API calls/s per action, 3,000 msg/s with batching, higher in high-throughput mode). Max message size 1 MiB, retention up to 14 days, visibility timeout up to 12 hours.
  • Decision rules: need replay, stream processing, multiple independent consumers or huge throughput, pick Kafka. Need complex routing, per-message priority or low-volume task dispatch, pick RabbitMQ. Need zero ops on AWS and a simple work queue, pick SQS (+ SNS for fan-out).
  • Briefly: Pulsar separates serving from storage (BookKeeper) and has built-in multi-tenancy and geo-replication; NATS JetStream is lightweight and low latency; Redis Streams with consumer groups suit modest volume when Redis is already there.
  • Delivery semantics: assume at-least-once everywhere. Exactly-once is end-to-end only with idempotent consumers (dedupe on a message id or a natural key) or transactional writes; Kafka transactions cover Kafka-to-Kafka.
  • Failure handling: bounded retries with backoff, then a dead-letter queue (DLQ) with alerts and a redrive tool; make poison messages visible instead of blocking the partition or queue.
  • Ordering needs a key: partition by entity id in Kafka, message group id in SQS FIFO. Global ordering means one partition and one consumer, which kills throughput.
  • Backpressure: monitor consumer lag / queue depth and scale consumers; in Kafka the useful parallelism is capped by the partition count.

In practice

  • Kafka at LinkedIn — Created there as a high-throughput log for activity data and metrics pipelines, which is why it is built around retention and replay rather than per-message routing.
  • RabbitMQ — Commonly used for background job dispatch and request routing between services where topic exchanges and per-message acks matter more than raw volume.
  • SQS + SNS — A common AWS pattern: SNS topic fans out to several SQS queues, each with its own DLQ and independent consumers.
  • Interview tip — Say "at-least-once plus an idempotent consumer" instead of promising exactly-once; interviewers listen for that.

Trade-offs

Pros

  • Kafka: replay, huge throughput, many independent consumer groups
  • RabbitMQ: flexible routing, priorities, per-message control
  • SQS: no servers, elastic, simple to operate and cost-effective at low volume

Cons

  • Kafka: operational weight, rebalancing, parallelism capped by partitions, weak per-message control
  • RabbitMQ: lower ceiling, no replay on classic queues, clustering takes care
  • SQS: no replay, AWS lock-in, FIFO throughput limits, limited routing

Cache-aside, read-through, write-through, write-behind and write-around, plus the failure modes: stampede, penetration, avalanche and hot keys.

The problem

Adding a cache is the first answer to a slow read path, and the follow-ups are always the same: how is it filled, how is it kept fresh, and what happens when it fails or a hot key expires. Have one default and a list of failure modes ready.

How it works

  • Cache-aside (lazy loading) is the default: read cache, on a miss read the DB and fill the cache; on write update the DB then delete the key. Only requested data is cached and a cache outage degrades to DB reads.
  • Read-through moves the load logic into the cache layer. Write-through writes cache and DB synchronously (fresh reads, slower writes). Write-behind writes the cache and flushes to the DB asynchronously (fast, but data loss if the cache dies). Write-around writes straight to the DB and lets reads fill the cache.
  • Invalidation: prefer delete-on-write over update-on-write, since concurrent updates can write stale values last. Even so a race exists (reader loads old row, writer deletes, reader fills old value), so keep a TTL as a safety net; delayed double-delete or CDC-driven invalidation narrows it.
  • Stampede (hot key expires, thousands of misses hit the DB): single-flight or a per-key lock so one caller reloads; jittered TTLs; early refresh or serving stale while revalidating.
  • Penetration (requests for keys that do not exist bypass the cache): cache a short-lived null result, validate input, or put a Bloom filter of existing keys in front.
  • Avalanche (many keys expire together or the cache dies): randomise TTLs, keep replicas or a multi-level cache, rate-limit and degrade to protect the DB.
  • Hot keys: replicate the key under several suffixes, add a small in-process cache with a short TTL, or push to the CDN. Big values and big keys need splitting.
  • Where to cache: client/browser, CDN for static and cacheable responses, application (in-process or Redis/Memcached), and the DB's own buffer pool. Nearer to the user is faster but harder to invalidate.
  • Eviction when memory is full: LRU (recency) is the default, LFU (frequency) resists scans; both are approximate in Redis. Always state the consistency window you accept.

Go sketch

Go
import (
    "math/rand/v2"
    "sync"
    "time"
)

type entry struct {
    val     string
    expires time.Time
}

type call struct {
    wg  sync.WaitGroup
    val string
    err error
}

// Cache is cache-aside with per-key single-flight loading.
type Cache struct {
    mu       sync.Mutex
    items    map[string]entry
    inflight map[string]*call
    ttl      time.Duration
    load     func(key string) (string, error)
}

func NewCache(ttl time.Duration, load func(string) (string, error)) *Cache {
    return &Cache{
        items:    map[string]entry{},
        inflight: map[string]*call{},
        ttl:      ttl,
        load:     load,
    }
}

// Get returns a fresh entry, or lets exactly one caller reload it.
func (c *Cache) Get(key string) (string, error) {
    c.mu.Lock()
    if e, ok := c.items[key]; ok && time.Now().Before(e.expires) {
        c.mu.Unlock()
        return e.val, nil
    }
    if cl, ok := c.inflight[key]; ok {
        c.mu.Unlock()
        cl.wg.Wait()
        return cl.val, cl.err
    }
    cl := &call{}
    cl.wg.Add(1)
    c.inflight[key] = cl
    c.mu.Unlock()

    cl.val, cl.err = c.load(key)

    c.mu.Lock()
    delete(c.inflight, key)
    if cl.err == nil {
        ttl := c.ttl + rand.N(c.ttl/10+1)
        c.items[key] = entry{val: cl.val, expires: time.Now().Add(ttl)}
    }
    c.mu.Unlock()
    cl.wg.Done()
    return cl.val, cl.err
}

// Invalidate is called after the database write succeeds.
func (c *Cache) Invalidate(key string) {
    c.mu.Lock()
    defer c.mu.Unlock()
    delete(c.items, key)
}

In practice

  • Facebook Memcache — The "Scaling Memcache at Facebook" paper uses cache-aside with delete-on-write and introduces leases to curb stale sets and thundering herds.
  • DynamoDB DAX — A write-through, read-through cache in front of DynamoDB that is API-compatible, so applications need few code changes.
  • CDNs — Cache at the edge keyed by URL with TTL and explicit purge; many support stale-while-revalidate to hide origin latency.
  • Redis — Typically used as the cache-aside store with `maxmemory` and an LRU/LFU policy; TTLs with jitter are the standard defence against synchronized expiry.

Trade-offs

Pros

  • Cuts read latency and shields the database from repeated reads
  • Cache-aside is simple and degrades gracefully when the cache is down
  • Single-flight and jitter remove most stampede and avalanche risk cheaply

Cons

  • Stale reads are inherent; the window depends on invalidation and TTL
  • Write-behind can lose acknowledged writes
  • Adds an invalidation problem, a cold-start problem and an extra system to run
  • Hot keys and big values can overload one cache node

Saga, 2PC, TCC and the outbox pattern: which to choose when one business operation spans several services, and when to avoid it altogether.

The problem

One business operation (place order, charge card, reserve stock) spans several services, each with its own database. A single ACID transaction is not available, so the interview asks how you keep the whole thing consistent when a step fails halfway.

How it works

  • First answer: avoid it. Keep data that must change together in one database or service boundary, or make each step idempotent with retries and accept eventual consistency. Cross-service transactions are a cost, not a feature.
  • Saga: a sequence of local transactions, each with a compensating action (refund, release stock) run in reverse if a later step fails. Compensations are semantic undo, not rollback, and intermediate states are visible.
  • Orchestration vs choreography: an orchestrator (workflow engine) calls steps and tracks state, easy to reason about and monitor. Choreography has services react to events, loosely coupled but the flow is hard to see and debug. Default to orchestration for anything longer than 2-3 steps.
  • Saga rules: every step and compensation must be idempotent and retryable; order steps so the pivot step (the point of no return) comes after the steps that can fail and before the retryable-only ones; use a persisted state machine and timeouts.
  • 2PC: a coordinator asks all participants to prepare (lock and promise), then commit. Gives atomicity, but locks are held across the round trips and a coordinator crash after prepare blocks participants. Fine inside one trust domain with XA-capable databases, poor across microservices.
  • TCC (Try-Confirm-Cancel): Try reserves resources (freeze funds), Confirm commits the reservation, Cancel releases it. Needs idempotent Confirm/Cancel, an empty Cancel when Try never arrived, and hanging prevention (reject a Try that arrives after its Cancel).
  • Outbox pattern: write the business row and an event row in the same local transaction, then a relay (polling or CDC such as Debezium) publishes to the broker. This solves the dual-write problem; consumers must be idempotent since delivery is at-least-once.
  • Decision rule: single DB if you can; outbox plus idempotent consumers for simple eventual flows; saga (orchestrated) for long business processes; TCC for money/inventory needing reservations; 2PC only among a few homogeneous databases.

Examples

Outbox: the state change and the event commit atomically; a relay publishes unsent rows.

sql
BEGIN;
UPDATE orders SET status = 'PAID' WHERE id = 42 AND status = 'PENDING';
INSERT INTO outbox (aggregate_id, event_type, payload, created_at)
VALUES (42, 'OrderPaid', '{"orderId": 42}', now());
COMMIT;

SELECT id, event_type, payload FROM outbox WHERE sent_at IS NULL ORDER BY id LIMIT 100;
UPDATE outbox SET sent_at = now() WHERE id = 1001;

In practice

  • Saga paper — Garcia-Molina and Salem (1987) introduced sagas to split long-lived transactions into sub-transactions with compensations.
  • Temporal / Step Functions — Workflow engines commonly used to run orchestrated sagas with durable state, retries and timeouts.
  • Debezium outbox — Reads the database log to publish outbox rows to Kafka without polling, a widely used way to implement the pattern.
  • Seata — An open-source framework popular in the Java ecosystem that offers TCC, saga and XA modes, which is why TCC comes up often in Chinese-market interviews.

Trade-offs

Pros

  • Saga avoids global locks and scales across services
  • Outbox guarantees the event is published if and only if the state change committed
  • TCC gives business-level isolation through reservations

Cons

  • Saga offers no isolation: dirty reads and lost updates need semantic locks or countermeasures
  • 2PC blocks on coordinator failure and holds locks across the network
  • TCC triples the code per service and needs anti-hanging logic
  • Every approach needs idempotency and good observability for stuck flows

Classic design questions

Read-heavy key-value redirect service: a 7-char base62 key space, a key-generation strategy, 301 vs 302, and a cache in front of a KV store.

The problem

Design `short.ly/aZ3k9Qx` -> a long URL. It looks trivial, so the interviewer is checking whether you size it, pick a key-generation scheme with its failure mode in mind, and know why a redirect status code is a product decision. The core is a read-heavy key-value lookup.

How it works

  • Requirements: create a short link (optional custom alias, optional expiry), redirect fast, basic click analytics; non-functional: redirect p99 in tens of ms, highly available, links never collide, keys not trivially guessable.
  • Estimates: assume 100 M new links/month (~40 writes/s) and 100:1 reads (~4k redirects/s avg, 15-20k/s peak). 5 years is ~6 B rows x ~500 B = ~3 TB, which fits a sharded KV store. 7 base62 chars = 62^7 ~ 3.5 x 10^12 keys, so 6 B uses ~0.2% of the space.
  • API: `POST /v1/links {url, alias?, expires_at?}` -> `201 {short_url}`; `GET /{key}` -> `301`/`302` with `Location`; `DELETE /v1/links/{key}`. Authenticate creation with an API key, never the redirect path.
  • Data model: one row per link, `key` (primary key) -> `long_url, owner_id, created_at, expires_at`; a KV or wide-column store is enough because the only query is by key. A separate click-events stream feeds analytics, off the redirect path.
  • Architecture: stateless app servers behind an LB, a Redis/CDN cache in front of the KV store (the key distribution is heavily skewed, so a small cache absorbs most reads), plus an ID/key service on the write path and an async queue for click events.
  • Deep dives: key generation. (a) Counter + base62, with each app server leasing a range of e.g. 10^6 ids from a coordinator: no collisions and cheap, but sequential keys are guessable (add a bijective shuffle or XOR with a secret). (b) Hash the URL and take 7 chars: stateless, but needs collision checks and retries. (c) Pre-generated key pool: simple and random, but one more component to run. Pick (a) unless unguessability is a hard requirement.
  • Bottlenecks: 301 is cached by browsers (less load, but later clicks never reach you, so analytics are lost); 302 hits you every time and lets you change or expire the target. Pick 302 if you count clicks or support expiry. Rate-limit creation per API key/IP, scan targets for malware, and treat expired keys as 404/410 via a TTL on the row plus a lazy check.

Examples

Go
import (
    "errors"
    "strings"
    "sync"
)

const alphabet = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz"

var errBadKey = errors.New("invalid base62 key")

// encodeBase62 turns a counter value into a base62 string.
func encodeBase62(n uint64) string {
    if n == 0 {
        return "0"
    }
    var buf [11]byte
    i := len(buf)
    for n > 0 {
        i--
        buf[i] = alphabet[n%62]
        n /= 62
    }
    return string(buf[i:])
}

// decodeBase62 is the inverse of encodeBase62.
func decodeBase62(s string) (uint64, error) {
    if s == "" || len(s) > 11 {
        return 0, errBadKey
    }
    var n uint64
    for _, c := range s {
        d := strings.IndexRune(alphabet, c)
        if d < 0 {
            return 0, errBadKey
        }
        n = n*62 + uint64(d)
    }
    return n, nil
}

// rangeAllocator hands out ids from a leased block and fetches a new block
// from a coordinator (for example a DB row or etcd) when the block runs out.
type rangeAllocator struct {
    mu    sync.Mutex
    next  uint64
    end   uint64
    lease func() (start, end uint64)
}

func (a *rangeAllocator) Next() uint64 {
    a.mu.Lock()
    defer a.mu.Unlock()
    if a.next >= a.end {
        a.next, a.end = a.lease()
    }
    id := a.next
    a.next++
    return id
}

Schema sketch: one row per link, plus the single-row range lease used by the key service.

sql
CREATE TABLE links (
    short_key  VARCHAR(10)   PRIMARY KEY,
    long_url   VARCHAR(2048) NOT NULL,
    owner_id   BIGINT,
    created_at TIMESTAMP     NOT NULL,
    expires_at TIMESTAMP
);

CREATE TABLE key_ranges (
    name      VARCHAR(32) PRIMARY KEY,
    next_free BIGINT      NOT NULL
);

UPDATE key_ranges
SET next_free = next_free + 1000000
WHERE name = 'links'
RETURNING next_free - 1000000 AS range_start, next_free AS range_end;

Create a link, then inspect the redirect without following it.

bash
curl -s -X POST https://short.ly/v1/links \
    -H "Authorization: Bearer $API_KEY" \
    -H "Content-Type: application/json" \
    -d '{"url":"https://example.com/very/long/path","alias":"launch","expires_at":"2027-01-01T00:00:00Z"}'

curl -sI https://short.ly/launch | grep -iE '^(HTTP|location|cache-control)'

In practice

  • Follow-up: how do you avoid guessable keys? — Counter ids are enumerable. Either encode the counter through a keyed bijection, use random keys from a pre-generated pool, or accept that links are public-by-obscurity and make sensitive links require auth.
  • Follow-up: what if one key suddenly gets 1 M req/s? — Serve it from the CDN/edge with a short TTL (a 302 with `Cache-Control: max-age` is allowed), add a per-node in-process cache, and make sure the hot key is not pinned to one cache shard.
  • Follow-up: multi-region? — Reads are served from regional caches and replicas; writes can go to one home region, or every region gets its own disjoint key range so no cross-region coordination is needed.
  • ID schemes at scale — Snowflake-style 64-bit ids (timestamp + machine id + sequence) are commonly used for ordered unique ids, but a 64-bit value needs 11 base62 chars, so shorteners usually use a smaller counter or leased ranges to stay at 7.

Trade-offs

Pros

  • Trivial access pattern (get by key), so a sharded KV store plus cache scales almost linearly.
  • Range-leased counters give collision-free keys with one coordinator call per million ids.
  • A 302 keeps analytics, expiry and target changes under your control.

Cons

  • Sequential base62 keys are enumerable unless you scramble them.
  • 302 costs a server round trip on every click and loses the browser-cache offload of 301.
  • A leased range that is never used (crashed server) wastes keys; acceptable only because the key space is huge.

WhatsApp/Slack-style chat: WebSocket gateways, a user-to-gateway routing table, per-conversation ordering, delivery acks and group fan-out.

The problem

Design a 1:1 and group chat service for hundreds of millions of users: messages arrive in order, are not lost when the recipient is offline, and show sent/delivered/read. The hard parts are millions of long-lived connections, routing a message to the right server, and per-conversation ordering.

How it works

  • Requirements: 1:1 and group chat, delivery states, offline delivery with push notification, media, multi-device, presence; non-functional: low latency (~100s of ms), no message loss, ordering within a conversation (not global), E2E encryption optional.
  • Estimates: assume 500 M DAU x 40 messages/day = 20 B/day ~ 230k msg/s avg (2-3x peak). At ~200 B stored per message that is ~4 TB/day, ~7 PB over 5 years before replication. If 100 M users are connected at once and a gateway holds ~50k connections, you need ~2k gateways.
  • API: a persistent WebSocket per device carrying frames `send {conv_id, client_msg_id, body}`, `ack {msg_id}`, `typing`, `presence`; plus REST for history (`GET /conversations/{id}/messages?before=<msg_id>&limit=50`), media upload (pre-signed URL) and conversation management.
  • Data model: messages keyed by `(conversation_id, message_id)` where `message_id` is a per-conversation monotonically increasing sequence (or a time-sortable id); a wide-column or log-structured store (Cassandra/ScyllaDB/HBase style) partitioned by conversation (plus a time bucket for huge ones). Separate tables: conversations/members, per-user inbox cursor (last read/delivered seq).
  • Architecture: clients -> LB -> stateless-ish connection gateways (hold sockets) -> chat service (assigns the sequence, persists, then fans out). A session store (Redis) maps `user_id/device -> gateway`; to deliver, look the target up and forward to that gateway, otherwise write to the offline inbox and trigger APNs/FCM push. Media goes to object storage with a CDN and only the URL travels in the message.
  • Deep dives: ordering and idempotency: the chat service (or the conversation's single partition owner) assigns the sequence, and the client's `client_msg_id` makes retries idempotent. Delivery: persist first, then ack the sender (`sent`); the recipient device acks (`delivered`), a read cursor update gives `read`; unacked messages are redelivered on reconnect from the last seq. Group fan-out: small groups fan out on write to each member's inbox; huge channels store one copy and members pull (fan-out on read) since writing 100k inboxes per message is too costly. Presence/typing: client heartbeat every ~30 s with a TTL in Redis, never persisted; throttle presence broadcasts to friends/active views.
  • Bottlenecks: gateway failure drops its connections, so clients reconnect with jittered backoff and resync from their last seq (no state lives only in the gateway). Hot conversations need a bounded partition (time buckets). Reconnect storms need connection-rate limits. For E2E encryption (Signal-protocol style) the server stores and routes ciphertext only, which rules out server-side search and complicates multi-device and large groups.

Examples

CQL-style schema: one partition per conversation (time-bucketed), newest message first.

sql
CREATE TABLE messages (
    conversation_id BIGINT,
    bucket          INT,
    message_id      BIGINT,
    sender_id       BIGINT,
    body            BLOB,
    PRIMARY KEY ((conversation_id, bucket), message_id)
) WITH CLUSTERING ORDER BY (message_id DESC);

CREATE TABLE inbox_cursor (
    user_id         BIGINT,
    conversation_id BIGINT,
    delivered_seq   BIGINT,
    read_seq        BIGINT,
    PRIMARY KEY (user_id, conversation_id)
);

History pagination by cursor (the last message_id seen), and a presence heartbeat key with a TTL.

bash
curl -s -H "Authorization: Bearer $TOKEN" \
    "https://chat.example.com/v1/conversations/42/messages?before=98123&limit=50"

redis-cli SET presence:user:1001 gw-17 EX 45

In practice

  • WhatsApp — Well known for running on Erlang with a very high connection count per server, and for end-to-end encryption based on the Signal protocol.
  • Discord — Its engineering blog describes storing messages by channel with time-based buckets to bound partition size, first on Cassandra and later on ScyllaDB.
  • Slack — Real-time events are pushed over WebSockets from a tier of gateway servers, while message history lives in sharded MySQL (Vitess).
  • Follow-up: ordering with multiple devices and retries? — Assign the sequence server-side, dedupe by `client_msg_id`, and render by sequence, not by client timestamp.
  • Follow-up: how do you deliver to a user with 3 devices? — The session table maps a user to several (device, gateway) entries; the chat service fans out to each and every device keeps its own cursor.

Trade-offs

Pros

  • Persistent sockets give low-latency push without polling.
  • Per-conversation sequences give the ordering users care about without global coordination.
  • Hybrid fan-out keeps small groups fast and huge channels affordable.

Cons

  • Stateful gateways make deploys and failures a reconnect storm; they need drain and jittered reconnect.
  • Fan-out on write multiplies storage and write load by group size.
  • E2E encryption removes server-side search, moderation and easy multi-device sync.

Twitter/Instagram-style home timeline: fan-out on write vs on read, the hybrid answer to celebrities, a per-user feed cache and cursor pagination.

The problem

Design the home timeline of a social network: a user opens the app and sees recent posts from accounts they follow, fast. The crux is the asymmetry: reads are far more frequent than writes, and a few accounts have tens of millions of followers.

How it works

  • Requirements: publish a post (text/media), follow/unfollow, home feed of followed accounts, newest or ranked first, infinite scroll; non-functional: feed load p99 < 200-500 ms, eventual consistency is fine (a post may take seconds to appear).
  • Estimates (assumptions): 300 M DAU, 1/3 post per user-day -> ~100 M posts/day (~1.2k/s); average 200 followers -> 20 B feed inserts/day (~230k/s) under fan-out on write; 10 feed opens per user-day -> 3 B reads/day (~35k/s). A cached feed of 800 post ids x 8 B = ~6.4 KB per user -> ~2 TB for 300 M users (ids only; post bodies are fetched from a post cache).
  • API: `POST /v1/posts`, `POST /v1/follow/{user_id}`, `GET /v1/feed?cursor=<opaque>&limit=20` -> `{items[], next_cursor}`. The cursor encodes the last item's (score or timestamp, post_id); offset pagination breaks when new posts shift the list.
  • Data model: `posts(post_id, author_id, body, media_urls, created_at)` in a sharded store; follow graph as `(follower_id, followee_id)` rows, indexed both ways; per-user feed cache `feed:{user_id}` as a Redis list or sorted set of post ids (score = time or rank), capped at ~500-1000 entries.
  • Architecture: post service writes the post, then emits an event; fan-out workers read the author's followers and push the post id into each follower's feed cache; the feed service reads the cache, hydrates ids to posts/authors via batched cache lookups, filters (blocked, deleted) and returns a page. Media is uploaded to object storage and served via CDN.
  • Deep dives: fan-out on write gives O(1) cheap reads but a write storm for big accounts and wasted work for inactive followers; fan-out on read (pull and merge at request time) has no write cost but slow, expensive reads. Hybrid: push for normal accounts, skip push for celebrities (say > 100k-1M followers) and merge their recent posts into the feed at read time; also only fan out to active users. Ranking: chronological is simple and cacheable; ranked feeds score candidates (recency, affinity, engagement) in a second stage, so cache candidate ids and rank on read.
  • Bottlenecks: the celebrity post (50 M followers = 50 M writes) is the classic hotspot, hence the hybrid; fan-out is async and retried, so a lagging queue means stale feeds rather than failed posts. Cache loss is recoverable by rebuilding a feed from the follow graph on demand. Deletes and privacy changes need a filter on read because the cache holds stale ids.

Examples

Feed cache as a Redis sorted set (score = post time), cursor paging, and trimming to a fixed length.

bash
redis-cli ZADD feed:1001 1760000000 post:9001
redis-cli ZREVRANGEBYSCORE feed:1001 "(1760000000" -inf LIMIT 0 20
redis-cli ZREMRANGEBYRANK feed:1001 0 -801

Follow graph indexed in both directions, and the keyset-paginated query behind a pull-based feed.

sql
CREATE TABLE follows (
    follower_id BIGINT NOT NULL,
    followee_id BIGINT NOT NULL,
    created_at  TIMESTAMP NOT NULL,
    PRIMARY KEY (follower_id, followee_id)
);

CREATE INDEX follows_by_followee ON follows (followee_id, follower_id);

SELECT p.post_id, p.author_id, p.created_at
FROM posts p
JOIN follows f ON f.followee_id = p.author_id
WHERE f.follower_id = 1001
  AND (p.created_at, p.post_id) < (TIMESTAMP '2026-10-08 12:00:00', 9001)
ORDER BY p.created_at DESC, p.post_id DESC
LIMIT 20;

In practice

  • Twitter / X — Publicly described its home timeline as fan-out on write into per-user Redis-backed timelines, with accounts that have very many followers merged in at read time (a hybrid).
  • Instagram / Facebook — Moved from a reverse-chronological feed to a ranked one; the ranking stage is separate from candidate retrieval, which is the structure to describe in an interview.
  • Follow-up: how do you handle unfollow and delete? — Filter on read against the follow graph and a tombstone check, and lazily remove the stale ids from the cache; do not try to rewrite millions of feeds synchronously.
  • Follow-up: new user with an empty cache, or cache loss? — Rebuild by pull (merge the recent posts of followees), write the result to the cache and serve; cap the work with limits per followee.

Trade-offs

Pros

  • Hybrid fan-out gives fast reads for most users and bounded write amplification for celebrities.
  • Caching ids only keeps memory small and lets post content be cached once.
  • Cursor pagination stays stable while new posts arrive.

Cons

  • Two code paths (push and pull) to build, test and keep consistent.
  • Eventual consistency: a post can appear late or briefly after being deleted.
  • A ranking stage adds latency, feature plumbing and cache-hit-rate trade-offs.

Inverted index with BM25 ranking, scatter-gather over document shards, a near-real-time indexing pipeline and typeahead.

The problem

Design search over a large corpus (products, documents, posts): full-text queries return ranked results in tens of milliseconds, new documents become searchable within seconds, and typing shows suggestions. The answer is an inverted index, distributed by sharding and scatter-gather.

How it works

  • Requirements: full-text query with filters and ranking, pagination, typeahead, near-real-time updates; non-functional: p99 < 100-200 ms, high availability, index freshness in seconds. Say up front that the search index is a derived view, not the source of truth.
  • Estimates (assumptions): 10 B documents x ~5 KB = ~50 TB raw; index size is the same order (often 0.3-1x depending on stored fields). At ~30-50 GB per shard that is ~1000+ primary shards, x2-3 with replicas; 10k QPS peak spread over every shard, since each query touches all of them.
  • API: `GET /search?q=red+shoes&filter=brand:acme&limit=20&cursor=...` -> `{hits[], total, next_cursor}`; `GET /suggest?prefix=red+sh`; indexing is internal (events), not a public write API. Deep pagination with from/size is expensive, so use `search_after`-style cursors.
  • Data model: the inverted index maps `term -> posting list [(doc_id, term_freq, positions)]`, plus doc values / stored fields for sorting, filters and snippets. Analysis (tokenize, lowercase, stem, stop-words, synonyms) must be identical at index and query time.
  • Architecture: source DB -> CDC/Kafka -> indexer workers (analyze, build) -> search shards; queries enter a coordinator that fans out to one replica of every shard, each shard returns its top-k with scores, and the coordinator merges them. Documents are sharded by `hash(doc_id)`, so every shard holds a full slice of all terms.
  • Deep dives: ranking: TF-IDF weights a term by frequency in the doc and rarity in the corpus, and BM25 adds term-frequency saturation and length normalization (the usual default); a second-stage re-ranker can use clicks and freshness. Document sharding (scatter-gather) scales writes and gives simple, balanced shards; term sharding makes multi-term queries cross-shard joins and has hot terms. Write path: new docs go to an in-memory buffer, then immutable segments, merged in the background; deletes are tombstones until a merge. Typeahead: a prefix index (trie or edge n-grams, FST-based completion) built from query logs and popular items, served from memory with a small cache on top.
  • Bottlenecks: tail latency, since a query is as slow as the slowest shard (use replicas, hedged requests, timeouts with partial results). Reindexing is a blue/green alias swap, not an in-place edit. Freshness vs relevance: more frequent refreshes mean more tiny segments and merge cost, so pick a refresh interval of ~1 s or more. If CDC lags the index is stale, so monitor the lag.

Examples

Elasticsearch-style: create an index with shards and a refresh interval, then run a bool query with a filter.

bash
curl -s -X PUT localhost:9200/products -H 'Content-Type: application/json' -d '{
    "settings": {"number_of_shards": 20, "number_of_replicas": 1, "refresh_interval": "1s"},
    "mappings": {"properties": {"title": {"type": "text"}, "brand": {"type": "keyword"}}}
}'

curl -s localhost:9200/products/_search -H 'Content-Type: application/json' -d '{
    "query": {"bool": {"must": {"match": {"title": "red shoes"}}, "filter": {"term": {"brand": "acme"}}}},
    "size": 20
}'

Core idea in 30 lines: an in-memory inverted index with AND queries by intersecting sorted posting lists.

Go
import (
    "sort"
    "strings"
)

type index map[string][]int

// add appends doc ids in increasing order, so posting lists stay sorted.
func (ix index) add(docID int, text string) {
    seen := map[string]bool{}
    for _, term := range strings.Fields(strings.ToLower(text)) {
        if !seen[term] {
            seen[term] = true
            ix[term] = append(ix[term], docID)
        }
    }
}

func intersect(a, b []int) []int {
    var out []int
    for i, j := 0, 0; i < len(a) && j < len(b); {
        switch {
        case a[i] == b[j]:
            out = append(out, a[i])
            i++
            j++
        case a[i] < b[j]:
            i++
        default:
            j++
        }
    }
    return out
}

// searchAll returns the docs containing every query term, rarest term first.
func (ix index) searchAll(query string) []int {
    terms := strings.Fields(strings.ToLower(query))
    sort.Slice(terms, func(i, j int) bool {
        return len(ix[terms[i]]) < len(ix[terms[j]])
    })
    var result []int
    for n, t := range terms {
        if n == 0 {
            result = ix[t]
        } else {
            result = intersect(result, ix[t])
        }
    }
    return result
}

In practice

  • Elasticsearch / OpenSearch / Solr — All are distributed layers over Lucene: an index is split into shards, each shard is a Lucene index of immutable segments merged in the background, and BM25 is the default similarity. Elasticsearch makes new documents searchable after a refresh, 1 s by default.
  • Elasticsearch sizing guidance — The official guidance is to keep shards in the tens of GB (roughly 10-50 GB) rather than thousands of tiny ones, because every shard has fixed overhead and every query visits all shards.
  • Follow-up: how do you keep the index in sync with the DB? — CDC (or an outbox) into Kafka, idempotent upserts keyed by doc id and version, plus a periodic full reconcile; never dual-write from the app.
  • Follow-up: typeahead at keystroke speed? — Serve from an in-memory prefix structure (trie or completion FST) over precomputed top suggestions per prefix, debounce on the client, and refresh the data offline from query logs.

Trade-offs

Pros

  • Inverted index makes term lookup O(1) and AND queries cheap via sorted-list intersection.
  • Document sharding scales both data and QPS, and replicas add read capacity and availability.
  • Immutable segments make writes fast, caches stable and recovery simple.

Cons

  • Eventually consistent with the source of truth; reindexing needs a plan.
  • Every query hits every shard, so tail latency and shard count matter.
  • Relevance tuning (analyzers, synonyms, boosts) is a continuous product job, not a one-off.

Multi-channel (push, SMS, email, in-app) delivery pipeline: queues per channel, preferences, dedup, retries with a DLQ and provider fallback.

The problem

Design a service that other teams call to notify users by push, SMS, email or in-app, from one-off transactional messages to a campaign hitting tens of millions of users. The hard parts are respecting preferences, not sending duplicates, surviving flaky third-party providers and not letting a campaign delay a password reset.

How it works

  • Requirements: send via push/SMS/email/in-app, templates with localization, user preferences and opt-out, scheduled and bulk sends, delivery status; non-functional: transactional messages in seconds, bulk throughput, at-least-once with duplicate suppression, no loss when a provider is down.
  • Estimates (assumptions): 100 M users x 5 notifications/day = 500 M/day ~ 6k/s average; a campaign to 50 M users in 10 minutes needs ~83k/s. Provider limits (not your CPU) set the real ceiling, so throughput planning is about per-provider rate limits and connection pools.
  • API: `POST /v1/notifications {user_id, template, params, channels?, priority, idempotency_key, send_at?}` -> `202 {id}`; `GET /v1/notifications/{id}` for status; `PUT /v1/users/{id}/preferences`; provider webhooks `POST /v1/webhooks/{provider}` for delivered/bounced/unsubscribed events.
  • Data model: `notifications(id, idempotency_key UNIQUE, user_id, channel, status, attempts, created_at)`; `user_prefs(user_id, channel, topic, enabled, quiet_hours)`; `devices(user_id, platform, token, last_seen)`; `templates(id, locale, version, body)`. Dedup keys live in a store with TTL.
  • Architecture: producers -> notification API (validate, check idempotency key, store, enqueue) -> router (apply preferences, quiet hours, rate limits, render template) -> one queue per channel and per priority (transactional vs marketing) -> channel workers -> provider adapters (APNs, FCM, an SMS and an email vendor) -> status updates; a scheduler enqueues delayed sends.
  • Deep dives: at-least-once plus dedup: queues redeliver, so workers are idempotent via `idempotency_key` (the same key returns the original result) and a sent-marker checked before the provider call; exactly-once is not achievable against an external provider. Retries use exponential backoff with jitter, a max attempt count, then a DLQ for inspection and replay; permanent errors (invalid token, hard bounce) are not retried and clean up the device or address. Isolate priorities with separate queues and worker pools so bulk cannot starve OTPs.
  • Bottlenecks: a provider outage is handled with a circuit breaker and a secondary provider (fallback per channel, e.g. a second SMS vendor) while preserving idempotency across them. Per-provider and per-user rate limits protect the provider and the user (daily caps, collapsing similar notifications). Fan-out of a campaign is done in batches by workers reading a segment, not by one giant request. Track delivery via provider callbacks, and expect that push is best-effort (no guaranteed delivery).

Examples

Go
import (
    "context"
    "errors"
    "math/rand"
    "time"
)

var errPermanent = errors.New("permanent failure")

// backoff returns the delay before retry n (1-based): exponential with
// full jitter, capped at max.
func backoff(n int, base, max time.Duration) time.Duration {
    d := base << (n - 1)
    if d <= 0 || d > max {
        d = max
    }
    return time.Duration(rand.Int63n(int64(d) + 1))
}

// deliver calls send until it succeeds, fails permanently, runs out of
// attempts (the caller then parks the message in a DLQ) or ctx ends.
func deliver(ctx context.Context, attempts int, send func() error) error {
    var err error
    for n := 1; n <= attempts; n++ {
        if err = send(); err == nil || errors.Is(err, errPermanent) {
            return err
        }
        select {
        case <-ctx.Done():
            return ctx.Err()
        case <-time.After(backoff(n, 200*time.Millisecond, 30*time.Second)):
        }
    }
    return err
}

Idempotent accept: the unique key turns a producer retry into a no-op.

sql
CREATE TABLE notifications (
    id              BIGINT PRIMARY KEY,
    idempotency_key VARCHAR(64) NOT NULL UNIQUE,
    user_id         BIGINT      NOT NULL,
    channel         VARCHAR(16) NOT NULL,
    status          VARCHAR(16) NOT NULL,
    attempts        INT         NOT NULL DEFAULT 0,
    created_at      TIMESTAMP   NOT NULL
);

INSERT INTO notifications (id, idempotency_key, user_id, channel, status, created_at)
VALUES (1001, 'order-42-shipped', 7, 'push', 'queued', NOW())
ON CONFLICT (idempotency_key) DO NOTHING;

A producer call with an idempotency key; retrying it is safe.

bash
curl -s -X POST https://notify.example.com/v1/notifications \
    -H "Authorization: Bearer $TOKEN" \
    -H "Content-Type: application/json" \
    -d '{"user_id":7,"template":"order_shipped","params":{"order":"42"},"priority":"high","idempotency_key":"order-42-shipped"}'

In practice

  • APNs and FCM — Both are best-effort: they hold or collapse messages for offline devices (APNs `apns-collapse-id`, FCM `collapse_key`) and report invalid tokens that you must delete. Never promise guaranteed push delivery.
  • Managed providers — Email and SMS usually go through vendors (e.g. Amazon SES/SNS, Twilio, SendGrid) that rate-limit you, report bounces and complaints by webhook, and need warm-up and sender reputation management for email.
  • Follow-up: how do you not send the same notification twice? — Idempotency key on accept, a sent-marker before the provider call, and idempotent consumers; accept that a crash between send and marker can still duplicate rarely, and use collapse keys so the device shows one.
  • Follow-up: how do you protect users from spam? — Per-user per-channel caps, quiet hours, digesting/collapsing, and honoring opt-out and unsubscribe immediately, including legal requirements for marketing email/SMS.

Trade-offs

Pros

  • Queue per channel and priority isolates failures and lets each channel scale and rate-limit independently.
  • Idempotency keys plus retries give safe at-least-once delivery without exactly-once machinery.
  • Provider adapters make fallback and vendor changes a local change.

Cons

  • Duplicates are reduced, not eliminated; a crash after sending can resend.
  • Many queues, pools and adapters to operate and monitor.
  • Delivery status is only as good as provider callbacks, and push has no delivery guarantee.

Metadata service plus chunked blob store: content-hash dedupe, resumable uploads via pre-signed URLs, erasure coding, sync and garbage collection.

The problem

Design a Dropbox/Drive/S3-like service: users upload and download files from many devices, files survive disk and datacenter failures, and clients sync changes. The central idea is to split a metadata service (small, strongly consistent) from a blob store (huge, cheap, durable) and break files into content-addressed chunks.

How it works

  • Requirements: upload/download (large files, resumable), folders and sharing, version history, multi-device sync; non-functional: durability above all (11 nines is the usual target wording), availability, low cost per GB. Clarify object (S3: flat key -> blob, HTTP API) vs file (hierarchy, POSIX-like) vs block (raw volumes) storage; this design is object-style with a file-sync layer on top.
  • Estimates (assumptions): 10 M users x 5 GB average = 50 PB logical. With 3x replication that is 150 PB raw; with 10+4 erasure coding (14/10 = 1.4x) it is 70 PB, a saving of ~80 PB. With 4 MB chunks that is 50 PB / 4 MB ~ 1.25 x 10^10 chunks; at ~100 B of metadata each that is ~1.25 TB of chunk index, which must be sharded.
  • API: `POST /v1/files {path, size, chunk_hashes[]}` -> which chunks are missing plus pre-signed upload URLs; `PUT <presigned-url>` per chunk (direct to blob store, in parallel, resumable); `POST /v1/files/{id}/commit`; `GET /v1/files/{id}` -> metadata and chunk list or a pre-signed download URL; `GET /v1/changes?cursor=` for sync.
  • Data model: metadata DB (sharded SQL/NewSQL, strongly consistent): `files(file_id, parent_id, name, owner, version)`, `file_versions(file_id, version, chunk_list, size, created_at)`, `chunks(hash, size, refcount, location)`. Blob store keyed by chunk hash (content-addressed).
  • Architecture: client chunks the file (4-8 MB) and hashes each chunk (SHA-256); the API checks which hashes already exist (dedupe) and returns pre-signed URLs only for the missing ones; the client uploads straight to the blob store and then commits the new version atomically in the metadata service. Downloads use pre-signed URLs, often via a CDN. A notification channel (long poll or WebSocket) tells other devices to pull changes.
  • Deep dives: durability: replication is simple and fast to read (3x), erasure coding stores k data + m parity shards (10+4 survives any 4 lost shards at 1.4x) but costs CPU and slower repair/read, so use replication for hot or small data and EC for cold or large. Placement: consistent hashing or a placement map with failure domains (rack/zone) so shards of one chunk are spread; background scrubbing checksums data and re-creates lost shards. Sync and conflicts: each file has a version; a commit that is based on a stale version becomes a conflicted copy (Dropbox-style) instead of overwriting. Delta sync only uploads changed chunks.
  • Bottlenecks: the metadata service is the choke point (every operation touches it), so shard by user/namespace and cache. Large uploads must never pass through the API tier (pre-signed URLs, multipart). Garbage collection: chunks with refcount 0 are deleted by a background mark-and-sweep after a grace period, because a racing upload may be about to reference them; never delete inline. Hot objects go behind a CDN; cross-region replication is asynchronous with a stated RPO.

Examples

Go
import (
    "crypto/sha256"
    "encoding/hex"
    "io"
)

const chunkSize = 4 << 20

type chunk struct {
    Hash string
    Size int
}

// splitChunks reads r in fixed 4 MiB pieces and returns each piece's
// SHA-256, which doubles as its dedupe key and blob-store name.
func splitChunks(r io.Reader) ([]chunk, error) {
    var out []chunk
    buf := make([]byte, chunkSize)
    for {
        n, err := io.ReadFull(r, buf)
        if n > 0 {
            sum := sha256.Sum256(buf[:n])
            out = append(out, chunk{Hash: hex.EncodeToString(sum[:]), Size: n})
        }
        if err == io.EOF || err == io.ErrUnexpectedEOF {
            return out, nil
        }
        if err != nil {
            return nil, err
        }
    }
}

// missing returns the chunks the server does not have yet.
func missing(chunks []chunk, have map[string]bool) []chunk {
    var out []chunk
    for _, c := range chunks {
        if !have[c.Hash] {
            out = append(out, c)
        }
    }
    return out
}

Metadata schema: versions point at ordered chunk lists; chunks carry a reference count for garbage collection.

sql
CREATE TABLE file_versions (
    file_id    BIGINT    NOT NULL,
    version    INT       NOT NULL,
    size_bytes BIGINT    NOT NULL,
    created_at TIMESTAMP NOT NULL,
    PRIMARY KEY (file_id, version)
);

CREATE TABLE version_chunks (
    file_id    BIGINT   NOT NULL,
    version    INT      NOT NULL,
    seq        INT      NOT NULL,
    chunk_hash CHAR(64) NOT NULL,
    PRIMARY KEY (file_id, version, seq)
);

CREATE TABLE chunks (
    chunk_hash CHAR(64) PRIMARY KEY,
    size_bytes INT      NOT NULL,
    refcount   INT      NOT NULL DEFAULT 0
);

SELECT chunk_hash FROM chunks
WHERE refcount = 0;

S3-style multipart upload: initiate, upload parts directly, then complete.

bash
aws s3api create-multipart-upload --bucket media --key videos/a.mp4
aws s3api upload-part --bucket media --key videos/a.mp4 --part-number 1 \
    --upload-id "$UPLOAD_ID" --body part-1.bin
aws s3api complete-multipart-upload --bucket media --key videos/a.mp4 \
    --upload-id "$UPLOAD_ID" --multipart-upload file://parts.json
aws s3 presign s3://media/videos/a.mp4 --expires-in 900

In practice

  • Amazon S3 — Documents a design target of 99.999999999% (11 nines) durability for objects, stores data redundantly across multiple devices and Availability Zones, offers strong read-after-write consistency, and supports multipart upload (parts of 5 MiB to 5 GiB, up to 10,000 parts). Say 'designed for', not 'guarantees'.
  • Dropbox — Its sync splits files into 4 MB blocks identified by a content hash so unchanged blocks are neither re-uploaded nor re-stored; it later moved block storage from S3 to its own system (Magic Pocket).
  • Facebook f4 — The f4 warm-blob paper uses Reed-Solomon (10, 4) erasure coding (1.4x within a datacenter) and reports a lower effective replication factor than keeping 3 full copies, while hot blobs stay replicated.
  • Follow-up: how do you resume a failed upload? — The client asks the server which chunk hashes it already has and uploads only the rest; with multipart uploads you list uploaded parts and continue, and you set a lifecycle rule to abort stale incomplete uploads.
  • Follow-up: how do you dedupe across users safely? — Cross-user dedupe by content hash saves storage but leaks 'someone already has this file' to a client that can probe by hash; mitigate by requiring proof of possession or limiting dedupe to within an account.

Trade-offs

Pros

  • Chunking plus content hashing gives dedupe, resumable and parallel uploads, and delta sync from one mechanism.
  • Splitting metadata from blobs lets each scale and be tuned independently (consistency vs cost).
  • Erasure coding cuts storage overhead from 3x to ~1.4x for cold data.

Cons

  • Small files suffer from per-chunk and metadata overhead; they are often packed or stored whole.
  • Erasure coding raises read/repair cost and latency, and needs enough nodes and failure domains.
  • Reference counting and garbage collection are easy to get wrong; a bug deletes user data.

Database selection

The ordered questions to ask and the rule for each answer, ending in a defensible default of PostgreSQL.

The problem

Interviewers do not want a feature list; they want to hear how you decide. Walk an ordered set of questions, state the rule for each answer, and land on a choice you can defend with numbers. The safe default is PostgreSQL until a measured need says otherwise.

How it works

  • Q1 Transactions, joins, constraints? If you need multi-row ACID, foreign keys or ad hoc joins, choose a relational database (PostgreSQL or MySQL). Money, inventory and bookings almost always land here.
  • Q2 Data shape and schema volatility? Aggregate-shaped records read and written as a whole (catalogs, profiles) with a changing schema fit documents (MongoDB, or JSONB in PostgreSQL). Tabular, relational data stays relational.
  • Q3 Dominant access pattern? Point lookups by key: key-value (Redis, DynamoDB). Range scans by partition then sort key: wide column (Cassandra, DynamoDB). Large aggregates over many rows: columnar (ClickHouse). Text relevance: a search engine.
  • Q4 Does one node suffice? A single well-provisioned relational primary comfortably handles terabytes and thousands to tens of thousands of QPS, with read replicas for reads. Shard only when a measured limit (write throughput, storage, failover time) is hit.
  • Q5 Write rate and regions? Sustained writes beyond one primary, or low-latency writes in several regions, push toward partitioned stores (Cassandra, DynamoDB global tables) or sharded SQL. Single-region read-heavy loads do not.
  • Q6 Consistency needs? Invariants that must never break (balances, unique usernames) want a single-writer ACID store. Feeds, counters and caches tolerate eventual consistency; say which anomalies you accept.
  • Q7 Team and operations? Pick what the team can run at 3 a.m. A managed service or a familiar engine beats a theoretically better one nobody knows. Say this out loud; it is a legitimate design input.
  • Default: start with PostgreSQL until a measured need says otherwise. It covers transactions, JSONB, full-text, geospatial and decent queues, so one system postpones operational sprawl, and migrating off it later is easier than unwinding a premature distributed store.
  • Polyglot persistence caveat: add a second store (cache, search index, warehouse) for a specific workload, keep one system of record, and feed the others by events or CDC. Every extra store costs sync logic, consistency gaps and on-call load.

In practice

  • Instagram — Documented in its engineering blog as running on sharded PostgreSQL, a standard example of scaling relational storage far past a single node.
  • Discord — Publicly described moving its message store from Cassandra to ScyllaDB, a Cassandra-compatible engine, to reduce operational pain at very large message volume.
  • Uber — Has written about moving a core system from PostgreSQL to MySQL, and about layering a sharded datastore on top of MySQL: the engine mattered less than the operational fit.
  • Interview note — Interviewers like to hear 'PostgreSQL first, here is the measured trigger that would make me shard or add a specialised store', not a list of NoSQL buzzwords.

Trade-offs

Pros

  • A fixed question order keeps the answer structured under pressure
  • Defaulting to PostgreSQL avoids premature distribution and operational sprawl
  • Each rule ties the choice to a stated requirement, so it is defensible

Cons

  • Defaults hide real exceptions: a 1M writes/s ingest path is not a PostgreSQL job
  • One database for everything eventually hits mixed-workload contention
  • Polyglot setups add sync lag and more failure modes to reason about

One card per database: best for, avoid when, scaling story and consistency stance.

The problem

You are asked 'which database?' for a component and have two minutes. Each database has a narrow sweet spot; name it, name when you would not use it, and say how it scales and what consistency it gives.

How it works

  • PostgreSQL: best for general OLTP with transactions, joins, constraints, JSONB and full-text. Avoid when sustained writes need many primaries or when scanning billions of rows for analytics. Scales vertically, with read replicas and partitioning; sharding needs extensions or app logic. ACID on a single primary, replicas lag.
  • MySQL: best for OLTP with simple, predictable queries and a mature replication ecosystem. Avoid when you need rich types or complex analytical SQL. Scales with read replicas and application or proxy sharding. ACID with InnoDB, replicas asynchronous or semi-synchronous.
  • MongoDB: best for aggregate-shaped documents with a changing schema. Avoid when you rely on many cross-document joins or invariants. Scales with replica sets and native sharding on a shard key (choose it carefully). Consistency is tunable via write and read concerns; multi-document transactions exist but cost more.
  • Redis: best for cache, sessions, counters, leaderboards, rate limits. Avoid as the only copy of durable data or data far above RAM. Scales with replicas and Cluster hash slots. Asynchronous replication means acknowledged writes can be lost on failover.
  • ClickHouse: best for aggregates over huge append-only data (events, logs, metrics). Avoid for frequent row updates, point lookups or transactions. Scales with shards and replicas. No multi-row ACID; replicas converge eventually.
  • Cassandra: best for very high write rates with key-based access and multi-datacenter availability. Avoid for ad hoc queries, joins or strong consistency. Scales horizontally by partition key on a ring. Consistency is tunable per query (`ONE`, `QUORUM`, `ALL`); model tables per query.
  • DynamoDB: best for predictable key access at any scale with little ops. Avoid when access patterns are unknown or heavy scans drive cost. Scales through automatic partitioning by partition key. Reads are eventually consistent by default; strongly consistent reads are optional, and transactions are supported within limits.
  • Elasticsearch / OpenSearch (search): best for full-text search, relevance ranking, facets and log search. Avoid as a system of record or for transactional needs. Scales by shards and replicas per index. Visibility is near-real-time; treat it as a derived index fed from the source of truth.

In practice

  • Redis in front of a relational primary — A very common pairing: PostgreSQL or MySQL as system of record, Redis for sessions, counters and hot reads.
  • ClickHouse beside OLTP — Analytics events land in ClickHouse while orders stay in the transactional store, so dashboards never scan the primary.
  • Cassandra and DynamoDB — Both descend from the Dynamo design (partitioned, replicated, tunable consistency); both reward modelling tables around known queries.
  • Search as a derived index — Elasticsearch/OpenSearch is usually fed from the database by events or CDC; if it is lost, you rebuild it.

Trade-offs

Pros

  • Cards give a fast, structured answer: best for, avoid when, scaling, consistency
  • Naming the 'avoid when' case signals judgement, not fandom

Cons

  • One line per database hides version and deployment differences
  • Cards say what is typical; measured workload data always overrides them

Index types, composite column order, covering and partial indexes, EXPLAIN basics, keyset pagination and secondary indexes in sharded stores.

The problem

'The query is slow' is the most common database follow-up. Know which index structure fits, how to order composite columns, how to read an EXPLAIN plan, and what every extra index costs on writes.

How it works

  • Structures: B-tree (default; equality, range, sorting; in-place updates), LSM-tree (writes buffered in memory then merged into sorted files; high write throughput, read and compaction cost; Cassandra, RocksDB), hash (equality only), inverted (term to document list; full-text), bitmap (low-cardinality columns, analytics).
  • Composite order: put equality columns first, then the range or sort column. The leftmost-prefix rule means an index on `(a, b, c)` serves `a`, `a,b` and `a,b,c` but not `b` alone. Order by selectivity only among equality columns.
  • Covering index: if the index holds every column the query needs (`INCLUDE` in PostgreSQL), the engine does an index-only scan and skips the table. Great for hot read paths, but wider indexes cost more storage and write work.
  • Selectivity and cardinality: an index on a column with few distinct values (a boolean, a status) rarely helps alone. Use partial indexes (`WHERE status = 'pending'`) or expression indexes (`lower(email)`) to index only what queries touch.
  • Write cost: every secondary index is another structure to update on each insert, update and delete, plus storage and cache pressure. Index for known queries, drop unused ones, and do not index tiny tables or columns that are never filtered.
  • Read the plan: run `EXPLAIN (ANALYZE, BUFFERS)`. Seq Scan on a large table, estimated rows far from actual rows (stale statistics), a Sort feeding a LIMIT, and high buffer reads are the red flags. Index Scan or Index Only Scan with few rows is the goal.
  • Smells: N+1 queries (one query per row of a list; fix with a join or `IN` batch), a missing index on a foreign key or filter column, a function on an indexed column (`WHERE date(created_at) = ...`) that defeats the index, and leading-wildcard `LIKE`.
  • Pagination: `OFFSET n` reads and discards n rows, so deep pages get slow and unstable under inserts. Keyset pagination remembers the last `(created_at, id)` and seeks past it using an index, at constant cost; the trade-off is no random page jumps.
  • Sharded stores: a local secondary index lives with each shard, so writes are cheap but a query without the shard key must scatter-gather all shards. A global secondary index is partitioned by the indexed value, so reads hit one shard but writes touch two and are usually asynchronous (stale).

Examples

Composite index: equality column first, range column second, then check the plan.

sql
CREATE INDEX idx_orders_user_created
    ON orders (user_id, created_at DESC);

EXPLAIN (ANALYZE, BUFFERS)
SELECT id, total
FROM orders
WHERE user_id = 42
  AND created_at >= now() - interval '30 days'
ORDER BY created_at DESC
LIMIT 20;

Partial and covering index: index only the hot slice, include the selected column.

sql
CREATE INDEX idx_orders_pending
    ON orders (created_at)
    INCLUDE (total)
    WHERE status = 'pending';

Keyset pagination: seek past the last row seen instead of using OFFSET.

sql
SELECT id, created_at, total
FROM orders
WHERE user_id = 42
  AND (created_at, id) < ('2026-01-15 10:00:00', 9000)
ORDER BY created_at DESC, id DESC
LIMIT 20;

In practice

  • PostgreSQL and MySQL (InnoDB) — Both default to B-tree indexes; PostgreSQL adds partial, expression and covering (`INCLUDE`) indexes and GIN for JSONB and full-text.
  • Cassandra and RocksDB — LSM-tree storage engines: writes go to a memtable and commit log, then flush to immutable sorted files merged by compaction.
  • DynamoDB — Offers local secondary indexes (same partition key) and global secondary indexes (different key, asynchronously updated), the standard example of the local vs global trade-off.
  • Interview note — Interviewers like to hear 'I would check EXPLAIN first and index for the actual query shape', and to see keyset pagination offered for any infinite-scroll feed.

Trade-offs

Pros

  • The right index turns a scan of millions of rows into a handful of page reads
  • Covering and partial indexes cut both read I/O and index size
  • Keyset pagination keeps deep pages at constant cost

Cons

  • Each index slows writes and consumes storage and memory
  • Composite indexes serve only the query shapes matching their prefix
  • Global secondary indexes in sharded stores are stale or expensive to keep consistent

Interview template

Clarify 5, estimate 5, high-level 10, deep dive 15, wrap-up 10: the exit criterion of each phase, how to steer time and how to compress to 35 or expand to 60.

The problem

A 45-minute system design interview is a timed conversation, not an exam with one right answer. The interviewer is judging whether you can structure an open-ended problem, spend time where it matters and communicate while you think. Running out of time in the middle of the deep dive is the most common way to fail a technically strong answer.

How it works

  • Agenda (45 min): Clarify 5, Estimate 5, High-level 10, Deep dive 15, Wrap-up 10. Say it out loud in the first minute: "I'll spend about five minutes on requirements, then sketch the design, then go deep on one or two parts. Tell me if you'd rather go elsewhere."
  • Clarify exit: do not leave until you can state the top 3 features, what is out of scope, the scale (DAU, read/write ratio) and the latency and availability stance, and the interviewer has nodded.
  • Estimate exit: do not leave until you have a QPS (average and peak), a storage figure per year and a bandwidth figure, and you know which of them will drive the design (for example "reads dominate, so cache").
  • High-level exit: do not leave until there is one diagram with APIs, data model and a traced request, and the interviewer confirms the direction. This phase is breadth only; park details with "I'll come back to this in the deep dive".
  • Deep-dive exit: do not leave until you have covered 2-3 hardest parts, each with bottleneck, options, decision and failure mode. Let the interviewer pick if they have a preference; their interest is the signal.
  • Wrap-up (last 5-10 min, can overlap): recap the design in 30 seconds, name the bottlenecks and single points of failure left, say how you would monitor it and what you would build next. Keep 2-3 minutes for the interviewer's questions.
  • Steer time: glance at the clock at each phase boundary, and if a phase overruns say "I'll take the simple option here and flag the alternative". When the interviewer interrupts, answer the question first, then say where you were so you return to the structure.
  • React to hints: a hint is a free pointer to what they want. Acknowledge it ("good point, that's a hot-key problem"), adopt it, and spend your next two minutes there rather than defending the earlier plan.
  • Signpost and keep the board clean: say "next I'll cover X" before each section, draw left to right (clients, edge, services, stores), label every arrow with its protocol or data, number the request flow, and write the key numbers in a corner so they stay visible.

Examples

Agenda printer: pass 35, 45 or 60 to see the compressed or expanded schedule.

bash
#!/usr/bin/env bash
# Print the interview agenda for 35, 45 (default) or 60 minutes.
set -eu
[ $# -ge 1 ] || set -- 45

case "$1" in
    35) m="3 4 8 12 8" ;;
    45) m="5 5 10 15 10" ;;
    60) m="5 7 13 20 15" ;;
    *) echo "usage: $0 [35|45|60]" >&2; exit 1 ;;
esac

set -- $m
printf '%-12s %2s min  exit when...\n' "Clarify" "$1"
printf '%-12s %2s min  QPS, storage, bandwidth known\n' "Estimate" "$2"
printf '%-12s %2s min  diagram + API + traced request\n' "High-level" "$3"
printf '%-12s %2s min  2-3 hard parts, decisions made\n' "Deep dive" "$4"
printf '%-12s %2s min  recap, risks, monitoring, next\n' "Wrap-up" "$5"

In practice

  • What strong candidates say — "I'll cover requirements, then a high-level design, then dive into the two riskiest parts. We have about 45 minutes, so I'll check in at each boundary." They state the plan, then follow it.
  • What interviewers listen for — Whether you can prioritise: skipping a standard component with one sentence ("auth is a standard JWT gateway, I'll not expand") signals judgement; spending 10 minutes on it signals the opposite.
  • Red flag — Still defining requirements at minute 15, or drawing the first box at minute 3 without a single question. Both mean the structure is missing. Also: ignoring a hint and continuing the original plan.
  • Compressing and expanding — 35 min: Clarify 3, Estimate 4, High-level 8, Deep dive 12, Wrap-up 8; do one deep dive. 60 min: Clarify 5, Estimate 7, High-level 13, Deep dive 20, Wrap-up 15; do three deep dives and add failure and cost discussion.

Trade-offs

Pros

  • Do more of: announce the plan, check in at phase boundaries and keep numbers visible on the board.
  • Do more of: park details explicitly ("deep dive later") so the high-level phase stays broad and finishes on time.
  • Do more of: treat interruptions and hints as steering and re-plan the remaining minutes out loud.

Cons

  • Mistake: following the template rigidly when the interviewer clearly wants to go deep on something earlier.
  • Mistake: spending 12 minutes on estimation arithmetic; round aggressively and move on.
  • Mistake: no wrap-up, so the interview ends mid-sentence in a detail with no summary of the design.

The clarification checklist with a ready-to-say question for each item and a template to state the agreed requirements back.

The problem

The prompt ("design Twitter", "design a file store") is deliberately vague. In the first five minutes the interviewer evaluates whether you ask what matters instead of assuming, whether you narrow the problem to something designable in 45 minutes, and whether your questions are tied to design decisions (every number you ask for should change the architecture).

How it works

  • Functional scope: ask for the top 3 features and what is explicitly out. Say: "I'll assume post, follow and read timeline are in; search, ads and DMs are out. OK?"
  • Users and clients: who uses it, how many, which platforms. Say: "Is this consumer mobile and web, or internal service-to-service? Do we need offline support?"
  • Scale: DAU, requests per user per day, data size per item, growth. Say: "Roughly how many DAU, and should I design for 10x growth over a few years?" If they shrug, propose a number and move on.
  • Read/write ratio: it decides cache, replicas and storage. Say: "I'd guess this is read-heavy, around 100:1. Does that match your expectation?"
  • Latency targets: ask for a p99 for the main path. Say: "Is p99 under 200 ms acceptable for reads? Writes can be a bit slower?"
  • Availability vs consistency stance: ask which failure is worse. Say: "If a partition happens, is it worse to show stale data or to fail the request?" Money and inventory usually mean consistency; feeds and likes mean availability.
  • Durability, geography, security: "Can we ever lose a write? Is the data single-region or global? Any compliance such as GDPR, PCI or data residency?"
  • Constraints: "Is there an existing stack or cloud I must use, and any budget or team-size limit?" Do not invent exotic infrastructure the team cannot run.
  • State the agreed requirements back in the template (see the snippet), write them on the board, and ask "anything to change before I move on?". Then start estimation.

Examples

Requirements template: fill it on the board, read it back, get a nod, then move to estimation.

yaml
# Requirements, stated back to the interviewer (example: news feed)
functional:
  in_scope:
    - post a short message with optional image
    - follow and unfollow users
    - read home timeline (newest first)
  out_of_scope:
    - search, ads, direct messages
users:
  dau: 50M
  clients: [ios, android, web]
scale:
  reads_per_user_per_day: 100
  writes_per_user_per_day: 1
  read_write_ratio: "100:1"
  growth: 10x in 3 years
nonfunctional:
  latency_p99_ms: {read: 200, write: 500}
  availability: "99.95%, prefer stale feed over failing"
  consistency: eventual for timeline, read-your-writes for own posts
  durability: no lost posts
  regions: multi-region reads, single write region
  compliance: GDPR delete requests within 30 days
constraints:
  stack: existing cloud, managed services preferred

In practice

  • What strong candidates say — "Before I design, let me narrow the scope. I'd like to confirm three features, rough scale and whether we favour consistency or availability." Every question is followed by a stated assumption, so the conversation never stalls.
  • What interviewers listen for — Questions that change the design: read/write ratio (cache or not), consistency need (SQL or Dynamo-style store), global vs single region (replication). Generic questions with no design consequence score little.
  • Red flag — Asking 15 questions with no pause, or none at all. Also silently assuming a scale ("it's like Facebook") and designing for it without agreement.

Trade-offs

Pros

  • Do more of: pair every question with a default assumption, so you can proceed if the interviewer says "up to you".
  • Do more of: tie each answer to a design consequence out loud ("read-heavy, so a cache").
  • Do more of: write requirements and numbers on the board; they become the yardstick for the later trade-off discussion.

Cons

  • Mistake: asking for every checklist item mechanically; pick the 5-6 that matter for this problem.
  • Mistake: accepting an enormous scope ("all of YouTube") instead of narrowing it to a designable core.
  • Mistake: forgetting non-functional requirements and discussing only features.

The order to draw the boxes, API first, data model, read and write path walkthroughs and one end-to-end request trace.

The problem

With requirements and numbers on the board, the interviewer wants a complete but shallow end-to-end design in about 10 minutes: the main components, how a request flows through them and where data lives. Depth comes later; here they check that the whole picture hangs together and that your APIs and data model follow from the requirements.

How it works

  • Define the API first: 3-5 endpoints, for example `POST /v1/posts`, `GET /v1/timeline?cursor=…&limit=20`. Name the resource, auth, and for writes an idempotency key (`Idempotency-Key` header) so retries are safe.
  • Pagination: prefer cursor (keyset) over offset for feeds and lists; mention a stable sort key. Say gRPC for internal service calls and REST for public ones, unless the problem says otherwise.
  • Data model and storage choice: list the 2-4 core entities, their keys and the main access patterns, then pick a store per pattern (relational for transactions, wide-column or KV for huge write volume, object store for blobs, search index for text). Justify with the read/write ratio.
  • Draw in this order, left to right: clients, then edge (DNS, CDN, load balancer, API gateway), then stateless services, then data stores, then async parts (queue, workers), then cache. Adding the cache last forces you to say what it caches and why.
  • Name each service by responsibility, not technology, and keep it to 4-6 boxes. Start with a modular monolith mindset: split only where scale, ownership or failure isolation demands it.
  • Walk the write path: client, gateway, service, validate, durable write, then enqueue any fan-out or indexing as async work. Say where the write is acknowledged (after the durable write, not after the fan-out).
  • Walk the read path: client, CDN or cache hit, otherwise service, then store, then fill cache. Mention the cache-miss path and where staleness can appear.
  • Trace one request end to end with real values ("user 42 posts, row is written to the posts table, an event goes to the queue, workers update followers' timelines"). Number the arrows on the board as you narrate.
  • Checkpoint: stop after the trace and ask "Does this match what you had in mind? Which part would you like me to go deeper on?". Mark open details with a star for the deep dive.

Examples

API first: an idempotent write and a cursor-paginated read.

bash
# API sketch, written on the board (news feed example)
curl -X POST https://api.example.com/v1/posts \
    -H "Authorization: Bearer $TOKEN" \
    -H "Idempotency-Key: 7c1f0a52-3d1e-4a53-9a58-1f2b7f0f9c11" \
    -d '{"text": "hello", "media_ids": []}'

# keyset pagination: the cursor encodes (created_at, id) of the last item
curl "https://api.example.com/v1/timeline?limit=20&cursor=MTcxMjM0NTY3OHw5OTk"

In practice

  • What strong candidates say — "Let me define the API first, then the data model, then draw the boxes." They narrate one request through the diagram with concrete values and then ask which part to expand.
  • What interviewers listen for — A clear separation of sync and async work, a justified storage choice per access pattern, and an explicit statement of where the write is acknowledged and where data may be stale.
  • Red flag — A diagram that is just a list of famous products (Kafka, Redis, Cassandra, Elasticsearch) with no data flow, or boxes with no stated responsibility.
  • Stripe-style idempotency — Public payment APIs commonly accept an idempotency key header so a client can safely retry a write; mentioning this for any money-moving or order-creating endpoint is an easy credibility signal.

Trade-offs

Pros

  • Do more of: APIs and data model before boxes; they anchor every later decision.
  • Do more of: label arrows with the protocol or payload and number the request flow.
  • Do more of: checkpoint with the interviewer once the end-to-end trace is done.

Cons

  • Mistake: drilling into one component (for example the schema of a table) before the whole picture exists.
  • Mistake: drawing 15 microservices for a 1k QPS product; start with few boxes and justify each split.
  • Mistake: leaving out the read path or the async path, so the design cannot be traced end to end.

How to pick the 2-3 hardest parts and run a drill on each, plus a scaling, reliability, observability and data-lifecycle playbook.

The problem

The deep dive is where seniority shows. The interviewer wants you to find the hardest parts of your own design from your numbers, reason about options and trade-offs, and talk about failure and operations, not only the happy path. You get time for two or three topics, so choose them on purpose.

How it works

  • Choose the 2-3 hardest parts from your estimates and requirements: the highest QPS path, the biggest data set, hot keys or celebrity fan-out, the strictest consistency need. If the interviewer points somewhere, go there first.
  • Run the same drill on each topic: bottleneck (from your numbers), options (2-3), trade-off, decision, failure mode, how you would monitor it. Saying the six words aloud keeps the answer complete and short.
  • Scale reads: cache first (cache-aside, TTL, hot-key protection), then read replicas, CDN for static or public content, and denormalised or precomputed views. Name the staleness you accept.
  • Scale writes and storage: shard with a partition key that spreads load and matches the main query (check for hot keys and celebrity users), batch writes, and move non-urgent work to queues and workers. Explain what happens when you reshard.
  • Reliability: replicate with a stated failover plan, retries with exponential backoff and jitter on idempotent calls only, timeouts everywhere, circuit breakers, rate limiting and load shedding, and graceful degradation (serve stale, hide non-critical features).
  • Consistency and correctness: say which data needs strong consistency (money, inventory) and which is fine with eventual; use idempotency keys, unique constraints and the outbox or saga pattern when a request touches several systems.
  • Observability: RED (rate, errors, duration) for services and USE (utilisation, saturation, errors) for resources, plus distributed tracing and alerts on SLO burn. Name the one or two metrics that would tell you this specific bottleneck is biting.
  • Data lifecycle: retention and TTLs, tiered or cold storage for old data, compaction, backups and restore tests, and deletion requests for compliance.
  • Security basics, one sentence each unless asked: authn/authz at the gateway, TLS in transit and encryption at rest, least privilege between services, and input validation and abuse limits on public endpoints.

Examples

One filled-in drill: bottleneck, options, trade-off, decision, failure mode, monitoring.

yaml
# Deep-dive drill: say these six words for every topic
topic: hot celebrity accounts in the timeline
bottleneck: "1 post fans out to 20M followers; 1 write = 20M timeline writes"
options:
  - fan-out on write for everyone
  - fan-out on read for everyone
  - hybrid: push for normal users, pull for celebrities at read time
trade_off: "push = fast reads, heavy writes; pull = cheap writes, slower reads"
decision: hybrid, threshold around 100k followers
failure_mode: "queue backlog delays timelines; degrade to pull-only and serve stale"
monitor: [fanout_queue_lag_seconds, timeline_read_p99_ms]

In practice

  • What strong candidates say — "From my numbers the bottleneck is X. Options are A and B. I'd pick A because of Y; the cost is Z, and if it fails we degrade by W. I'd watch metric M." It is short, structured and ends with an operational view.
  • What interviewers listen for — Whether the deep dives come from your own estimates rather than a memorised list, whether you mention hot keys and skew, and whether you can reason about what breaks when a node, a queue or a dependency goes down.
  • Red flag — Listing techniques ("we'll add Redis, sharding and Kafka") without saying which bottleneck each solves, or claiming a design has no downside.
  • Twitter/X home timeline — Commonly cited as the hybrid fan-out example: precompute timelines for most accounts at write time, and merge celebrity accounts at read time. Use it as an illustration, not as exact current behaviour.

Trade-offs

Pros

  • Do more of: derive the dive topics from your numbers and say so ("20M followers breaks fan-out on write").
  • Do more of: finish each dive with failure mode and monitoring, which most candidates skip.
  • Do more of: offer a simpler first version and the condition under which you would upgrade it.

Cons

  • Mistake: spreading across six topics at one-minute depth; two or three at depth beat six shallow ones.
  • Mistake: picking a shard key without checking skew or the main query pattern.
  • Mistake: retrying non-idempotent calls, or retries without backoff and jitter, which cause retry storms.

The mistakes that sink system design answers, and a last-5-minutes checklist: recap, bottlenecks, monitoring, next steps.

The problem

Most failed system design interviews fail on process, not knowledge: no requirements, no numbers, too much technology, no trade-offs. This entry lists the mistakes interviewers see most, how to recognise yourself making them, and a last-five-minutes checklist so the interview ends on a strong summary.

How it works

  • Jumping into components before requirements: if you drew a box before asking a question, stop and say "let me back up and confirm scope". It costs 20 seconds and shows self-awareness.
  • No numbers: every design needs QPS, storage and bandwidth to justify itself. If you caught yourself saying "it scales" without a figure, attach one.
  • Over-engineering: microservices, Kafka and a multi-region active-active setup for 10 QPS is a red flag. Start simple (one database, one cache), then state which number would force each upgrade.
  • Ignoring failure modes: for every component ask "what if this dies, is slow or returns garbage?". Name the timeout, retry, fallback and alert.
  • One database for everything, or buzzword stores: match the store to each access pattern and explain it in terms of reads, writes and consistency, not brand names.
  • Single points of failure and hot spots: check every box for redundancy (load balancer, queue, primary database) and check every partition key for skew, celebrity users and hot keys.
  • Not stating trade-offs: never present one option. Say "A is faster to build but B scales better; I'd pick A now because of the 1k QPS estimate".
  • Monologuing and ignoring hints: check in every few minutes ("does this direction work?") and treat interviewer questions as steering. Never drop a buzzword you cannot explain in two sentences.
  • Last 5 minutes checklist: (1) recap the design in 30 seconds along the request path, (2) name the remaining bottlenecks and single points of failure, (3) say how you would monitor it (RED and a few SLOs), (4) list next steps you would build or improve, (5) ask if they want to explore anything.

In practice

  • What strong candidates say — "The simplest thing that meets 1k QPS is one Postgres with a read replica. I'd move to sharding only past roughly X writes per second, and here is what would change." They justify scale decisions with their own numbers.
  • What interviewers listen for — Self-correction ("I forgot the failure case, let me add it"), explicit trade-offs, and the ability to explain any named technology in plain words when pushed.
  • Red flag — Saying "we'll use Kafka" or "we'll shard it" and, when asked why, being unable to explain partitions, ordering or the shard key. Also getting defensive when the design is challenged.
  • Wrap-up in practice — A 60-second close usually lands best: one sentence on what the system does, the request path in three steps, the two biggest risks, and what you would do with another week.

Trade-offs

Pros

  • Do more of: self-correct out loud; interviewers reward noticing a mistake before they point it out.
  • Do more of: start simple and name the trigger number for each upgrade.
  • Do more of: close with recap, risks, monitoring and next steps.

Cons

  • Mistake: memorised answers to famous questions that ignore the requirements given in this interview.
  • Mistake: spending the whole time on one beloved component (usually the database or cache) and never covering the rest.
  • Mistake: ending abruptly in a detail, with no summary and no questions for the interviewer.