Section 6: Bytes Never Meet the Index - High-Level Architecture

Executive Summary: Three layers with one rule between them. A stateless front-end fleet of ~1,000 nodes terminates TLS, verifies signatures and throttles; an index layer of ~1,000 partition servers owns ~40,000 key ranges and alone decides what exists; a data layer of 2,500 storage nodes in 10 clusters holds the bytes, which never pass through the index. Four walks fix the shape: Priya's 40 KB thumbnail costs one index round trip and three journal appends inside 200 ms; each 48 MB part of clip-00123.mp4 streams straight into 15 fragments and Complete writes a manifest without copying a byte; a GET spends 2 ms in the index and the rest of its 100 ms first-byte budget on one disk read; a LIST is one ordered range scan under 500 ms. The index is on every request's path, so it must be small, fast and strongly consistent; the data layer is on every byte's path, so it must be wide and simple.

The interviewer taps the record layouts from Section 5. "Two shapes of record. Draw the system that serves them at 5 million requests a second, then walk a request through it."

You: "Three layers, because each scales on a different number: the front-end on requests, the index on keys, the data layer on bytes."

The Three Layers

Front-end fleet. Stateless HTTP servers that do everything a request needs before it touches state: TLS, SigV4-style signature verification and bucket-policy evaluation (D8), per-partition and per-account token buckets (D9), and routing by a cached partition map (D15). Sizing is N15: 5M req/s at peak over 10K req/s per node is 500 nodes; with 2× headroom and N+1 AZ survival, **1,000 nodes**, ~333 per AZ, so losing an AZ leaves 667.

Index layer. 1,000 partition servers, each owning ~40 of the **40,000 range partitions** over (bucket_id, key). A partition has a single owner, which is what makes read-after-write and list-after-write strong (D5); its commit log is Paxos-replicated three ways into the data layer. A small, strongly consistent partition map service assigns ranges to owners. The average is gentle, ~125 req/s per partition against a limit of 3,500 writes/s plus 5,500 reads/s, but hot partitions run 10–100× that, and the skew is Section 8's whole subject.

Data layer. 2,500 storage nodes of 36 HDDs, an SSD stream tier for index logs and tables (the ~1,600 NVMe drives of N6, 3× replicated), a stripe/extent manager that allocates stripes and places their 15 fragments 5 per AZ on distinct racks and disks (D3), and journal nodes holding the three replicas of every small write (D4). Five background services keep it honest: the EC encoder packs journal bytes into stripes within ~10 minutes, the compactor reclaims space behind tombstones (D6), the scrubber re-reads every fragment every 30 days, repair rebuilds the ~5 disks that fail each day, and lifecycle (Section 12) walks the index change feed.

The whole shape, background services included:

Interviewer: "Ceph puts placement on the storage nodes with a hash and has no index tier. Why carry a separate one?"

You: "Ordered LIST and per-key throttling both need a range-ordered index (Section 8); Ceph's hash placement gives up both, and a 2 PB index on NVMe scales and fails on a different curve from 1.8 EB on HDD. What flips me is a small deployment with no ordered LIST and no tenants, where hashing key to node and skipping the index is honest engineering (D1)."