Section 3: Objects Are Tiny, Bytes Are Huge - Scale Estimation & Workload Analysis

Executive Summary: One region holds 10 trillion objects and about 1 EB, but the two numbers size different hardware: 90% of objects are under 100 KB and hold 5% of the bytes; 0.1% hold 80%. Object count sizes the index (2 PB of records, ~12 PB of SSD, ~40,000 range partitions); bytes size the disks (1.8 EB provisioned, 90,000 × 20 TB HDDs, 2,500 nodes in 10 clusters). Peak load is 5M req/s and 5.6 Tbps of egress, so bytes can never pass through the index. The average partition sees ~125 req/s against a 3,500 writes/s + 5,500 reads/s limit, so skew drives splitting. The fleet loses ~5 disks a day, so repair is steady-state work, and independent-failure math lands at 10^-17 per year, so correlated failure is the real enemy.

The interviewer has Section 2's requirements on the board and caps the marker. "Size it before you draw a box, and tell me which numbers you would actually build around." That is the test. Anyone can multiply; a Staff candidate says what the product changes. Every number below ends in a "so what."

Objects: Ten Trillion Keys

You: "One large region is the unit. The biggest public clouds report a few hundred trillion objects across a few dozen regions, so call it 10 trillion objects, 10^13."

So what: every request, even a 4 KB thumbnail read, has to find its object before a byte moves, so the index holds 10 trillion entries whatever those objects weigh. Object count sizes the index. Nothing else does.