Section 3: The Load Is Heartbeats, Not Locks - Scale Estimation & Workload Analysis

Executive Summary: One large cell serves 100,000 sessions. One KeepAlive per session per 12 s lease gives ~8,300 KeepAlives/s, which is most of the master's work. Reads add 1,000/s once a 99% client cache absorbs 100,000/s of demand, and consensus writes add 500/s at peak. The steady state comes to ~10,000 RPC/s, 20% of a single master's 50,000/s ceiling, and commits use 5% of log capacity. The whole database is about 1.3 GB, so it lives in RAM. So one master per cell is enough, the cache is mandatory, and lease length becomes a load knob. The numbers that can actually hurt are two bursts: the failover reconnect storm and hot-file invalidation.

The interviewer taps Section 2's requirements. "Now size it, and tell me what the arithmetic decides." Most candidates start counting locks. That's the trap: locks are the product, not the load.

Sessions: The Unit of Everything

You: "I'll size one large data-center cell. Every client process holds exactly one session, and its library multiplexes all its handles over it. Peak: 100,000 sessions per cell. orders-db alone is 24,000 of them: 4,000 tablet servers plus 20,000 front-ends."

You: "Tasks restart and roll out, so say a mean session lifetime of ~1 hour. 100,000 / 3,600 s ≈ 28 creates/s, and the same 28 expiries/s, so ≈ 56/s. I'll round that to 60/s in the write mix. Each one goes through consensus, because a session's existence is replicated state."