Section 10: Six Ways to Break It - Interviewer Deep-Dive Challenges

Executive Summary: The interviewer now tries to break the design with six escalations: a 2M req/s key, a global limit for Contoso, instant retries, a bad rule that throttles 40% of Directory traffic, the West Europe allocator dying at peak, and a customer who says they were throttled under their limit. None needs a new box: Sections 6–9 already built every mechanism.

Each question targets one assumption, and the score depends on whether you name it before you defend it.

Challenge 1: The Hot Key

"One tenant does 2M req/s on a single key. Your allocator melts, doesn't it?"

What they're really asking: does coordination cost grow with requests or with keys? In Section 3's Redis design this key alone needs 10–20× one shard.

The answer: the allocator never sees requests, only one report entry per LB per second. This key costs 500 entries/s on one of 16 shards, the same as any key that every LB sees, whatever its rate. The heat lands on the LB instead. Spread evenly, 2M req/s is ~4,000 req/s per LB, easy for one CAS loop (Section 7). Through only 40 LBs, as with Contoso, it is 50,000 req/s per LB. Once CAS retries show in tail latency, the LB splits the bucket across its W workers at share ÷ W, and a dry worker borrows from a sibling before it denies.

The senior line: "Leasing makes coordination cost scale with keys × LBs, not requests. A hot key becomes a per-core contention problem, which I fix locally."

Quick check before moving on: