Section 6: The Decision Never Leaves the Box - High-Level Architecture

Executive Summary: This section draws the platform as two planes. The data plane is a limiter library in the Envoy LBs and mesh sidecars that decides every request locally. The control plane is everything that changes slowly: rules, compiler, config distribution, a 16-shard regional allocator, a global allocator and telemetry. We trace Contoso's 02:00 ramp from first request to first 429, then defend the boxes that are deliberately missing: Redis, the per-request check RPC and a durable allocator.

The interviewer hands you the marker: "Draw it." First say the rule the drawing obeys, because Section 3 already priced it: a per-request central counter costs 15M ops/s in West Europe and a 0.5–1 ms round trip per call, the entire latency budget. So:

The one-line idea: every request is decided in the memory of the box it is passing through, and the only thing that crosses the network is a once-a-second conversation about how much each box may admit.

The Whole System on One Whiteboard

The sidecar talks to config, allocator and collectors as the edge LB does; those arrows are drawn once. Caller rules such as Billing → Directory are enforced at the callee's inbound sidecars, Directory's own pods, a bounded set. Sidecars on one node send reports through a per-node agent, so the allocator's reporter count stays in the low thousands, not the mesh's tens of thousands.