Section 9: Config Is Code That Skips Review - Deep Dive: Rule Configuration & Control Plane

Executive Summary: Sections 7 and 8 made each decision local and cheap. So a rule, once it reaches a load balancer, acts on every request with nothing in between. This section follows a rule from the owner's PUT through validation, versioned SQL, a per-cluster compiler, notify-plus-pull distribution, 24 hours of shadow mode and a staged global rollout. A separate emergency path with a mandatory TTL reaches every region in ≤ 60 s. The rule for this section: nothing reaches every load balancer at once, except an override that expires on its own.

Interviewer: "You've got 10 million rules and thousands of load balancers. How does Contoso's new 10,000 req/s rule get from the owner to the LBs in West Europe?"

You: "First, a reframe. The data plane does whatever its snapshot tells it to, at 30,000 decisions a second per LB, so a rule is effectively code that ships without a code review. My job here is to put a review, a test and a rollout in front of it, and still have a regional p99 of 5 seconds."

The lifecycle, one stage at a time

Author. Owners write rules through Section 4's Rules API or the portal. Product writes the SKU templates (Free, Standard, Premium, Enterprise). Across the fleet that comes to about 50K changes a day, roughly 0.6 a second on average, arriving in bursts at deploy time. The rate is trivial; the risk per write is not.

Validate. There are three checks, in increasing order of cost:

1. Schema: rate > 0, burst ≥ 1, and per is a known unit.
2. Conflicts: Section 5's precedence must settle the rule. A tie between equally specific rules that priority doesn't break is rejected here, not when a request hits it.
3. Blast radius: replay the rule against the last 24 hours of per-key-per-minute telemetry from Section 5 and report something like "this rule would throttle 30% of current Directory traffic, most of it from a handful of tenants." That is an estimate: it can't see dimensions that aren't keys yet, or per-second bursts, which is what shadow mode measures. Above a threshold, the write warns and needs a second approver. At :promote the check runs again on shadow data, now as a measurement, and refuses above the threshold unless a second approver signs.

Store. The rule lands in versioned SQL (200 B per rule, **2 GB globally**) with an append-only history. If-Match turns two owners editing at the same moment into a 412 instead of a silent overwrite.

Compile. No LB needs all 10M rules. The compiler builds one snapshot for each LB cluster, containing only the rules for services that cluster fronts: ≤ 100K rules, about 20 MB, with precedence already resolved. Each snapshot gets a monotonic version. Contoso's rule entered cluster WE-dir-07's snapshot as v812 in shadow mode, and its promotion becomes v813; the rule keeps its own version (1, then 2). The snapshot numbers are consecutive here only for readability, since other rules' changes land in between.

Quick check before moving on: