Section 4: Four Audiences, One Rule - API Design (with Pushback)

Executive Summary: A rate limiter has four contracts for four audiences: owners who write rules, the load balancer filter deciding every request, the throttled client that must slow down, and the allocator protocol that keeps 500 load balancers in agreement. We design each around Contoso's Directory API rule, then defend the four choices interviewers push on hardest: no per-request central check, 429 over 503, exposing remaining, and mandatory shadow mode.

The interviewer caps the marker: "Walk me through the API." A weak answer jumps straight to POST /check. A stronger one first names who sits on the other end of each contract.

You: "Four audiences. Owners need a careful, audited rules API. The load balancer needs a decision function that costs microseconds and never leaves the process. Clients need a standard rejection they can act on without our docs. And load balancers need a batched, once-a-second allocator protocol off the hot path."

Audience One: Owners and the Rules API

This is the control plane: about 50K changes a day from humans and CI pipelines. Correctness and auditability matter; latency doesn't. Contoso's rule:

PUT /v1/rules/dir-users-contoso
If-None-Match: *
Content-Type: application/json

{
  "scope":     { "service": "directory", "api": "GET /v1/users", "tenant": "contoso" },
  "limit":     { "rate": 10000, "per": "1s", "burst": 10000 },
  "algorithm": "TOKEN_BUCKET",
  "mode":      "SHADOW",
  "failMode":  "OPEN",
  "priority":  100
}

201 Created
ETag: "1"
{ "ruleId": "dir-users-contoso", "version": 1, "snapshot": "v812", "mode": "SHADOW", "propagation": "PENDING" }

• PUT on a caller-chosen ruleId makes the write idempotent. A CI pipeline retrying after a timeout lands the same rule, not a duplicate.
• If-None-Match: * makes this a create: if the rule already exists, the call gets 412 Precondition Failed. Later edits send If-Match with the rule's ETag, so a stale writer gets 412 instead of silently overwriting a colleague.
• mode defaults to SHADOW. A new rule logs "would throttle" decisions and enforces nothing.
• failMode is chosen per rule. Contoso's tenant rule is OPEN. The service-wide protective cap on Directory API (2M req/s per region) is STATIC, so it keeps its last shares even if the allocator vanishes.

The rest of the surface is small:

The rule's own version goes 1, then 2 on promotion. LBs run cluster snapshots: v812 is the Directory cluster's snapshot once Contoso's rule lands. Snapshot versions rise monotonically and every load balancer reports the one it runs, which is how propagation is tracked (regional p99 ≤ 5 s, staged global rollout ≈ 30 min). Section 9 builds that pipeline.