Section 1: Who Is Asking, and How Exact Is Fair? - Problem Understanding & Clarifications

Executive Summary: This section covers the opening ten minutes of a Staff/Principal interview on a cloud-scale rate limiting platform. We split the prompt into what, who, why and how, then run a clarifying exchange in which every answer settles a decision: dimensions, placement, latency, accuracy, scope, failure posture, authorship, and rate limits versus quotas. We close with an assumptions table and introduce Contoso, whose 02:00 sync job runs through the course.

The interviewer reads the prompt and stops talking:

"Design the rate limiting platform for a Microsoft-scale cloud. Ten thousand internal services and millions of external tenants call each other through our load balancers. Owners need per-tenant, per-caller and service-wide limits, enforced at the edge and between services, and the limiter can add at most about a millisecond — and must never be the reason we're down."

Most candidates hear "rate limiter" and reach for the textbook answer: a counter in Redis, one INCR per request. But the prompt contains a latency ceiling ("about a millisecond") and an availability rule ("never the reason we're down"), and the Redis answer quietly breaks both. A Staff candidate notices that before drawing anything.

Key Insight: A rate limiter's only job is to say "no", and it can fail three ways. Too often, and you throttle paying customers below their contract. Too slowly, and you add latency to every call in the cloud. Not at all, and you have a single point of failure for ten thousand services. The clarifying questions find out which failure is tolerable, and how much.

Decomposing the Prompt

Before asking anything, map the territory out loud:

• What it does: For every request, decide allow or deny against one or more limits and tell a denied caller when to come back. It also manages the rules: who sets them and how they reach the enforcement points.
• Who uses it: Service owners write rules and want to see what got throttled. Callers, external tenants and internal services, need a clear back-off contract. The load balancers run the decisions and need them in microseconds.
• Why it exists: To protect backends from overload, keep tenants from crowding each other out, and enforce the promises behind SKU tiers. Protection and fairness are different goals, and they will want different failure behaviour.
• How it's used: Inline, on every request through the edge and the service mesh, at tens of millions of requests per second. Nobody calls the limiter on purpose; it sits in the path of everything.

Quick check before moving on: