Section 7: Nobody Waits for the Slowest Box - Deep Dive: The Ads Front Door

Executive Summary: The front door owns two promises: answer inside 80 ms p99, and never break the feed. It stamps one absolute deadline on the request and gives every hop its remaining budget minus a reserve. It fans out to all 16 shards, proceeds with whatever returned when retrieval's 15 ms runs out, and hedges calls slower than the p95 (~8 ms), capped at 5% extra load. It sheds low-value placements first and walks a degradation ladder down to an empty slate. Degraded responses stay ≤ 0.5% on a normal day, and any valid response in time, even an empty one, counts toward 99.99% availability.

The architecture is on the board. The interviewer puts the marker down: "You've got two diagrams' worth of boxes up there. Where would you like to go deeper?"

You: "The front door. Every other box can be clever. This one decides what happens when a box is late, and at 1.2 million requests a second some box is always late. At ~$260K a minute, a front door that turns slowness into errors is the most expensive bug in the design. Retrieval and ranking come right after."

One Clock for the Whole Request

SelectAds arrives with the caller's deadline (Section 4). The front door takes the earlier of that and now + 80 ms, and the remaining budget travels as the gRPC deadline on every downstream call.

Interviewer: "Why not give each call a timeout and be done?"

You: "Timeouts compound. If retrieval gets 15 ms, light ranking 5 and heavy ranking 20, and retrieval runs 15 ms over, every later stage still spends its full allowance, and the request lands 15 ms late, past 80 even after the 12 ms of slack. An absolute deadline makes lateness visible downstream: the heavy ranker can see it has 14 ms left, not 20, and choose a smaller batch or give up early."

Each hop gets the smaller of its stage budget and what's left after reserving time for the stages behind it:

// Stage budgets from Section 3. Slack is 12 ms of the 80.
const RESERVE_AFTER = {           // ms that must remain for later stages
  retrieval: 5 + 10 + 20 + 5 + 3, // light, hydrate, heavy, auction, respond
  heavy:     5 + 3,               // auction + response/log hand-off
};

function hopDeadline(stage: "retrieval" | "heavy", stageBudgetMs: number,
                     requestDeadline: number): number {
  const mustFinishBy = requestDeadline - RESERVE_AFTER[stage];
  return Math.min(Date.now() + stageBudgetMs, mustFinishBy);
}
// If hopDeadline <= Date.now(), skip the stage and step down the ladder.

The 3 ms for response and log hand-off is never lent out, so a request that reaches the auction with nothing left still returns something on time. Maya's request r-7f3a never notices: at p50 it finishes in 38 ms. The machinery exists for the tail.

Quick check before moving on: