Section 7: Two Hundred Milliseconds Between Keys - Deep Dive: The Latency Path

Executive Summary: The candidate dives into the latency path first because the user feels it on every keystroke, and most of the 100 ms budget is spent before a request reaches our code. The client sends fewer requests and never renders a late one. The caches make short prefixes nearly free. Personalization merges on the client, so the 60% CDN hit rate holds.

Section 6's diagrams are on the board. The interviewer caps the marker: "Where would you like to go deeper?"

The trie tempts. You hold it back:

You: "I'd start with the latency path. Every user feels it on every keystroke, and the origin gets only 10 ms of a 100 ms budget. If I don't defend the other 90, the best index won't help. The prefix index comes next, in Section 8."

The Budget, and Where the Tail Hides

An average typist leaves about 200 ms between keystrokes, so suggestions must be on screen before the next key lands. Hence the SLO: p99 ≤ 100 ms from request sent to suggestions rendered. Add 50 ms of coalescing and it still fits the gap.

Section 2's budget table splits a cache miss into 50 + 20 + 10 + 10 = 90 ms, with 10 ms of slack. What matters here is who owns each leg. The 50 ms client-to-edge hop belongs to the user's network and our edge footprint. The 20 ms edge-to-origin hop is our backbone across 4 active-active regions. The 10 ms origin slice is Section 8's, and the lookup inside it is under 1 ms, typically ~100 µs. The last 10 ms of parse and render is the front end's. A CDN hit skips the backbone and origin legs: about 60 ms.

Interviewer: "That 50 ms first hop is optimistic on mobile."

You: "It's where the tail lives. The worst case is a cold connection: a fresh TCP and TLS handshake adds round trips before the first byte. So the client warms the connection when the search box gets focus, and every keystroke reuses it. And on genuinely slow networks no design hits 100 ms at p99. We track the SLO per network class, and on a slow class the client widens coalescing and leans on its cache rather than queueing requests."

That's why the transport is GET over reused HTTP/2 or HTTP/3, not WebSockets (D7, Section 4's first pushback).

Quick check before moving on: