Section 6: Two Halves That Never Call Each Other - High-Level Architecture
Executive Summary: This section draws the system as two halves with a wall between them. The read-only serving path (client, CDN edge, load balancer, suggestion nodes holding the index in memory, a separate history service) answers ~1M QPS at peak. The data path (log, data lake, daily build, validation gate, snapshot store, stream processor, policy service) learns from ~116K searches/s and produces three artifacts the serving path loads. We defend the missing database and queue, walk Maya's chicago mara through both halves, and spread it across four regions.
The interviewer pushes the marker across: "You've sized it and given me the API. Draw the system." The tempting picture is one loop: a search comes in, a counter goes up, the trie updates. Say why you won't. The read side is ~1M requests a second at peak, the write side ~116K logged searches a second, and the writes have nothing to say to the reader right now. A search submitted today should shape tomorrow's index after it has been counted, thresholded and checked. So I draw two halves, and only files cross between them.
The Serving Path
| Component | Job | Why it exists | What it is not |
|---|---|---|---|
| Client | Coalesces keystrokes within 50 ms, keeps a prefix cache, merges personal history | Cuts ~12 typed characters to ~8 requests per search | Not a ranker; it only adds its own history |
| CDN edge | Caches responses keyed on prefix, locale, client, safe-search, index version | Absorbs 60% of traffic; origin sees 400K of the ~1M peak | Not personalized; requests carry no identity |
| Regional L7 load balancer | Spreads misses across the region's 10 nodes | Every node is identical | Not prefix-aware; there are no shards |
| Suggestion service node | Walks the trie, merges trending, filters the blocklist, returns 10 | Origin p99 ≤ 10 ms, lookup < 1 ms | Not a database client; no network call per request |
| Personal history service | Returns a signed-in user's last 100 searches | Personalization without breaking the cache | Not on the per-keystroke path; ~35K reads/s at peak, not 400K |
Two choices carry the weight. The per-keystroke request carries no user identity (D5); a personalized response per keystroke would destroy the 60% hit rate (2.5× origin load) and put user data in every request. That flips only if learned personalization shows a CTR lift big enough to pay for a separate re-rank tier between the client and the cache. And the edge caches for 120 s, browser max-age=60 (D6): no edge cache means the origin eats the full 1M QPS, long TTLs stall trending and takedowns.
Interviewer: "Where's the database?"
You: "Deliberately absent. The index is ~20 GB and a node has 64 GB of RAM, so every node holds all of it in process memory: replicate, don't shard (D2). The obvious alternative, prefix → top-K in Redis, adds a network hop to every keystroke and over a billion keys (D1). Past 32 GB, half a node's RAM, I'd shard by locale, then by prefix range with replicated hot shards."
Interviewer: "And no queue in front of the nodes to absorb spikes?"
You: "No. A queue helps when work can wait. This work goes stale within one ~200 ms keystroke gap; a late suggestion is worse than none. The CDN and stateless replicas absorb spikes. The queue belongs on the data path (D10)."
Quick check before moving on:
Sign in to continue reading
The rest of this lesson is available with a free account. Signing in with Google or Microsoft is free.
Sign in to read the full lesson