Section 11: The Platform Is Speaking - Safety, Privacy & Failure Handling
Executive Summary: A search result is the web talking; a suggestion is the search engine putting words in the user's mouth. That makes typeahead a safety-critical system dressed as a convenience. This section layers the defense from raw logs to dropdown (privacy threshold, PII and policy filters, a serve-time blocklist for the 1-hour takedown SLA, manipulation resistance), then walks the failure modes under one rule: stale suggestions are acceptable, offensive or private ones are not, and when all else fails the dropdown disappears and the search box keeps working.
The interviewer leans back. "It's autocomplete. Worst case, someone sees a weird suggestion. Why would you spend real engineering on safety?"
Because nobody typed it. If Maya searches something ugly, that's Maya's query. If we suggest something ugly to the next person who types chicago m, that's us. Courts agree: in May 2013 Germany's Federal Court of Justice ruled that Google must remove defamatory autocomplete predictions once notified. The real question is where we filter, how fast, and what happens when the filter breaks.
Four Layers Between a Log and a Dropdown
Layer 1: the privacy threshold (D9). A query is suggestable only if at least 50 distinct users issued it in the 30-day window, counted with Section 5's HyperLogLog sketches. That shrinks ~1 billion distinct queries a day to a ~100 million corpus, and keeps one person's address or rare medical question away from strangers.
Layer 2: PII patterns. A leaked phone number can go viral and clear 50 users by lunchtime, so the build drops anything that looks like a phone number, email or street address, whatever its count.
Layer 3: policy at build time. Classifiers and the build-time blocklist remove hateful, sexually explicit, violent and dangerous predictions, plus defamatory pairings of a person's name with an accusation. Offline, it can afford heavy models.
Layer 4: the serve-time blocklist. It's versioned and pushed to all 40 nodes within 5 minutes. Each node filters its top-15 lists through it at lookup time. That's why K = 15, not 10: the five spares absorb removals, so a takedown leaves no gap.
"Why not filter only at build time and keep serving simple?" Because the base index rebuilds daily, so a court order arriving at 3 p.m. would wait for tomorrow's snapshot. D9 flips on regulation or a new surface with different privacy expectations, not on convenience.
Quick check before moving on:
Sign in to continue reading
The rest of this lesson is available with a free account. Signing in with Google or Microsoft is free.
Sign in to read the full lesson