Section 11: The Escalation Gauntlet - Interviewer Deep-Dive Challenges

Executive Summary: The interviewer spends the last stretch trying to break the design seven ways. Every answer follows the same pattern: the number first, then the mechanism Sections 7 to 10 already built, then the cost and the flip trigger from the decisions ledger. Two of the seven are won by saying the request belongs to a different system; the rest by naming exactly where a guarantee or a number stops.

The interviewer leans back. "Fine. Riya's inlinks job finishes in eight minutes and survives a dead rack. Now let me push on it."

Challenge 1: Just Use a Parallel Database

"Load the crawl into a parallel database and write SQL. Why build a framework?"

What they're really asking: is the restricted model a real choice, or did you not know databases exist?

You: "Three reasons, all from our numbers. First, the input is 10 TB of raw HTML with no schema, and Riya reads it once a night. Loading it would be a full pass over 10 TB before the first query runs, and the job itself is only one pass. Second, the work is parsing HTML in user code, which is a user function, not a relational operator. Third, failures. At 1% of machines per day, a 2,000-node query sees about 0.11 crashes per eight-minute run, and the longer jobs on this cluster see far more. If a query restarts on any failure, long jobs might never finish. Our design re-runs a dead worker's map tasks in parallel and loses at most about 25 s, and only if it dies as the map phase ends."

The flip is honest too: repeated, structured queries over the same data belong in a database, which is why Decision 1 named SQL as a rejected alternative rather than a wrong one.

The senior line: "A database wins when you query the same data many times. We read raw data once, at a scale where machines die during the job."