</> MAANG.io

Expert

Design landmark systems in depth, from Dynamo, GFS, Bigtable and Kafka to payments and ads pipelines.

0 of 232 solved 0%

Section 13: The Judges' Table - Final Summary & Candidate Scorecard

Executive Summary: This closing section turns the module into a way to grade yourself. We replay the arc as the signal the panel was recording, ledger the eleven decisions with their flip triggers, lay out a 100-point scorecard in Storm terms with score bands, and list the prompts that are this question in disguise.

The panel never graded the box labeled "count bolt." When sup-07 went dark at 20:14:05, they listened for "doomed trees fill every spout's pending slots and ingest nearly stops; about 60 s until Nimbus notices and its 4 workers move into spare slots; then 36–54 M posts of Kafka lag drain in 3–4.5 minutes, and the counts are complete once they do" versus "Storm is fault tolerant, so nothing is lost."

The Arc, Replayed as Signal

Five interview phases in sequence, each with the signal the panel records: framing in Sections 1-2 signals batch and exactly-once-everywhere killed early; estimation in Section 3 signals 100 executors with parse binding at 40; API and data in Sections 4-6 signal anchoring as opt-in and Nimbus off the data path; deep dives in Sections 7-9 signal the hot key fixed and the 20-byte acker priced; and failure and growth in Sections 10-12 signal the sup-07 timeline and successors tied to flip triggers.

  1. Framing (Sections 1–2): nightly batch died on "seconds," exactly-once everywhere on "±1%," a fire-and-forget firehose on "Kafka, 7 days."
  2. Estimation (Section 3): 600,000 posts/s → 1.2 M hashtag tuples/s → 16 + 40 + 16 + 16 + 1 + 12 ≈ 100 executors on 12 machines. Parse, CPU-bound on JSON, is the binding stage.
  3. API, data and architecture (Sections 4–6): emit(anchor, values) and ack(tuple) make reliability opt-in; the spout's message id is the Kafka offset; a 60-bucket ring is about 84 MB per count executor; Nimbus never touches a tuple.
  4. Deep dives (Sections 7–9): #worldcup at 150,000 tuples/s outruns count executor 9's 100,000, so anchored 100-ms pre-sums precede the fields grouping, cutting it to 400 tuples/s with at-least-once intact. p-77120's tree XORs to zero at about 20 bytes per pending tree. Trident buys exactly-once state with batch latency.
  5. Failure and growth (Sections 10–12): a dead machine nearly stalls the whole topology for 60–90 s, because trees touching its count executors fill every spout's pending slots, then the Kafka backlog takes about 3–4.5 minutes to drain. Its 4 workers land in spare slots (60 slots, 48 used), so no healthy worker restarts, and 132 cores remain for about 100. Heron, Flink and Kafka Streams each fixed one flip trigger.

Key Insight: None of those signals is "knows Storm's config names." Kafka offsets make replay possible, the acker decides when to replay, and the checkpoint bounds how much. None of them makes a count exact, and saying so is worth points.

Quick check before moving on:


Sign in to continue reading

The rest of this lesson is available with a free account. Signing in with Google or Microsoft is free.

Sign in to read the full lesson

Sign in to MAANG.io

Use your Google or Microsoft account — no password to remember.

Continue with Google Continue with Microsoft

Please accept the terms above to continue.