Section 13: The Judges' Table - Final Summary & Candidate Scorecard
Executive Summary: This closing section turns the module into a way to grade yourself. We replay the arc as the signal the panel was recording, ledger the eleven decisions with their flip triggers, lay out a 100-point scorecard in Storm terms with score bands, and list the prompts that are this question in disguise.
The panel never graded the box labeled "count bolt." When sup-07 went dark at 20:14:05, they listened for "doomed trees fill every spout's pending slots and ingest nearly stops; about 60 s until Nimbus notices and its 4 workers move into spare slots; then 36–54 M posts of Kafka lag drain in 3–4.5 minutes, and the counts are complete once they do" versus "Storm is fault tolerant, so nothing is lost."
The Arc, Replayed as Signal
- Framing (Sections 1–2): nightly batch died on "seconds," exactly-once everywhere on "±1%," a fire-and-forget firehose on "Kafka, 7 days."
- Estimation (Section 3): 600,000 posts/s → 1.2 M hashtag tuples/s → 16 + 40 + 16 + 16 + 1 + 12 ≈ 100 executors on 12 machines. Parse, CPU-bound on JSON, is the binding stage.
- API, data and architecture (Sections 4–6):
emit(anchor, values)andack(tuple)make reliability opt-in; the spout's message id is the Kafka offset; a 60-bucket ring is about 84 MB per count executor; Nimbus never touches a tuple. - Deep dives (Sections 7–9):
#worldcupat 150,000 tuples/s outruns count executor 9's 100,000, so anchored 100-ms pre-sums precede the fields grouping, cutting it to 400 tuples/s with at-least-once intact. p-77120's tree XORs to zero at about 20 bytes per pending tree. Trident buys exactly-once state with batch latency. - Failure and growth (Sections 10–12): a dead machine nearly stalls the whole topology for 60–90 s, because trees touching its count executors fill every spout's pending slots, then the Kafka backlog takes about 3–4.5 minutes to drain. Its 4 workers land in spare slots (60 slots, 48 used), so no healthy worker restarts, and 132 cores remain for about 100. Heron, Flink and Kafka Streams each fixed one flip trigger.
Key Insight: None of those signals is "knows Storm's config names." Kafka offsets make replay possible, the acker decides when to replay, and the checkpoint bounds how much. None of them makes a count exact, and saying so is worth points.
Quick check before moving on:
Sign in to continue reading
The rest of this lesson is available with a free account. Signing in with Google or Microsoft is free.
Sign in to read the full lesson