Section 4: Emit, Anchor, Ack - API Design (with Pushback)

Executive Summary: The developer surface is small. A spout implements open, nextTuple, ack(msgId) and fail(msgId). A bolt implements prepare and execute(tuple), and inside execute it calls emit(anchor, values) and then ack(tuple) or fail(tuple). A TopologyBuilder wires the components together with a parallelism hint each and a grouping on each edge. Four config knobs bound the whole thing: workers, ackers, max spout pending and the message timeout. Jordan's trending-hashtags topology fits in about 25 lines. Anchoring, acking, groupings and parallelism are all explicit, because each is a cost or correctness decision the framework can't make for you.

Interviewer: "Section 3 gave me about 100 executors. Now show me what Jordan actually types. What's the interface?"

You: "Two interfaces and a builder. I'll go through the spout, the bolt and the wiring, then defend the parts that look like extra work. Those are where you'll push."

The Spout: A Source That Remembers What It Sent

A spout is a pull loop the framework drives. open connects it, here to four of the 64 partitions of the posts topic. The framework calls nextTuple over and over, and each call should emit at most a tuple or two and return quickly. Every emit can carry a message id. Spout task 4 emits post p-77120 from partition 19 with the message id (19, its offset). The framework later calls back exactly one of two methods. ack(msgId) means the whole tree of work that grew from this post finished, so the offset is safe to commit. fail(msgId) means some part of it failed or timed out, so the spout re-reads that offset and emits it again. (A Kafka spout commits acked offsets periodically, every 5 s in our setting, so a restarted spout task resumes from its last committed offset; Section 10 prices that.)

Quick check before moving on: