Section 9: Six Ways to Push on a Pipeline - Interviewer Deep-Dive Challenges

Executive Summary: The design is on the board and the partner outage drained in 13 minutes. Now the interviewer pushes on it: exactly-once delivery, a broadcast once a second, a leak after unsubscribe, out-of-order events, "just use Kafka", and a crash halfway through a page. For each one you get what's really being tested, a model answer tied to the numbers ledger and the decisions behind it, and one sentence worth saying out loud.

Each challenge has a cheap answer and a priced one. Give the priced one. Here's where each answer lives:

Challenge One: "Give Me Exactly-Once"

"The fraud team says duplicates cause double holds on cards. Guarantee exactly-once."

What they're really asking: do you know why it's impossible over HTTP, and where the real fix lives?

We can't know whether a third party processed a request whose response never came back. That's the two generals problem:

Duplicates have three sources: a publisher retry, a fan-out re-expansion, and a delivery retry. We stay at-least-once (D4) and stamp the messageId in the envelope and in X-Notify-Message-Id. The fraud service records that messageId under a unique constraint in the same transaction as the card hold, so a second copy becomes a no-op. A publish-side dedup store would cost writes at 150,000/s and only remove one source of the three. The FIFO tier (Section 12) adds a 5-minute dedup window, and even that only narrows the publish side.

The senior line: "Exactly-once isn't a delivery property, it's a processing property. We deliver at least once with a stable ID, and the subscriber makes processing idempotent."