Section 1: Not a Queue, a Log - Problem Understanding & Clarifications

Executive Summary: A prompt that sounds like "design a message queue" is really a request for a durable, replayable log that many teams read independently. We break it into what, who, why and how, compare a queue, classic pub/sub and a log, and run a clarifying exchange in which each answer removes a class of design: per-message acks, global order, delete-on-consume, per-message sends and a cluster stretched across regions. What survives is a partitioned commit log whose consumers pull and track their own position. We meet Ari, whose order events travel through every section that follows.

The interviewer reads the prompt aloud:

"Every service here emits events — orders, clicks, database changes — and dozens of teams want to consume them independently, in order, and replay a week of history when they ship a bug. Design the system that carries all of it."

Three phrases do the work. Independently: one team reading an event must not take it from another, and a slow team must not hold up a fast one. In order: something is ordered, and the interview turns on what. Replay a week: the data outlives being read. A candidate who draws a broker that deletes each message on acknowledgement has broken that third phrase before the first box is on the board.

Key Insight: "Replay a week" is the sentence that decides the design. A queue hands out work and forgets it. This system remembers what happened, in order, for a fixed time, and lets each reader move through that history at its own pace. That is a log, and the rest of this module follows from taking the word seriously.

Decomposing the Prompt

• What it does: accepts events, stores them durably in order, and lets any number of readers fetch from any point in the retained history. The product is a commit log for the whole company, not a delivery service.
• Who uses it: producers like Ari's checkout service, which want a fast, durable "accepted" and then forget; and dozens of consumer teams (fulfillment, fraud, analytics) that read the same events at their own speed and occasionally rewind.
• Why it exists: with N producing systems and M consuming teams, point-to-point integration means up to N × M custom pipelines, each with its own format, failure modes and on-call owner. A shared log makes it N + M: each producer writes once, each consumer reads once.
• How it's used: high-throughput appends, many readers per event, most near the newest data, a few far behind or replaying.

The 2011 paper by Kreps, Narkhede and Rao describes LinkedIn feeding activity data and operational metrics to online consumers and offline Hadoop systems from one log. The N × M framing is how LinkedIn's engineers later summarized that motivation, not a quote from the paper.

On the left, a new consumer needs a pipe from every producer it cares about. On the right, it is one new reader of an existing log.

Quick check before moving on: