Publish–Subscribe Pattern
maang.io System Design Series
The publish–subscribe pattern decouples senders from receivers through topics: producers publish without knowing who listens, and the broker fans each message out to every subscriber. It's the shape behind event-driven architectures, notifications, and cache invalidation — and understanding push vs pull and durable vs ephemeral subscriptions is what separates a working design from a lossy one.
This deep dive comes in three parts:
- Foundations & Core Concepts — topics, fan-out, and pub/sub vs point-to-point.
- Internals — push vs pull, durable subscriptions, and filtering/routing, with worked examples.
- Interview — Staff/Principal Q&A: fan-out at scale and choosing a broker.
Publish–Subscribe Pattern — Part 2: Internals & Delivery Mechanics
maang.io System Design Series
What is this?
In Part 1 the broadcast tower was a black box: the DJ speaks, and somehow every tuned-in radio hears her. Now we climb the tower and look at the machinery. How does the broker actually remember who's subscribed? What happens, byte by byte, when a message is published? How does it reach ten subscribers without ten separate publishes? And what happens when a subscriber's radio was off — how does a durable subscription replay the news it missed?
The single most important internal fork in this whole topic is how the broker fans out. There are two families, and picking between them explains almost every behavioral difference across real systems:
- Copy-based brokers (RabbitMQ, ActiveMQ, Amazon SNS→SQS) — the broker physically copies each message into a per-subscriber mailbox.
- Log-based brokers (Apache Kafka, Google Pub/Sub, Apache Pulsar) — the broker appends each message once to a shared log, and every subscriber reads independently by tracking its own position.
That difference — copy the message N times vs store it once and let readers track a cursor — is the LSM-vs-B-tree-level trade-off of this topic. Everything below hangs off it.
The one-line idea: Fan-out is either eager (the broker copies each message into N subscriber queues at publish time) or lazy (the broker writes the message once to a log and each subscriber reads it on its own cursor). Eager fan-out is simple and pushes easily; lazy fan-out decouples storage from delivery and makes replay trivial. Almost every pub/sub design decision traces back to which one you chose.
Sign in to continue reading
The rest of this lesson is available with a free account. Signing in with Google or Microsoft is free.
Sign in to read the full lesson