Section 6: Where the Two Clocks Separate - High-Level Architecture
Executive Summary: This section draws the boxes and gives each one a single job. A replicated Message Log sits between the Publish API and everything else, so the publisher's clock stops at a 15 ms ack while the subscriber's clock keeps running. Delivery Queues are split by protocol so a 5 s webhook never paces a millisecond queue write, failures leave the hot path for the Retry Scheduler, and the control plane can fail without stopping a publish. We walk order #84721 through every box, defend each box with a number, and defend three things we left out.
The interviewer slides the marker across. "You've got the API and the tables. Show me the boxes, and walk one message through them."
You draw the data plane first, because that's where 150,000 publishes a second live.
The Data Plane: One Job per Box
| Box | Its one job | The number that justifies it |
|---|---|---|
| Publish API | Authenticate, validate, quota check, durable write, 202 |
p99 100 ms; 60 hosts at peak |
| Message Log | The durability point; separates the two clocks | 300 partitions, 30 brokers, 48 h retention |
| Fan-out Workers | Read a message, look up subscribers, filter, create tasks | 500 messages/s per partition at peak |
| Subscription Cache | Keep metadata reads off the store | Avoids 150,000 store reads/s |
| Page Queue | Split a big topic into 1,000-subscriber page tasks | app-promotions = 8,000 page tasks |
| Delivery Queues | Buffer tasks, one queue per protocol | 300K / 180K / 90K / 30K per s at peak |
| Delivery Workers | Make the attempt, classify the result | 180 hosts; 5 s HTTP timeout |
| Retry Scheduler | Hold failed tasks until they're due | 6,000 tasks/s in; at most ~6.1M pending, 12 GB (upper bound) |
| DLQ | Keep what we couldn't deliver, in the customer's account | After 10 retries or 24 h |
Two boxes are missing from the drawing on purpose. The endpoint breaker and the Parking Lot live inside the Delivery Workers' path, and they are Section 8's whole story.
Sign in to continue reading
The rest of this lesson is available with a free account. Signing in with Google or Microsoft is free.
Sign in to read the full lesson