Apache Kafka
maang.io System Design Series
Apache Kafka is not a queue but a distributed commit log: topics split into ordered, replicated partitions of immutable, offset-addressed records. Producers append, consumer groups read at their own pace, and retention keeps messages replayable. Master partitions, the in-sync-replica set, log segments, and exactly-once semantics, and you understand the backbone of modern event-driven and streaming systems.
This deep dive comes in four parts:
- Foundations & Core Concepts — topics, partitions, offsets, and consumer groups.
- Internals — brokers, leaders/ISR, the high-watermark, log segments, and retention, with worked numbers.
- Advanced Internals — the idempotent producer, transactions/exactly-once, and KRaft.
- Interview — Staff/Principal Q&A: ordering, delivery guarantees, and capacity reasoning.
Apache Kafka — Part 4: System Design & Interview Mastery
maang.io System Design Series
What is this?
You now know Kafka from the topic down to the high watermark. This chapter is where that knowledge earns the offer. An interviewer doesn't want you to recite "Kafka is a distributed commit log" — they want to watch you reach for it at the right moment, wire it into an architecture, and defend the trade-offs when they push back on ordering, exactly-once, or a hot partition. That's the gap between a memorized fact and a senior signal.
We'll do two things: see where Kafka fits in real designs (with diagrams), then run a mock interview — for each question, what they're really asking, a model answer, and the senior line that signals you've done this for real.
The one-line idea: interviewers use Kafka to test whether you can turn a synchronous, tightly-coupled design into an asynchronous, decoupled, replayable one — and then reason honestly about what that costs you in ordering, latency, and delivery semantics.
Keep the kitchen pass in mind one last time — the ordered rails, the shared readers, the backup copies. Every answer below is that picture, scaled.
Sign in to continue reading
The rest of this lesson is available with a free account. Signing in with Google or Microsoft is free.
Sign in to read the full lesson