Section 2: Ordered Per Key, Kept for a Week - Requirements (Functional & Non-Functional)

Executive Summary: Section 1 turned "carry the events" into a replayable, partitioned commit log. This section writes its contract: keyed partitions, reads from any offset, consumer groups that commit their own position, time retention and compaction. It makes three bounded guarantees (order within a partition, durability for writes acknowledged under acks=all, at-least-once by default) and puts numbers on everything else: 1.5 M messages/s and 7.5 GB/s of reads at peak, 7 days kept, produce p99 ≤ 20 ms, end-to-end p99 ≤ 100 ms, 99.99%, any one zone lost. It refuses priorities, delays, global order and content queries.

The interviewer caps the marker. "So it's a log. Now commit: what does it do, what does it promise, what won't it do?"

You: "Operations, demoed with Ari's orders topic; guarantees, worded precisely, because 'it's ordered' and 'it's exactly-once' sink candidates; then numbers and refusals."

Functional requirements: a log with a few verbs

• Topics, split into partitions. A topic is a named stream (orders). A partition is one ordered, append-only log: the unit of parallelism, ordering and placement. orders has 64.
• Produce with a key. produce(topic="orders", key="order-58213", value=…). The key picks the partition (murmur2 mod 64 → partition 37), so every event for one order lands in one log.
• Produce returns a position: partition 37, offset 1,284,553,901, the record's address for as long as it is retained.
• Consume from any offset. A reader asks for "partition 37 from offset N" and gets everything from there on, in order. It can start at the head, the oldest record, a saved offset or a timestamp ("Tuesday 09:00, before the bad deploy").
• Consumer groups. A group is one logical subscriber. Each partition goes to one member at a time, and fulfillment, fraud and analytics-sink each get every message independently.
• Offset commit. A group records "done up to here" per partition; a restarted member resumes there.
• Retention by time. Data older than 7 days is deleted, whether or not anyone read it.
• Compaction by key. Keyed state topics may instead keep at least the last record per key; a tombstone deletes a key.
• Admin. Create topics (partitions, replication factor, configs), move replicas, set quotas.

Interviewer: "Where's ack(messageId)? Where's 'redeliver this one in five minutes'?"

You: "Not in the list, on purpose (Decision 1: an append-only partitioned log where consumers pull and track their own offsets). The rejected alternative is the classic broker that tracks an ack per message per consumer: with five groups at 500,000 messages a second, that's 2.5 million state updates a second just to remember who read what. An offset is one number per partition per group. The flip trigger is real work-queue semantics (per-message ack, redelivery, priorities): a queue, or Kafka's much later share groups (Section 12)."

Quick check before moving on: