Section 3: One Stamp, Two Planes - Core System Design Deep Dive

Executive Summary: Section 2 cut the fleet into 12 stamps of 250k events/s. This section draws one stamp: a data plane that carries events through two Event Hubs, a Normalizer, a Detection Engine and an Alert Service to the SOC's tools, and a control plane that turns rule text into compiled rule sets live in every stamp within 60 seconds. We trace Contoso's tenth failed sign-in against the 22-second stage budget, then defend two hubs and rules-as-data under pushback.

The interviewer slides the marker back across the table. "You've told me it's 12 stamps. Draw me one. Show me how a sign-in becomes a page."

Weak answers become a list of Azure products. A stronger one starts with a split. Some boxes see every one of the stamp's 250k events per second, and their failure means missed attacks. Others see a few rule edits a minute, and their failure means a customer can't edit a rule for a while. Different scaling, different failure modes, so you draw two planes.

The Data Plane: One Event's Path Through Three Hubs

You draw left to right, saying one job per box as the pen moves:

• Ingest Edge: Event Hubs endpoints that receive each tenant's platform diagnostic streams, plus an HTTPS push API for custom sources. It takes tenantId from the delivery channel (an envelope property written by the trusted platform delivery pipeline, or the push API's verified token claims) and never from the event payload. Section 8 — Security, Compliance & Governance Architecture explains why that matters.
• raw-events hub: a durable, replayable log of exactly what arrived. An event the Ingest Edge acknowledges is never lost.
• Normalizer: a stateless Flink job that parses each source into the canonical schema and enriches it through the Enrichment Cache: in-process GeoIP/ASN, a 60 MB threat-intel Bloom filter, and the stamp's Redis for identity context and TI confirmation. Enrichment is local first and fail-open: if Redis is down, the event proceeds flagged unenriched. Fail-closed would let a cache outage stop all detection.
• normalized-events hub: the shared, clean stream that the Detection Engine, the Archive and the Retro-Hunt Service all depend on.
• Detection Engine: a stateful Flink job, a Router plus keyed Rule Evaluators, with rules arriving as broadcast state. Section 5 — Detection Engine Internals & Execution Flow owns it.
• detections hub → Alert Service: dedupe, suppression and correlation into incidents.
• Alert Store: Cosmos DB, partitioned by [tenantId, day]. Its change feed is the outbox, so a stored alert is always dispatched.
• Dispatcher → per-sink Service Bus queues → Connectors: Sentinel/SIEM, webhooks, Teams/email, ServiceNow, Logic Apps playbooks. A dead webhook fills only its own queue.

Key Insight: Every box in this stamp is drawn so that the slowest or most broken consumer hurts only itself. Hubs buffer between stages, the Enrichment Cache fails open, and sinks get their own queues. The Detection Engine never waits on Redis, Cosmos DB or a customer's webhook.