Section 2: Promises in Nines and Petabytes - Functional & Non-Functional Requirements

Executive Summary: This section turns Section 1's clarified scope into a contract. Nine functional requirements say what the platform does. A non-functional table says how well, in numbers: a 200 ms ack sent only after a durable write, P0 queryable in 30 seconds and everything else in 2 minutes, 5-second hot queries, 99.99% ingestion availability and zero loss of acknowledged data, at 10M events a second. Then the interviewer pushes on three dials (five nines, 30 days hot, sampling), and each is defended with arithmetic and a named trigger that would change the answer.

The interviewer sets the marker down: "Now what exactly are we promising, and to whom?" "Fast ingestion" can't be tested. "Ack p99 ≤ 200 ms, only after the batch is durable" can be, and an on-call engineer can be paged on it. Every number you write here becomes an obligation a later section has to meet.

The Functional Contract

You read the list aloud, numbered, so you can point back to it later:

1. Multi-tenant ingestion API. One API for 3M agents and the Azure service emitters (Entra ID sign-ins, Activity Log, diagnostics). The tenant comes from the caller's credential, never from the payload.
2. Batch acceptance with per-event validation. Good events are accepted. Bad ones come back as per-event rejects and land in a per-tenant ingestion-errors table. A partly bad batch never fails whole, and never silently.
3. Priority classes. Every event is P0 critical (sign-ins, audit and control-plane activity, alerts), P1 standard (process, file, DNS) or P2 bulk (network flows, verbose diagnostics). Its class decides its freshness, its place in the shedding order and its retention.
4. Tiered retention. Every class is searchable in seconds for 30 days and kept for 365 days. P0 is also kept immutable for 7 years.
5. KQL search, interactive and scheduled. Analysts run ad-hoc queries. Tenants register scheduled queries (analytics rules, dashboards) that run every 5 minutes.
6. Long-range search and archive restore. Beyond 30 days, queries run as async search jobs; P0 older than a year comes back through a restore job. Results stay readable through the normal query surface.
7. Tenant export and deletion. Tenants can export their data, and offboarding or a legal request can delete it, bounded by the P0 archive's immutability (a conflict policy decides, not code).
8. Stream hand-off. Downstream consumers, detection first, read the live stream without any change to ingestion.
9. Quotas and usage visibility. Each tenant has an ingestion quota (Contoso's contract: 10,000 events/s, bursting to 20,000) and can see its own volume, throttles, rejects, drops and freshness per table.

Interviewer: "Where's the detection engine?"
You: "Deliberately absent. Detection is a consumer, and FR 8 is its contract: a low-lag read of the stream. The Azure Real-Time Threat Detection System topic designs the engine; Section 11 plugs it in."

Key Insight: FR 2 and FR 9 separate a platform from a pipe. With 50,000 tenants, the most common support question is "where did my events go?" A rejects table and a per-table freshness view answer it before anyone opens a ticket.

Quick check before moving on: