Section 4: A Fire-and-Forget Front Door - API Design (with Pushback)

Executive Summary: This section designs the two contracts Section 3's numbers flow through, and they have almost nothing in common. The collection endpoint takes ~1.75M requests a second at peak from code we don't control, often while the page is being torn down. It answers 204, and that 204 means exactly one thing: the hits are durably in the log (or, during a declared regional outage, in an alarmed disk spool). The reporting API serves ~3.5K queries a second from signed-in analysts. It is structured, quota-metered, and honest about whether its numbers are provisional, sampled, thresholded, or folded into (other). Five pushbacks test the design, and each answer names the rejected alternative.

The interviewer taps the board: "Contracts. What does the tag send, and what does Dana call?" Say out loud that these are two APIs with opposite personalities. One faces anonymous browsers on 10M sites and must never slow a page down. The other faces paying customers and must never mislead them.

The Collection Contract: One Endpoint, One Answer

The tag batches events (about 2 per request on average) and sends them with navigator.sendBeacon, which survives page unload. The shape below is this design's own, not GA4's exact wire format:

POST /g/collect?v=2&tid=P-40213
Content-Type: text/plain

{"client_id":"c-7731","session_id":"s-1","events":[
  {"seq":1,"name":"page_view","age_ms":29000,
   "params":{"page_location":"/tents/ultralight-2p",
             "utm_source":"newsletter","utm_medium":"email"}},
  {"seq":2,"name":"scroll","age_ms":0,"params":{"percent":90}}]}

HTTP/1.1 204 No Content

Every field has a downstream job. tid plus client_id is the Kafka partition key, keeping one visitor's hits together. session_id is the client's own session, which the pipeline trusts but enforces. seq is the dedup key. age_ms is how long before the send each event happened: the page_view waited 29 seconds in the batch for the scroll. There's no absolute timestamp on purpose. Phone and laptop clocks are often minutes off and occasionally years off, but a device measures a short duration well. So the collector stamps received_ts on its own clock and computes event time as event_ts = received_ts − age_ms, the clock Section 8 runs on. There is no IP field: the collector reads it off the connection, looks up the geo, and discards it before anything is written.

The response is 204 No Content, and only after the batch is durably appended to Kafka with acks=all and 3 replicas: the leader acknowledges once all in-sync replicas (at least 2 of 3, since min.insync.replicas=2) hold the write, all inside a p99 budget under 50 ms. If the regional Kafka cluster is down, the collector spools to local disk, raises an alarm, and still returns 204. That's a declared, temporary durability downgrade, not a silent one. Only when Kafka and the spool are both exhausted does the collector answer 503, the last rung of the outage ladder Section 7 walks.

Three side doors complete the contract:

• GET pixel fallback for environments without sendBeacon. Events ride in the query string, and URL-length limits mean roughly one event per request.
• /debug/collect runs the same validation and returns the messages as JSON. It's for the developer wiring up Trailhead's tag.
• Measurement Protocol carries server-to-server hits, such as a refund from Trailhead's order system. It requires a per-stream api_secret and accepts events up to 72 hours old, which matches the pipeline's hard late-data cutoff.