Section 8: The Purchase That Arrived Tomorrow - Deep Dive: Event Time, Sessions & Late Data

Executive Summary: Section 7 made every hit durable. This section makes hits mean the right thing when they arrive out of order. We bucket by event time, let the client own the session while the server enforces it, and size the session state (~60 GB at peak). A 5-minute watermark lets streaming count about 99% of hits. Later hits take the batch path, which restates a property-day for 72 hours and then freezes it. Versioned cube writes make each step an atomic swap that Dana sees in dataState. We close with two pushbacks: why not wait 72 hours, and why not hold 72 hours of state?

The interviewer picks up Section 7's thread: "Your hits are durable. They're also out of order. The hiker's purchase is still on a train. What does 'Tuesday's revenue' even mean?"

Two Clocks on Every Hit

Every hit carries two times. Event time is when the thing happened on the device. Processing time is when our pipeline sees it. On a laptop over Wi-Fi they're milliseconds apart. For the Lisbon hiker they're 40 minutes apart: her purchase (seq 4) happened at 08:21 Lisbon time, sat in the tag's queue while the train had no signal, and reached a collector at 09:01.

You: "Every number Dana sees is keyed by event time. Bucket by arrival and the purchase counts in whatever hour the train found signal, and a Section 7 spool drain dumps an hour of traffic into one minute. Arrival time is a fact about our network; event time is a fact about her visit."

Client clocks can't be trusted, though. So in this design the tag sends each event's age (age_ms, milliseconds before the request was sent), and the collector computes event_ts = received_ts − age_ms on our own clock. Future timestamps are clamped to received_ts. Anything older than 72 hours is dropped and counted in late_dropped. Then event_date is the event's local date in the property's time zone: 08:21 in Lisbon is 01:21 in Denver, so the purchase belongs to Trailhead's Tuesday.

Quick check before moving on: