Section 8: Where Rides Refuse to Hold Still - Deep Dive: The Live Edge

Executive Summary: Section 7 indexed closed intervals. This section defends the part that moves: open rides, late end events, corrections, and rides whose end never arrives. We prove that write-before-remove in the indexer plus read-live-first in the query means no event committed at or before as_of is ever missed, at the price of occasional duplicates. Then we make every write order-proof with USING TIMESTAMP version, walk Maya's correction bucket by bucket, and bound the live set with a reaper.

The interviewer taps Redis and Cassandra on the whiteboard. "Every ride lives in one, then moves to the other. What happens to a query that runs while it moves?" The live edge is where rides cross from one tier to the other, 868 times a second at peak, while 3,600 "now" queries a second read across both tiers.

The Two-Tier Race

The candidate names the hazard first. Phase 1 reads live:{sf} (ZRANGEBYSCORE live:{sf} -inf <qe>); phase 2 reads the buckets. A ride that closes between the phases has left the live set but may not have reached its buckets. If the query sees neither, the ride vanishes.

The fix is two orderings that point in opposite directions:

• The indexer writes before it removes. On an end event it upserts events_by_id and every bucket row, waits for all of them to be acknowledged at LOCAL_QUORUM, and only then runs ZREM.
• The query reads in the opposite order. All of phase 1 finishes before any of phase 2 starts, and phase 2 reads at LOCAL_QUORUM, so its replicas overlap the indexer's.

Take any ride that overlaps the window and was committed at or before as_of, the indexer's applied-through watermark. If its ZREM ran before phase 1 read the live set, its bucket rows were already durable, so phase 2 finds them. If the ZREM ran after, phase 1 saw it open. There is no third case. Rewind the clock and say the safety dashboard refreshes the 17:10–17:40 query at 17:31:45.400Z, just as Maya's drop-off lands:

Maya appears twice: open, then closed. Flip the indexer's order, ZREM first, and the same timeline loses her whenever phase 2 reads the 17:00 bucket before the write lands.

Critical Insight: Across two uncoordinated stores, with stateless pages, you can't cheaply have both "no misses" and "no duplicates," so pick the failure a client can repair. A duplicate is fixed by deduping on event_id and keeping the closed copy. Nobody can fix a miss. That is the contract from Section 2: no misses for anything committed at or before as_of, possible duplicates across pages.

"Why not a transaction across both?" No transaction spans Redis and Cassandra, and even Cassandra's own lightweight transactions (Paxos, four round trips) would only order writes inside Cassandra. The flip trigger is the one from Section 2: a consumer that needs exactly-once listing gets server-side dedupe within a bounded query, or a point-in-time export.

It survives pagination. The cursor {as_of, phase, bucket_idx, last_start_ts, last_event_id} caps qe at as_of. SF's ~10,000 open rides span ten phase-1 pages, and a ride closing during page 4 is caught in phase 2, which runs after every phase-1 page. The ordering is per query, not per page.

Quick check before moving on: