Section 11: Data You Must Forget - Privacy, Consent & Data Governance

Executive Summary: Everything so far treated the immutable raw log as sacred: our source of truth and our disaster-recovery plan. This section covers the requirement that pulls the other way. Some data must never be stored, some needs permission, and some must be forgotten on request from storage we designed never to rewrite. We drop the IP at the collector, carry consent on every hit, treat client IDs as personal data, delete users in two stages (a query-time tombstone within 24 h, then a physical partition rewrite within 30 days), make retention a metadata operation, and suppress report rows small enough to point at one person.

The interviewer puts the marker down. "You've told me the raw log is sacred and nothing gets rewritten. Now a hiker in Lisbon wants her data gone. What do you do?"

You: "I'm not giving a legal opinion on GDPR. I'm designing so counsel has levers: data we never store, data we can find, and data we can remove on a deadline. The cheapest is data we never store, so I'll start at the collector."

The IP Never Reaches the Disk

The IP is the most sensitive field in a hit and useless once two lookups are done at the collector: geolocation and the data-center bot check. Then it's discarded, never written to Kafka, the spool or the raw store.

Interviewer: "Why not keep a hashed IP? Fraud teams love IPs."

You: "A hashed IPv4 is barely hashed: there are only 2³² addresses, so anyone holding the hash function can enumerate them. The honest choice is to keep nothing. The cost is that nobody can investigate by IP after the fact, so anything IP-based must be a rule that runs at the edge, inside the 50 ms budget, before the IP disappears. Section 7's bot check already works that way."

Quick check before moving on:

EU visitors' hits are steered to EU collectors by an EU-only anycast address range and geo-DNS. Plain anycast routes by network proximity, not jurisdiction, so the EU range is announced only from EU sites, and a regional failover moves EU traffic to another EU region, never out of the group. The raw IP is therefore processed and dropped inside the EU, and the regional Kafka and pipeline give us the option to keep enriched EU events in-region too. Whether that satisfies a given customer's obligations is a legal determination. The architecture's job is to make the answer yes, we can rather than we'd have to re-platform.