Section 12: Four Features, Zero Demolitions - Follow-up Feature Evolution
Executive Summary: With the core design defended, the interviewer asks what comes next. We add four features that Section 2 deliberately deferred: saved-search alerts (built as reverse search with a percolator, not a cron job), shipping and in-app payments (where a local product becomes national and the geo-sharding flip trigger fires), reputation tied to confirmed transactions, and promoted listings kept apart from organic ranking. Each lands additively, as a new consumer, index or table.
Fifty-five minutes in, the interviewer leans back: "Say this ships and works. Product comes back with four asks. Walk me through them." This tests whether your design has seams in the right places. If every feature needs surgery on the posting path, the core was too tightly coupled. If each one plugs into the event stream, you designed for growth without building it early.
Lead with this:
"Everything after a listing commits flows through the outbox and Kafka, so most new features should be new consumers of that stream. If one needs a change to
POST /v1/listings, I'll flag it as the expensive one."
Saved Searches and Alerts: Ask the Listing, Not the Query
Suppose Daniel had saved "road bike, within 10 miles, under $500" the week before. When Maya's Trek goes live at 7:40 pm, Daniel should get a push notification, not find it by chance twelve minutes later.
The obvious build is a scheduler that re-runs every saved search every 5 minutes. With 10 million saved searches, that's 10M × 288 runs a day, about 2.9 billion queries a day, roughly 10x all organic search traffic. You'd massively oversize the search cluster to hear "nothing new" almost every time.
Flip the direction. New listings are rare: ~120/s at peak. Saved searches are many and change slowly. So store the queries and run each new listing against them. Elasticsearch and OpenSearch support this directly with the percolator: saved searches are indexed as documents holding a query, and a percolate request takes one listing and returns the stored queries it matches.
The mechanism:
- Route saved searches by region, just like listings. A saved search is registered under the routing key of every region its circle touches. Daniel's Austin search lives on one shard. A new listing in Austin percolates against ~310K saved searches (10M / 32), not all 10M. Percolate candidate selection can't use
geo_distance, so each saved search also stores its geohash cells as a keyword field, and the listing's cell filters candidates: a Round Rock bike skips South Austin queries. - Pre-filter by category. Filtering on the stored query's
category_idmeans a road bike is never evaluated against apartment-rental queries. - Trigger on becoming
activeand on price drops, never onpending_review. The matcher consumes the moderation-passed event, so a scam held by Section 9's trust and safety pipeline never reaches anyone's phone. - Batch alerts. The first match sends immediately; further matches for that saved search within the hour fold into one digest push. A dozen pings during the Sunday burst gets notifications turned off.
- Recheck before sending. If Maya marks the bike sold before the batch goes out, the batcher's lookup against the detail page (the authoritative source for status) drops it.
The pushback: "Isn't the percolator itself expensive?"
"Per call, yes. But it runs once per new listing, ~120 times a second at peak, against one region's slice. The cron design ran 2.9 billion full searches a day. I'd benchmark percolate latency per shard, but the comparison isn't close."
Quick check before moving on:
Sign in to continue reading
The rest of this lesson is available with a free account. Signing in with Google or Microsoft is free.
Sign in to read the full lesson