Section 8: One Post, Eighty Million Followers - Deep Dive: Fan-Out and the Celebrity Problem

Executive Summary: A dispatcher reads each post event off the Post-Event Log and asks one question: which fan-out mode is the author in? Under 100K followers, it pushes. It pages the followers list 1,000 at a time, drops inactive followers with a 250 MB activity bitmap, and pipelines idempotent inserts to the feed-cache shards: ~700K inserts/s at peak on a fleet sized for 1M/s, done within 5 s p99. At 100K or more, it appends one post ID to the author's celebrity timeline and stops. 100K up / 90K down hysteresis stops accounts near the line from flapping. On awards night, ten celebrity posts cost ten writes, not a 13-minute backlog.

Section 7 saved this wall for its own dive. The interviewer opens it with one line.

Interviewer: "Dev posts his hike photo at 9:00:00. Walk me through how it gets into Maya's feed. Then do the same for @luna_sings."

Same API call, same Post Service, same kind of row. After the commit, the work differs by a factor of hundreds of thousands: Dev has 350 followers, @luna_sings has 80 million. Section 3 priced both pure strategies. Pull for everyone costs 35M timeline reads/s, and 87% of opens wait on a slow fetch. Push for everyone means one @luna_sings post, 80M active followers, takes 80 s of the whole fan-out fleet. This section builds the machine that does neither.

The Dispatcher Asks One Question

The publish path ends early on purpose. The Post Service commits Dev's row and an outbox record in one transaction on Dev's shard, then returns 201, inside the 500 ms publish budget however many followers Dev has. A relay moves the outbox record onto the Post-Event Log, so a crash between the two can delay the event but never lose it. The log is partitioned by author_id, so one consumer sees an author's events in order: a create always comes before its delete.

{ "type": "post_created", "post_id": "3157204128893075456",
  "author_id": "u_4471", "visibility": "followers" }

The post ID carries its creation time in its top 41 bits, so any worker can compute fan-out lag as now minus the ID's timestamp, with no lookup.

The dispatcher reads the author's fanout_mode from the accounts record, not the raw count. A classifier flips that mode using the follower count and the 100K/90K band below. Then the dispatcher branches:

The celebrity branch is one write. @luna_sings's 8:58 post was in celeb:{author_id}, her last 100 post IDs, by 8:58:02, inside the 2 s target. No feed changed. Her followers pick the post up when they open: Section 7's candidate assembly merges about 20 celebrity timelines per open in one batched multi-get, keeping posts from the 48-hour merge window.

(Synchronous fan-out was ruled out in Section 6: publish latency would grow with follower count.)