Section 8: When Nobody Can Read Everything - Deep Dive: The Five-Million-Viewer Stream

Executive Summary: Everything in Section 7 works for a 50-viewer stream. This section takes on v_8812 at its peak, 5M viewers posting 5,000 comments a second, through four mechanisms. Selection splits delivery into a shared broadcast lane and a per-viewer personal lane, with a reverse friend index that turns 2.5M checks per tick into 500 lookups. The fan-out tree cuts one router's 10 Gbps to 30 MB/s. Single-flight backfill absorbs a 50K-joins-per-second storm. Sub-sharding splits a 300 MB-per-minute hot partition.

The interviewer taps the board. "Your design works for Maya's cousin streaming a birthday party. Now Aria Vance goes live and five million people show up. Walk me through it."

You start with the number that rules out the naive answer. Five million viewers × 5,000 comments a second = 25 billion deliveries a second, from one stream. That is about 167 times the platform-wide budget of 150M (Section 3), and at 5,000 a second "show everything" is an unreadable blur anyway. The display cap, 10 comments a second per viewer, is what makes this stream possible. A random viewer of v_8812 sees 0.2% of its comments through the shared feed. The engineering question is which 0.2%.

Selection: Two Lanes, One Screen

Delivery splits into two lanes:

• The broadcast lane is the same for everyone: roughly the top 10 comments a second. The Channel Router that owns v_8812's log partition scores every comment. Signals: cleared safety, a quality model, author signals (verified, followed by the broadcaster), engagement, and language diversity.
• The personal lane is built per viewer at the Gateway: your own comments, your friends' comments, the broadcaster's and moderators' comments, and the pinned comment.

Personal-lane comments are always delivered and use up the 10-a-second budget first; the broadcast lane fills what's left. If personal traffic alone exceeds 10 a second it still goes out: the cap binds the broadcast lane, not the guarantee. That's how Maya's "Play the acoustic version!!" reaches Tomás in Lisbon. It will almost certainly never win a broadcast-lane slot against 4,999 rivals, and it doesn't need to: Maya is Tomás's friend.

Interviewer: "How does the gateway know Maya is Tomás's friend? You can't ask the Social Graph 5,000 times a second."

You: "It never asks at delivery time. Each gateway hosts about 5K viewers of v_8812 and receives its full firehose from a Regional Relay, 1.25 MB a second, so the personal lane needs no central per-viewer work. The naive check, for each local viewer and each comment in this 100 ms tick, is 5K × 500 = 2.5M checks per tick. So I invert it. At connect, the gateway loads the viewer's friend list from the Social Graph into a reverse friend index: author → local connections that count that author as a friend. Now it's one hash lookup per comment, 500 per tick. For about 10K large-stream viewers per gateway at ~300 friends and 16 bytes each, that's about 48 MB."

# Gateway: runs every 100 ms for each large video it hosts.
def tick(video, batch, broadcast_ids):
    ring[video].extend(batch)                        # 60 s ring buffer (Section 7)

    # Broadcaster and moderators are personal lane for everyone: never truncated.
    everyone_frame = encode_sse([c for c in batch
                                 if c.author_role in ("broadcaster", "moderator")])

    personal = defaultdict(list)
    for c in batch:                                  # ~500 comments per tick on v_8812
        for conn in friend_index.get(c.author_id, ()):    # 1 hash lookup per comment
            if conn.video == video:
                personal[conn].append(c)
        for conn in own_conns.get(c.author_id, ()):       # author's own devices
            personal[conn].append(c)

    # Lane marks are IDs; the bodies arrived earlier in the firehose.
    shared = [ring[video].get(cid) for cid in broadcast_ids]          # already cleared
    shared_frame = encode_sse(shared)                # serialized ONCE per gateway

    for conn in viewers[video]:                      # ~5K connections
        conn.write(everyone_frame)                   # personal lane: always sent,
        mine = personal.get(conn)                    #   uses the budget first
        if mine:
            conn.write(encode_sse(mine))
        conn.write_shared(shared_frame, room=conn.budget_left())   # cap binds this lane only

The last loop carries 50K deliveries a second (5K viewers × 10/s), nearly all of it buffer copies of one shared frame; the only per-viewer work is the personal lane, empty for most viewers on most ticks. The pinned comment goes out separately, as a pinned event when it changes.

The router sends each comment's body down the tree once (the firehose); about 2 s later, once Async Safety has cleared it, it sends the broadcast-lane selection as lane marks (comment IDs). Each gateway already holds the bodies in its ring buffer, so the selection costs about 10 tiny events a second.

Personal-lane comments go out once they pass the synchronous Safety Filter and meet the 2 s p99. Broadcast-lane comments are drawn only from those Async Safety has cleared, hence that lane's 4 s p99; Section 11 makes the reach argument.

Critical Insight: The lanes split the problem where cost and risk change. The broadcast lane is one central decision copied five million times, so it can afford to be slow and careful. The personal lane is millions of small local decisions, so it must be fast and cheap. Either alone fails: broadcast-only loses your friends; personal-only makes the stream feel empty.

The pushback

Interviewer: "Isn't sampling just hiding comments?"

You: "Every comment is accepted, made durable, stored, and visible in scroll-back and replay. Its author and every watching friend see it live. What we decline is 25 billion deliveries a second of blur. YouTube Live surfaces the same idea as 'Top chat' versus 'Live chat'. The broadcaster who wants every comment gets a different tool, in Section 9."

Interviewer: "Then why not personalize the ranking for each viewer?"

You: "Central per-viewer ranking is 5M viewers × 5K comments a second, the same 25-billion problem in a new form. The personal lane already captures the strongest signal, the social one. If we ever need per-viewer ML, it runs at the gateway over a short candidate list, never centrally. And a client can request a lower cap, for slow devices or accessibility."

Quick check before moving on: