Section 7: Fifty Thousand Open Doors per Machine - Deep Dive: Real-Time Delivery (Candidate-Driven)

Executive Summary: The interviewer hands over the wheel, and the candidate picks the four places where the delivery path would actually break in production: the transport and the Gateway that holds 50,000 connections, ordering and resume across reconnects, read-your-own-write for the author, and keeping comments in step with a video each viewer watches at a different delay. Each dive follows the same three beats (mechanism, number, trade-off), and most survive a round of pushback. The five-million-viewer stream is deliberately held back for Section 8.

The architecture is on the board, and Maya's comment has made its trip from São Paulo to Lisbon. The interviewer caps the marker: "Where do you want to go deeper?"

You: "The 5M-viewer stream deserves its own dive. First I want to prove the ordinary delivery path holds, because it carries every stream, and most have fewer than ten viewers. So: transport, ordering and resume, the author's own view, and video sync."

Key Insight: Naming what you are not diving into yet, and why, is part of the choice. It shows you have a map of the whole system.

Dive One: Transport and the Gateway

The traffic is lopsided. A viewer writes about once every 1,000 seconds (~17 minutes) but reads up to 10 comments a second. The transport should match that shape.

Interviewer: "Every chat system uses WebSockets. Why are you different?"

You: "Because this isn't chat. Upstream is one comment every 17 minutes, so an ordinary POST handles it, and that path already has auth, rate limiting and idempotency. Downstream, SSE gives me automatic reconnect and Last-Event-ID resume as part of the spec, so I don't have to invent them. It is plain HTTP, so proxies that mishandle protocol upgrades don't break it. I'd flip in two cases. If the native app already keeps a persistent MQTT or WebSocket connection for messaging, I'd multiplex onto that rather than open a second one. If we add features with high-rate upstream, like live games, WebSocket wins. The Gateway hides the transport either way."

The Gateway is an event-loop server. Each connection is a 20 KB state object (TLS buffers, send queue, subscriptions, friend list), not a thread. We tested each box to 100K connections and run it at 50K, so the survivors have room to absorb a failed neighbour's viewers. That works out to 1,000 Gateways across 6 regions, ~1 GB of connection state each, and ~150K deliveries/s (300 Mbps raw) per box. Any viewer can land on any Gateway: the connection belongs to the device, not the video. HTTP/2 multiplexes several video streams over one TCP connection, so when feed autoplay switches Maya's video, there's no new TLS handshake.

Heartbeats go out every 30 s. A quiet stream can go minutes without a comment, and NATs and load balancers drop idle-looking connections. The heartbeat keeps the path open and lets the client notice a dead one. Globally that's ~1.7M/s, about 34 MB/s: negligible.

Quick check before moving on: