Section 6: Seven Boxes and a Stopwatch - High-Level Architecture

Executive Summary: This section assembles the stores and surfaces of the previous sections into a system you can draw in two minutes: seven components, plus an eighth that exists only for people who are offline. We then walk a single location update — Maya's, at 2:14pm on her way to Dolores Park — from her phone to Raj's screen, tallying the milliseconds against the five-second arrival budget and discovering roughly forty times more headroom than we need. The lesson that falls out of the stopwatch is the one that sets up the central deep dive: this system is bounded by message volume, not by latency, and the most interesting boxes on the whiteboard are the ones that are not there.

The interviewer uncaps a marker and gives you the whiteboard. This is the moment candidates over-draw — twelve rectangles, three queues, a data lake off to the side, arrows crossing arrows. Resist it. A component earns its box by owning an access pattern that no other component owns, and by this point every access pattern has already been named. Draw the boxes that answer them and nothing else, and then let the interviewer try to add one.

The Seven Boxes

• Mobile client. Reads GPS and posts a point on a server-dictated interval — ten seconds while moving, thirty by default, three hundred while stationary. It holds one persistent WebSocket, renders distances it is handed, and computes nothing about other people.
• Load balancer and API gateway. Terminates TLS, validates the session token, and enforces connection-rate limits. Identity is established here and never re-asserted by a client payload.
• Location service. The only stateful-looking box on the write path, and it is stateless. Four jobs per update: coarsen the point to roughly a hundred-metre grid, write it to the location cache, read the last known positions of the sender's online friends, and publish to the channels of the handful who pass the geo-prefilter — an event carrying a computed distance, never coordinates. Distance arithmetic and the per-pair state machine both live here, at publish time. The entire feature's cleverness lives in that fourth job.
• Location cache. In-memory, keyed by user, ten-minute TTL, partitioned by user ID across the cache tier — which is why reading a publisher's friends' entries is a scatter-gather across many partitions rather than one local batch, a point Section 7 prices. The whole live set — ten million concurrent sharers at about a hundred bytes each — is roughly one gigabyte, which is why this is a messaging problem wearing a storage problem's clothes.
• Pub/sub broker. One channel per online user — the recipient's channel, which is the whole reason a publisher-side prefilter is expressible at all. Ten million live channels, sharded by user ID, carrying about 1.3 million messages a second at roughly 260 megabytes a second. Fire-and-forget, no durability, no replay.
• Session servers. Fifty thousand WebSockets each, so two hundred servers carry ten million concurrent clients, provisioned at two hundred and fifty-six for rolling deploys and headroom. At session establishment a session server subscribes to exactly one channel — its own user's — and relays whatever lands on it down that socket. Ten million online users means ten million subscriptions, one apiece, and one resubscribe per reconnect. A session server holds no durable state; it is a router with a socket.
• Friend-graph service. The source of truth for the mutual-friendship gate and the sharing opt-in, read once per session for about four hundred IDs, cached for the location service to consult on every publish, and invalidated by graph-change events. The graph service itself is never queried per update: 333 thousand graph queries a second for data that changes a few times a month would make it the first thing to fall over.

The eighth box is deliberately drawn with a dotted line, and it is worth being precise about why — because the usual reason given is wrong. A high-priority alert push to a reachable device is fast: well under a second, often a couple of hundred milliseconds. Latency is not what disqualifies mobile push as the live transport. Three other things do. It offers no delivery and no ordering guarantee, so you cannot build a state machine on the sequence it produces. The silent and background variants — the only ones that could carry routine positional updates without buzzing a pocket — are deliberately starved: iOS throttles and coalesces background pushes to roughly a couple per hour per app, and FCM normal-priority messages are simply held while the device sits in Doze. And an alert push is by definition user-visible, which makes it structurally unfit for a stream of "Maya moved forty metres" updates. That leaves it exactly right for the one case it owns: Raj has no open socket, so an arrival becomes a single alert push instead of a message dropped on the floor. Calling that out as an offline-only dependency, rather than letting it sit on the main path, is the difference between a design that understands its transports and one that lists them.

Quick check before moving on: