Section 6: One Brain, a Thousand Hands - High-Level Architecture
Executive Summary: The architecture has four kinds of box. A client library links into every application process. One master keeps all metadata in RAM and logs it to two remote replicas, with read-only shadow masters beside it. A Chubby cell decides which machine is master. 1,000 chunkservers in 40 racks store 64 MB chunks as Linux files. One rule organizes everything: control flows through the master, data never does. The master fields ~12,000 RPC/s at peak while ~500 GB/s of reads flow directly to chunkservers. Reading chunk 42 costs one cached lookup and one direct read; an append adds a lease, a pipelined push and a primary that picks the offset.
The interviewer points at Section 5's three tables. "Fine, the master knows everything. Now put the machines on the board and show me a byte moving."
You: "Four kinds of box, then the map task reading chunk 42 and a crawler appending to it. Watch for one thing: no data byte ever goes near the master."
The Whole Cluster on One Board
| Box | Count | Holds | Refuses to do |
|---|---|---|---|
| Client library | One per app process | Location cache (60 s TTL), chunk-index arithmetic | Cache file data |
| Master | 1 | Namespace, file→chunk lists, chunk table: ~80 GB in 256 GB of RAM | Carry a single data byte |
| Op-log replicas | 2 remote | Every mutation, flushed before the master acks | Serve clients |
| Shadow masters | A few | A log-tailing copy of metadata, < 1 s behind, plus a polled location map (this design) | Accept mutations |
| Chubby cell | 1 (shared) | The master lock with a 12 s session lease | Store file metadata |
| Chunkservers | 1,000 in 40 racks × 25 | ~1.5M replicas, ~90 TB each, with checksums and versions | Decide mutation order on their own |
One box goes beyond the 2003 paper, which had an external monitor restart the master and move a DNS alias. A Chubby lock for election and naming, and promoting a shadow through it, are this design's choices, and the Distributed Lock Service module covers how that lock works.
Sign in to continue reading
The rest of this lesson is available with a free account. Signing in with Google or Microsoft is free.
Sign in to read the full lesson