Section 9: At Least Once, Exactly Where We Say - Deep Dive: Record Append, Snapshot & the Consistency Model

Executive Summary: Record append carries the crawl. 500 crawlers append 1 MB records to one file with no locks, because the system picks the offset, not the client. When a record doesn't fit, the primary pads the chunk and answers RETRY_NEXT_CHUNK. Records are capped at 16 MB so padding stays small: about 0.8% at 1 MB per record. A failed attempt that gets retried can leave a duplicate on some replicas and garbage on another, so replicas are not byte-identical. What GFS promises is that each successful append was written at least once, atomically, at the same offset on every replica. Then: the paper's consistency table, crawler-217's record at 18 MB and 19 MB, and copy-on-write snapshot.

"Section 8 ordered writes at known offsets. But you have 500 crawlers on one file. Do they take a lock?"

You: "No. With write-at-offset they'd all queue behind a lock. So the client says 'put these bytes at the end' and the system says where they went."

How the Primary Places a Record

Record append is Section 8's write path plus one decision at the primary. The library uses PushData to send the record to all three replicas of the file's last chunk, then sends Append(handle, version, dataId, secondaries[]) to the primary, which checks whether the record fits.

A record never straddles a chunk boundary, and the cap of a quarter chunk, 16 MB, keeps padding small. With 1 MB records the average waste is half a record per chunk, 0.5 / 64 ≈ 0.8%, and even the worst case stays under 25%.

"So padding is what you pay for atomicity?"

You: "Yes. A record split across two chunks would be two mutations that could interleave with other appends. Padding keeps every record whole."

Quick check before moving on: