Section 5: A Tree of Tiny Files - Data Modeling

Executive Summary: The whole data model is one record type, the znode: a node in a hierarchical namespace holding a few kilobytes of data and a stat of counters and zxids. Two flags decide its life: ephemeral ties it to a session, and sequential appends an increasing counter. Every state change is stamped with a 64-bit zxid (epoch high, counter low) that puts all writes in one total order. The full tree, about 600 MB in our assumptions, lives in memory on every server, and survives restarts through an fsynced transaction log plus fuzzy snapshots that are safe only because every transaction is idempotent.

The interviewer points at the seven calls from Section 4. "Every one of those takes a path. What's actually at the end of it, and what's on disk when a server reboots?"

You: "A tree of tiny files: a name, a little data and a stat. No lock records, lease records or waiter queues, because Decision 1 put all of that in client recipes."

Znodes: Two Kinds, One Flag

Interviewer: "Why can't an ephemeral have children?"

You: "Because a session's death would then cascade through a subtree other sessions wrote into. An ephemeral means one thing: this node exists while that session is alive. Expiry deletes a flat list of nodes, nothing more, which is what makes Section 9's membership and election cheap."

The sequential flag is how the election gets its order. When sch-1, sch-2 and sch-3 each call create("/services/settlement/election/n_", ephemeral|sequential), the server appends the parent's next counter value and returns the real names: n_0000000041, n_0000000042, n_0000000043. The counter belongs to the parent, and in Apache ZooKeeper's implementation it comes from the parent's child-change count, so deletions leave gaps. Recipes compare numbers and never assume they're contiguous.

Here is Sam's corner of the tree:

The dotted edges are watches, not tree edges; they live in server memory, as we'll see below.