Section 5: The Row Key Is the Schema - Data Modeling

Executive Summary: Bigtable's data model is one sparse, sorted map: (row: string, family:qualifier, timestamp: int64) → bytes. Rows are sorted lexicographically and every single-row mutation is atomic. Column families are few, declared up front, and carry the policies: garbage collection, locality group, compression and access control. Qualifiers are unbounded and cost nothing when absent. Timestamps are 64-bit microseconds, so each cell keeps a short history. Because rows are range-partitioned into tablets in key order, the row key decides locality, compression, scan reach and hot spots. Webtable uses the reversed URL (com.cnn.www) so a whole site sits in one contiguous range. Profiles uses a random 64-bit user ID so a billion users spread evenly. Each user is one wide row whose cell versions are that user's event history, so a single atomic mutation covers both recent: and history:.

The interviewer points at the RowMutation from Section 4. "You've shown me the calls. Now show me the tables. Where's the schema?"

You: "Mostly in the row key. The engine stores rows sorted by key and cuts them into tablets by key range, so the key decides what's adjacent, what compresses together, what one scan can reach and which server gets the writes. Column families are the only part the engine knows about in advance. Everything else is free-form."

The Map, Precisely

• Row key: an arbitrary byte string up to 64 KB, typically 10–100 bytes. Reads and writes of one row are atomic, however many columns they touch (D3).
• Column family: declared with CreateColumnFamily and expected to number in the tens, at most hundreds. The family is the unit of policy: its garbage-collection rule, its locality group, its compression and its ACL.
• Qualifier: anything, created on first write, unbounded. A table can hold millions of distinct column keys while each row stores only the handful it uses. An absent cell takes no space.
• Timestamp: a 64-bit integer, by default the server's clock in microseconds, optionally supplied by the client (D12). Versions are stored newest first. Each family keeps either the last N versions or those younger than a maximum age.