Berkeley DB — Part 2: Internals & Implementation

maang.io System Design Series

What is this?

Back to the filing cabinet in your office. In Part 1 we saw the drawers (access methods) and the room they live in (the environment). Now we open the cabinet and watch a careful clerk — you — make a change correctly, so that even a power cut mid-edit can't lose or corrupt anything.

The trick the clerk uses is old and beautiful: before you ever touch the real folder in the drawer, you scribble the change into a bound logbook first — "at 3:04 I changed Alice's address from X to Y." Only then do you edit the folder. Why? Because if the lights go out while your hand is in the drawer, the folder might be half-edited and unreadable — but the logbook is intact and append-only, so when the power returns you re-read the logbook and redo (or undo) exactly what was in flight. That logbook is the write-ahead log (WAL), and it is the single most important internal idea in Berkeley DB.

The one-line idea: Berkeley DB updates its B-tree pages in place, but it never does so unsafely — every change is described in the write-ahead log and made durable there first, so a crash is always recoverable by replaying the log. Fast in-memory edits, a cheap sequential log append on the hot path, and correctness guaranteed by the WAL protocol.

This is the opposite internal bet from Cassandra's LSM-tree (which you met in the Cassandra topic). BDB keeps a classic B-tree that it modifies in place and leans on the WAL for durability; Cassandra and RocksDB never modify in place and lean on background compaction. Hold that contrast — Part 3 turns it into an interview answer.