RocksDB — Part 1: Foundations & Core Concepts

maang.io System Design Series

What is this?

Imagine the desk of someone who processes a torrent of paperwork all day. A slow worker would, for every new form, walk to the filing cabinet, find the exact folder it belongs in, and slot it in — a trip across the room per document. A fast worker never does that on the hot path. They keep an inbox tray on the desk for the newest forms (instant to drop into), and they also scribble every arrival into a carbon-copy journal so nothing is ever lost if the building loses power mid-shift. Only later, when the tray is full, do they photocopy the whole sorted tray into a fresh binder and shelve it. Filing never blocks receiving.

RocksDB is exactly that filing system — except it's bolted directly into your application's desk, not a records department you phone across the network. It's an embedded key-value store: a C++ library you link into your process, call functions on, and that persists data to your local disk. There's no server, no port, no network hop. You call Get("user:42") and it returns, in microseconds, from a file a few inches away.

The one-line idea: RocksDB is an embedded, persistent key-value store built on a Log-Structured Merge-tree (LSM-tree) — every write is a cheap append to a durable log and an in-memory buffer, the real filing happens later in the background, and the whole thing lives inside your process rather than behind a network socket.

If Part 2 of the Apache Cassandra topic felt familiar as you read that — good. Cassandra is an LSM-tree, and so is RocksDB. The difference is scope: Cassandra is a whole distributed database (ring, replication, coordinators) that contains an LSM storage engine; RocksDB is just that storage engine, extracted, hardened, and handed to you as a library so you can build your own database — or your own stream processor, or your own MySQL storage engine — on top of it.

1. Why RocksDB exists — the problem it solves

RocksDB was created at Facebook in 2012 as a fork of Google's LevelDB. LevelDB was an elegant embedded LSM store, but it was built for a single-threaded, spinning-disk world and choked under server workloads. RocksDB's whole reason for existing was to answer one question: how do we store data as fast as modern SSD/flash and many-core CPUs will physically allow, from inside a single process?

Before something like RocksDB, if you needed fast local persistence you had a few unhappy choices:

A B-tree store (like the classic Berkeley DB — a sibling topic in this tier). B-trees update in place: to change a key you find its page on disk (a random read) and rewrite it (a random write). On flash, random writes are the expensive operation, and they wear the device. Great for read-heavy, bad for write-heavy.
Roll your own file format. Now you own crash-recovery, compaction, caching, and concurrency — years of subtle work.
A full database server (Postgres, a Cassandra cluster). Correct, but you've added a network hop, an operational service, and latency for what might be a purely local lookup.

RocksDB fills the gap: industrial-strength local storage as a library. It absorbs write-heavy workloads by turning random writes into sequential ones (the LSM bargain), exploits SSD parallelism with multi-threaded background compaction, and exposes hundreds of tuning knobs so you can shape it to your workload. That's why it ended up as the storage engine inside so many bigger systems — you'll see the list in Part 3.

💡 Rule of thumb: reach for RocksDB when you need a fast local key-value store that outlives a process restart, and you're willing to build any higher-level features (queries, replication, transactions across machines) yourself or via the system embedding it. It gives you the hard 20% — durable, ordered, compacted storage — and stays out of your way for the rest.

2. Core concepts & vocabulary

Here's the mental model. Keep the desk analogy in mind — each term is one part of it.

Key-value, byte-string, sorted. Every entry is an opaque byte string key mapped to an opaque byte string value. RocksDB doesn't know or care about your schema — that's your job. Crucially, keys are kept in sorted order, so you can not only Get(key) but also iterate a range (Seek to a prefix and scan forward). That ordering is what makes it useful for more than a hash map.
MemTable — the inbox tray. An in-memory, sorted structure (a skip list by default) holding the most recent writes. Fast to insert into, fast to read from. There's one active memtable per column family.
WAL (Write-Ahead Log) — the carbon-copy journal. Before (or as) a write hits the memtable, it's appended to an on-disk write-ahead log so an acknowledged write survives a crash even though it's still only in RAM in the memtable. This is the same write-ahead discipline from the Write-Ahead Log (WAL) sibling topic and the ACID chapter: durable record first, then apply. Because it's a pure append, it's fast.
SSTable (SST file) — a filed, shelved binder. When a memtable fills up, it's frozen and flushed to disk as an immutable Sorted String Table. Once written, an SST file is never modified — updates and deletes become newer entries in newer files. Each SST is sorted and carries its own index and bloom filter so it's fast to search.
Levels (L0…Ln) — the tiered shelves. SST files are organized into levels of increasing size. L0 holds freshly-flushed files (whose key ranges can overlap each other); L1 and below hold files with non-overlapping key ranges within each level, so a given key lives in at most one file per level. Each level is ~10× larger than the one above.
Compaction — the nightly re-filing. Background threads merge SST files across levels, keeping only the newest version of each key, dropping deleted data, and re-sorting into tidy non-overlapping files. This is how the LSM pays down its debt (Part 2).
Column Families — labeled drawers in the same cabinet. A single RocksDB instance can hold multiple independent keyspaces called column families, each with its own memtable and its own SST files, but sharing one WAL so a write batch across families is still atomic and crash-consistent. Think of them as separate tables sharing one database.
Bloom filter & Block cache — the index card and the desktop. Each SST has a bloom filter (a tiny probabilistic "is this key possibly here?" structure — see the Bloom Filter sibling topic) so reads skip files that can't contain the key. A block cache keeps hot data blocks in RAM so repeat reads don't touch disk. Both exist to buy back the read cost the LSM design gives up.
Sequence numbers (MVCC). Every write gets a monotonically increasing sequence number. That's how RocksDB knows which of several copies of a key is newest, and it's the basis for snapshots — a consistent read view at a point in time. It's multi-version under the hood.

3. How you actually use it

RocksDB is a library, so "using it" means function calls, not a query language. Here's the shape of it (C++-flavored pseudocode, trimmed to the essentials):

// Open a database backed by a directory on local disk.
rocksdb::DB* db;
rocksdb::Options options;
options.create_if_missing = true;
rocksdb::DB::Open(options, "/data/mydb", &db);

// Put / Get — the two core operations.
db->Put(rocksdb::WriteOptions(), "user:42", "{\"name\":\"Ada\"}");
std::string value;
db->Get(rocksdb::ReadOptions(), "user:42", &value);   // value == the JSON

// Atomic multi-key write — all-or-nothing, one WAL append.
rocksdb::WriteBatch batch;
batch.Put("order:1001", "...");
batch.Delete("cart:42");                 // a delete is a tombstone, not an erase
db->Write(rocksdb::WriteOptions(), &batch);

// Ordered iteration — because keys are sorted, prefix scans are cheap.
auto it = db->NewIterator(rocksdb::ReadOptions());
for (it->Seek("user:"); it->Valid() && it->key().starts_with("user:"); it->Next()) {
    // walk every user in key order
}

Four things worth noticing, because they're the whole personality of the API:

Put and Get are the entire vocabulary — plus Delete, Merge (a read-modify-write optimization), and iterators. No SQL, no secondary indexes, no joins. You model those yourself by encoding structure into the key (e.g. user:42:orders:1001), exactly the query-first, key-designed thinking from the Cassandra topic.
A Delete doesn't erase anything immediately — it writes a tombstone, a marker that shadows the old value until compaction physically removes it. Same idea as Cassandra tombstones.
WriteBatch is your atomic unit — everything in the batch is applied together with a single WAL append, so a crash never leaves half of it.
You tune with Options. Those innocent-looking option objects hide the hundreds of knobs (buffer sizes, compaction style, compression, cache size) that let you shape RocksDB to a read-heavy vs write-heavy vs space-constrained workload. Part 2 covers the ones a Staff engineer must know.

⚠️ Because there's no server enforcing access, one RocksDB database can only be opened by one process at a time (it takes a file lock on the directory). It is embedded, single-writer, single-machine. If you're reaching for "how do two servers share this," you've stepped outside what RocksDB does — that's the job of the system built around it (Part 3).

4. When to reach for it — and when not

RocksDB is the right answer to a recognizable shape of problem:

Local, low-latency, durable state — you need microsecond point lookups against data that must survive restarts, on one machine. Stream processors keeping per-key state, embedded caches with persistence, a metadata store for a bigger system.
Write-heavy or mixed workloads on SSD — the LSM design soaks up writes that would punish a B-tree.
You're building a database (or database-like system) and need a battle-tested storage engine so you don't write compaction and crash-recovery yourself. This is RocksDB's most common role — it's the engine, not the car.

And the anti-signals — reach for something else when:

You need…	Reach for instead
A shared store multiple servers query over the network	A database server — Redis, Cassandra, DynamoDB, Postgres
Rich queries, joins, secondary indexes, SQL	A relational DB, or MyRocks/CockroachDB which add SQL on top of RocksDB
Read-mostly, in-place updates, minimal write volume	A B-tree store like Berkeley DB or LMDB (cheaper reads, no compaction)
Purely in-memory, no persistence needed	A plain hash map, or Redis without disk
Cross-machine replication / consistency out of the box	Cassandra, DynamoDB, or a system that wraps RocksDB with replication (TiKV, CockroachDB)

💡 The cleanest way to hold it: RocksDB is a component, not a product. If your answer to "how do I use RocksDB?" is "as the storage layer inside the thing I'm building," you're using it right. If it's "as my application's shared database," you probably want a database server that happens to use an LSM inside.

5. Key takeaways

RocksDB is an embedded, persistent, sorted key-value store — a C++ library you link into your process, forked from LevelDB, built to exploit SSD and many-core hardware. No server, no network, single-writer, single-machine.
It's an LSM-tree (same engine family as Cassandra): writes append to a WAL and an in-memory memtable, which later flushes to immutable SSTables, which compaction merges in the background. Cheap writes now, work deferred to later.
The API is tiny — Put/Get/Delete/Merge/iterators over byte-string keys kept in sorted order. You model structure by encoding it into keys; deletes are tombstones; WriteBatch is your atomic unit.
Column families are independent keyspaces sharing one WAL; bloom filters and a block cache buy back read cost; sequence numbers give MVCC and snapshots.
Reach for it as a building block — local durable state, write-heavy workloads, or the storage engine inside a larger system. Reach elsewhere when you need a networked shared store, SQL/joins, or built-in cross-machine replication.

Next, Part 2 opens the desk drawers: the exact write path and read path, the anatomy of an SSTable, how leveled vs universal compaction works, the write/read/space amplification trade-off at the heart of every tuning decision, and the knobs a Staff engineer reaches for when RocksDB misbehaves.

Intermediate