Caching — Part 2: Architecture & Internals

maang.io System Design Series

What is this?

Back to the desk drawer. In Part 1 we decided to keep a fast copy of hot data close by. But a drawer is small. The moment you try to put one more thing in a full drawer, you face the question that defines this entire chapter: what do you take out? Pull the wrong thing and tomorrow you're walking down the hall for it again. Keep the right things and the drawer feels magical — everything you reach for is already there.

This part is about the engineering inside the drawer: the rules for what to evict, the actual machines (Redis, Memcached) that hold the data, and what happens when one drawer isn't enough and you need a whole wall of them working together.

The one-line idea: a cache is finite, so its real intelligence is in what it chooses to forget — and once it grows beyond one box, in how it spreads keys across many boxes without going cold every time the cluster changes.

We'll keep the desk-drawer image, but now the drawer has a clear-out policy and, by the end, a whole filing system spread across the office.