Rate Limiting — Part 2: Architecture & Enforcement

maang.io System Design Series

What is this?

Back to our nightclub. In Part 1 we obsessed over the bouncer's clicker — how he counts. But there's a question we waved away that turns out to be just as important: where do you put the bouncer, and how many of them are there?

You could hand every guest a self-clicker and trust them to count themselves (cheap, but anyone can lie). You could station one bouncer at the club's front door who sees everyone (one consistent count, but a chokepoint). Or you could put a separate bouncer at the door of every room inside (each room protected, but now no one knows the building's total). And the moment you have more than one bouncer, a new headache appears: if the club hires three bouncers for one busy door and each keeps his own clicker, the "100 people max" rule quietly becomes 300.

That last problem — many bouncers, one shared limit — is the heart of distributed rate limiting, and it's where this chapter lives. We'll figure out where to enforce, the difference between each node counting locally versus all nodes sharing one distributed counter, how a shared store like Redis makes that count correct and atomic, and how to scope the key so you're limiting the right thing.

The one-line idea: the algorithm decides how to count; the architecture decides where the count lives and who shares it — and the hard part is keeping one honest count across many enforcing nodes.