Circuit Breaker — Part 1: Foundations & Core Concepts
maang.io System Design Series
What is this?
Look inside the electrical panel in your home. Behind that little grey door is a row of switches, and each one is a circuit breaker. When everything is fine, the breaker sits closed and electricity flows to your kitchen, your lights, your laptop charger. But the moment something goes wrong downstream — a short circuit, an overloaded outlet, a hair dryer that decides to die dramatically — the breaker trips. It snaps open and cuts the current. Not because it's broken, but precisely because it's working: it would rather kill the power to one room than let a fault upstream melt the wiring in your walls and burn the house down.
And here's the part people forget: after it trips, you don't leave it off forever. You walk over, wait a moment for things to settle, and flip it back on to see if the problem is gone. If the lights come back and stay on, great — you're back in business. If it trips again instantly, you leave it off and go find the real problem. That cautious "flip it back on and watch" step is the whole trick.
A software circuit breaker is exactly this idea, wrapped around a call to a remote dependency — a database, a payment gateway, another microservice. When that dependency starts failing, the breaker trips and stops you from calling it at all, failing fast instead of piling up. Then, after a cooldown, it lets a single test request through to check whether the dependency has recovered. Same three moves as the panel in your wall: flow, trip, cautiously re-test.
The one-line idea: A circuit breaker is a stateful proxy around a risky call that stops calling a failing dependency — failing fast instead of waiting on doomed requests — so one sick service can't drag the whole system down with it, then automatically probes to reopen the flow once the dependency heals.
We'll keep coming back to that electrical panel — closed, tripped, cautiously re-testing — because it maps perfectly onto the three states you're about to meet.
1. Why it exists: the cascading failure
To feel why this pattern matters, you have to picture what happens without it. This is the failure mode that has taken down more large systems than almost anything else, and interviewers love it because it separates people who've operated real systems from people who haven't.
Imagine a normal request path. Your web tier (Service A) calls a recommendations service (Service B) on every page load. Service B is usually fast — say 20 ms. One day, B's database gets slow, and B's responses balloon from 20 ms to 30 seconds before finally timing out.
Now watch what happens to A, the caller, even though A itself is perfectly healthy:
Every request that calls B now holds a worker thread (or connection, or memory) hostage for 30 seconds instead of 20 milliseconds. Requests arrive faster than they drain. Within seconds, A's thread pool is completely full of threads blocked on B. Now A can't serve anything — not even the pages that have nothing to do with recommendations. A is effectively down, purely because it kept politely waiting on a dependency that was never going to answer.
And it doesn't stop there. A's own callers (the load balancer, the gateway, other services) now see A timing out, and their thread pools start filling. The failure cascades outward, one healthy service at a time, until a single slow database has taken down an entire product. This is called a cascading failure or a retry storm when retries make it worse, and it is the nightmare the circuit breaker was invented to prevent.
The insight is brutal and simple: the dangerous thing was not B failing — it was A refusing to stop calling B. A slow dependency is far more dangerous than a dead one, because a dead one at least fails instantly. What A needed was something that would notice B is sick and say "stop — don't even try, just fail this call immediately and free the thread." That something is a circuit breaker.
💡 This is the same resilience thinking you met in the 101 resilience chapter — a system is only as available as its weakest synchronous dependency unless you deliberately decouple from it. The circuit breaker is the single most important tool for that decoupling in a synchronous, request/response world.
2. Core concepts & vocabulary: the three states
A circuit breaker is a small state machine with three states. Hold the electrical panel in your head as you read — the mapping is exact.
CLOSED — the normal, healthy state. The breaker is closed, so current flows: calls pass straight through to the dependency. The breaker just quietly watches, counting failures. As long as failures stay below the threshold, it does nothing but let traffic through. (Counter-intuitive naming alert: closed = calls allowed, exactly like a closed electrical circuit lets current flow. Beginners always get this backwards.)
OPEN — the tripped state. Too many recent calls failed, so the breaker opens the circuit and stops the flow. Every call now fails immediately — it "short-circuits," returning an error or a fallback in microseconds without ever touching the sick dependency. This is the whole point: A stops holding threads hostage waiting on B. The breaker stays open for a configured cooldown / reset timeout (say, 30 seconds), giving B room to recover without being hammered.
HALF-OPEN — the cautious re-test. When the cooldown elapses, the breaker doesn't just fling the doors open. It moves to half-open and lets a single trial request (or a small trickle) through as a probe. If that probe succeeds, the dependency looks healthy again → the breaker snaps back to CLOSED and normal traffic resumes. If the probe fails, the dependency is still sick → the breaker flips straight back to OPEN and starts a fresh cooldown, without exposing all your traffic to the failure again.
That's the entire mental model. Closed lets traffic flow and watches. Open fails fast and waits. Half-open sends one scout over the hill to check if it's safe. Everything else — the counting, the thresholds, the fallbacks — is detail hanging off these three states. We'll take them apart mechanically in Part 2.
A few more terms you'll want in your vocabulary:
- Failure threshold — the rule that decides when CLOSED trips to OPEN. Could be a raw count ("5 failures in a row") or, better, an error rate over a rolling window ("more than 50% of the last 20 calls failed"). Part 2 shows why the rate-based version is what production libraries actually use.
- Fallback — what you return instead of the real answer when the breaker is open (or the call fails). A cached value, a default, an empty list, a friendly "recommendations unavailable" — anything better than hanging or crashing. This is graceful degradation.
- Timeout — the breaker's essential partner. A breaker can only count a call as "failed" if the call is allowed to give up. An unbounded call never fails — it just hangs forever. Timeouts and breakers are a package deal; we'll return to this.
3. How you actually use it
In practice you almost never hand-write the state machine — you wrap the risky call using a resilience library. Here's the shape with resilience4j (the modern JVM standard, successor to Netflix's Hystrix), because reading it makes the concepts click:
// 1. Configure the breaker's policy
CircuitBreakerConfig config = CircuitBreakerConfig.custom()
.failureRateThreshold(50) // trip if >50% of calls fail
.slidingWindowSize(20) // …measured over the last 20 calls
.waitDurationInOpenState(Duration.ofSeconds(30)) // stay OPEN for 30s before probing
.permittedNumberOfCallsInHalfOpenState(3) // let 3 probe calls through
.slowCallDurationThreshold(Duration.ofSeconds(2))// count a >2s call as "failed" too
.build();
CircuitBreaker breaker = CircuitBreaker.of("recommendations", config);
// 2. Wrap the remote call — with a fallback for when it's open
Supplier<List<Item>> guarded = CircuitBreaker
.decorateSupplier(breaker, () -> recommendationClient.fetch(userId));
List<Item> recs = Try.ofSupplier(guarded)
.recover(throwable -> cachedOrEmptyRecommendations(userId)) // graceful degradation
.get();
Read what that policy says in plain English: "Watch the last 20 calls to the recommendations service. If more than half of them fail — or run slower than 2 seconds — trip open. Stay open for 30 seconds, then let 3 test calls decide whether to close again. And whenever you're open or a call fails, quietly serve cached recommendations instead of blowing up the page." That single block of config is doing everything the electrical panel does, automatically, thousands of times a second.
⚠️ Notice slowCallDurationThreshold. A production breaker doesn't only trip on errors — it trips on slowness too, because (remember Section 1) slow is the truly dangerous state. A dependency answering every call in 8 seconds is technically "succeeding" while it strangles your thread pool. Good breakers treat "too slow" as a failure. This is a Staff-level detail worth knowing exists.
4. When to reach for it — and when not
A circuit breaker earns its keep around a remote, synchronous call that can fail or hang and that you make often. Reach for one when:
- You make network calls to a dependency (another service, a database, a third-party API, a cache) on the request path — anything across a process or machine boundary.
- That dependency's failure shouldn't take you down — you have a sensible fallback, or you'd rather fail fast than hang.
- The call happens frequently enough that the breaker sees a meaningful signal. (A once-a-day cron call gets little value from a rolling-window breaker.)
Don't reach for one — or reach for something else — when:
| Situation | Why a breaker is wrong / not enough | Better fit |
|---|---|---|
| In-process / local calls | No network, no partial failure to protect against; you'd just add latency and complexity. | Plain error handling. |
| You want reliable delivery, not fast failure | A breaker drops the request; sometimes you must not lose it. | Asynchronous messaging / a queue — decouple in time (see that sibling topic). |
| Isolating one dependency's resources from another's | A breaker fails fast per call but doesn't cap how many threads B can consume. | Bulkhead pattern (separate thread pools per dependency) — pairs with a breaker. |
| Transient, one-off blips | Tripping the whole circuit is overkill for a single dropped packet. | Retry with backoff + jitter — but bounded, and behind a breaker so retries can't storm. |
| A single batch job | One infrequent call; a rolling window never fills. | A simple timeout + retry. |
The real-world answer is rarely "a breaker instead of these" — it's a breaker alongside them. Timeouts bound each call, retries with jitter absorb transient blips, bulkheads cap resource usage per dependency, and the circuit breaker sits on top deciding when to stop trying entirely. They're a resilience stack, and Part 3 shows how they compose. This connects directly to the load-balancing and health-check ideas from the 101 tier — a breaker is essentially a per-caller, per-dependency health check that reacts in milliseconds, far faster than a central load balancer ejecting an unhealthy node.
Key takeaways
- A circuit breaker is a stateful wrapper around a remote call that stops you from calling a failing dependency — it fails fast instead of letting doomed calls pile up and exhaust your threads.
- The problem it solves is the cascading failure: a slow dependency fills the caller's thread pool, the caller goes down, and the outage spreads upstream. A slow dependency is more dangerous than a dead one.
- It's a three-state machine: CLOSED (calls flow, watch for failures) → trips to OPEN (fail fast, wait out a cooldown) → HALF-OPEN (probe with one call) → back to CLOSED on success or OPEN on failure. Remember: closed = allowed, open = blocked, like an electrical circuit.
- Pair every breaker with a timeout (so calls can fail) and a fallback (so failing is graceful — a cached or default value), and treat slow calls as failures, not just errors.
- It's not a silver bullet — it lives in a stack with timeouts, retries+jitter, and bulkheads, and it's the wrong tool when you need reliable delivery (use a queue) rather than fast failure.
Next, Part 2 opens the box: the exact data structures behind the failure counter (the rolling window), the precise state-transition logic with real numbers, how the half-open probe is implemented without a thundering herd, and the tuning knobs and failure modes a Staff engineer must be able to defend.