Section 1: It's Not a Mutex, It's a Foundation - Problem Understanding & Clarifications

Executive Summary: This section covers the opening minutes of a prompt that sounds like one Redis command and is really the referee every other system in the company depends on. We look past the word "lock" to the four jobs clients actually hire the service for: leader election, small critical configuration, naming and group membership. Then we break the prompt down, run a clarifying exchange in which each answer kills or creates work, and argue for a lock service instead of a consensus library. We write the assumptions down and meet orders-db, whose three would-be masters recur in every section.

The interviewer reads the prompt aloud:

"Design a coarse-grained distributed lock service, like Google's Chubby, that thousands of internal services use to elect leaders, store small pieces of critical configuration and find each other. It must stay correct when machines crash, pause or get partitioned."

The tempting answer arrives in about four seconds: SET lock:orders-db NX PX 30000 on a Redis instance, release on exit, done. That's the trap. It treats the lock as the product. It isn't. Read the prompt again: elect leaders, store configuration, find each other. Nobody wants a lock for its own sake. They want exactly one storage master, exactly one scheduler, exactly one cron runner, and everyone else needs to learn who won. The Chubby paper (Burrows, OSDI 2006) records the surprise directly: Chubby was built for locking, and it ended up being used most as a name service.

Key Insight: Every primary-backup system needs exactly one primary, and it can't safely appoint itself. Two replicas that each conclude "the other one is dead, so I'm in charge" give you split brain: two masters writing the same tablets. A lock service is the outside referee that makes "exactly one" true. Everything in this design exists to keep that referee correct while machines crash, pause and get partitioned.

Decomposing the Prompt

• What it does: hold locks on named nodes, store small files next to them, and tell clients when either changes. Three primitives, lock, small file and change event, and every client job is a recipe built from them.
• Who uses it: infrastructure, not people. Storage-system masters, cluster schedulers, batch coordinators, every service that needs a primary. If the lock service is down, they can't fail over. So it sits at the very bottom of the dependency graph.
• Why it exists: to replace a dozen hand-rolled heartbeat-and-takeover schemes with one referee that has been proven correct and is run by one team.
• How it's used: rarely, but critically. A leader takes a lock and holds it for hours. Tens of thousands of followers read the leader's address once and cache it. Throughput is not the point. Being right and staying available is.

Leader election is lock plus file: the winner writes its address. Configuration is file plus event. Naming is file plus event, read through a cache. Membership is a directory of ephemeral nodes that disappear when their owner's session dies. One small set of primitives covers four products.