Section 5: The Row That Never Goes Backwards - Data Modeling
Executive Summary: The deploy system's data is small: about 100 MB of deploy records and 1 GB of instance events a day. What makes the model hard is that every row is a promise someone acts on. This section models seven entities: Service, Release, RolloutPlan, Rollout, DesiredState, ActualState and AuditEvent. It gives the rollout a state machine, uses a per-slot generation number to tell "the agent did what we asked" apart from "the host happens to match", and explains why all of it lives in a strongly consistent store. Current state is bounded by the fleet (400,000 slots). History is append-only and kept for at least a year.
The interviewer slides the marker back. "You have an API. What does it write?"
You: "Less than you'd think, and it all matters. Section 3 put the control plane at a few hundred writes a second. So this isn't a storage-scaling problem. It's a correctness problem: if one row is stale or goes backwards, a host runs the wrong binary. I'll split the model into three groups: what we ship, what we're doing with it, and what happened."
Seven Entities, Three Groups
| Group | Entity | Key fields | Mutability |
|---|---|---|---|
| What we ship | Service | owner team, default plan, capacity floor (the share of instances that must stay serving, set from its 25% headroom) | Rarely edited |
| Release | release_id, service, digest, commit, provenance, signature, created_at, status (candidate, deployed, bad) |
Immutable except status |
|
| RolloutPlan | stages, batch %, bake per stage, gate thresholds | Versioned; a rollout copies the version it started with | |
| What we're doing | Rollout | rollout_id, release, previous_release, plan snapshot, state, current stage and batch, lease owner and expiry, version |
Updated every loop tick |
| DesiredState | per host slot: service, digest, config_version, generation |
Written only by the orchestrator | |
| ActualState | per host slot: digest, phase, health, generation_seen |
Written only by the agent | |
| What happened | AuditEvent | who, what, when, why, evidence | Append-only |
The Release row is the immutable artifact from Section 4: one digest, built once, promoted through every stage. We never rebuild per environment and never accept a mutable tag like latest. Only status changes, and only forward. Here is Sam's release after CI registered it:
release_id: r-1432
service: checkout-api
digest: sha256:9f4e…
commit: a1b2c3d
signature: signed by CI, provenance attached
created_at: Tue 14:52 UTC
status: candidate # becomes bad at 16:48
The Rollout copies its plan instead of pointing at the live plan. If the checkout team edits their plan at 16:00, the rollout that started at 14:52 keeps the bake times it started with. Pointing at the live plan would let an edit change a rollout halfway through, and nobody could say afterwards which rules it ran under.
Key Insight: previous_release is the most important field on the Rollout. Rollback is a roll forward to N-1's digest, so the rollout has to know N-1 exactly, at the moment it starts. Reading "whatever ran before" from the hosts at rollback time is too late: by then half the hosts are on N.
Quick check before moving on:
Sign in to continue reading
The rest of this lesson is available with a free account. Signing in with Google or Microsoft is free.
Sign in to read the full lesson