Section 11: Year Two on the Train - Follow-up Feature Evolution

Executive Summary: The design is defended, so the interviewer turns to the roadmap: traffic-weighted canaries, feature-flag integration, error-budget-aware scheduling, migration orchestration, multi-GB ML model deploys and deploy analytics. Each gets three beats: what changes, what stays the same, and which earlier decision made it cheap. Most of them plug into a box that already exists. One flips a decision on purpose: a 20 GB model breaks the zone-cache budget and earns peer-to-peer distribution.

Interviewer: "It's a year later. The platform team has a roadmap. Walk me through what each item costs and what it touches."

You: "I'll price each one against the boxes we already have, and name the trigger for any decision it flips."

Traffic-Weighted Canaries: 1% of Requests, Not 1% of Instances

Today the canary is 1% of one zone's instances, minimum one. For checkout-api that's 2 of 200 instances, close to the intended 1%. For the average service, with about 22 instances per zone, the minimum of one instance means 1 ÷ 22 ≈ 4.5% of that zone's traffic hits the canary, with another 4.5% on the control. The small services get the biggest canaries, which is backwards.

With a service mesh, the canary instance keeps running, but the mesh sends it exactly 1% of the zone's requests, and the control gets the same 1%. The weight becomes one more field in desired state, next to the digest, and the mesh's sidecars watch it the way agents watch digests. Ramping 1% → 5% → 25% is a weight change, not a restart, and a bad canary is aborted by setting its weight to zero.

What stays the same: the control group, the five metrics, the two-bad-minutes rule and rollback to N-1. The health analyzer compares metrics tagged by version and doesn't care how the traffic arrived. The cost: the canary instance now carries far less than its share of load, so saturation metrics must be normalized per request, and the control needs exactly the same weight or the comparison is skewed.

Instance-only canaries stay the default for services without weighted routing; the mesh path turns on per service.