Section 1: A Car in a Box at 01:12:04 - Problem Understanding & Clarifications

Executive Summary: This section covers the first ten minutes of a Staff interview on a studio's annotation service. We split the prompt into what, who, why and how, and find the two kinds of writer (ML algorithms in bulk, humans one at a time) and the two kinds of reader (interactive tools and bulk training jobs). We name the trap, "it's just a tags table", and the three things it can't do. Then seven clarifying questions, the most important of which is what happens when an algorithm re-runs. We close with written assumptions and the running example: the film Night Heist, the char-detect algorithm moving from v3 to v4, and Priya, an editor on the trailer team.

The interviewer reads the prompt:

"Our studio runs dozens of ML algorithms and hundreds of human taggers over every film and episode we own. They produce annotations: 'a car is in this box at 01:12:04', 'this scene contains violence', 'Lena Cruz is on screen from 41:10 to 41:52', every subtitle line. Editors, artwork and trailer tools, and ML teams need to search all of it, for one title or across the whole catalog. Design the annotation service."

This is a real shape of system. Netflix has described one, called Marken, that stores and queries annotations on arbitrary entities, with versioned schemas and built-in support for temporal and spatial data. We'll borrow a few of its published facts and design the rest ourselves.

Decomposing the Prompt

• What it does: attach a typed fact to a piece of content, anchored in time ("41:10 to 41:52"), in space ("this box"), or neither ("contains violence"). Then find those facts again, for one title or across all of them. Two verbs: attach and find.
• Who writes: two very different producers. ML algorithms write in bulk, thousands of annotations per title in one run. Human taggers write one at a time and expect to see their edit right away.
• Who reads: editor, trailer and artwork tools ask interactive questions. ML teams read in bulk to train models. Personalization services read title-level tags.
• Why it exists: video is opaque. You can't grep a film. Annotations turn hours of pixels into something you can search.

The diagram below shows the prompt split four ways.