Section 8: From Draft to Every Edge in Sixty Seconds - Deep Dive: The Publish Pipeline & Read Path

Executive Summary: Publishing happens only ~1.2 times a second, but it is the most dangerous write in the system: it must be atomic, retry-safe, and on every CDN edge within 60 seconds. We walk through the publish transaction, the outbox behind every side effect, the cache policy that makes a 24-hour TTL safe, and Maya fixing a typo at 10:20 while her post serves 5,000 views a second.

The interviewer taps "Editor service": "Maya clicks Publish. Take me from that click to Daniel's phone in Berlin."

"Three promises: readers never see half a post, a retry never publishes twice, and every edge serves the new version within 60 seconds. The transaction owns the first two; the cache owns the third."

The Publish Transaction

The client sends POST /v1/posts/a81f3c9e2b7d/publish with {draftVersion: 540} and an Idempotency-Key header: exactly the version Maya was looking at.

def publish(public_id, draft_version, idem_key):
    if prior := idempotency.get(idem_key):
        return prior                                 # retry returns the original answer
    post = posts.by_public_id(public_id)             # 12-hex URL ID -> 64-bit post_id
    draft = kv.get("drafts", post.id)
    if draft.draft_version != draft_version:
        raise PreconditionFailed(412)                # draft moved since she clicked
    doc = validate_and_render_check(zstd_decode(draft.body_zstd))  # Section 10
    body_key = sha256(canonical_json(doc))           # content hash, not zstd bytes
    objects.put_if_absent(f"revisions/{body_key}", draft.body_zstd)   # BEFORE the txn

    with pg.transaction():
        first = posts.lock(post.id).first_published_at is None   # SELECT ... FOR UPDATE
        rev = insert_revision(post.id, kind="published", body_key=body_key,
                              draft_version=draft_version)
        update_posts(post.id, published_revision_id=rev.id, status="published",
                     first_published_at=coalesce_now())   # set only on first publish
        insert_outbox(aggregate_id=post.id,
                      type="post.first_published" if first else "post.revised",
                      payload={"revision_id": rev.id})
        idempotency.record(idem_key, {"revisionId": rev.id, "url": canonical_url(public_id)})

    cdn.soft_purge_async([f"post:{public_id}", f"pub-index:{post.publication_id}"])  # best effort
    return {"revisionId": rev.id, "url": canonical_url(public_id)}

Why each line is there:

• The body goes to object storage before the transaction. A failed transaction leaves only an orphaned object for garbage collection; the reverse order could commit a pointer to a missing body. Rewrites are harmless: the key hashes the canonical uncompressed JSON, so the same body keeps its key across compressor upgrades.
• Two IDs, translated once. URLs, API paths and surrogate keys carry the random 12-hex public_id, which leaks nothing about creation time or volume; tables, KV keys and Kafka use the 64-bit post_id.
• The pointer flip is a single-row update. Readers resolve posts.published_revision_id to an immutable object. They get the old revision or the new one and never a mix, because there is nothing to tear.
• The outbox row commits with the flip. Dual writes to Postgres and Kafka lose the event on a crash between them, or announce a phantom post if Kafka goes first and the transaction aborts. With the outbox, the event exists exactly when the publish does.
• Nothing slow runs inside the request. Fan-out to Maya's 40,000 followers, or a giant's 5 million, happens in consumers, not in a call with a 1-second p95 budget.

Critical Insight: Immutable revisions plus a pointer flip are what make aggressive caching safe. A cached page is a snapshot of one immutable revision, so it can only be outdated, never corrupt. Freshness becomes the purge's job, and purges can be retried.

Quick check before moving on: