</> MAANG.io

Expert

Design landmark systems in depth, from Dynamo, GFS, Bigtable and Kafka to payments and ads pipelines.

0 of 232 solved 0%

Section 4: One Row at a Time - API Design (with Pushback)

Executive Summary: Applications link a client library. They don't call a REST gateway and they don't send SQL. The library exposes three groups of calls. Admin calls create tables and column families and go through the master. Data calls go straight to the tablet server that owns the row: an atomic RowMutation, ReadRow and Scanner with filters, and two single-row read-modify-write primitives. Batch calls are MutateRows, which is atomic per row and not all-or-nothing, plus MapReduce adapters. Every guarantee the API makes stops at the edge of one row. This section shows the three running-example calls, defends that boundary against five pushbacks, and sets out which calls are safe to retry.

The interviewer points at Section 3's board. "Two million operations a second at peak. What do they look like when they arrive?"

You: "Every one of them names one row or one row range. That's the whole API in one sentence, and I'll defend the boundary it draws."

A Library, Not an Endpoint

The contract ships as a library linked into the crawler, the indexer and the search front-ends. The library holds the tablet-location cache (Section 7 walks the lookup). A cold client needs 3 round trips to find a tablet, a warm one needs none, and a Profiles front-end can cache all 200 K Profiles tablets in about 40 MB. After that, a call is one RPC to one tablet server, about 0.5 ms inside the datacenter.

I rejected a REST or gRPC gateway tier in front of the servers. On a 1 ms p50 read, an extra hop costs half the budget, and the gateway would have to scale to 2 M ops/s on its own. Flip trigger: callers outside the datacenter, or in languages we won't write a library for. Then a thin proxy goes in front for them, and the in-cell clients keep the direct path.

Quick check before moving on:


Sign in to continue reading

The rest of this lesson is available with a free account. Signing in with Google or Microsoft is free.

Sign in to read the full lesson

Sign in to MAANG.io

Use your Google or Microsoft account — no password to remember.

Continue with Google Continue with Microsoft

Please accept the terms above to continue.