API Gateway — Part 2: Architecture & Internals

What is this?

Part 1 gave you the gateway from the outside: the calm reception desk that hides fifty services behind one door. This part takes you behind the desk to watch a single request get processed — from the moment a TCP connection lands to the moment a response goes back out.

Think of it like following one letter through a mail-sorting facility. The envelope arrives, gets opened and checked, sorted to the right bin, sent down the right chute, and a reply comes back the same way. Nothing magic happens — but a lot of small, fast steps happen in a strict order, and the quality of a gateway lives entirely in how well it runs that assembly line under load.

The one-line idea: internally, an API Gateway is a high-throughput request pipeline — accept the connection, validate it, route it, enforce policy (rate limits, auth, retries, circuit breakers), forward it, and shape the response — all while staying fast, observable, and reconfigurable without dropping a single request.

We'll walk that pipeline stage by stage, then zoom out to how it survives across regions.