API Gateway — Part 1: Foundations & Fundamentals
What is this?
Imagine a large hospital. A patient walks in with a problem — maybe a broken arm, maybe a question about a bill, maybe they need a prescription refilled. They don't wander the corridors knocking on doors looking for the orthopedics ward, the billing office, and the pharmacy. They go to one reception desk. The receptionist checks who they are, figures out where they need to go, and sends them down the right hallway. Reception also quietly turns away people who shouldn't be there and keeps a log of who came through.
An API Gateway is that reception desk, for your backend. In a microservices world you might have fifty services — users, orders, payments, inventory, notifications — each living at its own address, each with its own quirks. You do not want a mobile app, a web app, and a partner integration each memorizing fifty addresses and re-implementing authentication fifty times. Instead, every client knocks on one front door. The gateway authenticates the caller, decides which service should handle the request, applies the rules (rate limits, quotas), forwards it, and shapes the response on the way back.
The one-line idea: an API Gateway is the single front door to a fleet of backend services — it hides their messy internal sprawl behind one stable, secure, well-governed entry point so clients deal with one thing instead of fifty.
If that sounds a lot like the load balancer from the earlier chapter, you're half right — and the difference between them is one of the most common interview questions in this whole topic. We'll draw that line sharply in Part 3. For now, hold this distinction: a load balancer asks "which server?"; a gateway asks "what is this request, who's asking, and what should happen to it?"
1. Why API Gateways exist
A monolith had it easy. One application, one entry point. Authentication, logging, and routing all happened in one place because there was only one place. When you broke that monolith into microservices, you got independent deployability and team autonomy — and you also scattered a pile of problems across the network.
Picture life without a gateway, where every client talks to every service directly:
Look at that mess and count what each client now has to do. It needs to discover which of the (constantly changing) service instances are alive. It has to implement authentication correctly against every service. It must handle retries and timeouts per service. And the instant you want to enforce a new rule — say, rate limiting — you have to ship that change into every client and every service at once. That's not a feature, that's a coordination nightmare.
The cross-cutting concerns that get painful the moment you go distributed:
- Service discovery — clients shouldn't need to know which instances are healthy or where they live today.
- Authentication and authorization — verifying identity in one trusted place beats reimplementing it in every service (and getting it subtly wrong in three of them).
- Cross-cutting policy — rate limiting, logging, and metrics should be enforced consistently, not copy-pasted per service.
- Protocol translation — clients speak HTTP/JSON; internal services might speak gRPC, GraphQL, or WebSockets. Something has to bridge that.
The gateway exists to pull all of that into one layer so your services can go back to doing the one thing they're good at: business logic.
2. Core problems API Gateways solve
2.1 Request routing
A gateway routes far more intelligently than "pick any healthy server." It reads the request and decides where it goes based on what it actually says:
- the URL path (
/api/v1/users/*→ User Service) - HTTP headers and method (an
X-API-Versionheader, aGETvsPOST) - client characteristics (mobile vs web, free tier vs enterprise)
- geography (route EU users to the EU region)
Here's a concrete request and the decisions a gateway pulls out of it:
POST /api/v1/users/123/profile
Host: api.example.com
X-API-Version: v1
X-Client-Type: mobile
Authorization: Bearer eyJhbGciOi...
From that one request the gateway reads the path (/api/v1/users/* → User Service), the version header (route to the v1 cluster), and the client type (maybe a slimmer mobile response). The client did nothing special — it just hit the front door, and the gateway did the thinking. This is exactly the Layer 7, content-aware routing idea from the load-balancer chapter, now made the gateway's whole personality.
2.2 Authentication and authorization
This is the gateway's highest-leverage job. Instead of every service validating tokens (and each getting it slightly wrong), the gateway does it once, at the door, and forwards a request the backends can trust.
The gateway handles token validation (JWT, OAuth 2.0, API keys), identity-based rate limiting, request-signature verification, and certificate-based auth. Crucially, by the time a request reaches the User Service, the User Service can assume the caller is who they claim to be. That trust boundary is the gateway's gift to every service behind it.
💡 A subtle but important rule: the gateway handles authentication ("who are you?") well. It can do coarse authorization ("is this caller allowed to touch the orders API at all?"). But fine-grained, data-level authorization ("can this user edit that specific order?") almost always belongs in the service that owns the data — the gateway doesn't know your business rules, and you don't want it to.
2.3 Protocol translation
Clients out on the internet overwhelmingly speak HTTP/JSON. Inside your system you may have gRPC services (fast, typed), GraphQL endpoints, or WebSocket streams. The gateway sits at the seam and translates, so a mobile app can make a plain REST call that ends up as a gRPC call internally.
This is the difference between an "outward-facing" contract (stable, simple, public) and your "inward-facing" reality (fast-moving, optimized, private). The gateway lets you evolve the inside without breaking the outside.
3. How a gateway is built: the proxy pattern
At its heart, an API Gateway is a sophisticated reverse proxy. The client thinks it's talking to the real service; it's actually talking to the gateway, which forwards the call. That indirection is the whole trick, and it buys you four things:
- Interception — sit in the middle and inspect or modify every request and response.
- Abstraction — hide service complexity, addresses, and protocols from the client.
- Decoration — add cross-cutting behavior (auth, logging, rate limiting) without touching the services.
- Remote access — make a distributed fleet feel like one local endpoint.
A request flows through a pipeline of stages, each doing one job before handing off to the next:
Think of it like an assembly line. Every request walks the same line; each station can stamp it, reject it (a rate-limited request never reaches auth), or enrich it. This pipeline shape is exactly what we'll crack open stage by stage in Part 2.
4. Flavors of gateway
Not all gateways play the same role. Three patterns show up repeatedly, and knowing which is which matters in design discussions.
| Type | Job | Hallmarks |
|---|---|---|
| Edge gateway | Face the public internet | DDoS protection, TLS termination, geographic distribution, high availability |
| Internal gateway | Service-to-service traffic | mTLS between services, circuit breaking, deep observability — often a service mesh |
| Hybrid | Both external and internal | Different security policies per side, multi-tenant, legacy/protocol bridging |
The terms you'll hear constantly: an edge gateway handles north-south traffic (clients in from the outside), while internal service-to-service calls are east-west traffic. Big systems run both — a hardened edge gateway at the perimeter and a lighter internal mesh inside. That split is a major theme of Part 3.
5. Security: defense in depth
Because the gateway is the first thing hostile traffic hits, it's the natural place to stack your defenses. The mental model is an onion: a request has to survive every layer before it reaches a service.
The ordering is deliberate and worth internalizing: cheap checks come first. You reject a DDoS flood or a rate-limited caller before spending CPU on an expensive cryptographic token validation. Pushing the costly work to the back of the line is what keeps the gateway fast under attack.
Different clients also bring different security realities the gateway must accommodate:
- Mobile apps — certificate pinning, limited secure token storage, device attestation.
- Browser apps — CORS enforcement, Content-Security-Policy headers,
SameSitecookies. - IoT devices — constrained CPU, certificate-based auth, lightweight protocols (MQTT/CoAP), offline tolerance.
⚠️ The gateway is your strongest security choke point and a juicy single point of failure. Everything funnels through it, so it must be highly available (a redundant, multi-zone deployment — never one box) and it must fail safely. A gateway that "fails open" under load and waves traffic through unauthenticated is worse than one that's simply down.
6. The ecosystem: what people actually use
You rarely build a gateway from scratch. The landscape splits three ways:
Cloud-managed — the provider runs it: AWS API Gateway, Google Cloud API Gateway / Endpoints, Azure API Management. Low operational burden, deep integration with the rest of that cloud, pay-per-use.
Open source / self-hosted — you run it: Kong (NGINX-based, rich plugin ecosystem), Envoy (the modern high-performance proxy under much of the rest), Traefik (auto-discovery, container-native), Istio Gateway (service-mesh ingress).
Custom — occasionally a team writes its own on Go, Node, or Spring Cloud Gateway for very specific needs. Reach for this last; the off-the-shelf options are battle-tested and you'll reinvent a lot of hard-won edge cases.
For interviews and most greenfield systems, the safe defaults are a managed cloud gateway or Envoy/Kong. Bring up a custom build only when you can name the specific requirement that off-the-shelf can't meet.
7. What "production-grade" demands
A gateway sits on the critical path of every single request, which sets a brutally high bar:
- Latency — gateway overhead should be tiny, on the order of single-digit milliseconds at p99. It's a tax on every call; keep it small.
- Throughput — a serious edge gateway handles 100K+ requests/second per cluster.
- Availability — 99.99%+. If the front door is closed, your whole system is "down" no matter how healthy the services behind it are.
- Observability — because every request passes through, the gateway is the best vantage point in your system for metrics, distributed tracing, and security monitoring.
That last point is a gift. The gateway sees everything, so it's where you naturally collect latency percentiles, error rates, auth failures, and rate-limit hits — and where you stamp the trace ID that lets you follow one request across every downstream hop. We'll lean hard on this in Parts 2 and 3.
💡 Notice the tension at the core of the whole topic: the gateway's greatest strength (everything goes through it) is also its greatest danger (everything goes through it). Every design decision about gateways is really a negotiation between centralized power and single point of failure. Hold that thought — it's the thread running through every chapter that follows.
8. Key takeaways
- An API Gateway is the single front door to a fleet of backend services — it hides internal sprawl behind one stable, secure entry point so clients deal with one thing instead of fifty.
- It exists to pull cross-cutting concerns — auth, routing, rate limiting, protocol translation, observability — into one consistent layer, freeing services to focus on business logic.
- Under the hood it's a smart reverse proxy running a pipeline: each request flows through stages (rate limit → auth → route → forward → respond), and each stage can inspect, modify, or reject it.
- Authenticate once at the door and forward trusted requests; keep coarse authorization at the gateway but leave fine-grained, data-level rules to the services that own the data.
- The gateway's power and its peril are the same fact — everything goes through it — so it must be highly available, fail safely, and serve as your best observability vantage point.