Load Balancer vs. API Gateway

You draw a load balancer in front of your servers. The interviewer asks whether you also need an API gateway. Both boxes sit in front of servers. Both receive a request and pass it on. Are they the same thing?

No, and the difference is one sentence. A load balancer picks a machine. An API gateway picks a service and applies policy.

Both are reverse proxies, servers that receive requests on behalf of the servers behind them. Proxy vs Reverse Proxy covers that category. This lesson separates the two jobs, because real systems use both, in a fixed order.

The Load Balancer

A load balancer spreads a stream of requests across a pool of identical servers. Identical is the important word. Every server behind it runs the same code, so any server can handle any request. The only question is which one, and an algorithm answers it: round robin, least connections, or least response time. The Load Balancing lesson covers those algorithms.

Its second job matters as much as the first. The balancer sends every server a health check, a small regular request that confirms the server still responds. When a server stops answering, the balancer takes it out of rotation. This is what lets one machine die without users noticing.

A load balancer does not examine what your request means. It does not read the path, and it does not check who you are. It moves traffic to a copy.

The API Gateway

An API gateway is the single entry point for your APIs. It sits in front of a set of different services and reads each request to decide which service should handle it. Because it already inspects every request, it is the natural place for shared work:

• Routing. Send /orders to the orders service and /users to the users service.
• Authentication. Check who the caller is once, at the edge, so the services can trust the request.
• Rate limiting. Reject a client that sends too many requests, before anything expensive runs.
• Caching. Return common responses without calling any service.
• Reshaping. Combine several service calls into one response for the client.

This list is why the two get confused. A gateway also spreads traffic, and a load balancer also sits in front of servers. The difference is what each one decides. The balancer decides which copy. The gateway decides which service, and what happens to the request on the way there.

Using Both, in Order

Real systems use both, and the order is standard. The load balancer sits in front of several gateway instances.

The reason is capacity and failure. Once the gateway handles authentication, rate limiting, and routing, every request passes through it. One gateway instance would be a single point of failure, one part whose failure stops the whole system. So you run several instances. The load balancer spreads traffic across them, and it drops any instance that fails its health check.

Behind the gateway, balancing happens again. Each service runs several copies, and requests to that service are spread across its copies. In a container platform, the platform usually provides this second layer for you.

You will sometimes see the reverse order, a gateway routing to a service that has its own dedicated balancer. That is the same idea drawn at a different depth, and it is not worth much interview time.

Choosing

• You have more than one copy of anything: add a load balancer. That is barely a decision.
• You have several services behind one public API: add a gateway, so clients do not track your services.
• Several services need the same policy, such as auth and rate limits: add a gateway and write that policy once.
• You have exactly one service: skip the gateway. It would add cost and an extra network hop, and give you nothing new.

Whether a gateway earns its place at all is its own decision. API Gateway vs Direct Service Exposure covers it.

💡 In the interview: when you draw both, describe each box in one clause. For example: "load balancer across the gateway instances, then the gateway does auth, rate limits, and routing". That one clause answers the follow-up before it is asked. Expect "what happens when a gateway instance dies?" Answer: it fails its health check, and the load balancer stops sending it traffic. Expect "why not one box?" Answer: one product can do both jobs, but the jobs stay separate: picking a copy and applying API policy. And if your design has a single service, say you are leaving the gateway out, and name what would make you add one.

Key takeaway: a load balancer spreads requests across identical copies and removes unhealthy ones, without reading the request. An API gateway is the entry point for different services: it reads each request, authenticates it, rate limits it, and routes it. Both are reverse proxies. Real systems use both in a fixed order, a load balancer in front of several gateway instances, with balancing again behind the gateway. Add a balancer whenever copies exist, and add a gateway when several services share one public API or one policy.