What is the difference between a reverse proxy and a load balancer?

A load balancer's job is to spread requests across backend replicas. A reverse proxy does that and more: TLS termination, request routing by URL, caching, header manipulation, observability. Modern products blur the line: NGINX and Envoy do both. The terminology has converged.

8 min read · Guide · Networking

How it works · Networking

Reverse proxies, the layer that sits in front of your servers.

A reverse proxy handles traffic at the edge of your service. It does five main jobs, and once you can name them the configs of Nginx, Envoy, HAProxy and Caddy look much more alike.

Parts01–10 InteractiveFunction picker PrereqHTTP · TLS · L4 vs L7

What is a reverse proxy?

A reverse proxy sits in front of one or more backend servers, receiving client requests and forwarding them to a backend it picks. The big three (NGINX, HAProxy, Envoy) handle TLS termination, load balancing, request routing, caching, rate limiting, and observability. Large providers such as Cloudflare, AWS and Google run reverse proxies at very large scale.

A forward proxy sits in front of clients and reaches arbitrary servers: corporate egress filters, Squid, the proxy your university used. A reverse proxy sits in front of servers and accepts arbitrary clients. The clients have no idea which backend is serving them; the backends have no idea which client originated the call. The proxy sits between the two.

Most internet-facing application services run behind one. The proxy is where TLS terminates, where routing decisions live, where rate limits enforce, and where the bulk of "edge" features get added without touching application code.

The five jobs a reverse proxy does

A reverse proxy is rarely deployed for one job alone, but naming the five separately, including load balancing, and recognising each by its config block, makes someone else's setup easier to read.

Function 01 · TLS termination

Decrypt at the edge.

The proxy holds the certificate and decrypts inbound traffic; backends speak plaintext on a private network. Centralizes cert management, ALPN selection, OCSP stapling, and TLS protocol upgrades. Often the only function a small reverse proxy is deployed for.

ssl_certificate /etc/ssl/cert.pem;
ssl_protocols TLSv1.2 TLSv1.3;
ssl_stapling on;
ssl_session_cache shared:SSL:50m;

L4 vs L7: moving bytes or reading requests

An L4 (transport) proxy moves bytes. It sees source IP, destination IP, ports, and the TCP stream, but not the HTTP path or method. It is cheap and fast, and cannot see inside TLS. AWS NLB, HAProxy in TCP mode, Envoy in TCP listener mode all do this.

An L7 (application) proxy understands the protocol: typically HTTP. It can route by path, rewrite headers, retry on 5xx, even rewrite responses. It is also more expensive, because it must terminate TLS to look inside, buffer the request to inspect it, and re-emit it. AWS ALB, Nginx, Caddy, Envoy in HTTP listener mode.

If you need to route by path, you need L7. If raw TCP is enough and TLS pass-through matters, L4 is cheaper.

Reverse proxy vs load balancer vs API gateway

On a whiteboard, all three are the same rectangle, with traffic in at the front, decisions in the middle and traffic out at the back. The names describe jobs rather than software, and one binary often does two or three of them.

reverse proxy

The front door

Defined by position, it is whatever stands between the internet and your backends. It terminates TLS, sets the forwarding headers, compresses, caches, hides the topology. It is the most general of the three names.

load balancer

The dispatcher

Its one job is to pick which healthy instance gets each request. Round-robin, least-connections, consistent hashing, plus health checks to prune the pool. A load balancer is a reverse proxy that decides where a request goes, not what it contains.

api gateway

The API contract layer

A reverse proxy with product features: API keys, per-customer rate limits, token validation, request transforms, usage metering, versioning. These are features product teams ask for, added to the same data path. See API gateways.

One nginx is all three more often than not. The server block terminating TLS is the reverse proxy; the upstream block with least_conn is the load balancer; add auth_request, a rate-limit zone, and a path rewrite and you are running a small API gateway. For one team with a handful of services this is a reasonable amount of infrastructure: three jobs, one config file, one thing to operate.

Split them when ownership splits. The edge tier (platform team, changes monthly, terminates TLS for the whole company) and the gateway (product teams, changes daily, holds per-endpoint policy) need different release cadences and different blast radii, since a gateway config mistake should not be able to take down TLS for everything behind it. A separate L4 load balancer in front (NLB, a keepalived pair) is worth it when the proxy tier itself must be replaceable without touching DNS. Split on ownership and blast radius, not on vendor categories.

Nginx, HAProxy, and Envoy do the same job differently

Each major proxy product makes a different trade-off, so choose by the one you want.

nginx

The default choice

Directive-based config, event-driven, 20+ years old. Good docs and a large module ecosystem. A reload (SIGHUP) is graceful: old workers finish their in-flight requests.

envoy

Dynamic config

Dynamic config via xDS API. Designed for service meshes. First-class stats, retries, circuit breaking, gRPC. The data plane behind Istio, Consul and AWS App Mesh.

haproxy

Load balancing first

L4 and L7. Handles high RPS with low resource use. Known for its stats and flexible ACLs. Supports sticky sessions and a runtime API for live updates without reloads.

Caddy is newer and defaults to automatic HTTPS via Let's Encrypt, uses declarative config and is written in Go. It has the smallest config of the group and suits small sites, though it is less common at high RPS. Traefik takes a different approach, using service discovery instead of a config file, and we compare Traefik vs Nginx head to head.

How the proxy tells the backend who really called

By convention, a reverse proxy adds X-Forwarded-For (originating client IP), X-Forwarded-Proto (http vs https), X-Forwarded-Host, and increasingly the standardized Forwarded header (RFC 7239). Backends rely on these headers to learn who the real caller is, because the TCP source they see is the proxy itself.

If the backend is reachable directly, an attacker can spoof these headers. The proxy should strip incoming copies and set them itself, and the backend should only trust values from the proxy.

Two ways a proxy checks a backend is healthy

Active health checks: the proxy probes each backend on an interval, marking it unhealthy after N failures. It is cheap and explicit, but the probe traffic adds some load.

Passive health checks: the proxy notes connection failures or 5xx responses and quarantines the backend (much like a circuit breaker). It adds no extra traffic but only reacts after real requests fail.

For graceful shutdown, the backend should stop accepting new connections, finish in-flight requests, then exit. The proxy needs drain support: pause sending new requests to a backend without yanking existing ones. Without it, every deploy drops a few requests.

Where reverse proxies sit: edge, cluster, or sidecar

Edge proxy: one big tier in front of everything, terminates TLS for the public internet. AWS ALB, Cloudflare, your Nginx fleet.

Cluster proxy: per-cluster ingress (Kubernetes Ingress controller, Istio gateway). Routes external traffic into the cluster.

Sidecar proxy: one per workload, transparent to the app. Service mesh data plane (Envoy in Istio and Consul Connect, linkerd2-proxy in Linkerd). It handles mTLS, retries and telemetry without app code (see service discovery).

Large organisations often run all three. Each layer adds cost and capability, so add layers deliberately.

NGINX vs HAProxy vs Envoy: choose by use case, not benchmark

NGINX · 2004: Originally a web server with reverse-proxy features bolted on; today the most-deployed reverse proxy on the web (~33% of all sites per W3Techs 2024). Configuration is the famous nginx.conf directive language. Good at static-file serving, simple HTTP routing, basic load balancing. Reloads are graceful. The open-source edition is free, and NGINX Plus adds metrics and active health checks.
HAProxy · 2001: The oldest of the three, born for L4/L7 load balancing. Known for throughput (100k+ requests per second per core), predictable latency and a low memory footprint (~50 MB for thousands of backends). Configuration is the haproxy.cfg file. Used at scale by Stack Overflow, Reddit, Imgur, GitHub for the front edge. Less suited to service mesh patterns, since it runs as a process rather than a control plane.
Envoy · 2016: Born at Lyft and donated to the CNCF, it is the data plane behind Istio and AWS App Mesh. Configuration is API-driven (xDS protocol), designed for control planes that push updates dynamically with no reload. Its memory footprint is higher than HAProxy's (~150 MB baseline) and it uses more CPU per connection, but its dynamic configuration is much stronger. It is the default in most service-mesh deployments.

Picking by use case. Static-file serving and simple TLS termination suit NGINX. Edge L4/L7 at very high throughput with a stable backend pool suits HAProxy. Service mesh, dynamic backends, gRPC and fine-grained traffic shifting suit Envoy. Performance differences in modern benchmarks are within roughly 20%, so choose by configuration model.

Three reverse-proxy stacks running at scale

Cloudflare: NGINX-customised at the edge. Cloudflare's edge fleet ran a deeply forked NGINX (~1,000 patches) for years; in 2022 they began migrating to "Pingora", a reverse proxy written in Rust and built for their workload (~1 trillion requests/day). Cloudflare reported about 70% less CPU and 67% less memory for the same traffic. Pingora handles ~40 million requests per second across the network.

Stripe: Envoy as the API gateway. Stripe terminated all incoming API requests at an Envoy fleet starting around 2019. The motivation was dynamic configuration, which lets Stripe shift traffic between API versions, route certain customers to canary backends, and apply per-endpoint rate limits, all pushed over xDS without a reload.

GitHub: HAProxy for raw throughput. GitHub has used HAProxy for its public web traffic for over a decade. Its engineering posts emphasise operational simplicity: a single binary, a single config file and hot reloads.

A closing note

Most engineers end up working with a reverse proxy sooner or later. Read the config from the top (listener, then routes, then upstream) and it breaks down into the five jobs above, applied in order.