Load Balancer

With only one server instance, both performance bottlenecks and outages concentrate in one place. Even if you add more instances, clients should not need to track and choose among all of them directly. Once you operate many instances, you need a shared layer that can select healthy targets and absorb change.

As horizontal scaling, autoscaling, and multi-zone deployments became normal, direct client-to-instance coupling stopped being workable. Instances now appear, disappear, and get replaced regularly. Load balancers became foundational because systems needed a stable front while compute topology kept changing underneath.

Requests arrive at the load balancer, which checks the target pool and forwards traffic to healthy instances according to its policy. Health probes remove failing targets from consideration, and the target set can change without affecting the public entry point. Some load balancers stay close to pure transport, while others can route based on richer HTTP information.

Load Balancer answers which instance of the same role should handle this request. API Gateway answers which service and policy path the request should follow. Service Discovery helps find live service locations; Load Balancer chooses among them. They solve related but different layers of routing.

Load balancers are standard in web APIs, internal service pools, and worker fleets with many equivalent instances. They are especially important for zero-downtime rollout and high availability. But if state is trapped inside instance memory, traffic distribution alone is not enough; the state model has to be reconsidered too.