Load balancing
Sharing requests fairly across servers, and how the choice of algorithm changes the result.
One server can only handle so many requests. When traffic grows, you run several copies of the server and put a load balancer in front. It receives every request and hands each to one of the servers. To the client it looks like one address; behind it the work is shared.
Common ways to pick a server
Step 1 of 3
Round robin
Take turns. Simple and fair in request count when requests cost about the same. It is blind to how busy each server is.
Fair in count is not the same as fair in work. In a delivery app, loading a restaurant list is quick, but generating an order-history export may take ten times longer. If requests alternate heavy and light and you have two servers, round robin sends every heavy request to server 1 and every light one to server 2: equal counts, very unequal load. Try it below.
Round robin versus least connections
Every other request is heavy (cost 10) and the rest are light (cost 1), across 2 servers. Run it with round robin, then switch to least connections. Compare the peak number of requests in flight.
- 10 cost, client u0
- 1 cost, client u1
- 10 cost, client u2
- 1 cost, client u3
- 10 cost, client u4
- 1 cost, client u5
- 10 cost, client u6
- 1 cost, client u7
- 10 cost, client u8
- 1 cost, client u9
- 10 cost, client u10
- 1 cost, client u11
With round robin one server piles up five long requests at once (peak 5), while with least connections the peak is 3 because the balancer notices the busy server and sends work elsewhere. Request counts are 6 and 6 either way, which is why counting requests alone can mislead you.
What to remember
A load balancer presents one address and shares requests over identical servers.
Round robin balances counts; least connections balances current work; hashing keeps a client on one server.
Equal request counts can still mean unequal load when requests differ in cost.
Make the balancer itself redundant.