Health checks and failover

How the balancer notices a dead server and stops sending it traffic.

Servers fail: a process crashes, a disk fills, a deploy goes wrong, a whole machine disappears. A load balancer that keeps sending requests to a dead server turns a small problem into user-visible errors for roughly one in N requests. So balancers run health checks: regular probes that decide which servers are fit to receive traffic.

Active health checks send a probe on a schedule, for example GET /health every 5 seconds with a 2-second timeout. After a number of consecutive failures (say 3) the server is marked unhealthy and removed from rotation; after a number of consecutive successes it comes back. Passive checks watch real traffic instead: if a server returns errors or times out repeatedly, it is ejected. Using both is common: active probes find dead servers quickly when traffic is low, passive ones react to real failures.

Failover is the wider idea: when something fails, traffic moves to a standby or to the survivors. For stateless servers it is automatic. For the balancer itself you run a pair or a managed multi-node service. For retries to be safe, requests that may be repeated (like placing an order) should be idempotent, which means a duplicate has the same effect as the original, usually enforced with an idempotency key. Also use connection draining on deploys: stop sending new requests to a server and let in-flight ones finish before stopping it.