System Design Series — Load Balancers

All about building robust, functional, and scalable systems.

Hola !!!

In the previous writeup we have learned about proxies, if you've not gone through it, please go through my profile and read that before starting the load balancers.

Load Balancers are one of the most important topics for system design interviews.

As the servers have limited resources, the more requests from other clients or the more requests from a single client, the more likely are for the servers to get overloaded. It might cause the failure of our system or might slow down our system. To tackle more number of clients we scale up our system. We can either scale vertically or horizontally. Vertical scaling means increasing the computational power of the system (single machine) and horizontally scaling means increasing the number of server instances (add more machines).In horizontal scaling, we increase the number of servers and if we assume that all added servers have the same power then our system can handle no of the server times requests that it used to server earlier. But how will the client know on which server it has to send the request such that the load can be balanced equally on all the servers?

This is where a load balancer comes into play. A load balancer is a server that sits in between a set of clients and a set of servers like in the image shown above and has the job to balance resources. Our clients will be sending requests to the load balancer and the load balancer's job would be to redirect these requests in a balanced or pre-configured way.

Load balancers make our overall system have better throughput. They decrease latencies as well. Load Balancers are kind of reverse proxies because they sit between clients and servers and they typically act on behalf of these servers.

If we randomly distribute a load of requests randomly over different servers, there might be a chance for any particular server to get overloaded. Another way a load balancer often selects a server is by following the Round Robin approach. Round Robin is basically a method in which the request may go from top to bottom to different servers and again from top to bottom, something like a circular way, that’s why it is called Round Robin. A slightly more complex strategy of Round Robin is called Rated Round Robin Server Selection Strategy where you place rates on servers and let’s say if a server has more ratings then more requests will be sent to that server. This is a used case where one particular server is more powerful than the rest. The order follows from top to bottom but when it is at that particular server (highly rated), it will send more number of request to this compared to other lower rated servers.

Another way for load balancers to select servers is based on performance or load. basically, your load balancer performs health checks of servers and based on that it selects a particular server.

You can find me on LinkedIn — https://www.linkedin.com/in/connectayush

Thank you :)