20 Load Balancing Interview Questions For System Design Interview

Load balancing is simple technique for distributing workloads across multiple machines or clusters. The most common and simple load balancing algorithm is Round Robin. In this type of load balancing the request is divided in circular order ensuring all machines get equal number of requests and no single machine is overloaded or underloaded.

The Purpose of load balancing is to

• Optimize resource usage (avoid overload and under-load of any machines)
• Achieve Maximum Throughput
• Minimize response time

Most common load balancing techniques in web based applications are

1. Round robin
2. Session affinity or sticky session
3. IP Address affinity

The main benefit of round‑robin load balancing is that it is extremely simple to implement. However, it does not always result in the most accurate or efficient distribution of traffic, because many round‑robin load balancers assume that all servers are the same: currently up, currently handling the same load, and with the same storage and computing capacity.

Round‑robin load balancing is one of the simplest methods for distributing client requests across a group of servers. Going down the list of servers in the group, the round‑robin load balancer forwards a client request to each server in turn. When it reaches the end of the list, the load balancer loops back and goes down the list again (sends the next request to the first listed server, the one after that to the second server, and so on).

A Reverse Proxy Server, sometimes also called a reverse proxy web server, often a feature of a load balancing solution, stands between web servers and users, similar to a forward proxy. However, unlike the forward proxy which sits in front of users, guarding their privacy, the reverse proxy sits in front of web servers, and intercepts requests.

A reverse proxy server acts like a middleman, communicating with the users so the users never interact directly with the origin servers. It also balances client requests based on location and demand, and offers additional security. A reverse proxy cache server can:

• Enhance performance by caching local content
• Disguise the characteristics and existence of origin servers
• Carry TLS acceleration hardware, enabling them to perform TLS encryption in place of secure websites
• Distribute load from incoming requests to each of several servers (load balancing)
• Compress content, optimizing it and speeding loading times
• Perform multivariate testing and A/B testing without inserting JavaScript into pages
• Protection from DDoS attacks and related security issues

Fail over means switching to another machine when one of the machine fails. Fail over is a important technique in achieving high availability. Typically a load balancer is configured to fail over to another machine when the main machine fails.

To achieve least down time, most load balancer support a feature of heart beat check. This ensures that target machine is responding. As soon as a hear beat signal fails, load balancer stops sending request to that machine and redirects to other machines or cluster.

Sticky session or a session affinity technique is another popular load balancing technique that requires a user session to be always served by an allocated machine.

In a load balanced server application where user information is stored in session it will be required to keep the session data available to all machines. This can be avoided by always serving a particular user session request from one machine. The machine is associated with a session as soon as the session is created. All the requests in a particular session are always redirected to the associated machine. This ensures the user data is only at one machine and load is also shared.

This is typically done by using SessionId cookie. The cookie is sent to the client for the first request and every subsequent request by client must be containing that same cookie to identify the session.

What Are The Issues With Sticky Session?

There are few issues that you may face with this approach

• The client browser may not support cookies, and your load balancer will not be able to identify if a request belongs to a session. This may cause strange behavior for the users who use no cookie based browsers.
• In case one of the machine fails or goes down, the user information (served by that machine) will be lost and there will be no way to recover user session.

A TCP load balancer is a type of load balancer that uses transmission control protocol (TCP), which operates at layer 4 — the transport layer — in the open systems interconnection (OSI) model. TCP load balancers are for applications that do not use HTTP. When deployed in front of a database cluster, a TCP load balancer spreads requests across all available server configurations.

The following variants to the round‑robin algorithm take additional factors into account and can result in better load balancing:

• Simple Round-Robin - Simplest methods for distributing client requests. It just forwards a client request to each server in turn. When it reaches the end of the list, the load balancer loops back and goes down the list again.
• Weighted Round-Robin – A weight is assigned to each server based on criteria chosen by the site administrator; the most commonly used criterion is the server’s traffic‑handling capacity. The higher the weight, the larger the proportion of client requests the server receives. If, for example, server A is assigned a weight of 3 and server B a weight of 1, the load balancer forwards 3 requests to server A for each 1 it sends to server B.
• Dynamic Round-Robin – A weight is assigned to each server dynamically, based on real‑time data about the server’s current load and idle capacity.

The biggest drawback of using the round robin algorithm in load balancing is that the algorithm assumes that servers are similar enough to handle equivalent loads. If certain servers have more CPU, RAM, or other specifications, the algorithm has no way to distribute more requests to these servers. As a result, servers with less capacity may overload and fail more quickly while capacity on other servers lie idle.

The weighted round robin load balancing algorithm allows site administrators to assign weights to each server based on criteria like traffic-handling capacity. Servers with higher weights receive a higher proportion of client requests. For a simplified example, assume that an enterprise has a cluster of three servers:

• Server A can handle 15 requests per second, on average
• Server B can handle 10 requests per second, on average
• Server C can handle 5 requests per second, on average

Next, assume that the load balancer receives 6 requests.

• 3 requests are sent to Server A
• 2 requests are sent to Server B
• 1 request is sent to Server C.

In this manner, the weighted round robin algorithm distributes the load according to each server’s capacity.

Sticky session means that when a request comes into a site from a client all further requests go to the same server initial client request accessed. I believe that session affinity is a synonym for sticky session, but there are different ways of implementing it:

1. Send a cookie on the first response and then look for it on subsequent ones. The cookie says which real server to send to. Bad if you have to support cookie-less browsers
2. Partition based on the requester's IP address. Bad if it isn't static or if many come in through the same proxy.
3. If you authenticate users, partition based on user name (it has to be an HTTP supported authentication mode to do this).
4. Don't require state. Let clients hit any server (send state to the client and have them send it back) This is not a sticky session, it's a way to avoid having to do it.

I would suspect that sticky might refer to the cookie way, and that affinity might refer to #2 and #3 in some contexts, but that's not how I have seen it used (or use it myself).