Load Balancing using Pramati Web Load Balancer

Satyajit Chetri, Product Engineering

Pramati Web Load Balancer is a software based web traffic management interceptor. Pramati Web Load Balancer offers much more than a traditional load balancing solution. Pramati Load Balancer's request processing intelligence makes it a powerful software Load Balancer bringing much more flexibility to the application layer

What is Load Balancing

Any method of directing traffic across more than one server to lower latency, distribute load, and reduce downtime is termed as Load Balancing.

This process enables distributing load across multiple physical servers while failover addresses where load should be directed during a server or service failure. Load balancing maximizes hardware performance by directing traffic to the most available server, which makes decisions based on current load rather than a pre-determined alternating schedule.

What are Web Load Balancers

A Web Load Balancer manages web traffic, balances incoming requests from a client over a number of web servers in the backend, and delegates the request to a backend server. Once the request is served, the Load Balancer returns the response from backend to the client.

Types of Load Balancers

Load Balancing can be done either through hardware or software solutions:

Hardware Load Balancers which can handle high volumes of traffic but lack configuration capabilities. They are best deployed at the edge of a hosting provider's network, making simple packet switching decisions at very high speed
Software Load Balancers which can be configured and made more intelligent to accommodate a wider range of load balancing options

Hardware Load Balancers are generally preferred over Software Load Balancers, because they guarantee a level of performance and scalability.

The Trouble with Hardware Load Balancers

Hardware Load Balancers are quite expensive for middle-sized businesses. These are based on DNAT or Destination NAT. Here, the request packets destination address is changed to an internal cluster address. On obtaining the response from the cluster, the destination address is changed to the client address.

The disadvantage is, it can only be used as a Round-Robin algorithm based network and can not perform request packets based extensive operations.

Pramati Web Load Balancer breaks away from the obvious task of load balancing and act upon intelligent management of cluster resources.

Load Balancing Related Terms

Session Stickiness

Whenever encryption or a transaction is involved, it is important to consistently direct the client to the server where the user's shopping cart is located or the encryption tunnel terminates. This phenomenon is the characteristic of session stickiness wherein related requests are forwarded to the same server for session integrity.

Session stickiness can be deployed transparently without modifying the existing application code.

URI based Redirector

The URI based Node Chooser lets Load Balancer to forward a request, based on URI pattern. This enables to forward requests to certain dedicated servers, which serves JSPs or HTML pages but not both.

The redirector supports regular expressions. It maps a particular URI based on a regular expression pattern that is included in the XML. This makes it possible for decisions to be made on the basis of context path, filetype, filename, directory location etc.

High Availability

High availability ensures that if one server cannot handle a request, other servers in the cluster are assigned to process the request seamlessly In a highly available system, if a single web server fails, then another server takes over, as transparently as possible, to process the request.

Failover

Failover is an imperative fault tolerance function of mission-critical systems that rely on constant accessibility. Upon failover, the Web Load Balancer automatically and transparently redirects requests from the failed server to other servers in the cluster or a different cluster, which mimics the operations of the original server. The Web Load Balancer detects the failover based on the response time of the nodes.

Pramati Web Load Balancer

Pramati Web Load Balancer offers much more than a traditional load balancing solution. Unlike other hardware load balancing solutions, Pramati Load Balancer's request processing intelligence makes it a powerful software Load Balancer bringing much more flexibility to the application layer.

Unlike most hardware Load Balancers, which are used to split out traffic based on its destination port and IP address; enabling routing packets to the correct cluster, the Web Load Balancer has an advanced traffic management unit, which inspects each request and direct it to the most appropriate server ensuring that sessions are honored and no individual server is overloaded.

The request processing intelligence makes it a powerful software Load Balancer bringing much more flexibility to the application layer. Pramati Web Load Balancer integrates with other commercial Application servers, thus making it a cost effective solution for many enterprises willing to adapt a sophisticated traffic management setup.

This Web Load Balancer provides all functionalities expected from a Software Load Balancer. It is functionally robust and highly configurable. In addition to being used with a Pramati Server Cluster, it is compatible with any HTTP server, including Apache, Pramati, Caudium, Cherokee, and IIS.

The only requirement for Pramati Load Balancer to start dispatching requests is the IP address of backend servers and ports on which they are running.

Features and Benefits

Some important features of Pramati Load Balancer are:

Dispatcher acts as a Reverse HTTP Proxy forwarding the requests to a group of internal nodes
A high-speed Node Chooser Module selects the most appropriate server with low latency
Dispatcher has an integrated polling mechanism for the servers enabling dead nodes to be removed from the list almost immediately when they crash or go offline
The nodes or servers are arranged logically into a group called Node Group, so request dispatching can be more context based than load based
Dispatcher uses efficient load scheduling algorithm and supports further plugging in of custom algorithms
Dispatcher is built on a robust Non-blocking, non-forking Java threaded model
Offers connection termination for rogue clients or zombies which try to SYN flood servers
Failover and High Availability are the prime feature of the Dispatcher
Easy configuration through a set of XML files
Cluster setup can be changed during Dispatcher runtime
Dispatcher works with any kind of backend server including IIS and Apache HTTP and HTTPS support
Provides flexibility and application layer control, cost- effectively
Supports a wide range of platforms and operating systems
Best deployed close to the servers it is managing, managing traffic and monitoring the host, closely
Provides load balancing in a cluster hosting a range of HTTP services
Ensures that each HTTP request is directed to the set of machines that provide the requested resource
Makes maximum use of the available resources, and is specifically optimized to scale well under heavy connection load
Ensures that the front-end services managed by the Load Balancers are always available, even if one of the balancers fails
Extracts web requests from a high-performance HTTP Keepalive connection and directs each individually instead of directing all requests to the same back-end machine
Ensures that all requests for the same resource are directed to the same cache server, for optimum response times and cache utilization
Probes each back-end server to verify that the required services are running

Important Aspects of Pramati Web Load Balancer

One interesting aspect of the Pramati Load Balancer is that unlike other commercial Load Balancers, it is not a separate utility. It is an integral feature of Pramati Application Server 3.5. The Server, when started in the Load Balancer mode, can forward requests to one or more "back-end" servers which can be:

Other instances of Pramati Server running at a different location, either in Standalone, or Cluster mode
Other web servers that use the HTTP protocol
In a special case, the backend server can be the same server instance as the LB. The LB itself can serve some of the requests, instead of sending them to other servers

Depending on context-based rules or simple availability, certain requests can be allocated to the Load Balancer itself. A member of a Pramati Cluster can serve as a Load Balancer, dispatching part of its requests to the other servers in the Cluster, in a RoundRobin fashion, and serving a part itself. This is possible because of the tight coupling of the Load Balancer with the web Container of the Pramati Server. Another way in which the Pramati Load Balancer can be used is an intelligent Web Load Balancer for a separate webserver.

Integration with Other Hardware and Software Solutions

Web Load Balancer can integrate with the existing clustering solution and comes along with Pramati Server. Web Load Balancer is also capable of working with other servers like Apache, IIS, Tomcat and much more.

The Pramati Services Framework enables services to be plugged into the server with minor configuration in the configuration file. Internally, Pramati web Container is also plugged in as a service, and can be unplugged from the core services anytime. The web Container comprises of the interceptor framework-a stack of interceptors that filter incoming web requests. One of the interceptors is the Web Load Balancer. Request Dispatcher is responsible for routing the requests based on a stack of `condition based redirectors' called node choosers.

Pramati Load Balancer can be deployed with servers other than Pramati Server and optimizes existing infrastructures. It can be obtained along with Pramati webGate plug-in and plugged into third party servers such as Apache, IIS etc.

This Web Load Balancer acts as an Interceptor for the Pramati Web Container. It is based on a unique node chooser algorithm that enables faster node selection.

Framework of Pramati Web Load Balancer

Pramati Load Balancer framework consists of the following components.

Request Dispatcher Interceptor

This Interceptor is part of the interceptor framework in the Pramati Web Container, and is activated only when the Server is started in Load Balancer mode. When the Request Dispatcher Interceptor is activated, it makes sure that all requests that come to the Server Instance are dispatched to the Web Load Balancer component rather than being served by the Web Container of which it is a part. In certain cases, the Load Balancer can allow requests to be processed by the web container that is, the rest of the Interceptor chain.

Node

Any server in the back-end, to which the Pramati Load Balancer dispatches requests, is referred to as a Node. A node can be another Pramati Server instance, an instance of an Apache server, or an IIS server, and suchlike. Every node managed by the Load balancer needs to have a unique name assigned by the Load Balancer administrator, mapped to a combination of IP-Address and Web-port. The Load Balancer allows configuration of its communication with the nodes, such as maximum socket pool size or socket time-out value.

Node Group

Nodes can be logically grouped into node groups. The Pramati Load Balancer treats all nodes in the same way i.e it makes no distinction between a Clustered node or a Standalone node, or even whether a node is a Pramati Server instance or some other Server. A node-group is a logical arrangement of nodes. Typically, the administrator would group all the members of a Cluster in a single node group to ensure easier administration.

Node Manager

The Node Manager is in charge of maintaining a record of which of the backend servers can deliver requests are alive at a particular point of time. When the Web Load Balancer is initialized, the Node Manager immediately checks and finds out the availability of the nodes. If a particular node is down, the Node Manager informs the Web Load Balancer and the Node Choosers, so that subsequent requests do not get routed to the failed node. It also checks all nodes at periodic intervals to find out whether the live nodes are still able to serve requests, and whether the dead nodes have come up again.

Node Chooser

Node Choosers are the decision-makers for the Web Load Balancer. The node chooser module comprises of a stack of node choosers arranged in a particular order, each assigned for a specific task. Whenever a request is passed to a node chooser, based on the algorithm and nature of the node chooser, one or more nodes are selected. If there are more than one node, which means that the current node chooser has deemed all those nodes eligible for processing the current request based on certain preconditions, then these nodes are passed on to the next node chooser.

The node choosers are characterized into two types.

Normal Node chooser

URL Mapping Node Chooser: It makes a decision based on the hostname and URL of the request, comparing them with the hurl rules laid down in the configuration file. A code snippet of the configuration for URI based redirection is given below:
<node-choser name="url-mapping-node-choser "class="com.Pramati.web.lb.nodechoser.UrlMappingNodeChoser" enable="true"><host name=www.pramati.com><uri pattern=".*jsp" node-ref="first,second" node-group-ref="Pramati_cluster"/><uri pattern=".*myApplication.*" node-ref="first"/></host></node-choser>
Session Stickiness Node Chooser: It makes a decision based on the jsession-id present in the request
Load Balancing Node Chooser: It implements a round robin algorithm

Failover Node chooser

The failover node chooser does not take part in the normal node choosing operation. Whenever there is a failover, the request is redirected to the failover node chooser automatically by the Load Balancer. The failover node chooser selects an alternate server and diverts the request. In the extreme case wherein even the other server is down or more than one node can server as failover node, the request is forwarded to the normal node choosers to select a node again.

This delegation process continues till the end of the node chooser stack until a single node is returned to which the request is forwarded. Most likely a single node will be chosen during the initial stages of the node chooser process. In a default configuration the first node chooser will be the url-mapping node chooser, followed by the session-stickiness node chooser, and then the load-balancing node chooser.

Working with Pramati Web Load Balancer

Pramati Web Load Balancer transparently sits in front of an internet server farm acting as a proxy by receiving the requests from the clients which are destined for certain internet services (current implementation supports only HTTP/HTTPS). The locations of the servers are mentioned in the Load Balancer configuration file, as nodes and node-groups.

When a request is received from the client, the Request Dispatcher Interceptor passes on the information to the Web Load Balancer receives the information about the status and availability of the nodes from the Node Manager. This information, along with the request is passed on to the node chooser module, and it decides which node the request is going to be sent. The Load Balancer writes the request to the backend and receives the response, which it writes back to the client that had made the request.

If a node goes down, and the Load Balancer was in the process of sending a request to or receiving a response from that particular node, the request has to be redirected to another node i.e failed over. The request is redirected to the failover node chooser automatically by the dispatcher. The failover node chooser selects an alternate server and diverts the request. In the extreme case wherein even the other server is down or more than one node can server as failover node, the request is forwarded to the normal node choosers to select a node again.

Note: Pramati Web Load Balancer, forwards a request to the backend server even if it is not a Pramati server for the request to forward. The dispatcher makes no assumption of the backend node. Only the IP address and the web port of the server are stored in the configuration file.

Providing High Availability for Web Load Balancer

The Dispatcher provides high availability for the servers clustered behind it, because one of its basic functions is to avoid choosing a failed server. The Dispatcher can also be configured to eliminate the Load Balancer itself as a single point of failure, as part of a comprehensive hardware and software implementation of high availability.

The current version of the Pramati Server supports starting multiple dispatchers for providing failover for dispatching. The Web Load Balancer provides failover and high availability by implementing a standby dispatcher. The high availability solution is a cluster of two Web Load Balancer, one of which is the Active node and the other a 'warm' standby node. The standby Load Balancer does not participate in any operation, except sync up, when the active RD is up. When the Active Load Balancer fails, a standby Load Balancer takes over transparently and functions as the Active RD. The first Load Balancer, when it resumes operation, subsequently functions as a standby Load Balancer. The standby Load Balancer remains aware of the status of the active Load Balancer and takes over in the event of a failure. Sync up is performed through RMI and is transparent to the user. Any connections, sessions, consumers or producers created on the main RD will be valid on the new RD also.

Usage Scenarios

Assume that there are four nodes A, B, C and D, which form part of a node group. When the request comes from the client:

The Request Dispatcher Engine forwards the request to the Normal Node Chooser Module, which has to perform the role of delegating a node for processing the request.
The request gets passed through the node chooser stack for resolving a single node to forward the request to. When the request is passed through the session stickiness node chooser, all the nodes (A, B, C and D) are resolved for a fresh request. These resolved nodes along with the request are passed on to the `URI based Redirector' which resolves A, B and D based on the URI.
These resolved nodes are passed to the Load Balancer node chooser which chooses node D based on some scheduling algorithm. The request is now forwarded to the node D.

Now, let us assume that node D has failed, or the Node Manager does not get a response from the node. An alert is sent to all the node choosers which updates their nodes list. The dead node D is permanently removed. The request along with the failed node information is passed to the failover node chooser which resolves A, B, earlier resolved by the URI based redirector. Since there is more than one resolved node, the node list is passed to the normal node chooser to narrow down the selection. A, B nodes are passed to the Session Stickiness node chooser which resolves A, B as it is a fresh request. The URI based redirector also resolves A, B as both the nodes can process the request pattern. A, B is then passed to the Load Balancer which selects A based on some scheduling algorithm. The request is passed to the node A.

The right node selected by the Load Balancer is not always the best node, since it is desirable for all eligible nodes to process their share of the load. Even the worst node needs to take up load. If traffic is only forwarded to the best node, it can be guaranteed that it will rapidly cease to be the best, and the load on the nodes will swing unevenly. Pramati Server's Web Load Balancer's scheduling algorithm for choosing the right node and its advanced smoothing techniques achieve and maintain an optimal balance of server load distribution in the shortest possible time.

The response time can be drastically reduced if the response is not routed through the actual path. i.e. the node, upon getting the request from the Load Balancer need not send the response back to the since Load Balancer can send it directly back to the client.

This cannot be achieved by Pramati Web Load Balancer as the client specific information including the IP addresses are not forwarded to the server and also this forces the sever to have a public link with the outside network which is not a healthy clustering solution.

For instance, an IIS or an Apache Server can delegate Client requests made to them to the Pramati Load Balancer which has been configured with a private IP address. Load Balancer performs the main function of

Deciding to which node among several backend the request should go, and sends the request to that backend server
It then processes the response and then sends it back to the original server from where the request came. This particular scenario does not require the Pramati Server to be started

Case 1

Web Load Balancer intercepts all requests to a set of homogeneous machines. In this case the Load Balancer would not serve any request on its own.

Typical Setup

Load Balancer for High Availability

Load Balancer for Load Balancer and High Availability

Case 2

The node, which runs the Web Load Balancer, would also be serving some request.

Typical Setup

Load Balancer as a means for allowing plug-ins to be written for Pramati Server

Load Balancer with default URI/Failover/Session stickiness plug-in, which allows forwarding requests based upon URI

Case 3

A pure Web Load Balancer for various machines based upon URI. For instance all JSP requests will be forwarded to one node and all ASP requests would be forwarded to one node, all PHP requests will be forwarded to one node and all static file requests will be forwarded to another node. RD in the Demilitarized Zone and all the nodes beyond the inner firewall.

Case 4

Web Load Balancer working along with a Pramati Plug-in for Apache/IIS to enable distribution of the load to backend Pramati Servers

Pramati Server webGate communicating with the Web Load Balancer, which distributes load and provides failover facility

Summary

Unlike other hardware based load balancing solutions, Pramati Web Load Balancer's request processing intelligence makes it a powerful software Load Balancer bringing much more flexibility to the application layer. Pramati Load Balancer integrates with other commercial Application servers, thus making it a cost effective solution for many enterprises willing to adapt a sophisticated traffic management setup.