Load Balancers in Microsoft Azure
Microsoft Azure offers a load balancing service for virtual machines (IaaS) and cloud services (PaaS) running in the Microsoft Azure cloud. Among other advantages, load balancing allows you to scale your applications and gives you the opportunity to respond more gently when an error or failure occurs.
Load balancing services can be enabled using the settings on the Microsoft Azure management portal or using the service model of your application. Once a hosted service with one or more endpoints is published to the cloud, it is automatically configured to use a load balancer provided by the Microsoft Azure platform. To get all the benefits of elasticity and scalability, you need to have at least two virtual machines configured for the same endpoint.
The diagram in the figure below shows an example of an application located in Microsoft Azure that uses a load balancer to address incoming traffic (at the address / port 1.2.3.4:80) to three virtual machines listening on port 80 (clickable).
')
The following are the main features of the load balancer in Microsoft Azure
Load balancing works with all types of services (IaaS and PaaS) and with all operating systems (Windows or any Linux distribution).
Applications in PaaS are configured using the service model . IaaS virtual machines are configured either through the management portal or using PowerShell.
The load balancer in Microsoft Azure is of type Layer-4. This means that it distributes the load among all available virtual machines by calculating the hash function of the traffic that arrives at this endpoint. This hash function is calculated in such a way that all packets arriving within a single connection (TCP or UDP) are sent to the same server. The Microsoft Azure Balancer uses a set of 5 fields (source IP address, source port, destination IP address, destination port, protocol type) to calculate the hash used in the matching traffic and the available server. In addition, the hash function was selected in such a way that the distribution of connections to the servers were quite random. However, depending on the type of traffic, it is possible that different connections will be tied to the same server. (It should also be noted that the distribution of connections to servers is NOT round-robin, and there is also NO request queue, as it was sometimes written earlier in some articles and blogs). The basic hash function allows you to get a good distribution of requests for a sufficiently large number of them from different sources.
The Microsoft Azure Load Balancing service supports TCP and UDP protocols. Clients can specify the type of protocol when setting up incoming endpoints in the service model, using PowerShell or in the management portal.
Hosted in the cloud service can specify multiple incoming endpoints and they will automatically be configured in the load balancer service.
Currently, several endpoints that have the same port and protocol are not supported. There is also a limit on the maximum available number of endpoints, which now can be up to 150 pieces.
Each service can specify up to 25 endpoints that will not participate in balancing. Such points can be used for internal communication between services.
The hosted service can indicate that this endpoint should have direct access to the virtual machine from outside bypassing the balancer. This will allow the application to manage the possible redirection of the user directly to the virtual machine, without using the services of balancing (and, as a consequence, the probability of requests getting to different nodes).
The load balancer works in conjunction with the Microsoft Azure Compute Service to ensure that the number of servers for endpoints scales up / down (either as the number of instances increases for the web or worker role or when new virtual machines are added to the same balancing group) The balancer automatically reconfigures itself for these changes.
The load balancer also silently changes its settings in response to preventive actions of the fabric controller or service updates of the client.
Microsoft Azure Load Balancer provides the ability to monitor the health of various service instances and remove unhealthy instances from the balancing rotation. There are three types of checks: Guest Agent check (for PaaS), HTTP check and TCP check. In the case of the Guest Agent , the balancer service runs a special agent on the corresponding virtual machine to obtain the status of the service. In the case of the HTTP test, the load balancer relies on polling the specified URL to determine the viability of the application. If the TCP test is selected, the balancer relies on the result of establishing a TCP connection on a specific port.
All return traffic originating from the service is Source NAT (SNAT) using the same virtual IP address as for incoming traffic. We will cover the SNAT in more detail in the following entries.
Microsoft Azure Load Balancer optimizes traffic between data centers in the same region so that placements that communicate with each other through a virtual IP address (VIP) and are in the same region, after establishing a TCP / IP connection, pass through the load balancer together.
Microsoft Azure Load Balancer allows you to change the VIP of two nodes by moving one node from the test state to the productive state and vice versa. The VIP change operation allows clients to use one VIP to communicate with the service, while the new version of the service is in the process of deployment (deploy). The new version can be placed and tested without affecting the main application, in a test environment. As soon as the new version passes all the checks, it can be put into commercial operation by “selecting” the VIP from the current virtual machine and assigning it a new one. In this case, all current connections to the old machine remain intact, and new connections are sent to the “new” server.
Next, we look at how most of the above can be used in the cloud service. The PaaS model of the environment we want to get is shown below:
This solution has two Frontend (FE) roles and one Backend (BE) role. The FE role opens up four balanced endpoints using the http, tcp, and udp protocols. One of the points is also used to monitor performance. The BE role opens up three endpoints with the http, tcp and udp protocols. Both roles (FE and BE) have one straight endpoint for the corresponding service instance.
The service described below is configured using the service model (some features of the scheme have been removed for readability)
Consider this model. We begin by defining the type of performance testing that the balancer should use to verify service health:
It says that we want to use HTTP testing using the relative path “Probe.aspx”. This test will later be connected to the endpoint.
Then we define the EF role as WorkerRole. It has several endpoints (http, tcp, udp)
Until we set up our own performance monitoring, performance testing is performed using the guest agent and can be changed by the service using the StatusCheck event.
Then we define an additional endpoint for port 80, which uses its own test (MyProbe)
The load balancer combines information about the endpoint and testing to get a URL like http: // {DIP of VM}: 80 / Probe.aspx, which will then be used to test service health. The service will understand (by logs?) That the same IP address periodically refers to it. These are the status requests from the host where the virtual machine is located.
The service must respond with HTTP code 200 for the balancer to consider it operational. Any other status code takes the virtual machine out of rotation.
Test setup also sets the frequency of polls. In our case, the balancer polls the service every 15 seconds. If no response is received within 30 seconds (two polling intervals), the test is considered failed and the virtual machine is taken out of rotation. Similarly, if the virtual machine is taken out of rotation, but a positive response starts to come from it, this service immediately returns to the rotation. If the service is long in the running / not running mode, the load balancer can decide for itself about the delay in putting into rotation until it responds positively to a sufficient number of tests.
The FE service also provides a set of direct endpoints, one for each instance, which connects to the instance directly at the specified port:
The definition above establishes a connection on ports 10110, 10111, etc. and transfers it to port 80 for all virtual machines with FE roles. This feature can be used in several scenarios:
Finally, the FE role provides an internal endpoint that can be used for interaction between FE and BE:
Each role can access its endpoints, as well as points that provide other roles using the RoleEnvironment class.
The BE role is also configured as WorkerRole. It does not provide any balanced endpoints, only internal points using http, tcp and udp:
The BE role also provides a direct endpoint, which allows you to connect directly to the BE instance:
The definition above establishes a connection on ports 10110, 10111, etc. and transfers it to port 80 for all virtual machines with BE roles.
We hope that the example above will show you how to use the various features of the load balancer together when modeling services.
In the following entries, we will demonstrate this example in action and show examples of source codes. We will also tell you more about:
Also please send your inquiries about what you would like to know in more detail.
Marios Zikos for the Microsoft Azure Networking team.
Load balancing services can be enabled using the settings on the Microsoft Azure management portal or using the service model of your application. Once a hosted service with one or more endpoints is published to the cloud, it is automatically configured to use a load balancer provided by the Microsoft Azure platform. To get all the benefits of elasticity and scalability, you need to have at least two virtual machines configured for the same endpoint.
The diagram in the figure below shows an example of an application located in Microsoft Azure that uses a load balancer to address incoming traffic (at the address / port 1.2.3.4:80) to three virtual machines listening on port 80 (clickable).
')
The following are the main features of the load balancer in Microsoft Azure
IaaS / PaaS support
Load balancing works with all types of services (IaaS and PaaS) and with all operating systems (Windows or any Linux distribution).
Applications in PaaS are configured using the service model . IaaS virtual machines are configured either through the management portal or using PowerShell.
Layer-4 balancer, hash distribution
The load balancer in Microsoft Azure is of type Layer-4. This means that it distributes the load among all available virtual machines by calculating the hash function of the traffic that arrives at this endpoint. This hash function is calculated in such a way that all packets arriving within a single connection (TCP or UDP) are sent to the same server. The Microsoft Azure Balancer uses a set of 5 fields (source IP address, source port, destination IP address, destination port, protocol type) to calculate the hash used in the matching traffic and the available server. In addition, the hash function was selected in such a way that the distribution of connections to the servers were quite random. However, depending on the type of traffic, it is possible that different connections will be tied to the same server. (It should also be noted that the distribution of connections to servers is NOT round-robin, and there is also NO request queue, as it was sometimes written earlier in some articles and blogs). The basic hash function allows you to get a good distribution of requests for a sufficiently large number of them from different sources.
Multiple protocol support
The Microsoft Azure Load Balancing service supports TCP and UDP protocols. Clients can specify the type of protocol when setting up incoming endpoints in the service model, using PowerShell or in the management portal.
Multiple endpoint support
Hosted in the cloud service can specify multiple incoming endpoints and they will automatically be configured in the load balancer service.
Currently, several endpoints that have the same port and protocol are not supported. There is also a limit on the maximum available number of endpoints, which now can be up to 150 pieces.
Internal endpoint support
Each service can specify up to 25 endpoints that will not participate in balancing. Such points can be used for internal communication between services.
Direct endpoint support
The hosted service can indicate that this endpoint should have direct access to the virtual machine from outside bypassing the balancer. This will allow the application to manage the possible redirection of the user directly to the virtual machine, without using the services of balancing (and, as a consequence, the probability of requests getting to different nodes).
Automatic configuration change when scaling up / down, updating and preventing the service
The load balancer works in conjunction with the Microsoft Azure Compute Service to ensure that the number of servers for endpoints scales up / down (either as the number of instances increases for the web or worker role or when new virtual machines are added to the same balancing group) The balancer automatically reconfigures itself for these changes.
The load balancer also silently changes its settings in response to preventive actions of the fabric controller or service updates of the client.
Monitoring
Microsoft Azure Load Balancer provides the ability to monitor the health of various service instances and remove unhealthy instances from the balancing rotation. There are three types of checks: Guest Agent check (for PaaS), HTTP check and TCP check. In the case of the Guest Agent , the balancer service runs a special agent on the corresponding virtual machine to obtain the status of the service. In the case of the HTTP test, the load balancer relies on polling the specified URL to determine the viability of the application. If the TCP test is selected, the balancer relies on the result of establishing a TCP connection on a specific port.
Source NAT
All return traffic originating from the service is Source NAT (SNAT) using the same virtual IP address as for incoming traffic. We will cover the SNAT in more detail in the following entries.
Traffic optimization within the data center
Microsoft Azure Load Balancer optimizes traffic between data centers in the same region so that placements that communicate with each other through a virtual IP address (VIP) and are in the same region, after establishing a TCP / IP connection, pass through the load balancer together.
Virtual IP exchange
Microsoft Azure Load Balancer allows you to change the VIP of two nodes by moving one node from the test state to the productive state and vice versa. The VIP change operation allows clients to use one VIP to communicate with the service, while the new version of the service is in the process of deployment (deploy). The new version can be placed and tested without affecting the main application, in a test environment. As soon as the new version passes all the checks, it can be put into commercial operation by “selecting” the VIP from the current virtual machine and assigning it a new one. In this case, all current connections to the old machine remain intact, and new connections are sent to the “new” server.
Example: Load Balancing Service
Next, we look at how most of the above can be used in the cloud service. The PaaS model of the environment we want to get is shown below:
This solution has two Frontend (FE) roles and one Backend (BE) role. The FE role opens up four balanced endpoints using the http, tcp, and udp protocols. One of the points is also used to monitor performance. The BE role opens up three endpoints with the http, tcp and udp protocols. Both roles (FE and BE) have one straight endpoint for the corresponding service instance.
The service described below is configured using the service model (some features of the scheme have been removed for readability)
<ServiceDefinition name="ProbeTenant"> <LoadBalancerProbes> <LoadBalancerProbe name="MyProbe" protocol="http" path="Probe.aspx" intervalInSeconds="5" timeoutInSeconds="100" /> </LoadBalancerProbes> <WorkerRole name="BERole" vmsize="Small"> <Endpoints> <InternalEndpoint name="BE_InternalEP_Tcp" protocol="tcp" /> <InternalEndpoint name="BE_InternalEP_Udp" protocol="udp" /> <InternalEndpoint name="BE_InternalEP_Http" protocol="http" port="80" /> <InstanceInputEndpoint name="InstanceEP_BE" protocol="tcp" localPort="80"> <AllocatePublicPortFrom> <FixedPortRange min="10210" max="10220" /> </AllocatePublicPortFrom> </InstanceInputEndpoint> </Endpoints> </WorkerRole> <WorkerRole name="FERole" vmsize="Small"> <Endpoints> <InputEndpoint name="FE_External_Http" protocol="http" port="10000" /> <InputEndpoint name="FE_External_Tcp" protocol="tcp" port="10001" /> <InputEndpoint name="FE_External_Udp" protocol="udp" port="10002" /> <InputEndpointname="HTTP_Probe" protocol="http" port="80" loadBalancerProbe="MyProbe" /> <InstanceInputEndpoint name="InstanceEP" protocol="tcp" localPort="80"> <AllocatePublicPortFrom> <FixedPortRange min="10110" max="10120" /> </AllocatePublicPortFrom> </InstanceInputEndpoint> <InternalEndpoint name="FE_InternalEP_Tcp" protocol="tcp" /> </Endpoints> </WorkerRole> </ServiceDefinition> Consider this model. We begin by defining the type of performance testing that the balancer should use to verify service health:
<LoadBalancerProbes> <LoadBalancerProbe name="MyProbe" protocol="http" path="Probe.aspx" intervalInSeconds="5" timeoutInSeconds="100" /> </LoadBalancerProbes> It says that we want to use HTTP testing using the relative path “Probe.aspx”. This test will later be connected to the endpoint.
Then we define the EF role as WorkerRole. It has several endpoints (http, tcp, udp)
<InputEndpoint name="FE_External_Http" protocol="http" port="10000" /> <InputEndpoint name="FE_External_Tcp" protocol="tcp" port="10001" /> <InputEndpoint name="FE_External_Udp" protocol="udp" port="10002" /> Until we set up our own performance monitoring, performance testing is performed using the guest agent and can be changed by the service using the StatusCheck event.
Then we define an additional endpoint for port 80, which uses its own test (MyProbe)
<InputEndpoint name="HTTP_Probe" protocol="http" port="80" loadBalancerProbe="MyProbe" /> The load balancer combines information about the endpoint and testing to get a URL like http: // {DIP of VM}: 80 / Probe.aspx, which will then be used to test service health. The service will understand (by logs?) That the same IP address periodically refers to it. These are the status requests from the host where the virtual machine is located.
The service must respond with HTTP code 200 for the balancer to consider it operational. Any other status code takes the virtual machine out of rotation.
Test setup also sets the frequency of polls. In our case, the balancer polls the service every 15 seconds. If no response is received within 30 seconds (two polling intervals), the test is considered failed and the virtual machine is taken out of rotation. Similarly, if the virtual machine is taken out of rotation, but a positive response starts to come from it, this service immediately returns to the rotation. If the service is long in the running / not running mode, the load balancer can decide for itself about the delay in putting into rotation until it responds positively to a sufficient number of tests.
The FE service also provides a set of direct endpoints, one for each instance, which connects to the instance directly at the specified port:
<InstanceInputEndpoint name="InstanceEP" protocol="tcp" localPort="80"> <AllocatePublicPortFrom> <FixedPortRange min="10110" max="10120" /> </AllocatePublicPortFrom> </InstanceInputEndpoint> The definition above establishes a connection on ports 10110, 10111, etc. and transfers it to port 80 for all virtual machines with FE roles. This feature can be used in several scenarios:
- Provide access to the specified instance and perform actions only with it
- Redirect the user application to a specific instance after it has passed through a balanced endpoint. This can be used for sticky sessions. But it should be borne in mind that this may lead to overload of this instance.
Finally, the FE role provides an internal endpoint that can be used for interaction between FE and BE:
<InternalEndpoint name="FE_InternalEP_Tcp" protocol="tcp" /> Each role can access its endpoints, as well as points that provide other roles using the RoleEnvironment class.
The BE role is also configured as WorkerRole. It does not provide any balanced endpoints, only internal points using http, tcp and udp:
<InternalEndpoint name="BE_InternalEP_Tcp" protocol="tcp" /> <InternalEndpoint name="BE_InternalEP_Udp" protocol="udp" /> <InternalEndpoint name="BE_InternalEP_Http" protocol="http" port="80" /> The BE role also provides a direct endpoint, which allows you to connect directly to the BE instance:
<InstanceInputEndpoint name="InstanceEP_BE" protocol="tcp" localPort="80"> <AllocatePublicPortFrom> <FixedPortRange min="10210" max="10220" /> </AllocatePublicPortFrom> </InstanceInputEndpoint> The definition above establishes a connection on ports 10110, 10111, etc. and transfers it to port 80 for all virtual machines with BE roles.
We hope that the example above will show you how to use the various features of the load balancer together when modeling services.
In the following entries, we will demonstrate this example in action and show examples of source codes. We will also tell you more about:
- How does SNAT work
- Custom tests
- Virtual networks
Also please send your inquiries about what you would like to know in more detail.
Marios Zikos for the Microsoft Azure Networking team.
Source: https://habr.com/ru/post/218797/
All Articles