We get the IP addresses of HTTPS clients from HAProxy (frontend) to Nginx (backend) in HTTP and TCP balancing modes

Quite often, it is required to balance the load between multiple web servers. In this case, as a rule, it is necessary that web applications obtain real IP addresses of clients, and not balancer IP.

In the case of balancing and terminating HTTP (S) traffic on HAProxy (Layer 7 [ 1 ]), this task is easily solved by adding the “X-Real-IP” header and processing it on Nginx using the ngx_http_realip_module [ 2 ] module. When balancing TCP traffic from HTTPS clients and transferring it to the web server directly without modification or termination (Layer 4 [ 3 ]), this header cannot be added, so you need to take advantage of the opportunities provided by the Proxy Protocol [ 4 , 5 , 6 ].

Consider both options (balancing L7 and L4) on the example of extracts from the haproxy 1.5.9 and nginx 1.6.2 configuration files.
')

Application level balancing (Layer 7): termination of HTTPS traffic to HAProxy and transfer via HTTPS to Nginx

In this example, HTTPS traffic from the client is terminated on HAProxy, modified and transmitted to Nginx also via HTTPS.

haproxy.cfg

global maxconn 4096 chroot /usr/share/haproxy uid 99 gid 99 daemon tune.ssl.default-dh-param 2048 defaults log global option redispatch option tcp-smart-accept option tcp-smart-connect retries 3 maxconn 2000 timeout connect 5000 timeout check 3000 timeout client 50000 timeout server 50000 frontend http_frontend *:80 mode http redirect scheme https code 301 if !{ ssl_fc } frontend https_frontend_ssl_terminate mode http bind *:443 ssl crt /etc/haproxy/ssl/public.example.com.pem option forwardfor header X-Real-IP default_backend web_server_http backend web_server_http mode http balance roundrobin #    backend  HTTPS server s1_https 192.168.1.10:443 ssl verify none server s2_https 192.168.1.20:443 ssl verify none 

nginx.conf

 server { server_name localhost; listen 443 ssl default_server; ssl_certificate /etc/nginx/ssl/internal.example.com.pem; ssl_certificate_key /etc/nginx/ssl/internal.example.com.key; #  HAProxy set_real_ip_from 192.168.1.254; real_ip_header X-Real-IP; root /usr/share/nginx/html; index index.html index.htm; error_page 500 502 503 504 /50x.html; location = /50x.html { root /usr/share/nginx/html; } location ~ /\.ht { deny all; } } 

Transport Level Balancing (Layer 4): Transmit TCP Traffic from HAProxy to Nginx

In this example, HTTPS client traffic is not modified (HAProxy interferes with the transport layer) and its termination occurs directly on Nginx.

haproxy.cfg

 global maxconn 4096 chroot /usr/share/haproxy uid 99 gid 99 daemon defaults log global option redispatch option tcp-smart-accept option tcp-smart-connect retries 3 maxconn 2000 timeout connect 5000 timeout check 3000 timeout client 50000 timeout server 50000 frontend http_frontend *:80 mode http redirect scheme https code 301 if !{ ssl_fc } frontend https_frontend_ssl_pass mode tcp bind *:443 default_backend web_server_tcp backend web_server_tcp mode tcp balance roundrobin # !   send-proxy  , #    ,   . #  Nginx     listen #  proxy_protocol. server s1_tcp 192.168.1.10:443 send-proxy server s2_tcp 192.168.1.20:443 send-proxy 

nginx.conf

 server { server_name localhost; # !    proxy_protocol      haproxy. #       . listen 443 ssl default_server proxy_protocol; ssl_certificate /etc/nginx/ssl/public.example.com.pem; ssl_certificate_key /etc/nginx/ssl/public.example.com.key; #  HAProxy set_real_ip_from 192.168.1.254; real_ip_header proxy_protocol; root /usr/share/nginx/html; index index.html index.htm; error_page 500 502 503 504 /50x.html; location = /50x.html { root /usr/share/nginx/html; } location ~ /\.ht { deny all; } } 

Conclusion

Using the settings described above, we were able to transmit the real IP addresses of clients when working on HTTPS to the Nginx web server located behind HAProxy. A similar approach can also be used when working with third-party load balancers, such as CloudFlare [ 7 , 8 ] and AWS ELB [ 9 , 10 ].

Literature

1. Application protocols of the OSI network model - en.wikipedia.org
2. Module ngx_http_realip_module - nginx.org
3. The transport layer of the OSI network model - ru.wikipedia.org
4. The PROXY protocol - haproxy.org
5. HAProxy Configuration Manual: send-proxy - cbonte.imtqy.com
6. Ngx_http_core_module module: listen directive - nginx.org
7. Getting Real IP Addresses Using CloudFlare, Nginx, and Varnish - danielmiessler.com
8. Getting Real IP Addresses Using Nginx and CloudFlare - babaei.net
9. Using Proxy Protocol With Nginx - chrislea.com
10. AWS Elastic Load Balancing - aws.amazon.com