I welcome categorically.
I hasten to inform the good news that after one and a half years (and not four), a stable version of HAProxy 1.6 with interesting functionality appeared.

Let me remind you that this is an ultra-fast solution that guarantees fault tolerance and ensures balancing and proxying of TCP and HTTP requests.


I kindly ask you to write about all found inaccuracies and errors in the LAN — I will promptly correct it.
')
In this article I will share what is remarkable about the release of version numbered 1.6 , which innovations you should pay attention to and briefly describe how to try these innovations. Examples in the article are present for familiarization, and their use does not relieve from the need to refer to the page of constantly updated documentation .

Finally, you can use quotes in arguments.

This is really good news. Now it is not necessary, when inserting into the configuration file, for example, headers, to escape spaces with a backslash.

reqirep "^Host: www.(.*)" "Host: foobar\1" 

 option httpchk GET / "HTTP/1.1\r\nHost: www.domain.com\r\nConnection: close" 

Lua

Apparently, April Fools' joke of the developers that they decided to rewrite the entire HAProxy to LUA had a positive effect on the functionality. And this may have become a major change in 1.6, like SSL was once in 1.5.
For example, take a look at the implementation of a “mirror” web server. He will return our headlines in the response body without any changes.

 global lua-load ./webmirror.lua frontend fe_habrahabr bind :81 name frontend_name http-request lua mirror default_backend be_habrahabr backend be_habrahabr server main_nginx 127.0.0.1:82 

- webmirror.lua

 function mirror(txn) local buffer = "" local response = "" local mydate = txn.sc:http_date(txn.f:date()) buffer = buffer .. "You sent the following headers/r/n" buffer = buffer .. "===============================================/r/n" buffer = buffer .. txn.req:dup() buffer = buffer .. "===============================================/r/n" response = response .. "HTTP/1.0 200 OK/r/n" response = response .. "Server: haproxy-lua/mirror/r/n" response = response .. "Content-Type: text/html/r/n" response = response .. "Date: " .. mydate .. "/r/n" response = response .. "Content-Length: " .. buffer:len() .. "/r/n" response = response .. "Connection: close/r/n" response = response .. "/r/n" response = response .. buffer txn.res:send(response) txn:close() end 

 $ curl -v 127.0.0.1:82 HTTP/1.0 200 OK Server: haproxy-lua/mirror Content-Type: text/html Date: Fri, 12 Mar 2015 13:06:44 GMT Content-Length: 208 Connection: keep-alive You sent the following headers =============================================== GET / HTTP/1.1 User-Agent: curl/7.41.0 Host: 127.0.0.1:82 Accept: */* =============================================== 

Or, for example, a tcp server:

 global lua-load hello_world.lua listen proxy bind 127.0.0.1:10001 tcp-request content use-service lua.hello_world 

- hello_world.lua

 core.register_service("hello_world", "tcp", function(applet) applet:send("hello world\n") end) 

Transfer Headers Between Sections (Contexts)

Previously, each context was isolated. In other words, it was not possible to use request headers for a response. But now you can.

 defaults mode http frontend fe_habr bind :9001 declare capture request len 32 # id=0 to store Host header declare capture request len 64 # id=1 to store User-Agent header http-request capture req.hdr(Host) id 0 http-request capture req.hdr(User-Agent) id 1 default_backend be_habr backend be_habr http-response set-header Your-Host %[capture.req.hdr(0)] http-response set-header Your-User-Agent %[capture.req.hdr(1)] server nginx1 10.0.0.3:4444 check 

Multiprocessing, peers and stick-tables

peer is another haproxy instance. For example, on another VM, in another DC.
stick-table is a flat database for storing information, for example, about the number of requests per second from one IP address, number of simultaneous sessions, error rate, session identifier for a cookie, etc.

In 1.5 there existed (in 1.6 remained) such a parameter as peers . Designed to synchronize stick-tables between balancers. And, unfortunately, when multiprocessing was enabled in haproxy (the nbproc parameter), this functionality started to work incorrectly due to its own table for each process in memory.
The solution came in the form of the bind-process parameter, an example will vividly show its use:

 peers article peer itchy 127.0.0.1:1023 global pidfile /tmp/haproxy.pid nbproc 3 defaults mode http frontend f_scalessl bind-process 1,2 bind :9001 ssl crt /home/bassmann/haproxy/ssl/server.pem default_backend bk_lo backend bk_lo bind-process 1,2 server f_myapp unix@/tmp/f_myapp send-proxy-v2 frontend f_myapp bind-process 3 bind unix@/tmp/f_myapp accept-proxy default_backend b_myapp backend b_myapp bind-process 3 stick-table type ip size 10k peers article stick on src server s1 10.0.0.3:4444 check 

Logs: syslog tags and new variables

From now on, for the convenience of filtering logs, you can apply different syslog tags for each frontend, backend, and process. If the parameter is not specified, the word haproxy will be used.

 frontend fe_habr_ssl log-tag SSL [...] frontend fe_habr log-tag CLEAR [...] 

New variables that can be used in the log-format parameter:

 %HM: HTTP method (ex: POST) %HP: HTTP request URI without query string (path) %HQ: HTTP request URI query string (ex: ?bar=baz) %HU: HTTP request URI (ex: /foo?bar=baz) %HV: HTTP version (ex: HTTP/1.0) 

DNS server names

In version 1.5 and earlier, if the DNS name was specified as a backend, HAProxy received the IP address at the start and used glibc (/etc/resolv.conf)

In 1.6, HAProxy asynchronously checks that the name matches the IP address on the fly and uses the explicitly specified DNS server. This eliminates the need to restart the balancer if the IP address of the server in the backend has changed (which often happens in Docker or Amazon Web Service environments).

Configuration example for Docker:

 resolvers docker nameserver dnsmasq 127.0.0.1:53 defaults mode http log global option httplog frontend fe_habr bind :80 default_backend be_habr backend be_habr server s1 nginx1:80 check resolvers docker resolve-prefer ipv4 

Now, if we restart the container with nginx with the “docker restart nginx1” command, we will see evidence of the functionality of this functionality in the logs:

(...) haproxy[15]: b_myapp/nginx1 changed its IP from 172.16.0.4 to 172.16.0.6 by docker/dnsmasq.


HTTP request processing rules

There are new rules for processing HTTP requests.

http-request: capture, set-method, set-uri, set-map, set-var, track-scX, sc-in-gpc0, sc-inc-gpt0, silent-drop
http-response: capture, set-map, set-var, sc-inc-gpc0, sc-set-gpt0, silent-drop, redirect

Wrestlers with DDoS should pay attention to an interesting parameter silent-drop . It can replace the reqtarpit/reqitarpit .
The effect is that the connection established by the client (ESTABLISHED) after applying silent-drop on HAProxy disappears from the netlist on the balancer, freeing resources. Thus, it is possible to repel attacks of much greater power without spending the precious balancer resources on it. But it is worth remembering that all firewalls, proxies, balancers through which this connection has passed will continue to hold this connection and may become a bottleneck (“bottleneck”) in protection.

Variables

Previously, HTTP headers were used to store temporary data in HAProxy. A striking example is the limitation of the number of requests per second in 1.5 .
Now there are variables.

Write the User-agent in lower case:

 http-request set-var(req.my_var) req.fhdr(user-agent),lower 

An example with contexts rewritten using variables

 global # variables memory consumption, in bytes tune.vars.global-max-size 1048576 tune.vars.reqres-max-size 512 tune.vars.sess-max-size 2048 tune.vars.txn-max-size 256 defaults mode http frontend f_myapp bind :9001 http-request set-var(txn.host) req.hdr(Host) http-request set-var(txn.ua) req.hdr(User-Agent) default_backend b_myapp backend b_myapp http-response set-header Your-Host %[var(txn.host)] http-response set-header Your-User-Agent %[var(txn.ua)] server s1 10.0.0.3:4444 check 

post office

HAProxy learned to send letters. For example, that the backend has stopped responding.
The example below probably covers all the features of this innovation. No authorization support.

 mailers mymailers mailer smtp1 192.168.0.1:587 mailer smtp2 192.168.0.2:587 backend be_habr mode tcp balance roundrobin email-alert mailers mymailers email-alert from haproxy@habrahabr.ru email-alert to admin@habrahabr.ru server srv1 192.168.0.30:80 server srv2 192.168.0.31:80 

HTTP request body processing

Now, in addition to processing HTTP headers, it is possible to process the request body.
Enabled in the frontend or backend section with the option http-buffer-request parameter

In version 1.5, it was possible to fight an attack like slowloris, in which the request headers from the attacker are transmitted as slowly as possible, on the verge of a connection timeout,
But no one interfered with the slowest transfer of the POST request body. Version 1.6 allows you to deprive the attacker and this opportunity.

By the way, using the option http-buffer-request it becomes possible to use methods such as req.body, req.body_param, req.body_len, req.body_size, etc.

Here is an example of how to block any mention of the “SELECT *” line in the body of POST requests:

 defaults mode http frontend f_mywaf bind :9001 option http-buffer-request http-request deny if { req.body -m reg "SELECT \*" } default_backend b_myapp backend b_myapp server s1 10.0.0.3:4444 check 

Converters

Used in ACLs and in every possible way simplified the configuration. For example, request routing without them:

 frontend ft_allapps [...] use_backend bk_app1 if { hdr(Host) -i app1.domain1.com app1.domain2.com } use_backend bk_app2 if { hdr(Host) -i app2.domain1.com app2.domain2.com } default_backend bk_default 

With converters:

 frontend ft_allapps [...] use_backend %[req.hdr(host),lower,map(/etc/haproxy/domain2backend.map,bk_default)] 

- domain2backend.map

 #domainname backendname app1.domain1.com bk_app1 app1.domain2.com bk_app1 app2.domain1.com bk_app2 app2.domain2.com bk_app2 

Convenient, isn't it?
So, in 1.6 there are even more of them and I will be grateful for someone's example in the comments.

Determining the client device

Quite unexpectedly for me, HAProxy was able to work with DeviceAtlas and 51Degrees to determine the type of device and transfer the backend result.

Configuration example for DeviceAtlas:

 global deviceatlas-json-file <path to json file> frontend www-only-ua bind *:8881 default_backend servers #   User-agent http-request set-header X-DeviceAtlas-Data %[req.fhdr(User-Agent),da-csv-conv(primaryHardwareType,osName,osVersion,browserName,browserVersion)] deviceatlas-json-file <path> frontend www-all-headers bind *:8882 default_backend servers #     http-request set-header X-DeviceAtlas-Data %[da-csv-fetch(primaryHardwareType,osName,osVersion,browserName,browserVersion)] 

For 51Degrees:

 global 51degrees-data-file '51D_REPO_PATH'/data/51Degrees-LiteV3.2.dat 51degrees-property-name-list IsTablet DeviceType IsMobile 51degrees-property-separator , 51degrees-cache-size 10000 frontend www-only-ua bind *:8082 default_backend servers #   User-agent http-request set-header X-51D-DeviceTypeMobileTablet %[req.fhdr(User-Agent),51d.single(DeviceType,IsMobile,IsTablet)] frontend www-all-headers bind *:8081 default_backend servers #      http-request set-header X-51D-DeviceTypeMobileTablet %[51d.all(DeviceType,IsMobile,IsTablet)] http-request set-header X-51D-Tablet %[51d.all(IsTablet)] # ,   51Degrees   http-request set-header X-51D-Stats %[51d.all(Method,Difference,Rank)] 

Attention! Support is not enabled by default. To work with it you need:

For DeviceAtlas:

Download API source code from DeviceAtlas
Compile HAProxy with the following parameters:

 $ make TARGET=<target> USE_PCRE=1 USE_DEVICEATLAS=1 DEVICEATLAS_SRC=<path to the API root folder> 

For 51Degrees:

 $ git clone https://github.com/51Degrees/Device-Detection 

Choose a method of work:
* Pattern - uses memory and processor evenly for work.

  $ make TARGET=linux26 USE_51DEGREES=1 51DEGREES_SRC='51D_REPO_PATH'/src/pattern 

* Trie is a high-performance algorithm that uses significantly more memory than Pattern

  $ make TARGET=linux26 USE_51DEGREES=1 51DEGREES_SRC='51D_REPO_PATH'/src/trie 

Saving the state of the backend servers

In 1.5, when, after receiving the reload or restart command, HAProxy assigned the UP status to all servers before performing the first check. What is unacceptable if the road is every second uptime service. In 1.6 it is possible to specify the path to the file where the backend information will be stored during the reboot.

 global stats socket /tmp/socket server-state-file /tmp/server_state backend bk load-server-state-from-file global server s1 10.0.0.3:4444 check weight 11 server s2 10.0.0.4:4444 check weight 12 

Before restarting, we save the state of the backends:

 socat /tmp/socket - <<< "show servers state" > /tmp/server_state 

The task is completed, at the start haproxy will read the file and instantly take note of it.

External checks

In 1.5, you can check the status of the backend servers by periodically connecting to the specified port.
In 1.6, for this purpose, you can additionally use third-party scripts:

 global external-check backend b_myapp external-check path "/usr/bin:/bin" external-check command /bin/true server s1 10.0.0.3:4444 check 

TLS / SSL

ECC and RSA support on the same IP address

It is believed that ECC protects content as well as RSA, but with a smaller key size, which means less time to process a request on the server. Unfortunately, not all clients support ECC, but I want to have compatibility with everyone.
For implementation, you need: ECC and RSA certificates for the domain, HAProxy version 1.6, and the following configuration:

 frontend ssl-relay mode tcp bind 0.0.0.0:443 use_backend ssl-ecc if { req.ssl_ec_ext 1 } default_backend ssl-rsa backend ssl-ecc mode tcp server ecc unix@/var/run/haproxy_ssl_ecc.sock send-proxy-v2 backend ssl-rsa mode tcp server rsa unix@/var/run/haproxy_ssl_rsa.sock send-proxy-v2 listen all-ssl bind unix@/var/run/haproxy_ssl_ecc.sock accept-proxy ssl crt /usr/local/haproxy/ecc.www.foo.com.pem user nobody bind unix@/var/run/haproxy_ssl_rsa.sock accept-proxy ssl crt /usr/local/haproxy/www.foo.com.pem user nobody mode http server backend_1 192.168.1.1:8000 check 

There is a benchmark result on the E5-2680v3 CPU and OpenSSL 1.0.2:

 256bit ECDSA: sign verify sign/s verify/s 0.0000s 0.0001s 24453.3 9866.9 2048bit RSA: sign verify sign/s verify/s 0.000682s 0.000028s 1466.4 35225.1 

Almost 15-fold increase when signing a response.

Forgery of SSL certificates on the fly

That allows you to use HAProxy in enterprises for analyzing the contents of requests.

Certificate Transparency Support (RFC6962)

When uploading .pem files (certificate chains with a key), HAProxy will try to find a file with the same name and the .sctl suffix in the same way. When it is detected, support for TLS Certificate Transparency is enabled. Requires OpenSSL version 1.0.2 and higher. Currently, the Certificate Transparency extension requires Chrome for EV certificates issued in 2015.

SNI support when connecting to backend with SSL

 backend b_myapp_ssl mode http server s1 10.0.0.3:4444 check ssl sni req.hdr(Host) 

HTTP reuse

By default, the connection established between HAProxy and the backend server belongs to the session that initiated it. The disadvantage of this approach is that this connection is idle between requests. In most cases, the reuse of these connections by other sessions will increase the performance of the backend.
Option

 http-reuse 

in 4 different modes provides the ability to use these idle connections.

Error 408

This error occurred in browsers due to a pre-connect connection timeout, designed to speed up surfing on the Internet.
In 1.5, it was treated with the line errorfile 408 /dev/null in the defaults section.
1.6 should use the option http-ignore-probes

---

In conclusion, I want to remind you that all new versions have full backward compatibility with old configuration files, and updating to the new version will not cause any headaches. And the possibilities presented above are only a small part of the work that has been done by the developers over the past year and a half.

Thank you for your attention to this review. I will be glad to answer questions in the comments and drugs.