Round-robin default gateway breaks source routing for OpenVPN server

On Linux (Debian 8) router, source routing is configured using iproute2. Both FORWARD and INPUT \ OUTPUT packets are correctly routed. After enabling the round-robin default gateway, problems began to arise when connecting OpenVPN clients. Case is quite rare, so I decided to share this solution with the community in the form of this note.

Round-robin default gateway is enabled with the following command:

ip route add default scope global nexthop via $GW1 dev $IF1 weight 1 nexthop via $GW2 dev $IF2 weight 1 nexthop via $GW3 dev $IF3 weight 1 

In reality, with routing, everything is a bit more complicated, but these particulars do not greatly influence the final output, so they can be omitted. As you can see, three gateways with the same weight are used alternately. OpenVPN server listens to all interfaces and works on the UDP protocol. In theory, it should work with these settings, but the clients stopped connecting to this server. Although the same SSH continued to work when connected via any of the interfaces. It can be assumed that the reason is in the UDP protocol, but checking on the DNS server showed that the responses to the UDP requests are returned from the necessary interfaces. Thus, when you enable the Round-robin gateway, source routing broke only for OpenVPN and the reason should be looked for in it.

When connecting, before starting a handshake, a login with a password is requested, and the authorization is successful, no further connection is made, the client's failed connection log does not say anything useful:

 TLS Error: TLS key negotiation failed to occur within 60 seconds (check your network connectivity) TLS Error: TLS handshake failed SIGUSR1[soft,tls-error] received, process restarting MANAGEMENT: >STATE:1451454269,RECONNECTING,tls-error Restart pause, 2 second(s) 

Interestingly, some clients (in fact, met only one) something interesting flew in the logs. When connected to the first interface $ IP1, the log produced the following:

 Incoming packet rejected from [AF_INET]$IP1:1194[2], expected peer address: [AF_INET]$IP2:1194 (allow this incoming source address/port by removing --remote or adding --float) Incoming packet rejected from [AF_INET]$IP1:1194[2], expected peer address: [AF_INET]$IP3:1194 (allow this incoming source address/port by removing --remote or adding --float) 

And so in a circle. Moreover, if the client replaces the IP address of the server with $ IP2, the response packets will come from $ IP1 and $ IP3, but not from $ IP2. Those. no matter what IP client connects, the answer never comes from the same address . Quite a strange behavior. I repeat, at the same time, routing from the source through other connections is working properly.
')
In the process of troubleshooting, it turned out that when the protocol is changed to TCP, the connection is successful and the reason for this must be found in the implementation of the work on UDP. After that, there was also a solution - the Multihome parameter on the server side of OpenVPN. Man openvpn of the multihome parameter:

--multihome
Configure a multi-homed UDP server. This is where you can use the IP address (eg, multiple interfaces, or secondary IP addresses)
specific address only. This is the case for the UDP reply packets. This is not supported on all
it is not enabled by default.
Note: this option is only relevant for UDP servers.
Note 2: If you have an IPv4 + IPv4 address, i.e.
IPv4-mapped case (some distributions have ported this to earlier kernel versions, though).

Now it is clear, the reason is in the principle of the OpenVPN server operation using the UDP protocol. The multihome option adds additional processing to udp packets that guarantee the sending of response packets from the address to which the request came. Specifically, what happens can be said after studying the sources of OpenVPN. Thus, when configuring the round-robin default gateway, for OpenVPN to work correctly, you must either use the TCP protocol or add the multihome parameter when using UDP.