IP Multipathing on Solaris 10

Hello dear chabroskoobschestvo.

It took one fine day to configure the network interface aggregation on the production server, but (much to the surprise), not finding a single article on Habré about the ipmp technology, I decided to write a small essay on installing and deploying IPMP (IP Multipathing) in Solaris 10. I want to immediately warn you that the article is mainly of practical workload, so I apologize in advance if somewhere it has not fully revealed the theoretical component of this topic.
And so, all the work was performed on the Oracle Sun T3-1 server, with the patched ILOM, Solaris 10 on board, and two igb0 and igb1 network cards.

root @ s1-solaris1 # cat / etc / release
Oracle Solaris 10 8/11 s10s_u10wos_17b SPARC
Copyright 1983, 2011, Oracle and / or its affiliates. All rights reserved.
Assembled 23 August 2011
')
root @ s1-solaris1 # uname -a
SunOS s1-solaris1 5.10 Generic_147440-09 sun4v sparc sun4v

But first, a little theory:

IPMP is a technology that combines physical channels (interfaces) for load balancing outgoing traffic and fault tolerance of incoming traffic. In solaris, ipmp is present since version 8 and is controlled by the in.mpathd daemon.


IPMP can work in two modes

- Probe mode
- Link-state mode

Probe mode is the mode in which the server sends icmp requests to a gateway or host on the network (in solaris, up to several such test hosts can be configured) and, based on this, it checks the link operability.

Link-state mode - checks the availability of the communication channel by the physical state of the network interface, i.e. What is the status of the interface, up / down.

Well, like everything, the theory is enough-), if that missed, please forgive.

And so, now the most interesting

In my case, the probe mode ipmp was configured, with the addresses:
igb0 - 10.0.0.1
igb1 - 10.0.0.2
igb0: 1 (ipmp virt-address) - 10.0.0.3

We give the interface configuration files to the following form:

root @ s1-solaris1 # cat /etc/hostname.igb0

s1-solaris1-igb0 netmask + broadcast + deprecated -failover group IPMP-1
addif s1-solaris1 netmask + broadcast +

root @ s1-solaris1 # cat /etc/hostname.igb1

s1-solaris1-igb1 netmask + broadcast + deprecated -failover group IPMP-1

Enter the group name convenient for you to the group IPMP-1 value. I will not paint all the values ​​(broadcast, deprecated, failover ...), because The article focuses on the practical application of ipmp, the definition of these values ​​can be seen in man ifconfig.

Also, we will need to edit the / etc / hosts file:

root @ s1-solaris1 # cat / etc / hosts
#
# Internet host table
#
:: 1 localhost
127.0.0.1 localhost
10.0.0.3 s1-solaris1 s1-solaris1.name.domain.ru loghost
10.0.0.1 s1-solaris1-igb0
10.1.0.2 s1-solaris1-igb1

And reboot the system, in order to make sure that the interfaces with us rise with the above settings

root @ s1-solaris1 # init 6

Check the output of ifconfig -a, we should have a virtual interface igb0: 1

root @ s1-solaris1 # ifconfig -a
lo0: flags = 2001000849 <UP, LOOPBACK, RUNNING, MULTICAST, IPv4, VIRTUAL> mtu 8232 index 1
inet 127.0.0.1 netmask ff000000
igb0: flags = 9040843 <UP, BROADCAST, RUNNING, MULTICAST, DEPRECATED, IPv4, NOFAILOVER> mtu 1500 index 2
inet 10.0.0.1 netmask ffffff00 broadcast 10.0.0.255
groupname IPMP-1
ether 0: 21: 28: d2: 1c: c6
igb0: 1: flags = 1000842 <BROADCAST, RUNNING, MULTICAST, IPv4> mtu 1500 index 2
inet 10.0.0.3 netmask ffffff00 broadcast 10.0.0.255
igb1: flags = 9040843 <UP, BROADCAST, RUNNING, MULTICAST, DEPRECATED, IPv4, NOFAILOVER> mtu 1500 index 3
inet 10.0.0.2 netmask ffffff00 broadcast 10.0.0.255
groupname IPMP-1
ether 0: 21: 28: d2: 1c: c7

The goal is achieved. The ibg0: 1 virtual interface works by referring to the igb0 interface. Check it with if_mpadm:

root @ s1-solaris1 # if_mpadm -d igb0

switch -d (detach) puts the interface offline.

We are looking at the output of ifconfig, igb1 will take over the function of the virtual interface by creating a sub interface igb1: 1

root @ s1-solaris1 # ifconfig -a
lo0: flags = 2001000849 <UP, LOOPBACK, RUNNING, MULTICAST, IPv4, VIRTUAL> mtu 8232 index 1
inet 127.0.0.1 netmask ff000000
igb0: flags = 9040843 <UP, BROADCAST, RUNNING, MULTICAST, DEPRECATED, IPv4, NOFAILOVER> mtu 1500 index 2
inet 10.0.0.1 netmask ffffff00 broadcast 10.0.0.255
groupname IPMP-1
ether 0: 21: 28: d2: 1c: c6
igb1: flags = 9040842 <UP, BROADCAST, RUNNING, MULTICAST, DEPRECATED, IPv4, NOFAILOVER> mtu 1500 index 3
inet 10.0.0.2 netmask ffffff00 broadcast 10.0.0.255
groupname IPMP-1
ether 0: 21: 28: d2: 1c: c7
igb1: 1: flags = 1000843 <BROADCAST, RUNNING, MULTICAST, IPv4> mtu 1500 index 3
inet 10.0.0.3 netmask ffffff00 broadcast 10.0.0.255

Bring it all back with the -r (reattach) switch

root @ s1-solaris1 # if_mpadm -r igb0

And finally, the probe based mode defaults the default routes to make sure the network card is active, and if not, then it will multicast address 224.0.0.1, so depending on the specific situation, you need to add this route:

root @ s1-solaris1 # route add default 10.0.0.10

That's all. Thanks for attention.