SMB Multichannel in Windows Server 2012
SMB Multichannel is one of the features of the SMB 3.0 protocol, designed to improve performance and fault tolerance at the network connection level when working with file servers. To use this feature, the SMB client and SMB server must support SMB 3.0, which, in turn, is implemented in Windows Server 2012 and Windows 8 .
SMB Multichannel allows you to simultaneously use multiple network connections to the file server, thereby providing:
For SMB Multichannel to work, two requirements must be met:
For SMB Multichannel to work, the client and server must support at least one of the following configurations:
As a result, SMB Multichannel will not work if the computer has one network adapter that does not support RSS, or if there are several adapters, but they work at different speeds. In the latter case, the traffic will be transmitted on the fastest adapter.
')
Some explanations regarding individual configurations.
Network adapter with RSS support
In the absence of SMB Multichannel, when an SMB client accesses a SMB server, an SMB session is opened, within which one TCP / IP connection is established. Traffic processing on this connection is performed by one of the available processor cores. If the network adapter does not support RSS, then in principle all network traffic is processed by one core, usually CPU0. If the network adapter supports RSS, the processing of network traffic is distributed among the available processor cores. However, without SMB Multichannel, only one connection is established in one session, and RSS cannot do anything to help this particular session.
In the case of SMB Multichannel, if the network adapter supports RSS, the SMB protocol establishes several connections within one session (by default, 4 to the RSS interface), and RSS, in turn, distributes traffic processing through these connections to the available cores.
Accordingly, even with the use of a single RSS adapter, SMB Multichannel makes it possible to speed up network operations and more effectively utilize the available bandwidth. And above all, this is true for 10GbE adapters.
What SMB Multichannel cannot provide for us in a configuration with one physical adapter is fault tolerance, since it is clear that if a single adapter fails, then we lose all the connections.
In Windows Server 2012 and Windows 8, the easiest way to determine which adapters and with which current settings support RSS is through the PowerShell cmdlet.
Two or more adapters combined using NIC Teaming
Using NIC Teaming is not a requirement for SMB Multichannel. If the system, for example, has two gigabit adapters, then SMB Multichannel can use them without any timing. However, combining adapters into a timing group will provide an additional advantage in terms of fault tolerance. Without a timeout, SMB Multichannel will only provide fault tolerance for SMB traffic, along with NIC Teaming you will get fault tolerance of any network traffic. At the same time both with timing and without it, SMB Multichannel aggregates the available bandwidths of the available adapters.
One or more network adapters with RDMA (Remote Direct Memory Access)
RDMA technology provides low latency when transferring data over a network and reducing processor load. SMB Multichannel establishes for each SMB session two connections per RDMA interface. If there are several RDMA adapters, fault tolerance is additionally provided. It should be noted that if you combine RDMA interfaces into a group using NIC Teaming tools in Windows Server 2012, then the OS treats these interfaces as non-RDMA. That is, in the timing, the capabilities of RDMA are not available to you. By analogy with RSS, you can get information about adapters with RDMA support and their configuration using the cmdlet
Multiple network adapters with the same speed
This is the easiest configuration option in which SMB Multichannel just works. If the adapters do not support RSS, then one connection is established per interface.
And now is the time to discuss the SMB Multichannel setup.
The beauty of technology is that it is automatically configured. If the OS detects one of the supported configurations (see above), then SMB Multichannel is automatically applied. However, you can make sure that the technology is enabled by
Additionally,
I have already mentioned the number of connections to be established for different types of network interfaces (4 per RSS-adapter, 2 per RDMA-adapter, 1 per ordinary NIC). By default, SMB Multichannel uses a maximum of 8 connections for each client / server pair. And this number can limit the number of connections per network interface. For example, if you have three RSS-adapters, then three connections will be established through the first, three through the second and two through the third.
It is recommended to use the default SMB Multichannel settings. However, these settings are subject to change. For example, the maximum number of connections per client / server pair is set as follows:
Finally, if for some reason you need to disable SMB Multichannel, for example, for testing or diagnostics, you can do this with the following two commands on the server and client, respectively:
The same commands with a value of $ true will re-enable the technology.
If your system has one of the supported configurations, then when you open a shared folder on the SMB server, Multichannel will establish several connections. You can verify this with
In my environment with two gigabit cards, it looks like this:
Next, by running the file copy from a network share, in Performance Monitor you can observe that copying is done using both network adapters.
Turning off one of the adapters, you will see that the copy operation is not interrupted, and the entire load will be transferred to the remaining adapter.
You can see how SMB Multichannel works in dynamics, and also compare how an OS behaves when technology is disabled, you can in my web caste (scroll before demonstrating SMB Multichannel).
Thus, with SMB Multichannel, you can improve the efficiency and reliability of network operations. In combination with other Windows Server 2012 technologies, such as SMB Scale Out, SMB Transparent Failover, and a number of others (we’ll talk about them in other posts), you can provide a high level of accessibility of running services at different levels with OS tools: disk subsystem, cluster node, network interface.
What are the benefits of SMB Multichannel?
SMB Multichannel allows you to simultaneously use multiple network connections to the file server, thereby providing:
- Increased throughput . The SMB client and server can transfer more data by simultaneously using multiple network connections established using one high-speed network adapter, such as 10GbE, or several network adapters. In the case of several network adapters, SMB Multichannel actually aggregates their bandwidths.
- Fault tolerance . Performing network operations, such as copying a file, are not interrupted if one or more connections are lost until at least one connection continues to exist. Loss of connection can occur either due to problems in the network environment (cable break, switch failure, etc.), or due to problems with the network adapter. Thus, when using multiple network adapters, SMB Multichannel provides fault tolerance at the physical network card level. But once again, only for SMB operations
What are the requirements for SMB Multichannel?
For SMB Multichannel to work, two requirements must be met:
- SMB 3.0 on the client and on the server. Currently this version of the protocol is implemented in Windows Server 2012 and Windows 8.
- One of the supported configurations discussed below
Supported Configurations
For SMB Multichannel to work, the client and server must support at least one of the following configurations:
- Multiple network adapters with the same speed
- One or more network adapters with RSS (Receive Side Scaling)
- Two or more network adapters grouped together using NIC Teaming
- One or more network adapters with RDMA (Remote Direct Memory Access)
As a result, SMB Multichannel will not work if the computer has one network adapter that does not support RSS, or if there are several adapters, but they work at different speeds. In the latter case, the traffic will be transmitted on the fastest adapter.
')
Some explanations regarding individual configurations.
Network adapter with RSS support
In the absence of SMB Multichannel, when an SMB client accesses a SMB server, an SMB session is opened, within which one TCP / IP connection is established. Traffic processing on this connection is performed by one of the available processor cores. If the network adapter does not support RSS, then in principle all network traffic is processed by one core, usually CPU0. If the network adapter supports RSS, the processing of network traffic is distributed among the available processor cores. However, without SMB Multichannel, only one connection is established in one session, and RSS cannot do anything to help this particular session.
In the case of SMB Multichannel, if the network adapter supports RSS, the SMB protocol establishes several connections within one session (by default, 4 to the RSS interface), and RSS, in turn, distributes traffic processing through these connections to the available cores.
Accordingly, even with the use of a single RSS adapter, SMB Multichannel makes it possible to speed up network operations and more effectively utilize the available bandwidth. And above all, this is true for 10GbE adapters.
What SMB Multichannel cannot provide for us in a configuration with one physical adapter is fault tolerance, since it is clear that if a single adapter fails, then we lose all the connections.
In Windows Server 2012 and Windows 8, the easiest way to determine which adapters and with which current settings support RSS is through the PowerShell cmdlet.
Get-NetAdapterRss Two or more adapters combined using NIC Teaming
Using NIC Teaming is not a requirement for SMB Multichannel. If the system, for example, has two gigabit adapters, then SMB Multichannel can use them without any timing. However, combining adapters into a timing group will provide an additional advantage in terms of fault tolerance. Without a timeout, SMB Multichannel will only provide fault tolerance for SMB traffic, along with NIC Teaming you will get fault tolerance of any network traffic. At the same time both with timing and without it, SMB Multichannel aggregates the available bandwidths of the available adapters.
One or more network adapters with RDMA (Remote Direct Memory Access)
RDMA technology provides low latency when transferring data over a network and reducing processor load. SMB Multichannel establishes for each SMB session two connections per RDMA interface. If there are several RDMA adapters, fault tolerance is additionally provided. It should be noted that if you combine RDMA interfaces into a group using NIC Teaming tools in Windows Server 2012, then the OS treats these interfaces as non-RDMA. That is, in the timing, the capabilities of RDMA are not available to you. By analogy with RSS, you can get information about adapters with RDMA support and their configuration using the cmdlet
Get-NetAdapterRdma Multiple network adapters with the same speed
This is the easiest configuration option in which SMB Multichannel just works. If the adapters do not support RSS, then one connection is established per interface.
And now is the time to discuss the SMB Multichannel setup.
SMB Multichannel setup
The beauty of technology is that it is automatically configured. If the OS detects one of the supported configurations (see above), then SMB Multichannel is automatically applied. However, you can make sure that the technology is enabled by
Get-SmbClientConfiguration or Get-SmbServerConfiguration Additionally,
Get-SmbClientNetworkInterface will show you whether RSS and RDMA are enabled for the available interfaces.I have already mentioned the number of connections to be established for different types of network interfaces (4 per RSS-adapter, 2 per RDMA-adapter, 1 per ordinary NIC). By default, SMB Multichannel uses a maximum of 8 connections for each client / server pair. And this number can limit the number of connections per network interface. For example, if you have three RSS-adapters, then three connections will be established through the first, three through the second and two through the third.
It is recommended to use the default SMB Multichannel settings. However, these settings are subject to change. For example, the maximum number of connections per client / server pair is set as follows:
Finally, if for some reason you need to disable SMB Multichannel, for example, for testing or diagnostics, you can do this with the following two commands on the server and client, respectively:
Set-SmbServerConfiguration -EnableMultiChannel $false Set-SmbClientConfiguration -EnableMultiChannel $false The same commands with a value of $ true will re-enable the technology.
SMB Multichannel in action
If your system has one of the supported configurations, then when you open a shared folder on the SMB server, Multichannel will establish several connections. You can verify this with
Get-SmbMultichannelConnection In my environment with two gigabit cards, it looks like this:
Next, by running the file copy from a network share, in Performance Monitor you can observe that copying is done using both network adapters.
Turning off one of the adapters, you will see that the copy operation is not interrupted, and the entire load will be transferred to the remaining adapter.
You can see how SMB Multichannel works in dynamics, and also compare how an OS behaves when technology is disabled, you can in my web caste (scroll before demonstrating SMB Multichannel).
Thus, with SMB Multichannel, you can improve the efficiency and reliability of network operations. In combination with other Windows Server 2012 technologies, such as SMB Scale Out, SMB Transparent Failover, and a number of others (we’ll talk about them in other posts), you can provide a high level of accessibility of running services at different levels with OS tools: disk subsystem, cluster node, network interface.
Source: https://habr.com/ru/post/151451/
All Articles