Cisco switch stacking. Part 2

Greetings to all!
In the first part of this article, we looked at the rather old stacking technologies for the Cisco 3750 family of switches, namely StackWise and StackWise Plus. Today I propose to continue the consideration of other technologies. Let me remind you, we still have StackWise-160, StackWise-480, FlexStack and FlexStack Plus.

StackWise-160

The StackWise-160 technology emerged with the advent of 3650 series switches. It is not compatible with other versions of stacking technologies. The main algorithms are borrowed from StackWise Plus. At the same time there are differences. The first is the new architecture of the switch itself, built on the basis of the new ASICs (Unified Access Data Plane (UADP) ASIC).
New ASICs are endowed with sufficient intelligence to provide switching functions. Because of this, unlike the 3750E / 3750X, there is no longer a dedicated switching fabric. An interesting point is that traffic switching between ports served by different ASICs is provided via a stack interface.
')
I would like to remind you that in the old ASICs (in the 3750E / 3750X switches), the elements responsible for processing incoming packets and the elements responsible for processing outgoing packets are separated. They do not have a common bus inside the ASIC. Therefore, even if a packet is transmitted between ports serviced by the same ASIC, this packet necessarily falls on the switching fabric.
The second difference with StackWise-160 is the increased throughput of the stacked bus. The capacity of the stack cable is now equal to 40 Gbps (full duplex). Thus, the throughput of the entire stack bus for StackWise-160 technology:
40 Gbit / s * 2 (in each direction) * 2 (number of ports) = 160 Gbit / s

It should be noted that, unlike the 3750 series of switches, the stack set for the 3650 is purchased separately.
For StackWise-160 technology, the general stack operation scheme at the program level has been changed. The Stateful Switch Over (SSO) scheme is now used for fault tolerance. As we remember, in previous technologies (StackWise and StackWise Plus) a simpler failover scheme was used. One of the switches is selected as the master (stack master). It performs logic operations (control-plane) for the entire stack. Only hardware tables (MAC tables and CEF tables (FIB / Adj)) are synchronized between stack switches. The remaining tables, including the routing table, are re-filled on the new master. Those. control-plane starts from scratch. On the 3650 switches, a more advanced scheme - Nonstop Forwarding with Stateful Switchover (NSF / SSO) - was used to ensure resiliency. There is no such thing as a master. Now use the Active-Standby scheme. One of the switches is selected by the main (Active), another one - by its hot standby (Standby), which synchronizes with the main all the necessary information (L2 and L3). Control-plane now works in Active-Standby mode. This minimized the time required for recovery in the event of a primary switch failure.

StackWise-480

Let's now take a look at StackWise-480 technology. Using it, you can stack 3850 series switches into a stack.
Switches 3650 and 3850 are very similar. Both of these series are based on UADP ASIC. Accordingly, the algorithms of the StackWise-480 and StackWise-160 stack are similar. Truth is the difference. The StackWise-480 technology uses three physical stack rings. This is achieved by the fact that within one stack cable for 3850 switches, there are three wires (Fig. 6). Each with a capacity of 40 Gbps (full duplex).

Capacity of the entire stack bus for StackWise-480 technology:

40 Gbit / s * 2 (in each direction) * 3 (number of wires) * 2 (number of ports) = 480 Gbit / s

At the logical level, the stack is represented in six ways (two logical paths per wire). Packages in three logical paths “spin” in one direction, and in the other three - in the other (Fig. 7).

The choice of path is carried out in the same way as before, with the help of tokens.

Flexstack

On this with an overview of the stacking technology family Stackwise suggest to finish. Let's now take a look at the 2960 switch family and the FlexStack and FlexStack Plus stacking technologies.

Stacking for the 2960 first appeared on the 2960-S switches. To integrate switches using the FlexStack technology, a stack module and specialized cables with a capacity of 10 Gbps (full duplex) are used. Each stack module has two ports. When combined, switches are connected to a ring (although this is not necessary). The total bandwidth of the entire stack bus is:

10 Gbit / s * 2 (in each direction) * 2 (number of ports) = 40 Gbit / s.
For the FlexStack stack, packet transfer between switches occurs device-by-device. The switch for each packet determines where to send it: to the regular or stack ports. This interaction resembles the work of several switches connected to each other via Ethernet. The difference is that the communication between stack switches is provided by the FlexStack protocol. The choice of which of the stack ports to send a packet is determined by a special algorithm resembling the work of OSPF. Those. the shortest path to the switch in the stack on which the destination port is located is selected. If any changes occur (for example, one of the switches failed or the stack cable is disconnected), this algorithm recalculates the paths again.

Separately, it should be noted a mechanism to prevent traffic looping. We are talking about traffic, where there is no exact recipient (broadcast, multicast, unknown unicast). Since this traffic hits all switches in the stack, there must be a mechanism that prevents it from looping if the switches are connected in a ring (all stack ports are involved). For this, the passive link mechanism is used. For each switch in the stack, a stack connection is selected that is located as far as possible from the switch in question. This connection becomes passive. Each switch in the stack will have its own connection. When passing through it, all broadcast, multicast and unknown unicast traffic is dropped, which is pushed onto the stack through the switch for which this connection is passive. Those. we just open the ring. But unlike STP, for each switch in the stack, the ring is opened in different places. This gives more efficient traffic than in the case of classic STP.

At the program level, the FlexStack operation scheme resembles StackWise / StackWise Plus. One of the switches is chosen by the master and the control-plane is launched only on it.

FlexStack Plus

The stack of FlexStack Plus differs from its predecessor in that up to 8 switches can be combined into a stack (for a FlexStack, this figure is 4) and the bandwidth over a dedicated cable is increased to 20 Gbit / s. Thus, the total bandwidth of the stack bus is - 80 Gb / s.

Conclusion

In conclusion, I would like to note a few points.
All the considered technologies allow to combine switches into a stack, which are next to each other. The maximum length of the stack cable is 3 meters. This, of course, is not very convenient. True, Cisco does not position the data of a series of switches for building distributed stacks. For this, in particular, Virtual Switching System (VSS) virtualization technology is proposed. Its support begins on Cisco 4k series switches.

Judging by the temporal characteristics of disaster recovery, StackWise technologies are faster than FlexStack technologies. This is partly due to the fact that FlexStack handles all types of failures programmatically. In principle, this is not surprising, since the Cisco 2960 series is younger. The most interesting of these are Stackwise-160 and 480, since they support “adult” SSO.

On the 3750X and 3850 switches, another type of stack appeared - the Power Stack (StackPower). It is designed to redistribute the "extra" power between the power supply units of the switches, combined into this type of stack. This allows you to ensure the operation of the switch stack, in case of failure of its local power supply. Or provide an additional PoE budget.

From practice I can say that I often had to push it with Stackwise stacks (and its extensions). There were no particular problems with their work. There are examples where such stacks worked fine for 5-7 years as a network core in relatively large networks. FlexStack is less common in practice. True, and with them there were no special problems.

UPD (March 2017)

Since IOS XE 16.3.3, there is support for StackWise Virtual technology. It allows you to stack two switches 3850-48-XS. This technology is not compatible with other versions of StackWise. Normal Ethernet ports (10 Gb / s or 40 Gb / s) are used as a stacked bus. You can use up to four ports. StackWise Virtual supports SSO / NSF (stateful switching and packet transfer at the moment of switching). StackWise Virtual in terms of work is similar to VSS technology.