Entertaining ten-gigabit

While our ships, now equipped with modern wireless technologies, continue to surf the Bolshoi Theater, less mobile and more active consumers of data traffic continue to use the good old Ethernet, which, in its current 10-gigabit form, can satisfy most of them needs. In this article we will give a practical overview of the ten-gigabit Ethernet (10Gbe) technologies in general and the Intel 10Gbe network adapters in particular.

Why 10Gbe?

Why is it about 10Gbe, and not about the latest 40- and 100-gigabit standards? The reason is simple: despite the age that is solid by telekomovskiy standards (the oldest ten-gigabit standard 802.3ae this year turned 10 years old) 10Gbe remains the most popular technology for the core of provider and large corporate networks, as well as data centers. It can be said that 100Gbe is too young and has a whole set of standard “childhood diseases” - the high cost of equipment, supply difficulties, a small number of offers, etc. However, the main reason, in my opinion, is that such speeds in most cases are simply not needed; With the exception of extra large data centers or giant cloud platforms, one or several 10Gbe lines do an excellent job with transmitting the traffic assigned to them.

Some optical terminology

In order not to be distracted further, we will immediately describe some of the optical terms that are found in this article. Optical fibers can be multimode and singlemode. In multimode cables, a beam of light from several rays propagates, constantly reflecting from the border areas; Rays with different wavelengths have different trajectories of motion. Such a nature of the movement of the rays leads to the dispersion of the signal encoded in the light beam - a distortion of its shape. Single-mode fiber is much thinner than multimode fiber, its thickness is comparable to the wavelength of light (the parameters of optical lines standard for telecommunications are given in the table below). Only one beam with a specific wavelength moves along it, without experiencing reflections. A light-emitting diode is usually used as a source of multimode light, while a single-mode light source is a laser.

	Single mode	Multimode
Core thickness, micron	eight	50
Fiber thickness, micron	125	125
Wavelength, microns	1.3 or 1.5	0.8
Damping, dB / km	~ 0.5	~ 3

Standards 10Gbe

The competition between the protocols of local networks is far in the past - Ethernet has won all the others completely and irrevocably. Perhaps this is even for the better, at least the manufacturers of network equipment are no doubt happy to see that working with one standard is much easier than “marrying” several. Anyway, the “unit of measurement” of the network can now be safely considered as one Ethernet port in one of its forms: 100 Mbps (Fast Ethernet, Fe), 1 Gbps (Gigabit Ethernet, Gbe) or 10 Gbps . Well, in turn, servers, workstations and other network equipment are connected to the switch ports. It is clear that hardly anyone will think of using 10Gbe for personal computers, but more powerful devices will appreciate such speeds.

Switch 10Gbe, empty sockets for environment modules are visible (they will be discussed below)

As already mentioned, the first standard 10Gbe was adopted in 2002. It became a logical continuation of the gigabit protocols (by the way, during the transition to 40Gbe, the continuity was interrupted). To date, several 10Gbe standards have been published, differing in signal transmission medium. Here are the main ones:

Standard	Transmission medium	Max. distance	Connector
10GBase-LR	Optics, single mode, 1.3 microns	10 km	SC / LC
10GBase-ER	Optics, single mode, 1.5 microns	40 km	SC / LC
10GBase-SR	Optics, multimode, 0.8 microns	300 m	SC / LC
10GBase-CX4	4-core copper cable	15 m	Infiniband
10GBase-T	Copper twisted pair	100 m	8P8C (RJ45)

As can be seen from the table, single-mode standards allow the signal to be transmitted over longer distances, but they are more expensive and more difficult to install — nevertheless, they have no equal for mainline needs. Multimode 10Gbe is widely used in local area networks within a single node, copper CX4 is used for wiring inside a single enclosure. 10GBase-T is more standardized out of respect for the twisted-pair past merits and is practically not used, as well as other, more exotic 10-gigabit standards.
In addition, there is the option of connecting 10Gbe cards with a copper cable Direct Attach with connectors in the form of SFP + modules - such lines are usually used for stacking, their length is about 10 m.

The color of the optical patch cord indicates its type: yellow - single-mode fiber, orange - multimode fiber. On top of the patch cord connectors FC, below - LC
')
Those wishing to apply this knowledge in practice, please pay attention to the last column. Optical 10Gbe interfaces have SC (thicker and shorter) or LC (longer and thinner) connectors. There is also the third most popular type of connector, FC with a spiral tie - it is used on passive optical equipment. Be careful when designing optical lines!

Media Modules

In full accordance with ISO / OSI covenants, while Ethernet frame processing is handled by the network interface processor, pairing with the physical medium is left to the physical layer (PHY) modules - they determine the ability of a particular network card to work with a specific transmission line. New speeds demanded new modules of the environment - and they, of course, were developed. The first was standardized module XENPAK; its main disadvantage later turned out to be a large physical size, which is why it is currently used in devices with low port density, say, routers. Next in line was the X2 standard: with an architecture identical to XENPAK, it was almost half the size, which allowed it to be used in multiport devices, such as switches. The XFP module was a fundamentally new solution, also progressive in size, but the most widespread standard was SFP + - it is now supported by most manufacturers.

Environment modules Left - XENPAK (type of SC connectors), on the right - SFP + (type of LC connectors)

Standardization of interfaces assumes that modules of different types and manufacturers should work normally with each other, however in life everything, as usual, is a bit more complicated, and compatibility problems still exist today.

Intel Network Adapters

I can not consider this fact well-known, but Intel is one of the oldest manufacturers of network controllers and adapters. Currently, the 10Gbe line of Intel X520 series on the Intel 82599ES controller is relevant, which looks like this:

P / n	Title	Port type
E10G42BTDA	Intel Ethernet Server Adapter X520-DA2	2 x Direct Attach
E10G42BT	Intel Ethernet Server Adapter X520-T2	2 x 10GBase-T
E10G41BFSR	Intel Ethernet Server Adapter X520-SR1	10GBase-SR
E10G42BFSR	Intel Ethernet Server Adapter X520-SR2	2 x 10GBase-SR
E10G41BFLR	Intel Ethernet Server Adapter X520-LR1	10GBase-LR

All of the above cards come with PHY modules already installed, the type of optical connector is LC. If someone suddenly needs a CX4 card, it is made on an Intel 82598EB chip and has the name "Intel 10 Gigabit CX4 Dual Port Server Adapter" (P / n EXPX9502CX4) - two CX4 ports. It is not difficult to see that there are no ER / ZR adapters in the line, they do not exist at all at the moment, in principle - this, of course, is a little sad.

Intel X520 Series Card, specifically - "Intel Ethernet Server Adapter X520-SR1" with one SR port

The list of operating systems compatible with the cards includes both Microsoft server operating systems (Windows Server 2008 in various versions) and * nix OS - RHEL / SLES Linux and FreeBSD.
A common theme in the development of server-based network cards, fully reflected in the X520 lineup, was the transfer of traffic management tools from software to hardware. The power of modern network processors allows them independently, without tugging at the central host, to perform a number of traffic manipulations, such as error control and integrity assurance, application of QoS policies, interaction with server virtual machines, etc.

Summing up our little reviewer, we can say this: over the 10 years of 10Gbe’s existence, its implementations have been honed to such an extent that it is no longer necessary to expect a revolution in this direction, yes, probably no one expects it. 10Gbe is a telecom's “workhorse”, the basis of its nodes and highways, and knowledge of its capabilities and limitations is a necessary component in the knowledge portfolio of any IT pros. The task of this educational program is to show where to dig if necessary and what to fear; I hope that it has been achieved.