Intel Omni-Path. Data is everywhere we have
The general clustering of everything and everything requires specific data exchange technologies, characterized primarily by low latency and high speed. The most famous and used standard in this area is Infiniband . Appearing back in 1999, during its development, Infiband increased its speed from 2 to 100 Gbit / s. Intel is a member of an association that develops Infiband and manufactures Infiband equipment. This, however, does not prevent it from supporting, with its own efforts, one more standard of high-speed switching networks - Intel Omni-Path . This title appeared several times in the posts of our blog, and each time the readers asked for details. Now we have the opportunity to talk about them.
Intel Omni-Path has a standard for technology of this kind of architecture. Its infrastructure consists of network adapters ( HFI, Host Fabric Interface ) for connecting compute and control nodes, switches that allow you to create the necessary topology and number of ports and controls for controlling and monitoring the entire factory.
')
From a network point of view, Omni-Path fits fairly well into the OSI reference model with one small nuance: the description of the protocol uses the term “ Link Transfer Protocol ” ( LTP ), which is responsible for the reliable delivery of second-level objects, flow control and connections. The second level is left to addressing, switching, and resource allocation.
When switching from the second to the half-level, data is cut into 64-bit pieces, a bit of the type is attached to them and a flit is obtained ( Flit ). The data unit that LTP works with consists of 16 flits, 14 CRC bits, and two reporting bits. One or more fleets are command and contain information of control protocols. If necessary, re-transmit a specific LTP ( Link Transfer Packet ) is sent an empty LTP with the necessary data in the command part. The reporting bits of 4 consecutive LTPs are added to an 8-bit report delivered in a command flit.
Handling small pieces of data allows you to keep the transmission delay under control, minimize their spread, provides the basis for QoS mechanisms. For traffic ranking, so-called “virtual lines” ( Virtual Lanes ) are used. For addressing factory subscribers, 24-bit identifiers are used.
The fourth and subsequent levels integrate the Omni-Path into the required software environment. Performance Scaled Messaging (PSM) - API and the corresponding Level 4 protocol designed for the needs of HPC - High Performance Computing. Open Fabrics Interface (OFI) is a universal framework and software installation for a wide variety of needs and protocols. Open Fabrics Alliance Verbs - API and the corresponding Level 4 protocol for implementing Remote Direct Memory Access (RDMA).
Intel Omni-Path technology exists not only in the form of beautiful images. In 2015-16, Intel released the full range of devices required for its use in practice. First, these are node end adapters - HFI. They exist in two modifications: in the form of a PCI Express x8 card (maximum data rate of 56 Gbit / s) and in the form of an x16 card (maximum data rate of 100 Gbit / s). Connection interface - QSFP28. Intel also releases QSFP cables of various lengths , copper passive and optical active.
For the organization of the required network topology Omni-Path, Intel offers two types of Intel Omni-Path Edge Switch 100 Series with 24 and 48 100G QSFP28 ports. Judging by the stated characteristics, the switches provide an honest duplex 100G per port without oversubscription. Also announced are two Intel Omni-Path Director Class Switch modular switches with a significantly larger number of ports (192 and 768) and significant oversubscription (no more than 25% utilization).
The company's plans include the integration of Omni-Path controllers into Intel Xeon processors and the active use of technology in Xeon Phi. We will wait for the next news from this area.
Intel Omni-Path Architecture
Intel Omni-Path has a standard for technology of this kind of architecture. Its infrastructure consists of network adapters ( HFI, Host Fabric Interface ) for connecting compute and control nodes, switches that allow you to create the necessary topology and number of ports and controls for controlling and monitoring the entire factory.
')
From a network point of view, Omni-Path fits fairly well into the OSI reference model with one small nuance: the description of the protocol uses the term “ Link Transfer Protocol ” ( LTP ), which is responsible for the reliable delivery of second-level objects, flow control and connections. The second level is left to addressing, switching, and resource allocation.
When switching from the second to the half-level, data is cut into 64-bit pieces, a bit of the type is attached to them and a flit is obtained ( Flit ). The data unit that LTP works with consists of 16 flits, 14 CRC bits, and two reporting bits. One or more fleets are command and contain information of control protocols. If necessary, re-transmit a specific LTP ( Link Transfer Packet ) is sent an empty LTP with the necessary data in the command part. The reporting bits of 4 consecutive LTPs are added to an 8-bit report delivered in a command flit.
Handling small pieces of data allows you to keep the transmission delay under control, minimize their spread, provides the basis for QoS mechanisms. For traffic ranking, so-called “virtual lines” ( Virtual Lanes ) are used. For addressing factory subscribers, 24-bit identifiers are used.
The fourth and subsequent levels integrate the Omni-Path into the required software environment. Performance Scaled Messaging (PSM) - API and the corresponding Level 4 protocol designed for the needs of HPC - High Performance Computing. Open Fabrics Interface (OFI) is a universal framework and software installation for a wide variety of needs and protocols. Open Fabrics Alliance Verbs - API and the corresponding Level 4 protocol for implementing Remote Direct Memory Access (RDMA).
Omni-Path Equipment
Intel Omni-Path technology exists not only in the form of beautiful images. In 2015-16, Intel released the full range of devices required for its use in practice. First, these are node end adapters - HFI. They exist in two modifications: in the form of a PCI Express x8 card (maximum data rate of 56 Gbit / s) and in the form of an x16 card (maximum data rate of 100 Gbit / s). Connection interface - QSFP28. Intel also releases QSFP cables of various lengths , copper passive and optical active.
For the organization of the required network topology Omni-Path, Intel offers two types of Intel Omni-Path Edge Switch 100 Series with 24 and 48 100G QSFP28 ports. Judging by the stated characteristics, the switches provide an honest duplex 100G per port without oversubscription. Also announced are two Intel Omni-Path Director Class Switch modular switches with a significantly larger number of ports (192 and 768) and significant oversubscription (no more than 25% utilization).
The company's plans include the integration of Omni-Path controllers into Intel Xeon processors and the active use of technology in Xeon Phi. We will wait for the next news from this area.
Source: https://habr.com/ru/post/370441/
All Articles