What could be a stack of technologies for high-frequency trading

In our blog on Habré, we already talked about which data centers are used to host hardware with the trading engines of exchanges and software for high-frequency trading. Today we will go ahead and talk about how the whole stack of technologies needed for HFT trading can be.

Forbes cites a story about this co-founder of the Robinhood service (we wrote about this project here ), Bulgarian Vlad Tenev.
')
Prior to starting work on his own mobile application for online trading, Tenev worked for Chronos Research, which developed ultra-fast software for HFT companies, banks and hedge funds. In many cases, the specific configuration of the infrastructure used for trade depended on the assets to be traded and the market strategies to be used. For example, an infrastructure suitable for work in the foreign exchange market may not always be competitive in the stock or derivatives market.

In 2011, Chronos developed two software platforms, united under the same name Zardoz (like the movie with John Connery) - one of them worked on standard x86 hardware and could be used for trading with delays of 15 microseconds. The other one was optimized for working with Tilera iron; using it, the delay could be reduced to values ​​less than 5 microseconds. Each of the systems was designed to be placed on colocation conditions in the data centers of the exchanges.

For private traders, the x86 architecture platform is of more interest, so Tenev describes it.

Data transfer

As a rule, a marketplace provides one physical connection (in the form of fiber) to the exchange engine, in which there is a "reduction" of applications from different sellers and buyers. To work with the Chronos system, the availability of a channel with a capacity of 10 gigabits was recommended, since in this case the delay of serialization was reduced, but rarely did users manage to get a channel wider than 1 gigabit. Through one physical connection, market data is transmitted (usually via TCP or UDP multicast) and orders are sent (via TCP or UDP unicast).

It would seem that in order to reduce the delay, the physical cross-cable should be inserted directly into the server's network card. However, in practice, this option is almost impossible to meet, since traders always use several physical servers to which dozens of cables from different exchanges are connected.

Equipment

To combine all this diversity, Chronos experts recommended using a high-speed switch, like a 24-port Arista. The use of such equipment contributed to the interport delay of 350 nanoseconds. Also, some traders used a switch from Blade Networks (purchased by IBM), which also worked on Fulcrum ASIC, and cost less.



In addition, for successful operation, the HFT trader needs a high-speed network card with a kernel bypass driver (kernel bypass driver). Tenev mentions Myricom’s 10G-PCIE2-8C2-2S network adapter for UDP traffic and the Flareon Ultra SFN7122F Solarflare Dual-Port 10GbE PCIe 3.0 Server I / O Adapter - Part ID: SFN7122F for TCP.

Soft

Each of these cards has the necessary drivers to bypass the kernel, which allows you to send and receive data via TCP and UDP in user space. Context switching is an expensive operation in terms of delays, so it should be avoided in HFT trading. Therefore, the processing of important data is made in the kernel.

To work with the Chronos system, the custom version of Gentoo Linux was also supplied, which was supported by the software developers.

The software worked in such a way that the modules responsible for direct trading worked with specific processor cores, and other processes were forbidden to use them (this is also necessary to avoid context switching). For these cores, the local timer and other interrupts were disabled.

The platform logic code was written in C and was event-oriented (in a series of articles we talked about creating a back-tester working under this scheme). The developers created their own “non-blocking” data structures - as a result, each thread worked in an infinite loop, polling its own FIFO for input.

As a result, as already mentioned, delays in processing applications did not exceed 15 microseconds.

Other materials on the topic of trading infrastructure:

• Trading and "iron": What are stock exchange data centers
• Development of trading systems for FPGA: Pros, cons and analysis of the architecture of the existing library
• Infrastructure and trading robots: What programming languages ​​are used in finance
• Squirrels, hackers and Facebook: Because of what the "exchange" falls