Financial data transfer protocols. Instructions for use
Many articles and “guides” have been written on how to become a trader and start trading on the Exchange. Algorithm trading, robots, option strategies - many have heard these terms. Unfortunately, in the majority of such “roadmaps” they forget to warn about the technical side of the question that many will have to face - the choice of a trading strategy and the appropriate data transfer protocol.
WHO TO BE?
To begin with, we determine the choice of direction in trade and evaluate our technical skills.
There are 4 main types of trade :
1 and 2 TYPES
In the first and second types of trading, you will interact only with a broker and use the software that it provides (therefore, this criterion should be given special attention when choosing a broker company). When selecting these types, solutions developed by the Exchange itself will not be available to you.
The software provided by the broker (client space for the broker system, technical analysis system, etc.) is installed on your PC or mobile device, and the connection (usually via the Internet) is made to the broker's servers.
Also in the second form, some brokers, in addition to the standard client software, can provide the ability to connect via the program interface (API) of their systems in order to develop the client's own trading software.
')
3 AND 4 TYPES
Here, customers use direct protocols provided by the exchange. In terms of technology and protocols, these types of trade do not differ from each other, except for the location of servers, software components and connection channels. It is important to take into account broker support for connection methods in general and specific stock exchange protocols, in particular.
A little bit about how to connect. What does the topology of the location of software components look like when using the most popular "Universal Connection" circuit? The broker’s office / data center hosts a router that is connected to the exchange infrastructure via a dedicated channel through the carrier’s L3. Three VLANa are organized on this router: DMZ (is a DMZ segment of the exchange trading network), CLT (network for client software), TST (segment for access to test exchange services is not so important in this context, therefore not shown in the diagram). The icon with the caption “VPT” is the system you have developed. The abbreviation stands for “external software and hardware” - this is our official term for any software that connects to the exchange infrastructure. For more information about Universal Connection, see Scheme 1.
Scheme 1.
In contrast to the connection using the Universal scheme, the layout of the server with a robot on collocation is much simpler:
Scheme 2.
Simply put, all you need to do (not counting the “paperwork” :)) is to place the server in the exchange data center (now it is the M1 data center in Varshavka, but moving to the DataSpace in Dubrovka, which will become the main ). With one network interface, the server looks at a pool of our access servers dedicated to high-speed robots, and the second interface, to the Internet, is the so-called control channel intended for servicing the robot.
It was a small digression in the direction of "iron-network" history, and now, the main thing ...
TRANSMISSION PROTOCOLS OF FINANCIAL DATA
Decided on the type of trade and connection? Now we select the appropriate protocol for transferring financial information.
At the moment, the exchange offers developers the following direct access to exchange markets:
ASTS Bridge
We immediately clarify a small terminological confusion in the names and translations: the exchange gateway is the Bridge. Gateway, which also really wants to be translated as a gateway, is another story ( see our first publication on Habré ). We call it the access server.
The gateway is the ASTS native trading and clearing system protocol that has existed since 1998 (previously the solution was called TEAP (TCP / IP version) or TEServer (RS-232 version no longer supported). Many developers know the protocol by the name MTESRL, by the name of the corresponding DLL. Due to the “nativeness” of this protocol, its main feature is the support of all transactions and all market data from all markets operating on the ASTS trading and clearing system.
The use of this protocol is recommended, first of all, for those who need access to clearing data and operations (viewing their positions, obligations, risk parameters, setting various kinds of limits, transferring securities and money between accounts, etc.), as well as participation in trades in negotiation deals modes (that is, not “quick” anonymous trades in the “glass”, but direct conclusion of deals with a specific counterparty).
The API is provided as a dynamic library - in 32-bit and 64-bit versions for Windows and Linux.
The connection architecture is as follows: the dynamic library falls into a package of software developed by you, this package is installed on a PC / server that has network access to the so-called server part of the gateway. The server part is a kind of proxy server that is located at the broker and is connected to the exchange infrastructure via dedicated network channels.
In the case of the above-mentioned HFT trading, when your software is installed in the data center of the exchange on colocation terms, an intermediate link in the form of the server part of the gateway is no longer needed - you connect directly to exchange Gateways.
An interesting feature of the gateway protocol is the support of "interfaces". An interface is a versioned set of user-accessible tables and transactions, with the appropriate structure and data types. Practically with each update of the trading and clearing system, new opportunities appear for users who require modifying the structure of tables or changing the format of transactions. The presence of versioned interfaces allows users who are not ready for changes to stay on the old version of the interface and not modify their software. At the moment, we support the ability to connect all versions of interfaces created over the past years (decades!), But next year we plan to become stricter - see below.
Plaza II
The main protocol of connection to the derivatives market is currently the Plaza II protocol.
To connect using this protocol, the exchange provides the CGate API. What does this give? On the one hand, it allows bidders to implement full functionality for access to trading, including clearing functionality for limiting sections, setting restrictions on instruments and viewing market maker obligations. On the other hand, it allows customers of bidders to implement their own high-speed robots with a minimum set of functions (to put an application / withdraw an application).
The API is provided as a set of dynamic libraries - in 32-bit and 64-bit versions for Windows and Linux.
The connection architecture provides two options:
With almost every release of the derivatives market, the exchange makes changes and improvements to its own program code, which is transmitted to customers in the form of an API. For the user, this looks like a new distribution with new versions of libraries inside. In addition to the code itself, the structure of the data given to users changes periodically with the release. Of course, in almost all cases we are trying to preserve the backward compatibility of the data structure and give users enough time to adapt to new structures.
FIX and FAST
These are standard international data exchange protocols. On the Runet and on Habré, in particular, there is plenty of information about them, so we will not overload our brief overview with them. Everyone can read them here and, for example, here .
Who is faster?
One of the most popular questions to our support team is: who is faster - FIX / FAST or the native protocol. Most often it is set by the creators of "robots". We can talk a lot and for a long time about speeds and delays in data transfer - this is a topic for an entire article (take note!). Therefore, for now, we restrict ourselves to statistics The statistics obtained as a result of the measurements shows that in the foreign exchange market in the first one and a half years of the existence of the FIX protocol, its share rose to 92% in terms of the number of applications, and over the next two years it exceeded 99%. In the stock market, the share of applications received through FIX reaches 60%, but this is explained by a different composition and number of investors. In the zone of collocation, almost all systems use FIX / FAST.
The situation on the derivatives market is slightly different: FIX is less popular here. Due to its architectural location in the system, its speed is slightly lower than the speed of the native protocol, but we are working to improve its latency performance.
ISS
Not to be confused with "IIS" ! :) This protocol is somewhat out of alignment, as it covers a segment of tasks related not to the implementation of transactions, but to work with stock data. In essence, this is an API to exchange web services, implemented according to the concept of Restful. It gives you the opportunity to obtain general market information - quotes, deals, indices, volumes, trading results, and so on - using the http / https protocol. The service is available only via the Internet, so minimizing delays in receiving data is not applicable to it.
This protocol is used to show stock quotes on websites (including all the data on moex.com are broadcast from there), downloads of trading results for analytics, drawing charts on various demo panels and scoreboards, and in any other working via the Internet applications. Also, from among the products available to all users through the EIS, our MOEX Trade Info information terminal and mobile applications work (taking this opportunity to say to all users of our iOS / Android applications, that yes, we know that the current versions of these applications are already morally obsolete: ) but we are working on completely new versions, taking into account all your wishes).
“Old school” : among the existing connection protocols, there are some that are gradually becoming history. For example, the library Client gate, which previously allowed to connect to the derivatives market. By the end of next year, it will be decommissioned and the development of systems for it will no longer be done.
Interface Support Policy
A project is currently underway on the Exchange to switch to a 2-year life cycle of software support and related exchange interfaces. More information about this is available on our website .
How to develop and test your software
Information on exchange protocols, as well as the provision of all related exchange software for development purposes, are open, free and accessible to everyone. The description of the main protocols can be found on our website , and more specific technical documentation on FTP .
For testing purposes, a number of test systems operating around the clock and technical support at help@moex.com are available, working from 8:00 to 24:00.
Upon completion of the development, in order for your system to be allowed to “combat” connection, it is necessary to go through the certification process. It consists in connecting to a test system with logging enabled and performing some typical operations. In the course of this test run, the technical support staff checks the correctness of the implementation of working with the API (we do not look at the source codes and your software, everything is only based on the logs).
WHO TO BE?
To begin with, we determine the choice of direction in trade and evaluate our technical skills.
There are 4 main types of trade :
- Manual trading (long-term investment / few transactions). It uses the "client" of the brokerage system. It does not require virtually any special knowledge and high costs.
- Algorithmic trading with a strategy that does not require high-speed connections . This type of trading involves the use of automation equipment from a broker. They will be cheaper for the money, and also do not require special skills.
- Algorithmic trading with high-speed strategy . This type of trading provides a samopisny robot on top of the exchange gateways (through the broker's infrastructure, or through a remote connection directly to the exchange servers). This option is more expensive for money than the first two, and requires programming skills.
- HFT trading . This type of trading involves collocation, or its own development on top of the exchange gateways. A rather expensive option, similar to the third.
1 and 2 TYPES
In the first and second types of trading, you will interact only with a broker and use the software that it provides (therefore, this criterion should be given special attention when choosing a broker company). When selecting these types, solutions developed by the Exchange itself will not be available to you.
The software provided by the broker (client space for the broker system, technical analysis system, etc.) is installed on your PC or mobile device, and the connection (usually via the Internet) is made to the broker's servers.
Also in the second form, some brokers, in addition to the standard client software, can provide the ability to connect via the program interface (API) of their systems in order to develop the client's own trading software.
')
3 AND 4 TYPES
Here, customers use direct protocols provided by the exchange. In terms of technology and protocols, these types of trade do not differ from each other, except for the location of servers, software components and connection channels. It is important to take into account broker support for connection methods in general and specific stock exchange protocols, in particular.
A little bit about how to connect. What does the topology of the location of software components look like when using the most popular "Universal Connection" circuit? The broker’s office / data center hosts a router that is connected to the exchange infrastructure via a dedicated channel through the carrier’s L3. Three VLANa are organized on this router: DMZ (is a DMZ segment of the exchange trading network), CLT (network for client software), TST (segment for access to test exchange services is not so important in this context, therefore not shown in the diagram). The icon with the caption “VPT” is the system you have developed. The abbreviation stands for “external software and hardware” - this is our official term for any software that connects to the exchange infrastructure. For more information about Universal Connection, see Scheme 1.
Scheme 1.
In contrast to the connection using the Universal scheme, the layout of the server with a robot on collocation is much simpler:
Scheme 2.
Simply put, all you need to do (not counting the “paperwork” :)) is to place the server in the exchange data center (now it is the M1 data center in Varshavka, but moving to the DataSpace in Dubrovka, which will become the main ). With one network interface, the server looks at a pool of our access servers dedicated to high-speed robots, and the second interface, to the Internet, is the so-called control channel intended for servicing the robot.
It was a small digression in the direction of "iron-network" history, and now, the main thing ...
TRANSMISSION PROTOCOLS OF FINANCIAL DATA
Decided on the type of trade and connection? Now we select the appropriate protocol for transferring financial information.
At the moment, the exchange offers developers the following direct access to exchange markets:
| Protocol | Markets | Available features |
|---|---|---|
| ASTS Bridge | Stock, Currency | Trading + obtaining all market data. 100% support for all operations. |
| Plaza II | Urgent | Trading + obtaining all market data. 100% support for all operations. |
| FIX | Stock, Currency, Urgent, UTS | Trading operations in the main trading modes (without the support of negotiation transactions), Trade Capture (stock and currency only), Drop Copy |
| FAST | Stock, Currency, Urgent | Obtaining anonymous market data. |
| Information and statistical server (ISS) | Everything | Obtaining anonymous market data through exchange web services. |
ASTS Bridge
We immediately clarify a small terminological confusion in the names and translations: the exchange gateway is the Bridge. Gateway, which also really wants to be translated as a gateway, is another story ( see our first publication on Habré ). We call it the access server.
The gateway is the ASTS native trading and clearing system protocol that has existed since 1998 (previously the solution was called TEAP (TCP / IP version) or TEServer (RS-232 version no longer supported). Many developers know the protocol by the name MTESRL, by the name of the corresponding DLL. Due to the “nativeness” of this protocol, its main feature is the support of all transactions and all market data from all markets operating on the ASTS trading and clearing system.
The use of this protocol is recommended, first of all, for those who need access to clearing data and operations (viewing their positions, obligations, risk parameters, setting various kinds of limits, transferring securities and money between accounts, etc.), as well as participation in trades in negotiation deals modes (that is, not “quick” anonymous trades in the “glass”, but direct conclusion of deals with a specific counterparty).
The API is provided as a dynamic library - in 32-bit and 64-bit versions for Windows and Linux.
The connection architecture is as follows: the dynamic library falls into a package of software developed by you, this package is installed on a PC / server that has network access to the so-called server part of the gateway. The server part is a kind of proxy server that is located at the broker and is connected to the exchange infrastructure via dedicated network channels.
In the case of the above-mentioned HFT trading, when your software is installed in the data center of the exchange on colocation terms, an intermediate link in the form of the server part of the gateway is no longer needed - you connect directly to exchange Gateways.
An interesting feature of the gateway protocol is the support of "interfaces". An interface is a versioned set of user-accessible tables and transactions, with the appropriate structure and data types. Practically with each update of the trading and clearing system, new opportunities appear for users who require modifying the structure of tables or changing the format of transactions. The presence of versioned interfaces allows users who are not ready for changes to stay on the old version of the interface and not modify their software. At the moment, we support the ability to connect all versions of interfaces created over the past years (decades!), But next year we plan to become stricter - see below.
Plaza II
The main protocol of connection to the derivatives market is currently the Plaza II protocol.
To connect using this protocol, the exchange provides the CGate API. What does this give? On the one hand, it allows bidders to implement full functionality for access to trading, including clearing functionality for limiting sections, setting restrictions on instruments and viewing market maker obligations. On the other hand, it allows customers of bidders to implement their own high-speed robots with a minimum set of functions (to put an application / withdraw an application).
The API is provided as a set of dynamic libraries - in 32-bit and 64-bit versions for Windows and Linux.
The connection architecture provides two options:
- Your PBX installed on your PC / server connects to the broker’s servers and enters the exchange infrastructure through the broker’s server.
- Your PBX directly (using the Internet / dedicated communication channels or access from the collocation zone) goes to the public servers of the exchange and works directly with the exchange, bypassing the broker's infrastructure.
With almost every release of the derivatives market, the exchange makes changes and improvements to its own program code, which is transmitted to customers in the form of an API. For the user, this looks like a new distribution with new versions of libraries inside. In addition to the code itself, the structure of the data given to users changes periodically with the release. Of course, in almost all cases we are trying to preserve the backward compatibility of the data structure and give users enough time to adapt to new structures.
FIX and FAST
These are standard international data exchange protocols. On the Runet and on Habré, in particular, there is plenty of information about them, so we will not overload our brief overview with them. Everyone can read them here and, for example, here .
Who is faster?
One of the most popular questions to our support team is: who is faster - FIX / FAST or the native protocol. Most often it is set by the creators of "robots". We can talk a lot and for a long time about speeds and delays in data transfer - this is a topic for an entire article (take note!). Therefore, for now, we restrict ourselves to statistics The statistics obtained as a result of the measurements shows that in the foreign exchange market in the first one and a half years of the existence of the FIX protocol, its share rose to 92% in terms of the number of applications, and over the next two years it exceeded 99%. In the stock market, the share of applications received through FIX reaches 60%, but this is explained by a different composition and number of investors. In the zone of collocation, almost all systems use FIX / FAST.
The situation on the derivatives market is slightly different: FIX is less popular here. Due to its architectural location in the system, its speed is slightly lower than the speed of the native protocol, but we are working to improve its latency performance.
ISS
Not to be confused with "IIS" ! :) This protocol is somewhat out of alignment, as it covers a segment of tasks related not to the implementation of transactions, but to work with stock data. In essence, this is an API to exchange web services, implemented according to the concept of Restful. It gives you the opportunity to obtain general market information - quotes, deals, indices, volumes, trading results, and so on - using the http / https protocol. The service is available only via the Internet, so minimizing delays in receiving data is not applicable to it.
This protocol is used to show stock quotes on websites (including all the data on moex.com are broadcast from there), downloads of trading results for analytics, drawing charts on various demo panels and scoreboards, and in any other working via the Internet applications. Also, from among the products available to all users through the EIS, our MOEX Trade Info information terminal and mobile applications work (taking this opportunity to say to all users of our iOS / Android applications, that yes, we know that the current versions of these applications are already morally obsolete: ) but we are working on completely new versions, taking into account all your wishes).
“Old school” : among the existing connection protocols, there are some that are gradually becoming history. For example, the library Client gate, which previously allowed to connect to the derivatives market. By the end of next year, it will be decommissioned and the development of systems for it will no longer be done.
Interface Support Policy
A project is currently underway on the Exchange to switch to a 2-year life cycle of software support and related exchange interfaces. More information about this is available on our website .
How to develop and test your software
Information on exchange protocols, as well as the provision of all related exchange software for development purposes, are open, free and accessible to everyone. The description of the main protocols can be found on our website , and more specific technical documentation on FTP .
For testing purposes, a number of test systems operating around the clock and technical support at help@moex.com are available, working from 8:00 to 24:00.
Upon completion of the development, in order for your system to be allowed to “combat” connection, it is necessary to go through the certification process. It consists in connecting to a test system with logging enabled and performing some typical operations. In the course of this test run, the technical support staff checks the correctness of the implementation of working with the API (we do not look at the source codes and your software, everything is only based on the logs).
Source: https://habr.com/ru/post/261369/
All Articles