Ways to transfer financial data # 2: FAST protocol

In one of the past topics, we looked at the FIX protocol , which was created to transmit financial information and automate communications in the stock market. However, this protocol was not the most ideal tool in the face of ever increasing volumes of financial data; therefore, as its development, a new standard was created - the FAST protocol (FIX Adapted for STreaming). Today we will talk about this technology.

Story

In November 2004, the then CEO of Acrhipelago Holding Mike Cormack at a FIX community conference called FPL (FIX Protocol Limited) in New York announced that the current version of the protocol does not cope with the increased financial information generated by the stock market. When transferring large amounts of data using FIX, there were significant delays in their processing, which caused losses to traders and made it impossible for them to develop existing trading strategies.

')

The classic message passing format Tag = Value, used in FIX, was too cumbersome to process it quickly. Shortly after this speech, the first steps were taken to remedy the situation.



When creating the FAST protocol, the developers aimed to achieve the possibility of transferring large amounts of data, avoiding the appearance of delays in obtaining information. The protocol was developed by a FIX community working group called Market Data Optimization Group (mdowg), which was formed in 2004.



In 2005, the experts of the group presented a pilot project (proof of concept) of the protocol, and a year later the first version of FAST 1.0 was released. Later, several updates were released, and currently, the majority of financial market players use protocol version 1.2. There are also several open protocol implementations (for example, OpenFAST , OpenFAST.NET, and QuickFAST ).

How it works

According to the FIX protocol standard, each message has the format Tag = Value SOH, where Tag is the number of the field to be transmitted, Value is its value, and SOH is the delimiter character (for more details, see our past material ). An example of writing a message in the Tag = Value syntax:



35=x|268=3 (message header) 279=0|269=2|270=9462.50|271=5|48=800123|22=8 (trade) 279=0|269=0|270=9462.00|27



The FAST protocol allows you to get rid of redundancy using a template that describes the structure of the message as a whole. This method is called implicit tagging, because FIX tags in the transmitted data are only “implied”. In summary, the Tag = Value syntax changes as follows:

• A template describes a structured set of fields with their operators;
• The sequence of fields in the message corresponds to the sequence of tags in the template;
• Only data changes are transmitted in the message.

The FAST protocol is used to obtain market data.

What are the stock exchange sites FAST

The FAST protocol is supported by a number of the world's largest stock exchanges, including:

• NYSE Archipelago ;
• CME Group (CME) ;
• International Securities Exchange (ISE) ;
• NasdaqOMX ;
• SIX_Swiss_Exchange ;
• Eurex ;
• Xetra (trading system) ;
• Bombay_Stock_Exchange (BSE Ltd, India);
• BATS ;
• ICAP ;
• OPRA ;
• BVMF ;
• Nordic Growth Market (NGM) ;
• Shanghai Stock Exchange (SSE, China) ;
• Moscow Exchange (MICEX-RTS) .

Let us consider in more detail the implementation of FIX on the Moscow Exchange.

Implementation of the Moscow Exchange

Moscow Exchange has implemented the ability to provide access to financial information under the FAST protocol . Traders can receive FAST tables of financial instruments (prices, trading volumes, etc.), quotes, all transactions, indices, as well as information about all active, impersonal orders. According to the information on the website of the Exchange, the average delay in publishing updates is less than 300 microseconds (for the stock and currency markets).



UDP channels are used to transmit financial data, and a set of undelivered data is transmitted via TCP. The data is structured into a group of flows (channels or “feeds”) that contain information about a group of financial instruments. These tools are automatically grouped by the Exchange's trading system in accordance with certain parameters.







UDP can reduce delays in the transmission of information, and their duplication reduces the loss of messages. If the scheme is described in more detail, the FAST service consists of the following channels:

• Market Data Incremental - a system of UDP channels for receiving data changes on current trading;
• Incremental Refresh - this channel allows you to receive information about current trading on the market;
• Instruments Replay - the stream is used to receive reports on financial instruments admitted to trading;
• Market snapshot - the stream is used to restore current information on all tools;
• Instruments Incremental - a channel for receiving changes in the parameters of the instruments (change in the collateral and price limits);
• Message Replay - allows you to request messages that were skipped in the Incremental Refresh stream.

Image / moex.com (click to open in full size)



All messages sent from the Moscow Exchange trading system are encoded in FAST format. A specific feature of the Moscow Exchange format is the addition of a preamble of 4 bytes to the message. This preamble contains a tag number 34 (SeqNum), which allows you to get the sequence number of messages without having to decode the entire message - this translates into additional time savings on processing data streams.







As a rule, the transfer of the FAST template is carried out using XML. A typical use case of the protocol (connection before the start of the trading day) can be described by the following sequence of actions:

1. Downloading a configuration XML file with a list of multicast IP addresses and other connection parameters (streams, ports, etc.).
2. Download FAST template from an FTP resource.
3. Start “listening” to the Snapshot stream, and then Incremental Refresh to get the data.

An example of a template in XML:

 <!-- Market Data - Incremental Refresh Generic --> <template xmlns="http://www.fixprotocol.org/ns/fast/td/1.1" name="X-Generic" id="2104"> <string name="MessageType" id="35"> <constant value="X"/> </string> <string name="ApplVerID" id="1128"> <constant value="9"/> </string> <string name="BeginString" id="8"> <constant value="FIXT.1.1"/> </string> <string name="SenderCompID" id="49"> <constant value="MOEX"/> </string> <uInt32 name="MsgSeqNum" id="34"/> <uInt64 name="SendingTime" id="52"/> <sequence name="GroupMDEntries"> <length name="NoMDEntries" id="268"/> <uInt32 name="MDUpdateAction" id="279" presence="optional"/> <string name="MDEntryType" id="269" presence="optional"/> <string name="MDEntryID" id="278" presence="optional"/> <int32 name="RptSeq" id="83" presence="optional"/> <uInt32 name="MDEntryDate" id="272" presence="optional"/> <uInt32 name="OrigTime" id="9412" presence="optional"/> <uInt32 name="SettlDate" id="64" presence="optional"/> <string name="SettleType" id="5459" presence="optional"/> <uInt32 name="MDEntryTime" id="273" presence="optional"/> <uInt32 name="EffectiveTime" id="5902" presence="optional"/> <uInt32 name="StartTime" id="9820" presence="optional"/> <string name="Symbol" id="55" presence="optional"/> <decimal name="MDEntryPx" id="270" presence="optional"/> <decimal name="MDEntrySize" id="271" presence="optional"/> <string name="QuoteCondition" id="276" presence="optional"/> <string name="TradeCondition" id="277" presence="optional"/> <string name="OpenCloseSettlFlag" id="286" presence="optional"/> <string name="OrdType" id="40" presence="optional"/> <decimal name="NetChgPrevDay" id="451" presence="optional"/> <decimal name="AccruedInterestAmt" id="5384" presence="optional"/> <decimal name="ChgFromWAPrice" id="5510" presence="optional"/> <decimal name="ChgOpenInterest" id="5511" presence="optional"/> <decimal name="BidMarketSize" id="5292" presence="optional"/> <decimal name="AskMarketSize" id="5293" presence="optional"/> <int32 name="TotalNumOfTrades" id="6139" presence="optional"/> <decimal name="TradeValue" id="6143" presence="optional"/> <decimal name="Yield" id="236" presence="optional"/> <decimal name="TotalVolume" id="5791" presence="optional"/> <int32 name="OfferNbOr" id="9168" presence="optional"/> <int32 name="BidNbOr" id="9169" presence="optional"/> <decimal name="ChgFromSettlmnt" id="9750" presence="optional"/> <int32 name="SumQtyOfBest" id="10503" presence="optional"/> <string name="OrderSide" id="10504" presence="optional"/> <string name="OrderStatus" id="10505" presence="optional"/> <decimal name="MinCurrPx" id="10509" presence="optional"/> <uInt32 name="MinCurrPxChgTime" id="10510" presence="optional"/> <uInt32 name="VolumeIndicator" id="7017" presence="optional"/> <decimal name="Price" id="44" presence="optional"/> <int32 name="PriceType" id="423" presence="optional"/> <decimal name="NominalValue" id="9280" presence="optional"/> <decimal name="RepoToPx" id="5677" presence="optional"/> <decimal name="BuyBackPx" id="5558" presence="optional"/> <uInt32 name="BuyBackDate" id="5559" presence="optional"/> <string name="TradingSessionID" id="336" presence="optional"/> <string name="TradingSessionSubID" id="625" presence="optional"/> </sequence> </template> 

How to connect to FAST

ITinvest provides its customers with access to the Moscow Exchange markets through direct connection using the FIX and FAST protocols. In addition, special IT services have been created for high-frequency traders and algorithmic traders, ranging from server collocation in the M1 data center to providing access to virtual machines for hosting robot (rates are listed on the site ).



For those who have not read our previous material on the FIX protocol, we attach the video again with a story about solutions for obtaining direct access to the stock market.

Other protocols

In addition to the FIX and FAST protocols supported by the international community, so-called native protocols are used to directly connect to the domestic markets, which arose before the merger of the MICEX and RTS exchanges into the Moscow Exchange.



In the markets related to the RTS Stock Exchange (FORTS - futures and options , Standard), the Plaza II protocol is used to directly perform transactions and obtain data in the connection mode. To perform trading operations and obtain stock exchange data on the sites previously belonging to the MICEX exchange (currency and stock markets), a bi-directional MICEXBridge gateway (TEAP) is used.



These protocols will be discussed in our next articles. In addition, we describe the protocol Simple Binary Encoding , which, to a certain extent, is the successor of the FIX case.



Today, thank you for your attention, we will be happy to answer questions in the comments.



PS If you notice a typo or mistake - write a personal message, and we will fix everything promptly.