Superficially about the basics of market architecture and algorithmic trading
Many people know that one of the first things they say in a technical college is to forget everything that went through at school. This recommendation is relevant here. Sometimes it is useful to start from scratch.
At the moment, all markets are automated. For this reason, some economic explanation of pricing are certain rudiments. Rulat algorithms + some manual intervention.
The task of each trading algorithm is always the same - to bring money to the owner. The algorithm is the better, the more money he is able to bring.
Among the algorithms on the market are the so-called market-maker algorithms. You can explain it on your fingers from a simple example to a more complicated one:
')
Imagine that you have a task to create a new symbol for trade. Let there be people who for some reason want to trade it. What is required of you? You need to form your character from your Level2 applications at any time. Those. fill the symbol with prices and liquidity. At first, you can make a completely stupid MM-algorithm - Level2 does not change. Ie the client bought or sold, after which you added liquidity to the original Level2. Obviously, such an algorithm will always give the owner money. But the problem is that people are not complete idiots, and they will not trade on a constant symbol - there is not even the potential to earn them.
So you need to somehow make people make deals. The next dull MM algorithm can be a simple sine wave - Level2 goes up and down. Many people will also merge, but there are geniuses who will see the pattern and begin to bend the owner of the MM algorithm.
It turns out that you need to come up with something so that the MM-algorithm has the maximum difference between merging and earning. Here and begin to develop various mat. models that are far from trivial.
Well, in the market, of course, there are many MM-algorithms with different owners. There are large owners (non-sick banks) who also have an insider - they know which traders (because they are their clients) where they stand and how they traded. Therefore, mat. models are well thought out. But the task is always the same - to squeeze out as much money as possible from the meat (simple market participants).
Those. You understand that even you can be a market maker, you only need a good MM algorithm and the solution of some organizational issues.
It seems, it became clear that everything is kept in our automated time on algorithms. There are many types of them. Let's try to consider a purely technical algorithm for creating a trading platform. The simplest algorithm of this type is the exchange one. About him and talk.
So, there is some kind of symbol that will be traded only on our exchange. And there are many who want to trade it. And this means that there are ready-made MM-algorithms and meat, without which everything is completely meaningless (besprofitno).
The exchange algorithm is purely technical, i.e. brings profit to its owner by the fact that everyone uses his results, paying a commission. In this case, even a negative commission can be embedded in the algorithm, for example, for MM algorithms. The commission grid is again a kind of uncomplicated mate. model.
So, there is Level2 (FOREX terminology, it is also called MarketDepth on the exchanges) - these are the levels: the purchase / sale price and how much (volume). Each such level (price + volume) is called a gang. The best gangs for sale and purchase are called Bid and Ask (some call Offer, but this is just terminology).
Level2 is formed from client requests (limitnik) - this is MM-algorithms + some more types of algorithms and meat. Spread is the difference between the current Ask and Bid (in particular, for this reason there is a very inaccurate phrase in its wording that MM earns on a spread).
Any limitnik exchange algorithm almost always puts on the appropriate gang in the glass. It is for this reason that by placing the limitnik inside the spread its corresponding narrowing occurs. I will not explain it on my fingers, the algorithm is simple (you can google or ask).
If a limitnik is placed worse than the current corresponding price (for example, SellLimit <= Bid), then the exchange execution algorithm does not execute at a price worse than the market price, but begins to lick (by volume) the gang from the best towards deterioration until the entire limitnik is filled, until the gang runs out to the level at which this limitnik is set.
Of course, there are sometimes secret moments in the exchange algorithms, when information about your application, before getting into the glass, goes, for example, to the MM-algorithm. And in advance (ahead of time), he simply acts so that you do not get a positive slippage from the limiter at a price worse than the current price, and take it as a net profit for yourself. This is some of those. insider, which, of course, is bought, i.e. has a very clear price tag (they rule the money). Do not forget about the main task of any algorithm - money. Therefore, there should not be anything surprising - the truth of life.
On exchanges during a trading session, a pre-set limitnik is executed in 99% of cases exactly at the price - without slippage. This means that the price to cross your limitnik, it must be executed.
Limits in Level2 are covered by market orders. Types of orders is full, as the exchange wants (prescribe). Same algorithm. For example, MT5 introduced its own types of orders that the exchange could well legalize - to implement in its exchange execution algorithm. A market order is a derivative of a limit order: the limiter is worse than the current price, and this is almost not measured. That's how much you need to execute the market - so much you get, but only with good negative slippage. The main users of market orders are meat.
Obviously, the price can move along almost any trajectory without making deals - we look at the MM-algorithms. If the transaction is made, then its price and volume is called Last. And this infa is also broadcast by the exchange. Last-Data Flow is called T & S (Time & Sales).
Some exchanges (far from all) add a conditional flag to Last-Data - the direction of the transaction (Buy or Sell). This marker is a kind of classifier: for passive and active participants. Those. on those whose limitnik is full, and those who filled it. Obviously, this classifier is a temporary filter: whose execution of the two sides of the transaction was initiated by a later / early trade order.
It is clear that the exchange algorithm must be consistent - i.e. a queue is formed from applications and they are processed sequentially. But there are variants of the so-called snapshots, when applications accumulate for a relatively short time, and then are issued at level2 times with the corresponding execution.
The correct exchange algorithm does not allow the pricing of the public situation Bid> = Ask. In the algorithm itself, as applications are received, at the initial stage a glass is formed, in which situations are often Bid> = Ask. In such a situation, the executive part of the exchange algorithm is turned on, whose task is to resolve this situation to the Ask> Bid state. And only after settling the already formed glass with the correspondingly formed Last-data becomes public - accessible to all.
SellLimit is always executed at the Bid price, BuyLimit is executed at the Ask.
But only these Bid and Ask are non-public prices of the initially formed glass, as described above.
If you expose SellLimit - this is the desire to sell you, which is equal to the offers for others to buy from you. For this reason, SellLimit falls into Ask Gangs. For example, if you put SellLimit inside the spread, then the best Ask-gangs are formed with the level and volume of your limiter. Those. putting inside a SellLimit spread, you change the Ask price. If someone wants to buy at the Ask-price, he will flood your limitnik. To say in this case that SellLimit is executed at the Ask price, or it is executed without a spread - a very vague wording. It is better to simply understand the mechanism, as elsewhere.
I will give an example of execution. You put SellLimit inside the spread, which means Ask equals SellLimit. Now you set BuyLimit equal to Ask. In this situation (see the first paragraph) in the exchange algorithm, it turns out that Bid is equal to BuyLimit. Those. in the end, it turns out that Bid = Ask. Everything went to resolve the situation, until Ask becomes more Bid. Until the algorithm resolves, no one will see the correct glass. For simplicity, let the volumes SellLimit and BuyLimit equal Vol. It turns out that both limitik collapse, Bid and Ask become equal to the next best gang in the glass, i.e. Ask> Bid. Next, the Last-Data contains the execution price, which is equal to your SellLimit (== BuyLimit), the volume Vol, and the BUY direction (because BuyLimit was sent later to SellLimit).
Note that if in the same situation you first send BuyLimit and then SellLimit, then the result will be the same - you buy / sell yourself, losing on a double commission. But only in Last-data the direction flag will be opposite - SELL.
Classification of traders can be engaged indefinitely. But among prof. market participants (brokers, platform developers, etc.) are divided into two types of traders: clickers (GUI-clickers) and algorithmic traders. The names fully speak about their features. The division into experienced and novices (they also use "noobs"), as a rule, is not conducted.
If the task is to attract traders, then the clickers are the easiest to attract and algo-traders are much more difficult.
A nice nice GUI is made to the clickers, secondary attention is paid to the trading conditions. For this reason, in particular, clickers are on average meat (a source of profit for other market participants). The competition for clickers is high.
Algae traders are very picky about trading conditions, the GUI is not decisive. The competition for algorithmic traders is low, although serious efforts are spent on it.
Mat. The expectation of all algorithmic traders is much higher than the mat. waiting for clickers. The number of algorithmic traders is much less, but the turnover is higher.
It so happened that the opinion of algorithmic traders listen to a much lesser extent than the clickers. Platform developers suffer this, and brokers suffer the same. The opinion of algorithmic traders can even extend to how, for example, the platform is sold to a broker - for a commission on turnover or a monthly fee. Those. All nuances of possible changes in trading conditions are taken into account.
Most often, market participants are set to satisfy only one type of trader.
Obviously from what was written earlier, market makers are algorithmic traders.
This term is used by market makers. Toxic flow is a systemic profitable flow of trade orders. It comes from the very ones that make money on the imperfections of the MM algorithms. In practice, 99% of the toxic flow falls on algorithmic traders.
All three models are absolutely market - it is very important to understand, because no fair market or dishonest. Everything is one and intertwined.
The first differs from the other two in that it does not earn on commissions from the turnover of customers, but on their discharge. Let us analyze everything in one real example (all the data are public), names are missing.
A certain broker declares that his turnover is> $ 100 billion / month, while his earnings are $ 80 per month. Obviously, this is a market maker, because even the most inadequate STPs do not earn more than $ 65 per mio, not to mention such a high monthly turnover.
It turns out that $ 80 per leman is the rate of discharge of their customers. This means that with an average lot of 0.5, the average client makes a deal with a profit of ~ -0.5 points of EURUSD.
If you dodge, then the average trade in a lot / pip can be approximately calculated by the ticket in MT4. Yes, many tickets will go to canceled deposits and I / O operations, but this is no match for the number of market orders that are clickers' favorites.
We think further, once a monthly turnover> 100 yards, it means MoneyIn - MoneyOut> $ 8 mio. Those. a broker at the sink of their clients monthly earns $ 8 mio.
The question is where it is possible to find a monthly amount of hamsters. And do not forget that MoneyOut is on average about two times smaller (he invented) MoneyIn. Those. the broker monthly attracts about $ 16 mio new live money.
How he does it?!
The average hamster is somewhere around $ 200 (also invented by himself, never heard from anyone). So even by the most conservative estimates (MoneyOut = 0), you need to strip 40,000 hamsters a month.
In fact, hamsters for that are called that they merge many many times. And carry their average $ 200 on a schedule - with each RFP over the years. This is a payment for the regular pleasure of feeling like a clicker. Just as they spend every month on movies, beer, etc.
If a toxic stream is suddenly found among hamsters, it is quickly and easily detected, after which it is strongly recommended to switch to the STP-type or close the account.
This type of broker actually benefits other types of brokers (STP and ECN / STP). These brokers are not market makers competitors. They even do a great blessing: they produce a clear separation “ECN for pros” and “Standard for beginners”. Since this allows the customer base on the Standard to get rid of toxic. It is even advantageous to them that people go to others, because toxica in the end less. And you can earn more with less risky as a result of MM.
STP and ECN / STP can experience serious problems with a high proportion of toxics. They need the clickers at least in order to dilute the toxic in front of their LPs, which are almost always reduced to large market makers. And they act in the image and likeness of small market makers (as described above).
The fact is that on the FOREX at this stage the MM algorithms of large owners - banks prevail. If you make money on FOREX, then it is almost always a loss of these banks, more precisely, of their algorithmic departments, in which millions of budgets are spent on something, in particular, so that you do not earn.
To profit from the drain of another simple trader can now basically only indirectly: the trader will merge the MM algorithm, and you will throw up this profit from him. To put it bluntly, making money on FOREX is robbing something stolen.
But all this sounds strange only if you look through rose-colored glasses at many events taking place in the world. For a long time, the market (stock or FOREX) has become a legal tool for taking money from the public. But it is not all that bad.
Let's go back to our MM broker. Does he have a grain scheme? Not really. The fact is that a MM broker is able to earn even a lot more than $ 80 with mio at a very volatile and trending market. But go into a decent minus in the long-sluggish market. These are the predictable features of the average hamster trade. Those. The brokerage MM model is a risky model, but at the same time it can be very profitable under favorable market conditions.
In order for the MM-broker to refuse the MM-model, he needs to believe that the STP-model is able to give at the commission not much less. This means that he needs to increase at times the momentum. It is practically impossible to do on the huge base of clickers, since turnover is done by algorithmic traders. Algo-traders will only go when trading conditions are among the best. Well, this in turn can occur with serious financial investments in STP-infrastructure.
Moreover, if there is such a huge herd of meat in the form of clickers, then why donate them to the snickering MM-banks, receiving only a commission from their drains. If you can take everything completely yourself. In general, such simple reasoning makes it possible to understand that a MM broker with a huge meat base becomes a hostage of himself.
There can still be ridiculous, if the base has some kind of national mentality. For example, the number of Russian hamsters is reduced by an order of magnitude if you ask them to fill in a little more fields in the questionnaire, as regulators require. In many ways, this forces MM brokers to go offshore.
But again, not everything is so bad ...
Quanta are, as a rule, educated people with an analytical mindset. Unfortunately, in life it happens that a good education, possession of exact sciences, etc. it is not enough for success. Usually, the more effort spent on learning something, the greater the importance attached to it. Among scientists there is a huge number of mediocre personalities who cope well with the career features of their profession. These are publications in journals, participation in summits, lectures, etc. One of the near-science themes is a comprehensive study of trading activities. Everything is done as taught. Those. vlob applied mat. apparatus, associations with physics are built, various market models are created, etc. Such scientists are called quanta. A large part of them are careerists. But there are exceptions, as in any case.
You can direct a different quantum heresy by giving the term cointegration precise mathematical definitions. But all this has little to do with the practice of trading. For a simple explanation of cointegration is possible. Imagine that you have a lot of trade symbols. If there is a possibility (the quantity of each symbol) to buy / sell these symbols in such a way that the profit fluctuates around some not very variable value, then such symbols are cointegrated. Of course, quite childish, but the essence conveys by 90%. This combination of symbols is called a cointegrated portfolio. A classic example of such a portfolio: an asset and futures on it.
Any portfolio has Bid and Ask-prices. Moreover, it has its Level2 (easily calculated from the Level2 characters included in it). If there is an opportunity for a cointegrated portfolio to make a deal so that after it is closed there is a positive profit, then this possibility is called arbitrage. Such a co-integrated portfolio is called an arbitrage portfolio (it has just been invented).
The lower the trade costs (quality of execution, commission), the more chances the cointegrated portfolio will become arbitrage.
Markup - price deterioration (Level2) by a certain amount. For example, a broker may lower prices to make a profit. It can go to the marketing step - resetting the commission (clickers do not like difficulties), but entering it into a brand. There are also positive (profitable) markups for a trader, but their understanding is possible only with a good understanding of pricing, so for now let's miss.
Suppose there is a desire to do arbitration. To do this, you need to at least create a cointegrated portfolio. The simplest cointegrated portfolio consists of two symbols of the same name: one for one broker, the other for the other.
Take, for example, so popular EURUSD and give the symbols for convenience the corresponding names: EURUSD1 and EURUSD2. The most important point that needs to be fully realized is that EURUSD1 and EURUSD2 are completely different characters.They could even be called brokers in a different way, to have (by an order of magnitude, for example) different prices and other differences. Only one thing is important - they are cointegrated. But for simplicity, we will consider the elementary case: EURUSD1 and EURUSD2.
Before comparing prices, an algorithmic markup is made on them in order to bring in them all possible trade costs (quality of execution for each broker and commission for each broker). We will further assume that all prices are already marked.
So, in each broker you have trading accounts with certain money. If it is very primitive to look at arbitration, then it is required to find the moments Ask1 <Bid2 and Ask2 <Bid1. And in these moments open / close opposite positions in each of the brokers.
This is the simplest and frontal implementation. Let us make a small digression towards a more generalized and universal vision of such trade.
In this case, the co-integration of the portfolio suggests that Synth = EURUSD1 / EURSD2 hovers around one. This Synth has its own Synth_Bid and Synth_Ask (Synth_Level2) prices. If it is possible to build a ZigZag with vertices on Synth_Bid and lowlands on Synth_Ask, then our portfolio Synth is arbitrage. But this is a distraction.
Let us return, nevertheless, to a more familiar for the majority look at the trade. In fact, in some cases it is justified to create something high-level for the convenience of trading. And for arbitration this high level is done like this:
They take the masked Level2_1 and Level2_2 and simply combine into Level2_All, to which the created artificial high-level symbol EURUSD_All begins to correspond. Very simple trading functions are written that are able to trade EURUSD_All. For example, if you want to sell EURUSD_ALL, then OrderSend (EURUSD_All, OP_SELL) sends a SELL order to the broker who has the highest bid price, i.e. his bid price is on the best gang in Level2_All.
Here we must now say a few words about Level2_All. In his internal representation of the gangs now contains not only the prices and volume, but also the name of the source of this data.
With this implementation, you only need to wait for the situation when Ask_All <Bid_All and at this moment simultaneously open multidirectional positions on EURUSD_All. As a result, receiving high-level profits and the lack of open positions on EURUSD_All. Convenient, isn't it?The adviser in such a high-level language would occupy 10 lines: he saw negative spread, trade it, wait further.
If we descend from a high level of vision of such trade downwards, then we will notice that at the moment when we do not have positions on EURUSD_All, we will have an open position on EURUSD1 and opposite to it on EURUSD2. This in turn will cause the natural distortions of Equity1 and Equity2. Yes, roughly speaking, Equity_All = Equity1 + Equity2 will grow as trading progresses, but we know that Equity1 and Equity2 must be at least positive. And our distortions may well simply reset the account on one of the brokers, although the other will grow.
How to solve this problem? The first thing that comes to mind is to pour money from one broker to another. Those.order the withdrawal of funds from the broker, where more money and send them to where it is less. But it is long, very long. Moreover, it is also extremely expensive - bank and other types of transfers are not free.
But imagine that someone appears who says that all distortions will level for a small fee. This one is clearing.
It was easy to see how convenient to trade high-level. It is convenient to trade not only arbitration, but generally any strategy, since EURUSD_All prices based on their construction can not be worse than EURUSD1 and EURUSD2. Those.the artificial symbol is more profitable and the profit on it is obviously higher. Here comes the idea of ​​creating such artificial high-level symbols for any kind of strategies. At the same time, by the same principle, to take not two sources (brokers in the example) of cointegrated symbols, but any number of them. Obviously, the more sources, the more profitable (the prices are better) high-level symbols.
Such high-level formations are called liquidity aggregators. Obviously, this algorithm is a kind of software that allows trading this way. Among the companies participating in the life of the market there are software companies that deal with exactly such aggregation algorithms. By giving its customers the opportunity not to bother with low-level trading, but to trade at a high level, wasting their intellectual powers on other things.
Among these software companies, Currenex and Integral are best known. They sell their software for a commission on turnover. For this reason, in particular, some write their own aggregators to be more competitive.
I wanted you to create a trading platform with little effort. Very rude it looks like this. You come to a well-established software-aggregator company and declare your desire. Those answer that without problems, but it is necessary to fulfill some conditions. To begin with, you must have agreements with source companies that you want to aggregate. Next, you need one single account with a prime broker who already has agreements with your source companies and is able to organize clearing so that there is no imbalance. Clearing is also part of the commission.
Further, the aggregation software company adjusts its aggregation software with your sources and prime broker, with whom it has contracts for the possibility of such activity, and gives you a turnkey trading platform.
Note that the sources may be the same aggregators, their sources are also some aggregators. All of this can end up intertwining and repeating, which sometimes causes an overestimation (duplication of sources) of volumes on some bands of your Level2_All. And, of course, at the very ends of this network are banks with their own MM-algorithms.
Market participants are always divided into two categories: creating (PriceGiver) market offers (prices and quantity) and agreeing (PriceTaker) with them.
The market offers themselves are bids: ready to buy / sell so much at such and such a price or better. The combination of these applications (Level2) conditionally shows the current market liquidity.
Liquidity Provider (LP) - this is how the aggregator names its price sources, which it combines into one single virtual (high-level) Level2. Obviously, LP is PriceGiver. A significant share of PriceGivers is occupied by MM-algorithms.
STP is an aggregator where the customer is allowed to be only PriceTaker-ohm. This means that customers are not able to form their applications for which someone could agree. They can only accept offers from PriceGivers.
As mentioned earlier, the aggregator is a kind of high-level virtual trading service. Therefore, all limit orders of STP aggregator clients are virtual and are stored on the aggregator server.
As a result, if someone takes an STP aggregator as an LP, he will not be able to trade with his clients. Almost all MM algorithms running on an STP aggregator will not give a positive profit.
In a simple STP, all virtual client trade orders come to his PriceGivers as market orders. For example, the client has SellLimit. As soon as Bid> = SellLimit occurs, a market order SELL is sent to the LP to which the Bid corresponds. The response of LP to such a trade order is the result of the execution of the client’s SellLimit.
Obviously, with such an implementation, slippage occurs, the average of which is less than zero. Those. customers incur losses.
Here, all virtual client trade orders come to its PriceGivers, as limit orders (a market order is a limiter, at a decent price (determined by the STP aggregator) worse than the current one).
Let us analyze the execution algorithm using the example above:
At the moment of Bid> = SellLimit, the virtual SellLimit is frozen (taken out of the account by the execution system, the client cannot do anything with it) and the same SellLimit is sent to the corresponding LP. Moreover, all these actions do not affect pricing. Those.a bit later, freezing (for example, at 1 ms) Bid may form better (from other LPs), but it will almost (there are nuances) not be involved in the execution of our client SellLimit.
LP, where SellLimit was sent, replies that it executed one part of SellLimit, and did not execute the rest for various reasons - reject. After that, the client receives a high-level open SELL position for the volume filled and defrozen SellLimit for the rest.
The speed and quality of the LP response depends on many factors. The duration of the response can be up to several seconds. There are situations when an answer from LP does not come.
Note that (with such an implementation of STP), the limiters of the aggregator clients do not slip into the negative zone. Moreover, there are frequent positive slippings that cover a significant part of the fixed trading costs - the commission.
Here you can see how much the performance and pricing of exchanges and FOREX differ.
ECN / STP - as an aggregator, algorithmically identical to STP. The only difference is that a new virtual LP - LP0 is added to the aggregation: requests from the customers of the aggregator.
PriceGivers also become ECN / STP aggregator clients. This means that customers can actually make transactions between themselves directly.
The greater the turnover is in LP0, the more profitable the ECN / STP aggregator is, since unlike other LPs, LP0 does not require a fee for its use by the aggregator. Moreover, the flow of trade orders attributable to LP0 can in no way be characterized as toxic. Sincethis is already a trade between clients, even if there are MM algorithms among them.
It turns out such a trouble-free LP.
But it must be clearly understood in order to be able to be PriceTaker for LP0-PriceGiver of ECN / STP aggregator, you need to be its client. Those.Customers of other aggregators cannot sometimes take advantage of the excellent (best) LP0 price offers.
For example, you trade in two aggregators. In ECN / STP, put a limiter inside the spread, improving Bid or Ask. But in another aggregator you cannot buy / sell yourself at this price, because they are not visible. Those. you buy / sell, most likely, at a less favorable price than the one offered by your limitnik.
The strength of ECN / STP is the ability to exchange liquidity.
Imagine that some STP aggregator took as an LP an ECN / STP aggregator. This automatically indicates that STP aggregator clients become a PriceTakers ECN / STP aggregator, in particular (and other LPs from the STP aggregator list). It turns out that in the example above with two aggregators, you would see your application in another STP aggregator in a STP aggregator. And if it were the best in the final STP-Level2, then you could make a deal with yourself. Those.your application would be available to a lot more PriceTakers, which means that the likelihood of the execution of your application would increase.
It is necessary to understand that each aggregator has its own PriceTakers base, and such a simple technique allows you to exchange these bases, increasing the likelihood of the execution of PriceGivers applications. And it is also profitable PriceTakers, because prices are better.
Of course, if two ECN / STP aggregators mutually make each other their LP, then a serious discrepancy will occur in everyone's Level2 - recursively liquidity will be increased several times. Therefore, ECN / STP aggregators exchange (more precisely, they can) among themselves domestic (LP0) liquidity through certain technical agreements - tags.
Obviously, the ECN / STP aggregator has some insider information: who and how trades through it. A competent owner of such information creates a classifier of customer profitability. For example, as a result of the analysis, he learns that the aggregator's clients are stably (and at the right speed) poured onto the GBPJPY from 12 to 17 o'clock in each environment. What it means is draining - it means that their money flows into LP, which at that time had the most competitive prices.
This begs the question, why give someone a drain when you can take it away? Some will go according to the gray scheme - they will disable ECN / STP at this time and in fact will turn into a DC with a market-making scheme. But there is an absolutely mutually beneficial white path - the HFT MM algorithm. Let it be ours.
In our example, every Wednesday from 12 to 17 o'clock we put our BuyLimit and SellLimit inside GBPJPY-spread at a minimum distance from the external (not formed by us) spread and follow all the time (here we need HFT - fast dragging of orders) . It improves prices - narrows the spread. And merging customers will make deals at better prices. However, (see the mention of the analysis of the discharge rate), their discharge will flow to us already, since they will make deals with us.
Mutual benefit is that we get a profit, and merging customers lose money at a slower rate than if we did not use the HFT MM algorithm. Well, if the owner of this MM-algorithm is the ECN / STP aggregator itself, then the commission, naturally, is zero for using its LP0. And the client will pay the aggregator in full for the transaction. Those. in fact, for such an owner, the commission is even negative.
It is not necessary to have the aforementioned insider information. You can get it yourself a simple way - sounding.
To do this, you will need to spend money, of course: a spread is created according to the scheme above, but only with a minimum lot. ECN / STP customers will always make deals with you, thus you will receive insider information. Further, everything is the same as in the scheme above.
Adaptability is that insider information is taken by a sliding window, i.e. adapts (controversial, of course) to the mood of customers and the market.
Note that any HFT MM algorithm will almost never be toxic, since only PriceTakers takes money, among which there are almost no HFT MM algorithms.
It should probably be mentioned that these elementary HFT MM algorithms were invented by me immediately. And HFT exploited MM algorithms are much more intelligent. However, their essence is roughly conveyed.
An ECN / STP aggregator is always beneficial when its clients include HFT MM algorithms, since they provide a high turnover. However, for independent HFT MM algorithms, trading costs in the form of commission are a significant obstacle to uncover their full potential. For this reason, a marketing scheme is created, which can have quite a clear mate. calculations when the HFT MM algorithm is given a negative commission. Those.the aggregator for all applications of the HFT MM-algorithm that were executed on LP0, does not charge him a fee, moreover, pays him a part of the fee, taken in full from the other side of the transaction - PriceTaker. A similar scheme was casually mentioned in the description of the exchange algorithm.
On FOREX, not all ECN / STP aggregators provide T & S data. There may be several reasons for this. For example, hide the momentum and do not explicitly analyze the toxic coming from ECN / STP. However, some ECN / STP aggregators do provide such information.
T & S is available on the stock exchanges, but the data on the classification of transactions (PriceGivers or PriceTakers) do not always have official (stock) origins. Most often this is an independent algorithmic classification.
In the case of both the stock exchange and the ECN / STP aggregator, the correctly classified T & S data shows when PriceTakers / PriceGivers lose / win. In this case, to create an independent simple HFT MM-algorithm is not required (with proper technical infrastructure - speed), even sounding and any analysis of the dynamics of Level2. Since properly converted and analyzed T & S data - the aforementioned insider.
Institutional FOREX off-market quotes are not rare. They become a sore point when deals are made on them. And especially for patients when these deals have a high (in absolute terms) profit, regardless of its sign.
For various reasons, MM-algorithms can fail, giving out non-market quotes. In fact, they are quite market, since they offer to buy / sell for them. But in the concluded agreements, as a rule, there is an obvious loophole when the deals made on non-market quotations can be canceled (and often canceled).
Non-market prices are always determined retroactively by comparison with price histories from several sources. If a strong release is compared with the rest, it means an off-market market.
The right of lastluk is to refuse execution at the price offered by them. This is used by some LPs in the direction of toxic customers: if a request from a toxic came - not to execute, if from a merging one - to execute.
Because of this, the quality of performance greatly decreases, increasing the number of rejects. They are struggling with algorithmic pricing in some aggregators.
There are sites without blastas, but there with obvious toxicity, as a result, what has already been described earlier: either the site refuses the toxic client, or the LP, which incurs losses from this toxicant.
As already mentioned, the aggregator is a high-level union of cointegrated symbols. Note that there is no restriction on the reality of such symbols.
This means that if there are synthetic cointegrated symbols, then they can not only be created, but also aggregated.
The virtual aggregator does this - it aggregates cointegrated synthetic feeds (price streams) into one high-level (virtual, of course) symbol that can be traded inside one site.
Virtual aggregators have the same property as classical ones - they mask toxicity, create the best prices, increase instant liquidity, increase the liquidity ceiling of the vehicle, etc.
Most often, the creation of virtual aggregators falls on the shoulders of algorithmic traders. The most advanced classical aggregators can also partially do this: aggregate not only the clearly existing symbols of the LP, but also their synthetics.
A primitive example of visualizing the simplest part of a virtual aggregator is to combine synthetic feeds:
The most adequate assessment of any character occurs when analyzing its high-level aggregated namesake.
Imagine that we have a certain sequence of Level2 values ​​— large vectors, where each price level has its own volume (gangs). Let us number the elements of this sequence from zero (current), towards the past.
Sranivaem between themselves Level2 [0] and Level2 [1]. If Bid [0]> = Ask [1], then all gangs from Level2 [1] _Ask that are not higher than Bid [0] fall into the T & S. Similarly for the situation Ask [0] <= Bid ​​[1]. In other cases, nothing gets into the simulated T & S.
This algorithm is easy to check for ling. links to official T & S data exchanges. Then, to introduce some improvements through uneven distributions across gangs, etc ... In general, the topic is always delving into the rough model above.
Vector_Ask [0] = Level2 [0] _Ask - Level2 [1] _Ask. This vector adds up all negative values ​​from the best (for the price) gang, until the first non-negative gang is encountered. This amount is entered into T & S [0]. With Bid - likewise.
It is more difficult here, because It is necessary to combine two almost contradictory methods for ECN and STP. I will not paint.
Applicability in the MM-algorithms already described. Also, for example, it is possible to estimate price levels of accumulations of serious volumes and analyze their impact on distant (seconds, minutes, hours) price targets.
On FOREX, in order to evaluate the relationship between all symbols, it is sometimes reasonable to use Level2 not of symbols, but of currency pairs (to build a synthetic Level2).
This is the name of the rule (algorithm) for the distribution of the filling (when executed) of limit orders located on the same gang.
For example, on the best gang there are several applications at once, the total volume of which is larger than the incoming market order volume from PriceTaker. It is clear that PriceTaker will receive the full volume of execution at the price of the best gang. The question arises, how to distribute this volume between applications that were on the gang?
And here each system has its own vision of how to do it right.
This is a classic allocation rule, when, at the time of execution, all applications have an execution rate corresponding to that part of the volume that the applications contribute to the total gang volume.
This means that it does not matter if you break your limitnik into pieces, in what sequence these pieces are sent. The performance will always be the same.
The time for setting the limiter dominates here. Limits are poured in priority: until the previously set limit ones are filled in completely, other limiters will not be poured.
With this allocation rule, PriceGivers try to set their limiters ahead of the rest to increase the likelihood of their execution. As a result, this causes a narrowing of the spread and a decrease in volumes on gangs, since if someone submitted their request for a new gang (reduced the spread), then the other participant does not want to join this gang, because anyway, he will be an outsider in front of a smarter one.
When submitting an application for a new gang, it is sent either in full volume, or broken into pieces in descending order, putting them in the same sequence in the queue. This allows, if necessary, to remove part of the volume of its application (according to the stack principle), without losing the priority of execution.
The term composed on the go. The priority is not on the time of issuing, but on the volume of the application (or another custom feature - Weight).
This allocation encourages PriceGivers to fill the best gangs with maximum liquidity. For PriceTakers this is one of the best rules.
Also the term composed. This is a modification of Pro-Rata, where applications are no longer given equal distribution weights, but depending on the Weight attribute.
For example, any distribution (mat. Term) by the time for issuing applications can be used: the earlier it is set, the higher the weight. In particular, the use of mat is common. linear distribution of time (Time Pro-Rata).
The same is done for Volume and even for more complex features (for example, Weigth = TimeLive * Volume).
With this approach, various mathematically justified methods of optimal bidding are applied — maximizing the probability of the highest in the total amount of execution of applications from a market participant. Those. there are algorithms, how to break an application into pieces and in what sequence to send them to the gangs.
Each site comes up with its own rules of allocation, or uses the ones that have already been developed. Depending on the goals that she pursues. It is clear that the main goal is to increase profits, i.e. increase in turnover. And here comes the nontrivial question of choosing an allocation algorithm. In order for PriceGivers to motivate and PriceTakers activity does not diminish. It all depends on the base of the existing clientele of the site and development prospects.
Most often, do not bother and choose simple options: Pro-Rata and FIFO.
The only algorithm where there are no collisions with the minimum allowable volume (lot) is FIFO-variants.
When in the T & S-data there is a classification by directions, you can talk about the general indicators of trade PriceTakers, or PriceGivers.
We will assume further that the mentioned direction concerns PriceTakers. This is actually a convention, because PriceGivers-directions - just the opposite sign.
Recall that each element of the T & S sequence is data on a transaction made by someone from PriceTakers: price, volume and its direction.
This means that we are quite able to build the dynamics of change in the total Equity of all PriceTakers for each character individually and, accordingly, for any set of characters. Moreover, on FOREX, considering all the crosses and majors at the same time, you can build the dynamics of change in the total currency profile of PriceTakers: how much what currency is “on hand” at PriceTakers.
There is such a thing as Level3 - this is complete information about the current state of each market participant. Roughly speaking, this is a well-guarded insider. However, as mentioned above, we can get close to it by calculating Equity_PriceTakers.
Let me remind you, the actual purpose of PriceTakers is meat for PriceGivers, among which there are very strong market participants. Those. self algorithmic pricing serves to enrich PriceGivers by draining PriceTakers.
So PriceGivers will do everything to make Equity_PriceTakers decrease. Therefore, the use of such an easily calculated algorithmic insider seems reasonable from the point of view of the nature of modern electronic finance. of the market.
Each exchange works on some trading engine. Suppose the owner of the exchange realized the serious imperfection of the current engine that threatens (or retards) the business. He faces the task of writing a new perfect exchange engine.
It may well be that the new engine is incompatible (it happens) with the old one. How to transfer the exchange to a new engine?
The problem is that the old engine has all the current liquidation. And if the old just turn off and give a new one. Then there will be practically no new liquidation (killing of the stock exchange) - all cannot instantly be reorganized.
Therefore, make the ECN / STP-scheme of the new engine. When the old engine is connected via STP to the new one as the only LP. And the new one also gives its internal ECN.
This allows you to trade immediately through two engines.
After a more advanced engine begins to gain popularity (because it is more profitable to trade through it - less trading costs, more opportunities), liquidity from the old one begins to flow into it (it gets into the internal ECN of the new engine).
As a result, from the old engine, the liquidation virtually all goes to a new one. After that, the old one closes completely. And there is only a new one without an external LP. Those. it turns out an updated stock exchange (ECN) - absolutely everything on the new perfect engine.
It is important to understand that the exchange engine (or the engine of any other trading platform) has nothing to do with the trading API. Those. no one but the exchange itself is able to influence the work of this internal engine directly. This can only be done indirectly.
For this, the owner of the engine writes bridges between his internal engine and the external trading API. Most often limited to only one such real trading API - FIX API (ALMOST obligatory - standard). But there are cases when there are several real trading APIs.
Trading platforms with their own trading APIs are not real trading APIs of the platform, but their wrapper: sometimes more high-level, sometimes less functional, but more simple in work. For example, in almost all trading platforms, the available trading APIs are actually wrappers. Therefore, trading through them increases trading costs due to latency wrappers.
There are a lot of trading platforms. Not only cointegrated FIs can bargain on some of them, but generally the same FIs. Those. The same FI is traded at different sites.
In order to absorb these sites (pour all their liquidation on their site), you need to create an ECN / STP aggregator, where all the victim sites act as LPs. This method creates a new trading platform with more favorable prices - dumping prices.
Thanks also to competent marketing of the promotion of a new site, algo-takers, then algo-givers, then takers, etc. go to dumping because of dumping. Pouring liquidation like a snowball. From a certain moment the process becomes irreversible. And the victim sites are completely absorbed.
In the examples above, you can see the main historical significance of ECN / STP schemes. They serve the purpose of centralizing the market, more efficient pricing, increasing liquidity and improving prices.
Matching is the previously described convergence of two multidirectional trading applications within the ECN. Note that STP execution of requests is not a match.
In any ECN (stock exchange) there is the possibility of a statistical classification of the conducted match into groups of market participants, within which the greatest turnover of the match occurs. For example, if two market participants, thanks to a match, transfer money to each other through mutual matching, then a similar classifier will separate them into a separate group.
Any ECN is strategically beneficial so that it has as many brokers as possible (independent customer databases). This is to reduce the likelihood of creating a hypothetical competitor.
A competitor is born most often through self-identification of brokers, which occurs as follows. Relatively large brokers classify the match of their customer databases. If it is possible to identify large groups of such clients within their databases, then an obvious thought arises: why should these clients pay ECN for matching among themselves, when it happens all the same between broker clients? Of course, this is a pragmatic question.
The answer is to create your own ECN / STP engine, where ECN serves to match intra-broker clients, and STP to execute on an external ECN (to avoid confusion, we will call the external ECN exchange later in the text (most often it is)). This solution gives several advantages (and disadvantages) both for the broker and for his clients.
The broker already has its own trading platform, which in relation to the stock exchange has its own name - dark-pool. Accordingly, the commission from the match of its customers now goes to the owner of this platform, and not to the owner of the exchange.
Broker clients sometimes get better execution (than on the exchange), if it occurs within the ECN, because The latency (technical / computing time delays) of the exchange for broker clients is always higher than the latency of the client database of the broker’s clients.
But there are ambiguous moments for customers. If an STP transaction is carried out, then the guaranteed execution of clients is no longer available. Those. FillRate (workmanship) <100%. As it was already discussed in detail earlier, this is a consequence of the fact that customers' requests in the STP scheme are no longer visible in the Level2 exchange. Those. exchange clients cannot respond to them. Roughly speaking, dark-pools clients become just PriceTakers (not PriceGivers) for the exchange.
However, they leave PriceGivers right inside the dark-pool, and this is not enough. Since We do not forget that before the creation (and its cause) of the dark-pool, there was an adequate self-identification described above. It should also be said that the dark-pool allows you to give its customers anonymity before the exchange, because All STP transactions now come from only one name — the name dark-pool. Moreover, because of its isolation with respect to the exchange, dark-pool creates additional protection against highly aggressive priceGivers (MM-algorithms and other HFT-representatives of algorithmic trading), which not without reason attracts many market participants to go through trading from exchanges to dark-pools.
According to the previously described absorption scheme of exchanges, a large dark-pool is able to absorb the exchange and actually begin to fulfill its role. In this case, the regulatory burden of the exchange does not apply to dark-pools, since they are practically pure electronic (virtual) formations. Of course, the regulatory paw in the future will not bypass them, as it always happened when something new gained serious weight.
As you can see, the creation of dark pools is a natural evolutionary process taking place within the market — its decentralization. The non-zero probability of decentralization causes centralized exchanges to improve their trading conditions. Those. decentralization for exchanges is a kind of mechanism that increases the effect of this type of market. However, there are negative aspects of excessive decentralization, which were shown on the example of FOREX. There, the same ECN / STP scheme, paradoxically, serves to centralize the market and to improve the trading conditions accordingly.
Those. through ECN / STP formations, one can decentralize the market by creating dark-pools, or centralize it — creating high-level aggregation from the same dark-pools, in particular.
The best trading conditions are somewhere in the middle between the paradigms of decentralized and centralized markets.
Any algorithmic trader is faced with the task of translating the tester robot into a combat state, ready to work in the real market.
In fact, there is only one way to competent translation. Fortunately, it is almost universal.
The combat robot is divided into two parts: a tester and a synchronizer.
The tester shows the current trading history of the tester robot (up to the present).
The synchronizer coordinates this data with the current real trading environment, trying to fit it with a virtual one (obtained in the tester).
For example, the virtual environment shows that now at such and such a level is limitnik. The task of the synchronizer is to make it so that at this price level there is a similar limiter on the real market.
Universal synchronizer can not be. But there are only two fundamentally different approaches to synchronization.
The classic version is implemented in one form or another in all signal services that are so popular now. Such a scheme is beneficial services, because The synchronizer is as simple as possible, and they are not very concerned about the clients' trade costs.
The second scheme, as far as I know, is not applied anywhere. Perhaps one of the very boring algorithmic traders in his home realized ...
Above said what costs are so fashionable now signal services using a primitive classical synchronization scheme: through a market order.
It is worth mentioning one more minus of such synchronization - the manipulative component. The technique described below can remind someone layering . But still it is different. I will say in advance that the following only makes sense for ECN (stock exchanges) and ECN / STP platforms.
Imagine that you are a supplier of signals (preferably free) and you have subscribers (there will be more of them then), the size of which is higher than yours. Then from time to time you have the opportunity to include a simple transfer of money from the accounts of your subscribers to your master account:
Expose BIG BuyLimit and SellLimit to the inside of the spread and make a “trigger” - a small SELL (sell yourself - BuyLimit). This causes an avalanche of SELL orders from your subscribers. Your BuyLimit starts to fill up, i.e. grow BUY-posture. What causes an avalanche again, but already BUY-orders from your subscribers. Those.Your SellLimit starts to fill up.
As a result, having caused similar recursive avalanches (the basis of which still lies in the latency of signaling services), you receive an overflow of money from your subscribers to your master account. This improves the performance of your master account, attracting even more subscribers.
Of course, it is not necessary to put limit account on the master account. This can be done on your personal friend account. Then get overflow on him. But the master account will have a small drawdown, which may alienate subscribers.
There may be other options for such frauds, but their core-idea will be unchanged, as written above.
MM algorithms are almost impossible to synchronize profitably through the classical scheme. Therefore, it is advisable to refer to the previously mentioned alternative synchronization scheme.
The market is characterized by the presence of various kinds of patterns at each point in time. If we denote N (t) as a function of the number of patterns depending on time and plot it. Then failures will be immediately apparent - a sharp decrease in the number of market patterns.
These failures almost always correspond to important events,> 90% of which are scheduled - economic news on a schedule.
When searching for market patterns, it is reasonable to discard similar periods of N (t) failures, both planned and force majeure, from analysis. Periods of apparent absence of the desired patterns are also thrown away - a special automated filter.
Such actions can significantly increase the likelihood of finding patterns.
Therefore, during the trading of the market patterns found, any trading activity stops before the release of the planned news and some time after.
Thanks to sergeev , from whom it started to start and systematically continue writing this educational program.
At the moment, all markets are automated. For this reason, some economic explanation of pricing are certain rudiments. Rulat algorithms + some manual intervention.
The task of each trading algorithm is always the same - to bring money to the owner. The algorithm is the better, the more money he is able to bring.
Market makers
Among the algorithms on the market are the so-called market-maker algorithms. You can explain it on your fingers from a simple example to a more complicated one:
')
Imagine that you have a task to create a new symbol for trade. Let there be people who for some reason want to trade it. What is required of you? You need to form your character from your Level2 applications at any time. Those. fill the symbol with prices and liquidity. At first, you can make a completely stupid MM-algorithm - Level2 does not change. Ie the client bought or sold, after which you added liquidity to the original Level2. Obviously, such an algorithm will always give the owner money. But the problem is that people are not complete idiots, and they will not trade on a constant symbol - there is not even the potential to earn them.
So you need to somehow make people make deals. The next dull MM algorithm can be a simple sine wave - Level2 goes up and down. Many people will also merge, but there are geniuses who will see the pattern and begin to bend the owner of the MM algorithm.
It turns out that you need to come up with something so that the MM-algorithm has the maximum difference between merging and earning. Here and begin to develop various mat. models that are far from trivial.
Well, in the market, of course, there are many MM-algorithms with different owners. There are large owners (non-sick banks) who also have an insider - they know which traders (because they are their clients) where they stand and how they traded. Therefore, mat. models are well thought out. But the task is always the same - to squeeze out as much money as possible from the meat (simple market participants).
Those. You understand that even you can be a market maker, you only need a good MM algorithm and the solution of some organizational issues.
Let's try step by step to model on your computer the exchange (the simplest option) closed market (from one FI).
Initial data:
- Thousands of robot traders.
- each robot has the same initial capital.
- there is no price and, accordingly, its history.
- no trading costs (commissions, etc.).
How to launch thousands of robots so that they start trading among themselves?
Let's set the initial level (not the price) of the average price - unit. First, we launch robots that immediately submit limit orders. The formation of the Bid and Ask price history begins. For a while there will be no deals, but prices will move along any trajectory.
If the trajectories (two) are horizontal lines, it will mean that the market is completely dead. To revive it, we will launch robots, which will bring the trajectories out of horizontal position. Here we may encounter the fact that the trajectories endlessly rush to one of the sides. So it is necessary to set (not necessarily explicitly) some boundaries of the trajectories. Now we have a more or less tolerable story. At the same time, no deals have been made yet.
We launch robots that, on the basis of a formed history, do their dirty work - they trade. Send deals. Robots that put up limit orders can merge. Then the price will disappear and everything will stop. It will be necessary for certain such robots to give a disproportionately high capital (this will significantly increase the initial (50/50) probability of earnings), compared to the others. Let's call these robots MM robots. And in their algorithm we lay the guarantee of the presence of their applications. Is there a possibility of merging MM robots? Of course have. So you need to somehow guarantee the absence of mergers for MM-robots.
You can go in two ways. Introduce such trading costs so that they cover the slow discharge of the MM robot. Either get insider trading on other robots and, on the basis of it, model and change the price so that there is a positive MO. It is logical to do both.
We introduce initial trading costs for all robots for trading operations, and transfer them to the accounts of MM robots. Initial costs do Koef * MO MM robots. Of course, Koef> 1. Inside also learned how to use, so MM robots slowly merge the rest of the robots.
Is it possible to make robots trade among themselves indefinitely? This can not be done, because Certain robots will accurately merge, retiring from the game forever. The profit from the drain will be redistributed among other robots, i.e. the average capitalization will increase over time, and the number of market participants will fall.
How to avoid it? There is only one way - the introduction of new robots with new capital. So it is always required at a certain stage to feed our market with new money and new robots.
Well, sort of breathing ...
And here come the people to whom we all show and convince that this is all real. They bring in money and start trading. Can human trading actions be modeled by a robot? It is difficult to say, because There is no limit to perfection, but you can definitely create a behavioral model of a person with a high degree of coincidence. It turns out, again we get to the stage of the presence of only robots. So our market will breathe and live even with people.
What simple conclusions can be drawn from even such a primitive that was voiced above?
- The market is a tool for withdrawing money in favor of MM robots.
- The market is dead without new participants and their new capital.
- The market is dead without insider trading costs.
- Prices are formed on the basis of insider trading.
Area-forming exchange algorithm
It seems, it became clear that everything is kept in our automated time on algorithms. There are many types of them. Let's try to consider a purely technical algorithm for creating a trading platform. The simplest algorithm of this type is the exchange one. About him and talk.
So, there is some kind of symbol that will be traded only on our exchange. And there are many who want to trade it. And this means that there are ready-made MM-algorithms and meat, without which everything is completely meaningless (besprofitno).
The exchange algorithm is purely technical, i.e. brings profit to its owner by the fact that everyone uses his results, paying a commission. In this case, even a negative commission can be embedded in the algorithm, for example, for MM algorithms. The commission grid is again a kind of uncomplicated mate. model.
So, there is Level2 (FOREX terminology, it is also called MarketDepth on the exchanges) - these are the levels: the purchase / sale price and how much (volume). Each such level (price + volume) is called a gang. The best gangs for sale and purchase are called Bid and Ask (some call Offer, but this is just terminology).
Level2 is formed from client requests (limitnik) - this is MM-algorithms + some more types of algorithms and meat. Spread is the difference between the current Ask and Bid (in particular, for this reason there is a very inaccurate phrase in its wording that MM earns on a spread).
Any limitnik exchange algorithm almost always puts on the appropriate gang in the glass. It is for this reason that by placing the limitnik inside the spread its corresponding narrowing occurs. I will not explain it on my fingers, the algorithm is simple (you can google or ask).
If a limitnik is placed worse than the current corresponding price (for example, SellLimit <= Bid), then the exchange execution algorithm does not execute at a price worse than the market price, but begins to lick (by volume) the gang from the best towards deterioration until the entire limitnik is filled, until the gang runs out to the level at which this limitnik is set.
Of course, there are sometimes secret moments in the exchange algorithms, when information about your application, before getting into the glass, goes, for example, to the MM-algorithm. And in advance (ahead of time), he simply acts so that you do not get a positive slippage from the limiter at a price worse than the current price, and take it as a net profit for yourself. This is some of those. insider, which, of course, is bought, i.e. has a very clear price tag (they rule the money). Do not forget about the main task of any algorithm - money. Therefore, there should not be anything surprising - the truth of life.
On exchanges during a trading session, a pre-set limitnik is executed in 99% of cases exactly at the price - without slippage. This means that the price to cross your limitnik, it must be executed.
Limits in Level2 are covered by market orders. Types of orders is full, as the exchange wants (prescribe). Same algorithm. For example, MT5 introduced its own types of orders that the exchange could well legalize - to implement in its exchange execution algorithm. A market order is a derivative of a limit order: the limiter is worse than the current price, and this is almost not measured. That's how much you need to execute the market - so much you get, but only with good negative slippage. The main users of market orders are meat.
Obviously, the price can move along almost any trajectory without making deals - we look at the MM-algorithms. If the transaction is made, then its price and volume is called Last. And this infa is also broadcast by the exchange. Last-Data Flow is called T & S (Time & Sales).
Some exchanges (far from all) add a conditional flag to Last-Data - the direction of the transaction (Buy or Sell). This marker is a kind of classifier: for passive and active participants. Those. on those whose limitnik is full, and those who filled it. Obviously, this classifier is a temporary filter: whose execution of the two sides of the transaction was initiated by a later / early trade order.
It is clear that the exchange algorithm must be consistent - i.e. a queue is formed from applications and they are processed sequentially. But there are variants of the so-called snapshots, when applications accumulate for a relatively short time, and then are issued at level2 times with the corresponding execution.
Execution of limit orders on the exchange
The correct exchange algorithm does not allow the pricing of the public situation Bid> = Ask. In the algorithm itself, as applications are received, at the initial stage a glass is formed, in which situations are often Bid> = Ask. In such a situation, the executive part of the exchange algorithm is turned on, whose task is to resolve this situation to the Ask> Bid state. And only after settling the already formed glass with the correspondingly formed Last-data becomes public - accessible to all.
SellLimit is always executed at the Bid price, BuyLimit is executed at the Ask.
But only these Bid and Ask are non-public prices of the initially formed glass, as described above.
If you expose SellLimit - this is the desire to sell you, which is equal to the offers for others to buy from you. For this reason, SellLimit falls into Ask Gangs. For example, if you put SellLimit inside the spread, then the best Ask-gangs are formed with the level and volume of your limiter. Those. putting inside a SellLimit spread, you change the Ask price. If someone wants to buy at the Ask-price, he will flood your limitnik. To say in this case that SellLimit is executed at the Ask price, or it is executed without a spread - a very vague wording. It is better to simply understand the mechanism, as elsewhere.
I will give an example of execution. You put SellLimit inside the spread, which means Ask equals SellLimit. Now you set BuyLimit equal to Ask. In this situation (see the first paragraph) in the exchange algorithm, it turns out that Bid is equal to BuyLimit. Those. in the end, it turns out that Bid = Ask. Everything went to resolve the situation, until Ask becomes more Bid. Until the algorithm resolves, no one will see the correct glass. For simplicity, let the volumes SellLimit and BuyLimit equal Vol. It turns out that both limitik collapse, Bid and Ask become equal to the next best gang in the glass, i.e. Ask> Bid. Next, the Last-Data contains the execution price, which is equal to your SellLimit (== BuyLimit), the volume Vol, and the BUY direction (because BuyLimit was sent later to SellLimit).
Note that if in the same situation you first send BuyLimit and then SellLimit, then the result will be the same - you buy / sell yourself, losing on a double commission. But only in Last-data the direction flag will be opposite - SELL.
Returning to the question of the price at which limitniki are executed:
If you look at the bars of a small TF on some weakly-liquid symbol, you will see that the Bid bars are cut from below (BuyLimit), and the Ask bars from above (SellLimit).
Consider again the situation SellLimit. In the sidebar tester, SellLimit is executed only when its HighBid> = SellLimit. Please note that HighBid (as well as LowAsk) is practically not cut on exchanges. And running ahead - not at all trimmed to ECN / STP. Those. if you need to test the strategy with limitikami, then for you the main information on the execution of SellLimit is the value of the Bid-price, or rather its High. This may be another argument in favor of the assertion that SellLimit is executed at Bid prices.
Distracting a bit, we can say that from the same considerations ZigZags are built with vertices on Bid-data and low-lying on Ask-data. And it is on the basis of such a construction that the maximum potential profitability is estimated.
Types of traders
Classification of traders can be engaged indefinitely. But among prof. market participants (brokers, platform developers, etc.) are divided into two types of traders: clickers (GUI-clickers) and algorithmic traders. The names fully speak about their features. The division into experienced and novices (they also use "noobs"), as a rule, is not conducted.
If the task is to attract traders, then the clickers are the easiest to attract and algo-traders are much more difficult.
A nice nice GUI is made to the clickers, secondary attention is paid to the trading conditions. For this reason, in particular, clickers are on average meat (a source of profit for other market participants). The competition for clickers is high.
Algae traders are very picky about trading conditions, the GUI is not decisive. The competition for algorithmic traders is low, although serious efforts are spent on it.
Mat. The expectation of all algorithmic traders is much higher than the mat. waiting for clickers. The number of algorithmic traders is much less, but the turnover is higher.
It so happened that the opinion of algorithmic traders listen to a much lesser extent than the clickers. Platform developers suffer this, and brokers suffer the same. The opinion of algorithmic traders can even extend to how, for example, the platform is sold to a broker - for a commission on turnover or a monthly fee. Those. All nuances of possible changes in trading conditions are taken into account.
Most often, market participants are set to satisfy only one type of trader.
Obviously from what was written earlier, market makers are algorithmic traders.
Toxic stream
This term is used by market makers. Toxic flow is a systemic profitable flow of trade orders. It comes from the very ones that make money on the imperfections of the MM algorithms. In practice, 99% of the toxic flow falls on algorithmic traders.
Brokers work patterns
- Marketing.
- STP.
- ECN / STP.
All three models are absolutely market - it is very important to understand, because no fair market or dishonest. Everything is one and intertwined.
The first differs from the other two in that it does not earn on commissions from the turnover of customers, but on their discharge. Let us analyze everything in one real example (all the data are public), names are missing.
A certain broker declares that his turnover is> $ 100 billion / month, while his earnings are $ 80 per month. Obviously, this is a market maker, because even the most inadequate STPs do not earn more than $ 65 per mio, not to mention such a high monthly turnover.
It turns out that $ 80 per leman is the rate of discharge of their customers. This means that with an average lot of 0.5, the average client makes a deal with a profit of ~ -0.5 points of EURUSD.
If you dodge, then the average trade in a lot / pip can be approximately calculated by the ticket in MT4. Yes, many tickets will go to canceled deposits and I / O operations, but this is no match for the number of market orders that are clickers' favorites.
We think further, once a monthly turnover> 100 yards, it means MoneyIn - MoneyOut> $ 8 mio. Those. a broker at the sink of their clients monthly earns $ 8 mio.
The question is where it is possible to find a monthly amount of hamsters. And do not forget that MoneyOut is on average about two times smaller (he invented) MoneyIn. Those. the broker monthly attracts about $ 16 mio new live money.
How he does it?!
The average hamster is somewhere around $ 200 (also invented by himself, never heard from anyone). So even by the most conservative estimates (MoneyOut = 0), you need to strip 40,000 hamsters a month.
In fact, hamsters for that are called that they merge many many times. And carry their average $ 200 on a schedule - with each RFP over the years. This is a payment for the regular pleasure of feeling like a clicker. Just as they spend every month on movies, beer, etc.
If a toxic stream is suddenly found among hamsters, it is quickly and easily detected, after which it is strongly recommended to switch to the STP-type or close the account.
This type of broker actually benefits other types of brokers (STP and ECN / STP). These brokers are not market makers competitors. They even do a great blessing: they produce a clear separation “ECN for pros” and “Standard for beginners”. Since this allows the customer base on the Standard to get rid of toxic. It is even advantageous to them that people go to others, because toxica in the end less. And you can earn more with less risky as a result of MM.
STP and ECN / STP can experience serious problems with a high proportion of toxics. They need the clickers at least in order to dilute the toxic in front of their LPs, which are almost always reduced to large market makers. And they act in the image and likeness of small market makers (as described above).
The fact is that on the FOREX at this stage the MM algorithms of large owners - banks prevail. If you make money on FOREX, then it is almost always a loss of these banks, more precisely, of their algorithmic departments, in which millions of budgets are spent on something, in particular, so that you do not earn.
To profit from the drain of another simple trader can now basically only indirectly: the trader will merge the MM algorithm, and you will throw up this profit from him. To put it bluntly, making money on FOREX is robbing something stolen.
But all this sounds strange only if you look through rose-colored glasses at many events taking place in the world. For a long time, the market (stock or FOREX) has become a legal tool for taking money from the public. But it is not all that bad.
Let's go back to our MM broker. Does he have a grain scheme? Not really. The fact is that a MM broker is able to earn even a lot more than $ 80 with mio at a very volatile and trending market. But go into a decent minus in the long-sluggish market. These are the predictable features of the average hamster trade. Those. The brokerage MM model is a risky model, but at the same time it can be very profitable under favorable market conditions.
In order for the MM-broker to refuse the MM-model, he needs to believe that the STP-model is able to give at the commission not much less. This means that he needs to increase at times the momentum. It is practically impossible to do on the huge base of clickers, since turnover is done by algorithmic traders. Algo-traders will only go when trading conditions are among the best. Well, this in turn can occur with serious financial investments in STP-infrastructure.
Moreover, if there is such a huge herd of meat in the form of clickers, then why donate them to the snickering MM-banks, receiving only a commission from their drains. If you can take everything completely yourself. In general, such simple reasoning makes it possible to understand that a MM broker with a huge meat base becomes a hostage of himself.
There can still be ridiculous, if the base has some kind of national mentality. For example, the number of Russian hamsters is reduced by an order of magnitude if you ask them to fill in a little more fields in the questionnaire, as regulators require. In many ways, this forces MM brokers to go offshore.
But again, not everything is so bad ...
Quanta
Quanta are, as a rule, educated people with an analytical mindset. Unfortunately, in life it happens that a good education, possession of exact sciences, etc. it is not enough for success. Usually, the more effort spent on learning something, the greater the importance attached to it. Among scientists there is a huge number of mediocre personalities who cope well with the career features of their profession. These are publications in journals, participation in summits, lectures, etc. One of the near-science themes is a comprehensive study of trading activities. Everything is done as taught. Those. vlob applied mat. apparatus, associations with physics are built, various market models are created, etc. Such scientists are called quanta. A large part of them are careerists. But there are exceptions, as in any case.
Cointegration
You can direct a different quantum heresy by giving the term cointegration precise mathematical definitions. But all this has little to do with the practice of trading. For a simple explanation of cointegration is possible. Imagine that you have a lot of trade symbols. If there is a possibility (the quantity of each symbol) to buy / sell these symbols in such a way that the profit fluctuates around some not very variable value, then such symbols are cointegrated. Of course, quite childish, but the essence conveys by 90%. This combination of symbols is called a cointegrated portfolio. A classic example of such a portfolio: an asset and futures on it.
Arbitration
Any portfolio has Bid and Ask-prices. Moreover, it has its Level2 (easily calculated from the Level2 characters included in it). If there is an opportunity for a cointegrated portfolio to make a deal so that after it is closed there is a positive profit, then this possibility is called arbitrage. Such a co-integrated portfolio is called an arbitrage portfolio (it has just been invented).
The lower the trade costs (quality of execution, commission), the more chances the cointegrated portfolio will become arbitrage.
Markap
Markup - price deterioration (Level2) by a certain amount. For example, a broker may lower prices to make a profit. It can go to the marketing step - resetting the commission (clickers do not like difficulties), but entering it into a brand. There are also positive (profitable) markups for a trader, but their understanding is possible only with a good understanding of pricing, so for now let's miss.
We are trading arbitration
Suppose there is a desire to do arbitration. To do this, you need to at least create a cointegrated portfolio. The simplest cointegrated portfolio consists of two symbols of the same name: one for one broker, the other for the other.
Take, for example, so popular EURUSD and give the symbols for convenience the corresponding names: EURUSD1 and EURUSD2. The most important point that needs to be fully realized is that EURUSD1 and EURUSD2 are completely different characters.They could even be called brokers in a different way, to have (by an order of magnitude, for example) different prices and other differences. Only one thing is important - they are cointegrated. But for simplicity, we will consider the elementary case: EURUSD1 and EURUSD2.
Before comparing prices, an algorithmic markup is made on them in order to bring in them all possible trade costs (quality of execution for each broker and commission for each broker). We will further assume that all prices are already marked.
So, in each broker you have trading accounts with certain money. If it is very primitive to look at arbitration, then it is required to find the moments Ask1 <Bid2 and Ask2 <Bid1. And in these moments open / close opposite positions in each of the brokers.
This is the simplest and frontal implementation. Let us make a small digression towards a more generalized and universal vision of such trade.
In this case, the co-integration of the portfolio suggests that Synth = EURUSD1 / EURSD2 hovers around one. This Synth has its own Synth_Bid and Synth_Ask (Synth_Level2) prices. If it is possible to build a ZigZag with vertices on Synth_Bid and lowlands on Synth_Ask, then our portfolio Synth is arbitrage. But this is a distraction.
Let us return, nevertheless, to a more familiar for the majority look at the trade. In fact, in some cases it is justified to create something high-level for the convenience of trading. And for arbitration this high level is done like this:
They take the masked Level2_1 and Level2_2 and simply combine into Level2_All, to which the created artificial high-level symbol EURUSD_All begins to correspond. Very simple trading functions are written that are able to trade EURUSD_All. For example, if you want to sell EURUSD_ALL, then OrderSend (EURUSD_All, OP_SELL) sends a SELL order to the broker who has the highest bid price, i.e. his bid price is on the best gang in Level2_All.
Here we must now say a few words about Level2_All. In his internal representation of the gangs now contains not only the prices and volume, but also the name of the source of this data.
With this implementation, you only need to wait for the situation when Ask_All <Bid_All and at this moment simultaneously open multidirectional positions on EURUSD_All. As a result, receiving high-level profits and the lack of open positions on EURUSD_All. Convenient, isn't it?The adviser in such a high-level language would occupy 10 lines: he saw negative spread, trade it, wait further.
If we descend from a high level of vision of such trade downwards, then we will notice that at the moment when we do not have positions on EURUSD_All, we will have an open position on EURUSD1 and opposite to it on EURUSD2. This in turn will cause the natural distortions of Equity1 and Equity2. Yes, roughly speaking, Equity_All = Equity1 + Equity2 will grow as trading progresses, but we know that Equity1 and Equity2 must be at least positive. And our distortions may well simply reset the account on one of the brokers, although the other will grow.
How to solve this problem? The first thing that comes to mind is to pour money from one broker to another. Those.order the withdrawal of funds from the broker, where more money and send them to where it is less. But it is long, very long. Moreover, it is also extremely expensive - bank and other types of transfers are not free.
But imagine that someone appears who says that all distortions will level for a small fee. This one is clearing.
Simple aggregator
It was easy to see how convenient to trade high-level. It is convenient to trade not only arbitration, but generally any strategy, since EURUSD_All prices based on their construction can not be worse than EURUSD1 and EURUSD2. Those.the artificial symbol is more profitable and the profit on it is obviously higher. Here comes the idea of ​​creating such artificial high-level symbols for any kind of strategies. At the same time, by the same principle, to take not two sources (brokers in the example) of cointegrated symbols, but any number of them. Obviously, the more sources, the more profitable (the prices are better) high-level symbols.
Such high-level formations are called liquidity aggregators. Obviously, this algorithm is a kind of software that allows trading this way. Among the companies participating in the life of the market there are software companies that deal with exactly such aggregation algorithms. By giving its customers the opportunity not to bother with low-level trading, but to trade at a high level, wasting their intellectual powers on other things.
Among these software companies, Currenex and Integral are best known. They sell their software for a commission on turnover. For this reason, in particular, some write their own aggregators to be more competitive.
Creating a trading platform
I wanted you to create a trading platform with little effort. Very rude it looks like this. You come to a well-established software-aggregator company and declare your desire. Those answer that without problems, but it is necessary to fulfill some conditions. To begin with, you must have agreements with source companies that you want to aggregate. Next, you need one single account with a prime broker who already has agreements with your source companies and is able to organize clearing so that there is no imbalance. Clearing is also part of the commission.
Further, the aggregation software company adjusts its aggregation software with your sources and prime broker, with whom it has contracts for the possibility of such activity, and gives you a turnkey trading platform.
Note that the sources may be the same aggregators, their sources are also some aggregators. All of this can end up intertwining and repeating, which sometimes causes an overestimation (duplication of sources) of volumes on some bands of your Level2_All. And, of course, at the very ends of this network are banks with their own MM-algorithms.
PriceTaker and PriceGiver
Market participants are always divided into two categories: creating (PriceGiver) market offers (prices and quantity) and agreeing (PriceTaker) with them.
The market offers themselves are bids: ready to buy / sell so much at such and such a price or better. The combination of these applications (Level2) conditionally shows the current market liquidity.
Liquidity provider
Liquidity Provider (LP) - this is how the aggregator names its price sources, which it combines into one single virtual (high-level) Level2. Obviously, LP is PriceGiver. A significant share of PriceGivers is occupied by MM-algorithms.
STP
STP is an aggregator where the customer is allowed to be only PriceTaker-ohm. This means that customers are not able to form their applications for which someone could agree. They can only accept offers from PriceGivers.
As mentioned earlier, the aggregator is a kind of high-level virtual trading service. Therefore, all limit orders of STP aggregator clients are virtual and are stored on the aggregator server.
As a result, if someone takes an STP aggregator as an LP, he will not be able to trade with his clients. Almost all MM algorithms running on an STP aggregator will not give a positive profit.
Simple STP aggregator
In a simple STP, all virtual client trade orders come to his PriceGivers as market orders. For example, the client has SellLimit. As soon as Bid> = SellLimit occurs, a market order SELL is sent to the LP to which the Bid corresponds. The response of LP to such a trade order is the result of the execution of the client’s SellLimit.
Obviously, with such an implementation, slippage occurs, the average of which is less than zero. Those. customers incur losses.
Other STP aggregator
Here, all virtual client trade orders come to its PriceGivers, as limit orders (a market order is a limiter, at a decent price (determined by the STP aggregator) worse than the current one).
Let us analyze the execution algorithm using the example above:
At the moment of Bid> = SellLimit, the virtual SellLimit is frozen (taken out of the account by the execution system, the client cannot do anything with it) and the same SellLimit is sent to the corresponding LP. Moreover, all these actions do not affect pricing. Those.a bit later, freezing (for example, at 1 ms) Bid may form better (from other LPs), but it will almost (there are nuances) not be involved in the execution of our client SellLimit.
LP, where SellLimit was sent, replies that it executed one part of SellLimit, and did not execute the rest for various reasons - reject. After that, the client receives a high-level open SELL position for the volume filled and defrozen SellLimit for the rest.
The speed and quality of the LP response depends on many factors. The duration of the response can be up to several seconds. There are situations when an answer from LP does not come.
Note that (with such an implementation of STP), the limiters of the aggregator clients do not slip into the negative zone. Moreover, there are frequent positive slippings that cover a significant part of the fixed trading costs - the commission.
Here you can see how much the performance and pricing of exchanges and FOREX differ.
.
, . , — LP. , (FillRate) STP-. ( ) , STP-. , , , .
, STP-, , FillRate, . . Those. , .
, , — .
ECN/STP
ECN / STP - as an aggregator, algorithmically identical to STP. The only difference is that a new virtual LP - LP0 is added to the aggregation: requests from the customers of the aggregator.
PriceGivers also become ECN / STP aggregator clients. This means that customers can actually make transactions between themselves directly.
The greater the turnover is in LP0, the more profitable the ECN / STP aggregator is, since unlike other LPs, LP0 does not require a fee for its use by the aggregator. Moreover, the flow of trade orders attributable to LP0 can in no way be characterized as toxic. Sincethis is already a trade between clients, even if there are MM algorithms among them.
It turns out such a trouble-free LP.
But it must be clearly understood in order to be able to be PriceTaker for LP0-PriceGiver of ECN / STP aggregator, you need to be its client. Those.Customers of other aggregators cannot sometimes take advantage of the excellent (best) LP0 price offers.
For example, you trade in two aggregators. In ECN / STP, put a limiter inside the spread, improving Bid or Ask. But in another aggregator you cannot buy / sell yourself at this price, because they are not visible. Those. you buy / sell, most likely, at a less favorable price than the one offered by your limitnik.
Metamorphosis ECN / STP
The strength of ECN / STP is the ability to exchange liquidity.
Imagine that some STP aggregator took as an LP an ECN / STP aggregator. This automatically indicates that STP aggregator clients become a PriceTakers ECN / STP aggregator, in particular (and other LPs from the STP aggregator list). It turns out that in the example above with two aggregators, you would see your application in another STP aggregator in a STP aggregator. And if it were the best in the final STP-Level2, then you could make a deal with yourself. Those.your application would be available to a lot more PriceTakers, which means that the likelihood of the execution of your application would increase.
It is necessary to understand that each aggregator has its own PriceTakers base, and such a simple technique allows you to exchange these bases, increasing the likelihood of the execution of PriceGivers applications. And it is also profitable PriceTakers, because prices are better.
Of course, if two ECN / STP aggregators mutually make each other their LP, then a serious discrepancy will occur in everyone's Level2 - recursively liquidity will be increased several times. Therefore, ECN / STP aggregators exchange (more precisely, they can) among themselves domestic (LP0) liquidity through certain technical agreements - tags.
Reality.
LP- -. , . LP , . , , : LP (.. ), — . , .. . .
, , FOREX. ECN/STP-, - PriceTeker- PriceGivers. Those. , -. , retail-FOREX USD. — , ( ) STP ECN/STP-.
HFT -
Obviously, the ECN / STP aggregator has some insider information: who and how trades through it. A competent owner of such information creates a classifier of customer profitability. For example, as a result of the analysis, he learns that the aggregator's clients are stably (and at the right speed) poured onto the GBPJPY from 12 to 17 o'clock in each environment. What it means is draining - it means that their money flows into LP, which at that time had the most competitive prices.
This begs the question, why give someone a drain when you can take it away? Some will go according to the gray scheme - they will disable ECN / STP at this time and in fact will turn into a DC with a market-making scheme. But there is an absolutely mutually beneficial white path - the HFT MM algorithm. Let it be ours.
In our example, every Wednesday from 12 to 17 o'clock we put our BuyLimit and SellLimit inside GBPJPY-spread at a minimum distance from the external (not formed by us) spread and follow all the time (here we need HFT - fast dragging of orders) . It improves prices - narrows the spread. And merging customers will make deals at better prices. However, (see the mention of the analysis of the discharge rate), their discharge will flow to us already, since they will make deals with us.
Mutual benefit is that we get a profit, and merging customers lose money at a slower rate than if we did not use the HFT MM algorithm. Well, if the owner of this MM-algorithm is the ECN / STP aggregator itself, then the commission, naturally, is zero for using its LP0. And the client will pay the aggregator in full for the transaction. Those. in fact, for such an owner, the commission is even negative.
Adaptive HFT MM Algorithm
It is not necessary to have the aforementioned insider information. You can get it yourself a simple way - sounding.
To do this, you will need to spend money, of course: a spread is created according to the scheme above, but only with a minimum lot. ECN / STP customers will always make deals with you, thus you will receive insider information. Further, everything is the same as in the scheme above.
Adaptability is that insider information is taken by a sliding window, i.e. adapts (controversial, of course) to the mood of customers and the market.
Note that any HFT MM algorithm will almost never be toxic, since only PriceTakers takes money, among which there are almost no HFT MM algorithms.
It should probably be mentioned that these elementary HFT MM algorithms were invented by me immediately. And HFT exploited MM algorithms are much more intelligent. However, their essence is roughly conveyed.
One of the mutually beneficial options for attracting HFT MM-algorithms
An ECN / STP aggregator is always beneficial when its clients include HFT MM algorithms, since they provide a high turnover. However, for independent HFT MM algorithms, trading costs in the form of commission are a significant obstacle to uncover their full potential. For this reason, a marketing scheme is created, which can have quite a clear mate. calculations when the HFT MM algorithm is given a negative commission. Those.the aggregator for all applications of the HFT MM-algorithm that were executed on LP0, does not charge him a fee, moreover, pays him a part of the fee, taken in full from the other side of the transaction - PriceTaker. A similar scheme was casually mentioned in the description of the exchange algorithm.
HFT MM Algorithms: Exchange vs FOREX
On FOREX, not all ECN / STP aggregators provide T & S data. There may be several reasons for this. For example, hide the momentum and do not explicitly analyze the toxic coming from ECN / STP. However, some ECN / STP aggregators do provide such information.
T & S is available on the stock exchanges, but the data on the classification of transactions (PriceGivers or PriceTakers) do not always have official (stock) origins. Most often this is an independent algorithmic classification.
In the case of both the stock exchange and the ECN / STP aggregator, the correctly classified T & S data shows when PriceTakers / PriceGivers lose / win. In this case, to create an independent simple HFT MM-algorithm is not required (with proper technical infrastructure - speed), even sounding and any analysis of the dynamics of Level2. Since properly converted and analyzed T & S data - the aforementioned insider.
Off-market
Institutional FOREX off-market quotes are not rare. They become a sore point when deals are made on them. And especially for patients when these deals have a high (in absolute terms) profit, regardless of its sign.
For various reasons, MM-algorithms can fail, giving out non-market quotes. In fact, they are quite market, since they offer to buy / sell for them. But in the concluded agreements, as a rule, there is an obvious loophole when the deals made on non-market quotations can be canceled (and often canceled).
Non-market prices are always determined retroactively by comparison with price histories from several sources. If a strong release is compared with the rest, it means an off-market market.
LastLook
The right of lastluk is to refuse execution at the price offered by them. This is used by some LPs in the direction of toxic customers: if a request from a toxic came - not to execute, if from a merging one - to execute.
Because of this, the quality of performance greatly decreases, increasing the number of rejects. They are struggling with algorithmic pricing in some aggregators.
There are sites without blastas, but there with obvious toxicity, as a result, what has already been described earlier: either the site refuses the toxic client, or the LP, which incurs losses from this toxicant.
Trader protection.
An un-licensed profitable trader at the institutional FOREX can easily "strip" through the mechanisms of Off-market and LastLook. Therefore, it is reasonable to choose a place to trade through the most advanced aggregator, which is adjacent to another with a strong legal department.
Those. in addition to the fact that the trader will be anonymously represented in the general flow of the aggregator. He will still receive free legal professional assistance in the analysis of problems with the cancellation of transactions. Free, because legal the department does not uphold the rights of the trader, but the face of the much larger owner of the aggregator, on whose behalf, in particular, this same trader made deals. For this, of course, must be the principal position of the owner.
Virtual aggregator
As already mentioned, the aggregator is a high-level union of cointegrated symbols. Note that there is no restriction on the reality of such symbols.
This means that if there are synthetic cointegrated symbols, then they can not only be created, but also aggregated.
The virtual aggregator does this - it aggregates cointegrated synthetic feeds (price streams) into one high-level (virtual, of course) symbol that can be traded inside one site.
Virtual aggregators have the same property as classical ones - they mask toxicity, create the best prices, increase instant liquidity, increase the liquidity ceiling of the vehicle, etc.
Most often, the creation of virtual aggregators falls on the shoulders of algorithmic traders. The most advanced classical aggregators can also partially do this: aggregate not only the clearly existing symbols of the LP, but also their synthetics.
A primitive example of visualizing the simplest part of a virtual aggregator is to combine synthetic feeds:
The most adequate assessment of any character occurs when analyzing its high-level aggregated namesake.
A rough algorithm for obtaining T & S data from Level2
Imagine that we have a certain sequence of Level2 values ​​— large vectors, where each price level has its own volume (gangs). Let us number the elements of this sequence from zero (current), towards the past.
ECN (Exchange)
Sranivaem between themselves Level2 [0] and Level2 [1]. If Bid [0]> = Ask [1], then all gangs from Level2 [1] _Ask that are not higher than Bid [0] fall into the T & S. Similarly for the situation Ask [0] <= Bid ​​[1]. In other cases, nothing gets into the simulated T & S.
This algorithm is easy to check for ling. links to official T & S data exchanges. Then, to introduce some improvements through uneven distributions across gangs, etc ... In general, the topic is always delving into the rough model above.
STP (preferably a lot of LP)
Vector_Ask [0] = Level2 [0] _Ask - Level2 [1] _Ask. This vector adds up all negative values ​​from the best (for the price) gang, until the first non-negative gang is encountered. This amount is entered into T & S [0]. With Bid - likewise.
ECN / STP
It is more difficult here, because It is necessary to combine two almost contradictory methods for ECN and STP. I will not paint.
Why T & S?
Applicability in the MM-algorithms already described. Also, for example, it is possible to estimate price levels of accumulations of serious volumes and analyze their impact on distant (seconds, minutes, hours) price targets.
On FOREX, in order to evaluate the relationship between all symbols, it is sometimes reasonable to use Level2 not of symbols, but of currency pairs (to build a synthetic Level2).
Order allocation
This is the name of the rule (algorithm) for the distribution of the filling (when executed) of limit orders located on the same gang.
For example, on the best gang there are several applications at once, the total volume of which is larger than the incoming market order volume from PriceTaker. It is clear that PriceTaker will receive the full volume of execution at the price of the best gang. The question arises, how to distribute this volume between applications that were on the gang?
And here each system has its own vision of how to do it right.
Pro-data
This is a classic allocation rule, when, at the time of execution, all applications have an execution rate corresponding to that part of the volume that the applications contribute to the total gang volume.
This means that it does not matter if you break your limitnik into pieces, in what sequence these pieces are sent. The performance will always be the same.
FIFO
The time for setting the limiter dominates here. Limits are poured in priority: until the previously set limit ones are filled in completely, other limiters will not be poured.
With this allocation rule, PriceGivers try to set their limiters ahead of the rest to increase the likelihood of their execution. As a result, this causes a narrowing of the spread and a decrease in volumes on gangs, since if someone submitted their request for a new gang (reduced the spread), then the other participant does not want to join this gang, because anyway, he will be an outsider in front of a smarter one.
When submitting an application for a new gang, it is sent either in full volume, or broken into pieces in descending order, putting them in the same sequence in the queue. This allows, if necessary, to remove part of the volume of its application (according to the stack principle), without losing the priority of execution.
Volume (Weigth) FIFO
The term composed on the go. The priority is not on the time of issuing, but on the volume of the application (or another custom feature - Weight).
This allocation encourages PriceGivers to fill the best gangs with maximum liquidity. For PriceTakers this is one of the best rules.
Weight Pro-Rata
Also the term composed. This is a modification of Pro-Rata, where applications are no longer given equal distribution weights, but depending on the Weight attribute.
For example, any distribution (mat. Term) by the time for issuing applications can be used: the earlier it is set, the higher the weight. In particular, the use of mat is common. linear distribution of time (Time Pro-Rata).
The same is done for Volume and even for more complex features (for example, Weigth = TimeLive * Volume).
With this approach, various mathematically justified methods of optimal bidding are applied — maximizing the probability of the highest in the total amount of execution of applications from a market participant. Those. there are algorithms, how to break an application into pieces and in what sequence to send them to the gangs.
What is used and why
Each site comes up with its own rules of allocation, or uses the ones that have already been developed. Depending on the goals that she pursues. It is clear that the main goal is to increase profits, i.e. increase in turnover. And here comes the nontrivial question of choosing an allocation algorithm. In order for PriceGivers to motivate and PriceTakers activity does not diminish. It all depends on the base of the existing clientele of the site and development prospects.
Most often, do not bother and choose simple options: Pro-Rata and FIFO.
The only algorithm where there are no collisions with the minimum allowable volume (lot) is FIFO-variants.
PriceTakers vs PriceGivers
When in the T & S-data there is a classification by directions, you can talk about the general indicators of trade PriceTakers, or PriceGivers.
We will assume further that the mentioned direction concerns PriceTakers. This is actually a convention, because PriceGivers-directions - just the opposite sign.
Recall that each element of the T & S sequence is data on a transaction made by someone from PriceTakers: price, volume and its direction.
This means that we are quite able to build the dynamics of change in the total Equity of all PriceTakers for each character individually and, accordingly, for any set of characters. Moreover, on FOREX, considering all the crosses and majors at the same time, you can build the dynamics of change in the total currency profile of PriceTakers: how much what currency is “on hand” at PriceTakers.
Level3--
There is such a thing as Level3 - this is complete information about the current state of each market participant. Roughly speaking, this is a well-guarded insider. However, as mentioned above, we can get close to it by calculating Equity_PriceTakers.
Let me remind you, the actual purpose of PriceTakers is meat for PriceGivers, among which there are very strong market participants. Those. self algorithmic pricing serves to enrich PriceGivers by draining PriceTakers.
So PriceGivers will do everything to make Equity_PriceTakers decrease. Therefore, the use of such an easily calculated algorithmic insider seems reasonable from the point of view of the nature of modern electronic finance. of the market.
Organization of the exchange of the new engine
Each exchange works on some trading engine. Suppose the owner of the exchange realized the serious imperfection of the current engine that threatens (or retards) the business. He faces the task of writing a new perfect exchange engine.
It may well be that the new engine is incompatible (it happens) with the old one. How to transfer the exchange to a new engine?
The problem is that the old engine has all the current liquidation. And if the old just turn off and give a new one. Then there will be practically no new liquidation (killing of the stock exchange) - all cannot instantly be reorganized.
Therefore, make the ECN / STP-scheme of the new engine. When the old engine is connected via STP to the new one as the only LP. And the new one also gives its internal ECN.
This allows you to trade immediately through two engines.
After a more advanced engine begins to gain popularity (because it is more profitable to trade through it - less trading costs, more opportunities), liquidity from the old one begins to flow into it (it gets into the internal ECN of the new engine).
As a result, from the old engine, the liquidation virtually all goes to a new one. After that, the old one closes completely. And there is only a new one without an external LP. Those. it turns out an updated stock exchange (ECN) - absolutely everything on the new perfect engine.
It is important to understand that the exchange engine (or the engine of any other trading platform) has nothing to do with the trading API. Those. no one but the exchange itself is able to influence the work of this internal engine directly. This can only be done indirectly.
For this, the owner of the engine writes bridges between his internal engine and the external trading API. Most often limited to only one such real trading API - FIX API (ALMOST obligatory - standard). But there are cases when there are several real trading APIs.
Trading platforms with their own trading APIs are not real trading APIs of the platform, but their wrapper: sometimes more high-level, sometimes less functional, but more simple in work. For example, in almost all trading platforms, the available trading APIs are actually wrappers. Therefore, trading through them increases trading costs due to latency wrappers.
Takeover exchanges
There are a lot of trading platforms. Not only cointegrated FIs can bargain on some of them, but generally the same FIs. Those. The same FI is traded at different sites.
In order to absorb these sites (pour all their liquidation on their site), you need to create an ECN / STP aggregator, where all the victim sites act as LPs. This method creates a new trading platform with more favorable prices - dumping prices.
Thanks also to competent marketing of the promotion of a new site, algo-takers, then algo-givers, then takers, etc. go to dumping because of dumping. Pouring liquidation like a snowball. From a certain moment the process becomes irreversible. And the victim sites are completely absorbed.
In the examples above, you can see the main historical significance of ECN / STP schemes. They serve the purpose of centralizing the market, more efficient pricing, increasing liquidity and improving prices.
Matching
Matching is the previously described convergence of two multidirectional trading applications within the ECN. Note that STP execution of requests is not a match.
Match classifier
In any ECN (stock exchange) there is the possibility of a statistical classification of the conducted match into groups of market participants, within which the greatest turnover of the match occurs. For example, if two market participants, thanks to a match, transfer money to each other through mutual matching, then a similar classifier will separate them into a separate group.
Self-identification
Any ECN is strategically beneficial so that it has as many brokers as possible (independent customer databases). This is to reduce the likelihood of creating a hypothetical competitor.
A competitor is born most often through self-identification of brokers, which occurs as follows. Relatively large brokers classify the match of their customer databases. If it is possible to identify large groups of such clients within their databases, then an obvious thought arises: why should these clients pay ECN for matching among themselves, when it happens all the same between broker clients? Of course, this is a pragmatic question.
Dark-pool
The answer is to create your own ECN / STP engine, where ECN serves to match intra-broker clients, and STP to execute on an external ECN (to avoid confusion, we will call the external ECN exchange later in the text (most often it is)). This solution gives several advantages (and disadvantages) both for the broker and for his clients.
The broker already has its own trading platform, which in relation to the stock exchange has its own name - dark-pool. Accordingly, the commission from the match of its customers now goes to the owner of this platform, and not to the owner of the exchange.
Broker clients sometimes get better execution (than on the exchange), if it occurs within the ECN, because The latency (technical / computing time delays) of the exchange for broker clients is always higher than the latency of the client database of the broker’s clients.
But there are ambiguous moments for customers. If an STP transaction is carried out, then the guaranteed execution of clients is no longer available. Those. FillRate (workmanship) <100%. As it was already discussed in detail earlier, this is a consequence of the fact that customers' requests in the STP scheme are no longer visible in the Level2 exchange. Those. exchange clients cannot respond to them. Roughly speaking, dark-pools clients become just PriceTakers (not PriceGivers) for the exchange.
However, they leave PriceGivers right inside the dark-pool, and this is not enough. Since We do not forget that before the creation (and its cause) of the dark-pool, there was an adequate self-identification described above. It should also be said that the dark-pool allows you to give its customers anonymity before the exchange, because All STP transactions now come from only one name — the name dark-pool. Moreover, because of its isolation with respect to the exchange, dark-pool creates additional protection against highly aggressive priceGivers (MM-algorithms and other HFT-representatives of algorithmic trading), which not without reason attracts many market participants to go through trading from exchanges to dark-pools.
Dark-pools development
According to the previously described absorption scheme of exchanges, a large dark-pool is able to absorb the exchange and actually begin to fulfill its role. In this case, the regulatory burden of the exchange does not apply to dark-pools, since they are practically pure electronic (virtual) formations. Of course, the regulatory paw in the future will not bypass them, as it always happened when something new gained serious weight.
Centralization vs Decentralization
As you can see, the creation of dark pools is a natural evolutionary process taking place within the market — its decentralization. The non-zero probability of decentralization causes centralized exchanges to improve their trading conditions. Those. decentralization for exchanges is a kind of mechanism that increases the effect of this type of market. However, there are negative aspects of excessive decentralization, which were shown on the example of FOREX. There, the same ECN / STP scheme, paradoxically, serves to centralize the market and to improve the trading conditions accordingly.
Those. through ECN / STP formations, one can decentralize the market by creating dark-pools, or centralize it — creating high-level aggregation from the same dark-pools, in particular.
The best trading conditions are somewhere in the middle between the paradigms of decentralized and centralized markets.
Real-Time Tester
Any algorithmic trader is faced with the task of translating the tester robot into a combat state, ready to work in the real market.
In fact, there is only one way to competent translation. Fortunately, it is almost universal.
The combat robot is divided into two parts: a tester and a synchronizer.
The tester shows the current trading history of the tester robot (up to the present).
The synchronizer coordinates this data with the current real trading environment, trying to fit it with a virtual one (obtained in the tester).
For example, the virtual environment shows that now at such and such a level is limitnik. The task of the synchronizer is to make it so that at this price level there is a similar limiter on the real market.
Synchronizer types
Universal synchronizer can not be. But there are only two fundamentally different approaches to synchronization.
- Classic - through the market (a la limitnik at a price worse than the current). This is the simplest scheme, when the trading environment is copied through the markets. The advantages of this method are full repetition and simple visualization. Cons - negative slippage. Those.for vehicles with low mat. waiting is a bad option.
- Through limit orders. When, for example, the copied BUY is considered as BuyLimit on the opening price of the copied position. All copied limit orders are also taken into account. The advantage of this method is the leveling of negative slippage. The downside is that limit orders rejects can distort results.
The classic version is implemented in one form or another in all signal services that are so popular now. Such a scheme is beneficial services, because The synchronizer is as simple as possible, and they are not very concerned about the clients' trade costs.
The second scheme, as far as I know, is not applied anywhere. Perhaps one of the very boring algorithmic traders in his home realized ...
Manipulating the use of signaling services
Above said what costs are so fashionable now signal services using a primitive classical synchronization scheme: through a market order.
It is worth mentioning one more minus of such synchronization - the manipulative component. The technique described below can remind someone layering . But still it is different. I will say in advance that the following only makes sense for ECN (stock exchanges) and ECN / STP platforms.
Imagine that you are a supplier of signals (preferably free) and you have subscribers (there will be more of them then), the size of which is higher than yours. Then from time to time you have the opportunity to include a simple transfer of money from the accounts of your subscribers to your master account:
Expose BIG BuyLimit and SellLimit to the inside of the spread and make a “trigger” - a small SELL (sell yourself - BuyLimit). This causes an avalanche of SELL orders from your subscribers. Your BuyLimit starts to fill up, i.e. grow BUY-posture. What causes an avalanche again, but already BUY-orders from your subscribers. Those.Your SellLimit starts to fill up.
As a result, having caused similar recursive avalanches (the basis of which still lies in the latency of signaling services), you receive an overflow of money from your subscribers to your master account. This improves the performance of your master account, attracting even more subscribers.
Of course, it is not necessary to put limit account on the master account. This can be done on your personal friend account. Then get overflow on him. But the master account will have a small drawdown, which may alienate subscribers.
There may be other options for such frauds, but their core-idea will be unchanged, as written above.
MM algorithms are almost impossible to synchronize profitably through the classical scheme. Therefore, it is advisable to refer to the previously mentioned alternative synchronization scheme.
ECN () — . . , .
, - — . , , , - . — .
( HFT)
The market is characterized by the presence of various kinds of patterns at each point in time. If we denote N (t) as a function of the number of patterns depending on time and plot it. Then failures will be immediately apparent - a sharp decrease in the number of market patterns.
These failures almost always correspond to important events,> 90% of which are scheduled - economic news on a schedule.
When searching for market patterns, it is reasonable to discard similar periods of N (t) failures, both planned and force majeure, from analysis. Periods of apparent absence of the desired patterns are also thrown away - a special automated filter.
Such actions can significantly increase the likelihood of finding patterns.
Therefore, during the trading of the market patterns found, any trading activity stops before the release of the planned news and some time after.
PS
Thanks to sergeev , from whom it started to start and systematically continue writing this educational program.
Source: https://habr.com/ru/post/202402/
All Articles