How to create a trading robot using genetic programming

Good day. In this article I will talk about creating a system in which genetic algorithms write robots. In theory, these robots could trade on the exchange.

I am a fan of three things - artificial intelligence, high-performance machines and the practical application of any knowledge. Having some free time, I designed a small puzzle, purchased iron and sat down to create.
')
The project arose from the desire to try to practice genetic programming. The first option was to create a bot for any game, but I stopped at trading robots, where the exchange is also a kind of game.

This article assumes that you are familiar with the concept of genetic algorithms or genetic programming. And also what trading robots do.

Where to begin?

I started by studying the platform for creating MetaTrader5 robots. The MQL5 language is positioned as similar to C ++, with minor differences in syntax. In simple terms, the platform has functions for accessing market data and functions for carrying out trading operations. After studying and testing several dozen simple robots, work began on identifying a common elementary base on which these algorithms are built.

For the convenience of working with logic inside the genetic algorithm, I had to create my own meta-language over MQL, let's call it SadLobster. Without this generalization, it would be terribly difficult to force the machine to write code according to the rules of a programming language created for a person. The whole project was designated as a prototype, so that it would be easier to accept many compromises and simplifications. Otherwise, this development phase would never end.

How does one robot work

Let's see what the simplified version of the robot that will be created looks like.
(I had to throw away too much to make the article look like)


The functions boolA__3 and priceA__10 process the information received from the quote charts.
The boolA__3 function is launched to check if there is a signal for placing an order. The first time we check if there is a buy signal. The second time we run with the value invert = 1 and check if there is a sell signal.

The priceA__10 function determines at what price an order should be placed.

Sadlobster
The second feature of the SadLobster language is that its syntax is compatible with C ++. That is, the same code that I use for testing in MQL can be run through C ++ tester, which was written separately.

MQL tester vs C ++ tester

• This tester is two orders of magnitude faster than MQL and has the necessary API so that it can be controlled by a genetic algorithm.
• MQL also provides excellent opportunities for debugging and checking the correct operation of robots.

When applied to trading robots, there is such a term “ grail ” - this is a robot that earns a lot and stably even outside the training set. During the development, I met their outlines several times. And each of them was the result of a vulnerability in the C ++ tester. As evolution progressed, robots found vulnerabilities in the testing framework — performed impossible operations or found a way to look into future data and many other tricks. (I think the potential of genetic programming in testing is greatly underestimated.) Here MQL came to the rescue. Running the robot there, he lost the magic properties of the Grail, because there most of the vulnerabilities are already covered.

A language consists of a list of functions that can be used. The simplest are AND, OR, CREATE_LINE, IS_INSIDE, ...

And the functions of access to data quotes and technical indicators - HIGH, LOW, FRACTAL, MA, MACD_SIGNAL. These functions will be listed in list 1.

Simulation of trading history

The robot runs on a period of history, for example, from 2014 to 2016. There is a trading simulation. All his transactions are recorded and a report is generated for them. My report looks like this:



or so

 1.82 14.66 64.1% 1.02 -383[+0.99] 451 (30.8%) +6613 : 179F <736c> 

These numbers mean: profitability, expectation of winnings, share of profitable deals, ratio of average profitable deal to average loss, drawdown, number of deals, time percentage in the market, net profit, robot identifier.

The report shows a good robot or not. I will try to tell about the strategy tester and its implementation another time.

Fitness function

An interesting module that needs attention is the fitness function. To evaluate the results of trading, we need to simulate it, and then analyze all transactions. There is the widest field for creativity. The results depend on the fact that you will be considered the best robot. And the more complex the system, the more difficult it is to do. Since it is impossible to describe the behavior of the desired program with a single number.

The first decision - the more the robot earned, the better it is. But this raises the question of risks. Such a robot is completely unviable. Less risk - less profit, more risk more profit.

Trading robots have several different characteristics. The simplest of them are the profit factor (PF) and the expectation of profit per trade (EP), the maximum drawdown of funds, LR correlation, the Sharpe ratio.

This is how the MetaTrader report on the work of one of the created robots looks like:



Each of the parameters has its own coefficient of importance. In proportion to these numbers, the fitness function for each robot is calculated. After which there are well-known processes of crossing and mutation. And additionally set the threshold for the minimum number of transactions. From 0.2 to 2 transactions per day, minimum.

 self.KOEF = [2, 4, 1, 1, 1, 1, 0, 0, 2] self.KEYS = ['PF', 'EP', 'win_persent', 'p_wiin_div_loss', 'max_dd', 'deals', 'profit', 'pfMonth', 'LR'] 

Dynamics and Results of the Genetic Algorithm Launch

Graphic representation of the evolution or training schedule



The red line on the left is the profit factor of the best robot, and the blue line is the cross test of the top 10 robots. 20 iterations are usually enough to evaluate the result. The first ten iterations can be ignored, because there all restrictions are not imposed on robots. At iterations from 10 to 20, we see how the forward results are improving.

On the right is a histogram of the monthly profitability of the best of the robots in points. It shows three years of study on the left, and one year of cross-test on the right.

I also tried to avoid re-optimization, so I scored all the floating parameters with constants, with the expectation that there were enough degrees of freedom due to the combination of functions.

About complexity

The robot algorithm for simplicity has no internal memory or states. This feature also helps to cache the results of calculations on each bar. Which greatly speeds up the calculations. Trying to use only functions with complexity O (1) or O (n) in logic, I strongly limited the functional. But this required computational resources.

Random Tree Generation

How to get the function in the form in which it is presented in the first listing?

1. It is necessary to create a list of possible functions and describe them.
2. Collect a random expression tree which is logic
3. Convert to code

Here are some of the interface functions that are used in the logic of robots. Each function name is a kind of macro, accessible from both MQL and C ++ test frameworks. Implementations differ due to language differences. Call it a list of 1.


Consider the simple MORE_I function.

 {'name':'MORE_I', 'input':['DEF_PRICE','DEF_PRICE','invert'], 'result':'DEF_BOOL', 'price':4} 

This function takes two price parameters (and an auxiliary parameter invert, you can ignore it). It returns a boolean value. The price parameter means a certain abstract complexity of this function, which was intended to control the complexity of the entire logic of each robot.

And here there is a good olympiad problem : it is necessary to collect all possible logics with the given complexity and type of result from the source functions. Logic is an expression of the type F (X) -> Y.

Example - we want the function of deciding whether to enter a long position. We need a boolean solution - DEF_BOOL, then the options are as follows:

 return MORE_I(_CLOSE(__value(1)), _HIGH(__value(1)), invert);//1 return MORE_I(_CLOSE_D1(1), _HIGH(__value(1)), invert);//2 //__value    “1”    . //PS           . 

Trying to finish the prototype, I was very wrong with the random () function where I would have to use smarter logic. But the whole idea was to start the car entirely and, having weighed it with tests, to start iterative improvements. Below is a description of the algorithm on which I stopped.

The task of the algorithm is to generate a function that will return DEF_BOOL. The expression notation is LISP-like: [Function Name, param1, param2 ...]. Parameters that begin with DEF are a type. The expression in which there is such a parameter is not final, requires clarification. The notation does not indicate the type of return value as unnecessary.

1. Let's create a pool of such expressions, where we will generate them.
2. We check if there are functions in our pool without parameters that need clarification. If so, select it and return as result. If not, continue.
   
3. Randomly choose one of the following possible actions - add one more function (4) to the pool or fill in the existing unspecified parameters (5).
   
4. Add a new expression. Since we only need functions that will return the type DEF_BOOL, we select all such functions from the list of SP1. Now select a random function and write it to the pool in the form ['IS_INSIDE', 'DEF_HPRICE_LEVEL', 'DEF_PRICE'].
   
5. Extend the existing function. In the IS_INSIDE function, two parameters need clarification. We are looking for a function that can fill the parameter DEF_PRICE in SP1.
   We get ['IS_INSIDE', 'DEF_HPRICE_LEVEL', ['MA_HI_I_v2', '8', 'dsD1', '1', 'invert']]].
6. Returning to paragraph 2.

The result of the algorithm will be a similar expression.

 ['IS_INSIDE', ['MAKE_HPRICE_LEVEL', ['MA_CLOSE_v2', '3', 'dsD1', '1'], ['STDDEV8_RANGE_MAX_END', '10']], ['MA_HI_I_v2', '8', 'dsD1', '1', 'invert']] 

To create another function of the same type, the pool can be reused without zeroing, which significantly speeds up the work. Also, the function can be disassembled and create from it a pool that will be used for crossing or mutation functions.

This is the third implementation of the algorithm, the first two were not so successful. It was very useful to get acquainted with the 4th volume of Knut, namely chapter 7.2.1.6 Generation of all trees. If you need an improved version, be sure to re-read it again. The disadvantages of this algorithm are:

1. We need to make sure that SP1 is capable of generating expressions in the right quantity and variety. For this, I just have a test that shows that out of 10,000 generated functions, 90% are unique.
   
2. It is also not clear what the distribution of basic functions in the expression.
   
3. I would like to know how many different functions can generate a specific list of basic functions.
   
4. Ps. This, by the way, is one of those parts of the system where we replaced the whole power of the person’s analytical mind with the simple function Random (). The person who creates the robot should already know the answer to the question. How? this robot will work and why . The GA here simply serves as an optimized brute force.
   

Broadcast to the final form

Next, this LISP-like expression is turned into a listing in SadLobster, where each indivisible expression is a new variable. The logical expression remains the same.

 DEF_BOOL boolA_001(bool invert) { DEF_PRICE var_2 = MA_HI_I(8, dsD1, 1, invert); DEF_PIPS_DOUBLE var_1 = STDDEV8_RANGE_MAX_END(10); DEF_PRICE var_0 = MA_CLOSE(3, dsD1, 1); DEF_HPRICE_LEVEL var_3 = MAKE_HPRICE_LEVEL(var_0, var_1); DEF_BOOL var_4 = IS_INSIDE(var_3, var_2); return var_4; } 

SadLobster is not Haskell with pure functions.

Although I aspired to this. One of the problems that arise when creating a language is error handling. Immediately, there was a desire to use the mechanism of execs, but MQL does not support them. The most frequent problem is a failed object. It would be ideal to use nil values, we will not complicate ahead of time. This can be improved in future versions. And in the current implementation, it is simply checked whether the object is valid, if not, then the function ends immediately. This is done by a macro of type CHECK_LINE_OR_FALSE.

 DEF_PERIOD var_1 = makePeriodSinceLastDay(ds); DEF_POINTS var_0 = GetZZPoints(5, ds, 0); DEF_LINE var_3 = MAKE_SUPPORT(var_0, var_1, 4, !invert); CHECK_LINE_OR_FALSE(var_3); 

Expression optimization

Consider the option when the expression looks like this:

 ['IS_INSIDE', ['MAKE_HPRICE_LEVEL', ['GET_MEDIAN_CLOSE_PRICE', ['makePeriodToday', 'ds'], 'ds'], //1 ['MA_RANGE_v2', '7', 'dsD1', '1']], ['GET_MEDIAN_CLOSE_PRICE', ['makePeriodToday', 'ds'], 'ds']] //2 

Expressions 1 and 2 are the same. After translating and extracting variables, var_2 is used in both places and no duplication of code.

  DEF_PIPS_DOUBLE var_1 = MA_RANGE(7, dsD1, 1); DEF_PERIOD var_0 = makePeriodToday(ds); DEF_PRICE var_2 = GET_MEDIAN_CLOSE_PRICE(var_0, ds); DEF_HPRICE_LEVEL var_3 = MAKE_HPRICE_LEVEL(var_2, var_1); DEF_BOOL var_4 = IS_INSIDE(var_3, var_2); 

Development requires infrastructure

I wanted to create a very robust base for designing robots. Analyzing the examples of PBX orders on the freelance exchange, I have embedded new features / requirements from the TOR into the overall system. So I tried to expand the diversity in the behavior of robots, because the diversity in the code base could lead to the creation of the same algorithmic patterns.

At some point, and this is normal, the development emphasis shifted towards the writing of analytical tools, to automate the analysis of what these or other algorithms do. These are mostly single page scripts of the type:

1. Log data to the database while the GA is running
2. Remove from the database and process
3. Display graphically with mathplotlib

Here is an example of one of them, shows the result of trading hundreds of robots superimposed on one chart, to assess the distribution of the executed orders.

A few words about performance

Testing is very fast for several reasons:

• All robots are compiled into machine code.
• Testing run multithreaded.
• Parallel even linking process.
• From the strategy tester a lot of checks have been cut.
• Used caching for heavy functions
• Testing of robots is very rough, there are no scalpers or HFT, the analysis takes place on the hourly charts.
• I used a 12-thread processor with overclocking up to 4GHz Intel Core i7-5820K for testing.

How it works?

I want to clarify that, depending on the settings of the GA, of which there are a lot, you can get robots with diametrically different characteristics. Suppose that it is important for us to get a robot that will have a positive return on the results of the next year after the training, and made a lot of transactions to evaluate the non-randomness of the results.

Let's look at this experiment - we run the GA 15 times, because each GA is a series of very many random events of generation, mutation, crossing and roulette.

I want to clarify that Money Management is not used in the works and the trade is conducted with the same minimum volume.



$ 158 is the average profit per month for training, $ 21 is the average profit for the next 12 months. The results balance around zero profit plus margin of error. On the other hand, it can be compared with a random robot that will simply lose on the spread. Do not forget that the game on the stock exchange is a game with a negative amount. In another period of study, the results are likely to be different.

Hepienda will not

It turned out to make GA create robots with a specific task. This project expanded my understanding and expertise in the topic described above. And then a terrible thing happened - the goal of the project was achieved. The project for generating robots is ready. This article draws a line on the work done.

I want to divide the conclusion into two points

Subjective - in the course of the work there are a lot of options that could be tested in the framework of this system, for which it was created. For example:

• Use random data, or not random, to see how the system is trained in intelligible patterns.
• Expand the arsenal of basic logic by orders of magnitude.
• Run training on all available data at once.
• Run an experiment in the form of a real evolution, where each iteration at the input is supplied with new data without repetitions.
  

Objective - technical analysis of the instrument is like driving a car looking into the rearview mirror. A simple pattern for a trading robot could not be found. Without a complete market model, it is not clear why certain robots work, and when it will stop.

And most importantly, I see the future of this project in the sandbox format for the development of AI in the field of writing algorithms.

I am pleased to answer your questions, suggestions and comments.

Thank you for your time.

My blog on Medium