Hacker tools to trade
In order to carry out our hacker financial experiment (to make money on it), we need a program that can conduct research, testing, training, and trading using an algorithm. None of the existing programs to date truly covers all of these areas. However, you can combine different systems and try to solve the problem. Fortunately, there are a couple of suitable tools. By and large, I will use Zorro Trader and the R language, but sometimes some other tools.
The spectrum of programming languages is not as wide as it might seem initially. Moreover, you can not program at all in the classical sense, using various kinds of strategy constructors or the same Excel. But, unfortunately, such an approach is applicable for fairly primitive strategies, and I have not heard of any such profitable system. For real work and research will have to learn how to program for real.
Freedom to choose a programming language is even less. You always want to take a language with a simple and beautiful syntax. Probably one of the best options where simplicity and object orientation are combined - Python. The language has a rich library of statistical and indicative functions, which is why many programmers begin their journey with it. However, they all eventually wonder at the slowness of the program they have developed. No funds invested in expensive and ultra-fast computers here will not help. The reason for the slowness, in fact, is very simple: the simpler the syntax of the language, the more time it takes to execute the algorithm that is written on it.
Basically, the speed of the algorithm depends on whether the language in which it is written is compiled or interpreted. For example, C, Pascal or Java - compiled programming languages. This means that the program code will be executed directly on the processor (in the case of C, Pascal) or in the virtual machine (in the case of Java). At the same time, the languages Python, R, MATLAB are interpretable, that is, the code will not work by itself. It will require a kind of translator. The interpreted language works much slower and at the same time requires more resource from the processor and RAM than the compiled language. But at the same time they are interactive, that is, you can enter commands directly into the console. There are compromise languages. For example, C # is an intermediate language between compiled and interpreted. A C # program will be compiled into an intermediate, hardware-independent code, which, depending on the implementation, will be either interpreted or converted into machine code. C # is about 4 times slower than C, but at the same time 30 times faster than Python.
')
Below is a table of the performance of two test programs written in several languages: the solver of Sudoku and the cycle of multiplication of matrices of size 1000 by 1000. Results in seconds:
Why is the speed of the program is so important for trading systems? Strategy development often happens empirically. That is, you are looking for the appropriate system parameters from many options and test each time. Suppose that one test of parameters for a C program takes 1 minute. In this case, the same strategy, but already written in EasyLanguage, will require about 30 minutes for the test. In Python - 2 hours, and on R - 10 hours! If I did a test for my article The Trend Experiment , I would have waited for the results so far. Now you understand why many trading platforms use their own compiled programming language or C language variants for working strategies. In this case, high-frequency trading strategies are usually implemented in C or directly in machine language.
Even compiled languages have large differences in the speed of work due to the difference in the implementation of trade and analysis. For example, I took the simplest strategy from here and tested it on different trading platforms (10-year backtest on tick data) and got a completely different speed.
However, these differences are not so critical. There is one trick to overcome the slow speed of the language. Even in interpreted languages there are libraries of functions that are often written in C / C ++. Scripts that do not go step by step on historical data, but only call the necessary functions from libraries that process data simultaneously, will work with comparable speed in all languages. And indeed, some trading systems can be programmed in this so-called vector mode (you will see this below with an example of R code). Unfortunately, this method works only with simple systems and only for testing, it is not applicable for developing strategies for combat work.
Zorro-Trader is a software for financial analysis and algorithmic trading. A kind of Swiss knife, as you can use it for all kinds of quick tests. I choose it because:
An example of a strategy written in C is the classic intersection of simple moving averages (SMA):
You can see that Zorro Trader offers a very simple method for implementing a strategy. But here you will encounter a lack of the C language: you cannot call external libraries as easily as in Python or R. The use of machine learning packages based on C / C ++ takes quite a long time to implement. Fortunately, for this purpose, Zorro Trader can call R functions.
R is a script interpreter for data analysis and graphing. This is not a programming language with a sequential syntax, but rather a conglomerate of operators, functions, and data structures, which has grown greatly in 20 years. It is not very logical structured and more difficult to learn than a conventional programming language, however, it has some unique advantages. I will use it in this article when it comes to complex analysis or machine learning tasks. This is my tool because:
An example of the intersection of SMA in R for a vector backtest:
You can see that the vector code, in fact, simply consists of function calls. At the same time, it works as fast as the same equivalent in C. But such a function is very difficult to read and must be rewritten for real trading, and things like stop-loss need to be removed, since they will not work on vector backtest That is, how good R is for interactive data analysis, as it is useless for writing trading strategies, although some packages, for example Quantstart, assume rudimentary functions for testing and optimization. They require a “special” programming style and, at the same time, do not imitate trade very realistically and at the same time remain too slow for serious tests. That is, while some functions and packages of the R language can work quite quickly, since they are written in C, the language itself will not be able to conduct serious testing quickly. But Zorro Trader and R perfectly complement each other.
In addition to languages and platforms, often need support tools that can be small, simple, cheap (or free), but to solve some problems quickly. To edit scripts, I do not use the Zorro Trader editors or the R console. I use Notepad ++. For interactive work with R, I can recommend RStudio. It is very useful to develop a strategy to have a tool for comparing files, for example, you want to compare the logs of various system options and check which option is better to open a transaction, determine the time difference and evaluate the effects of differences. For this, I use Beyond Compare.
In addition to Zorro trader and R, there is another relatively new software development system that I plan to consider better in the near future: TSSB for generating and testing trading systems with free parameters using advanced machine learning algorithms. David Aronson and Timothy Masters participated in its development, so I am sure the system will not be as useless as most other trading system generators. However, there is another limitation: TSSB cannot trade, therefore you cannot really trade on your ingenious systems. Although, maybe I can find a way to combine TSSB with Zorro Trader.
Language selection
The spectrum of programming languages is not as wide as it might seem initially. Moreover, you can not program at all in the classical sense, using various kinds of strategy constructors or the same Excel. But, unfortunately, such an approach is applicable for fairly primitive strategies, and I have not heard of any such profitable system. For real work and research will have to learn how to program for real.
Freedom to choose a programming language is even less. You always want to take a language with a simple and beautiful syntax. Probably one of the best options where simplicity and object orientation are combined - Python. The language has a rich library of statistical and indicative functions, which is why many programmers begin their journey with it. However, they all eventually wonder at the slowness of the program they have developed. No funds invested in expensive and ultra-fast computers here will not help. The reason for the slowness, in fact, is very simple: the simpler the syntax of the language, the more time it takes to execute the algorithm that is written on it.
Basically, the speed of the algorithm depends on whether the language in which it is written is compiled or interpreted. For example, C, Pascal or Java - compiled programming languages. This means that the program code will be executed directly on the processor (in the case of C, Pascal) or in the virtual machine (in the case of Java). At the same time, the languages Python, R, MATLAB are interpretable, that is, the code will not work by itself. It will require a kind of translator. The interpreted language works much slower and at the same time requires more resource from the processor and RAM than the compiled language. But at the same time they are interactive, that is, you can enter commands directly into the console. There are compromise languages. For example, C # is an intermediate language between compiled and interpreted. A C # program will be compiled into an intermediate, hardware-independent code, which, depending on the implementation, will be either interpreted or converted into machine code. C # is about 4 times slower than C, but at the same time 30 times faster than Python.
')
Below is a table of the performance of two test programs written in several languages: the solver of Sudoku and the cycle of multiplication of matrices of size 1000 by 1000. Results in seconds:
Tongue | Sudoku | Matrices |
C, C ++ | 1.0 | 1.8 |
Java | 1.7 | 2.6 |
Pascal | - | four |
C # | 3.8 | 9 |
Javascript | 18.1 | sixteen |
Erlang | 18 | 31 |
Python | 119 | 121 |
Ruby | 98 | 628 |
Matlab | - | 621 |
R | - | 1738 |
Why is the speed of the program is so important for trading systems? Strategy development often happens empirically. That is, you are looking for the appropriate system parameters from many options and test each time. Suppose that one test of parameters for a C program takes 1 minute. In this case, the same strategy, but already written in EasyLanguage, will require about 30 minutes for the test. In Python - 2 hours, and on R - 10 hours! If I did a test for my article The Trend Experiment , I would have waited for the results so far. Now you understand why many trading platforms use their own compiled programming language or C language variants for working strategies. In this case, high-frequency trading strategies are usually implemented in C or directly in machine language.
Even compiled languages have large differences in the speed of work due to the difference in the implementation of trade and analysis. For example, I took the simplest strategy from here and tested it on different trading platforms (10-year backtest on tick data) and got a completely different speed.
Zorro Traid: ~ 4 seconds (Lite-C, option C)
MT4: ~ 110 seconds (on mql4, another version of C)
MultiCharts: ~ 155 seconds (EasyLanguage, C / Pascal mix).
However, these differences are not so critical. There is one trick to overcome the slow speed of the language. Even in interpreted languages there are libraries of functions that are often written in C / C ++. Scripts that do not go step by step on historical data, but only call the necessary functions from libraries that process data simultaneously, will work with comparable speed in all languages. And indeed, some trading systems can be programmed in this so-called vector mode (you will see this below with an example of R code). Unfortunately, this method works only with simple systems and only for testing, it is not applicable for developing strategies for combat work.
Selection of tools
Zorro-Trader is a software for financial analysis and algorithmic trading. A kind of Swiss knife, as you can use it for all kinds of quick tests. I choose it because:
- It's free (if you are rich))
- C scripts are easy to write and work very quickly. You can implement the system or idea literally within 5 minutes
- Open architecture - you can add anything with a DLL
- Minimalist - only frontend for programming language
- Allows you to automate experiments
- Very accurate and realistic mechanism for simulating real trading
- Integrated Portfolio Management
- There are all basic data processing functions and statistical functions (most with source code).
- The product is constantly being developed and maintained (new versions usually come out every 2-3 months).
- Last but not least, I know the platform thoroughly, because I wrote a tutorial on it ...
An example of a strategy written in C is the classic intersection of simple moving averages (SMA):
function run() { vars Close = series(priceClose()); vars MA30 = series(SMA(Close,30)); vars MA100 = series(SMA(Close,100)); Stop = 4*ATR(100); if(crossOver(MA30,MA100)) enterLong(); if(crossUnder(MA30,MA100)) enterShort(); } You can see that Zorro Trader offers a very simple method for implementing a strategy. But here you will encounter a lack of the C language: you cannot call external libraries as easily as in Python or R. The use of machine learning packages based on C / C ++ takes quite a long time to implement. Fortunately, for this purpose, Zorro Trader can call R functions.
R is a script interpreter for data analysis and graphing. This is not a programming language with a sequential syntax, but rather a conglomerate of operators, functions, and data structures, which has grown greatly in 20 years. It is not very logical structured and more difficult to learn than a conventional programming language, however, it has some unique advantages. I will use it in this article when it comes to complex analysis or machine learning tasks. This is my tool because:
- It's free (“Software is like sex: it's always better when it's free”)
- R scripts can be very short and effective (as soon as you are used to its syntax).
- It is a global standard for data analysis and machine learning.
- Open architecture - you can add almost any module.
- Minimalist - only a console with an interpreter.
- There are mountains of packages for all conceivable and unimaginable mathematical and statistical problems.
- Constantly developed and supported by the global scientific community (about 15 new packages are released almost every day).
An example of the intersection of SMA in R for a vector backtest:
require(quantmod) require(PerformanceAnalytics) Data <- xts(read.zoo("EURUSD.csv", tz="UTC", format="%Y-%m-%d %H:%M", sep=",", header=TRUE)) Close <- Cl(Data) MA30 <- SMA(Close,30) MA100 <- SMA(Close,100) Dir <- ifelse(MA30 > MA100,1,-1) # calculate trade direction Dir.1 <- c(NA,Dir[-length(Dir)]) # shift by 1 for avoiding peeking bias Return <- ROC(Close)*Dir.1 charts.PerformanceSummary(na.omit(Return)) You can see that the vector code, in fact, simply consists of function calls. At the same time, it works as fast as the same equivalent in C. But such a function is very difficult to read and must be rewritten for real trading, and things like stop-loss need to be removed, since they will not work on vector backtest That is, how good R is for interactive data analysis, as it is useless for writing trading strategies, although some packages, for example Quantstart, assume rudimentary functions for testing and optimization. They require a “special” programming style and, at the same time, do not imitate trade very realistically and at the same time remain too slow for serious tests. That is, while some functions and packages of the R language can work quite quickly, since they are written in C, the language itself will not be able to conduct serious testing quickly. But Zorro Trader and R perfectly complement each other.
More hacker stuff.
In addition to languages and platforms, often need support tools that can be small, simple, cheap (or free), but to solve some problems quickly. To edit scripts, I do not use the Zorro Trader editors or the R console. I use Notepad ++. For interactive work with R, I can recommend RStudio. It is very useful to develop a strategy to have a tool for comparing files, for example, you want to compare the logs of various system options and check which option is better to open a transaction, determine the time difference and evaluate the effects of differences. For this, I use Beyond Compare.
In addition to Zorro trader and R, there is another relatively new software development system that I plan to consider better in the near future: TSSB for generating and testing trading systems with free parameters using advanced machine learning algorithms. David Aronson and Timothy Masters participated in its development, so I am sure the system will not be as useless as most other trading system generators. However, there is another limitation: TSSB cannot trade, therefore you cannot really trade on your ingenious systems. Although, maybe I can find a way to combine TSSB with Zorro Trader.
Source: https://habr.com/ru/post/398749/
All Articles