Event-oriented Python backtesting step by step. Part 3

In previous articles, we talked about what an event-oriented backtesting system was, explored the class hierarchy that needs to be developed for it, and discussed how such systems use market data in the context of historical testing and for “live” work on the exchange.

Today we describe the NaivePortfolio object, which is responsible for tracking positions in the portfolio and, based on incoming signals, generates orders with a limited number of stocks.
')

Position Tracking and Order Processing

The order management system is one of the most complex components of an event-oriented back tester. Its role is to track current market positions and their market value. Thus, on the basis of the data obtained from the corresponding backtester component, the liquidation value of the position is calculated.

In addition to analyzing positions, the portfolio component must take into account risk factors and be able to use techniques for determining the size of positions — this is necessary for optimizing orders sent to the market through a broker's trading system.

A Portfolio object must be able to process SignalEvent objects, generate OrderEvent objects OrderEvent and interpret FillEvent objects to update positions. Thus, it is not surprising that Portfolio objects are usually the most voluminous elements of the backtesting system in terms of lines of code.

Implementation

Let's create a new file portfolio.py and import the necessary libraries - the same implementations of the abstract base class that we used earlier. You need to import the floor function from the math library to generate integer orders. FillEvent and OrderEvent objects FillEvent also OrderEvent — a Portfolio object handles each of them.

 # portfolio.py import datetime import numpy as np import pandas as pd import Queue from abc import ABCMeta, abstractmethod from math import floor from event import FillEvent, OrderEvent 

An abstract base class for Portfolio and two abstract methods update_signal and update_fill . The first handles new trading signals that are taken from the event queue, and the last one works with information about the executed orders received from the engine handler object.

 # portfolio.py class Portfolio(object): """  Portfolio          : , , 5 , 30 , 60   . """ __metaclass__ = ABCMeta @abstractmethod def update_signal(self, event): """  SignalEvent         . """ raise NotImplementedError("Should implement update_signal()") @abstractmethod def update_fill(self, event): """           FillEvent. """ raise NotImplementedError("Should implement update_fill()") 

The main object of today's article is the NaivePortfolio class. It is designed to calculate the size of positions and reserved funds and process trade orders in a simple manner — simply by sending them to a brokerage trading system with a specified number of shares. In the real world, everything is more complicated, but such simplifications help to understand how the portfolio order processing system should function in event-oriented products.

NaivePortfolio requires the amount of initial capital - in the example it is set at $ 100,000. You must also set the day and time to start work.

Portfolio contains all_positions and current_positions . The first item stores a list of all previous positions recorded by the time stamp of a market event. Position is simply the amount of a financial instrument. Negative positions mean that shares are sold short. The second element stores a dictionary containing the current positions for the last update of the bars.

In addition to the items responsible for the positions, the portfolio stores information on the current market value of open positions (holdings). “Current market value” in this case means the closing price obtained from the current bar, which is approximate, but rather plausible at the moment. The all_holdings element stores a historical list of the cost of all positions, and current_holdings stores the most recent dictionary of meanings:

 # portfolio.py class NaivePortfolio(Portfolio): """  NaivePortfolio    (..   -)    /   ,   .       BuyAndHoldStrategy. """ def __init__(self, bars, events, start_date, initial_capital=100000.0): """          .           ( ,     ). Parameters: bars - The DataHandler object with current market data. events - The Event Queue object. start_date - The start date (bar) of the portfolio. initial_capital - The starting capital in USD. """ self.bars = bars self.events = events self.symbol_list = self.bars.symbol_list self.start_date = start_date self.initial_capital = initial_capital self.all_positions = self.construct_all_positions() self.current_positions = dict( (k,v) for k, v in [(s, 0) for s in self.symbol_list] ) self.all_holdings = self.construct_all_holdings() self.current_holdings = self.construct_current_holdings() 

The following construct_all_positions method simply creates a dictionary for each financial instrument, sets each value to zero and then adds the date and time key. Python dictionary generators are used.

 # portfolio.py def construct_all_positions(self): """   ,  start_date   ,      . """ d = dict( (k,v) for k, v in [(s, 0) for s in self.symbol_list] ) d['datetime'] = self.start_date return [d] 

Metozh construct_all_hldings is similar to the one described above, but adds some additional keys for free funds, commissions and the balance of the account after the transactions, the total commission paid and the total amount of existing assets (open positions and money). Short positions are considered as "negative". The values ​​of starting cash and total account equal to the initial capital:

 # portfolio.py def construct_all_holdings(self): """      ,  start_date   ,      . """ d = dict( (k,v) for k, v in [(s, 0.0) for s in self.symbol_list] ) d['datetime'] = self.start_date d['cash'] = self.initial_capital d['commission'] = 0.0 d['total'] = self.initial_capital return [d] 

The construct_current_holdings method is almost identical to the previous one, except that it does not “wrap” the dictionary in the list:

 # portfolio.py def construct_current_holdings(self): """  ,         . """ d = dict( (k,v) for k, v in [(s, 0.0) for s in self.symbol_list] ) d['cash'] = self.initial_capital d['commission'] = 0.0 d['total'] = self.initial_capital return d 

At each heartbeat, that is, for each request for market data from a DataHandler object, the portfolio must update the current market value of the held positions. In the real trading scenario, this information can be downloaded and parsed directly from the brokerage system, but for the backtesting system it is necessary to calculate these values ​​separately.

Unfortunately, due to bid / asset spreads and liquidity, there is no such thing as a “current market value”. Therefore, it is necessary to evaluate it by multiplying the amount of the asset held by the “price”. In our example, the closing price of the previous bar is used. For intraday trading strategies, this is a fairly realistic approach, but for trading on time intervals for more than a day, everything is not so likely since the opening price may differ significantly from the opening price of the next bar.

The update_timeindex method update_timeindex responsible for processing the current value of new positions. It receives the latest prices from the market data processor and creates a new dictionary of tools that represent current positions, and equating “new” positions to “current” positions. This scheme only changes when a FillEvent . After this, the method all_positions current position set to the all_positions list. Then, the present value values ​​are updated in a similar way, except that the market value is calculated by multiplying the number of current positions by the closing price of the last bar ( self.current_positions[s] * bars[s][0][5] ). New values ​​obtained are added to the all_holdings list:

 # portfolio.py def update_timeindex(self, event): """           .   , ..        (OLHCVI).  MarketEvent   . """ bars = {} for sym in self.symbol_list: bars[sym] = self.bars.get_latest_bars(sym, N=1) # Update positions dp = dict( (k,v) for k, v in [(s, 0) for s in self.symbol_list] ) dp['datetime'] = bars[self.symbol_list[0]][0][1] for s in self.symbol_list: dp[s] = self.current_positions[s] # Append the current positions self.all_positions.append(dp) # Update holdings dh = dict( (k,v) for k, v in [(s, 0) for s in self.symbol_list] ) dh['datetime'] = bars[self.symbol_list[0]][0][1] dh['cash'] = self.current_holdings['cash'] dh['commission'] = self.current_holdings['commission'] dh['total'] = self.current_holdings['cash'] for s in self.symbol_list: # Approximation to the real value market_value = self.current_positions[s] * bars[s][0][5] dh[s] = market_value dh['total'] += market_value # Append the current holdings self.all_holdings.append(dh) 

The update_positions_from_fill method determines exactly what FillEvent (buy or sell) was, and then updates the current_positions dictionary by adding or removing the appropriate number of stocks:

 # portfolio.py def update_positions_from_fill(self, fill): """   FillEvent     ,     . Parameters: fill - The FillEvent object to update the positions with. """ # Check whether the fill is a buy or sell fill_dir = 0 if fill.direction == 'BUY': fill_dir = 1 if fill.direction == 'SELL': fill_dir = -1 #     self.current_positions[fill.symbol] += fill_dir*fill.quantity 

The corresponding update_holdings_from_fill method update_holdings_from_fill similar to the one described above, but it updates the holdings value. To simulate the cost of execution, the method does not use the price associated with the FillEvent . Why so prosithodit? In the backtesting environment, the execution price is actually unknown, which means it must be assumed. Thus, the exercise price is set as the “current market price” (closing price of the last bar). The value of the current positions for a particular instrument is then equated to the price of execution multiplied by the number of securities in the order.

After determining the exercise price, the current value of holdings, available funds and total values ​​can be updated. Also the total commission is updated:

 # portfolio.py def update_holdings_from_fill(self, fill): """   FillEvent    holdings   . : fill -  FillEvent,    . """ # Check whether the fill is a buy or sell fill_dir = 0 if fill.direction == 'BUY': fill_dir = 1 if fill.direction == 'SELL': fill_dir = -1 # Update holdings list with new quantities fill_cost = self.bars.get_latest_bars(fill.symbol)[0][5] # Close price cost = fill_dir * fill_cost * fill.quantity self.current_holdings[fill.symbol] += cost self.current_holdings['commission'] += fill.commission self.current_holdings['cash'] -= (cost + fill.commission) self.current_holdings['total'] -= (cost + fill.commission) 

Next, the abstract method update_fill from the abstract base Portfolio class is implemented. It simply executes the two previous methods update_positions_from_fill and update_holdings_from_fill :

 # portfolio.py def update_fill(self, event): """            FillEvent. """ if event.type == 'FILL': self.update_positions_from_fill(event) self.update_holdings_from_fill(event) 

A Portfolio object must not only trigger FillEvent events, but also generate an OrderEvent when receiving SignalEvent signal events. The generate_naive_order method uses a signal to open a long or short position, the target financial instrument and then sends the corresponding order with 100 shares of the desired asset. 100 here is an arbitrary value. In the course of real trading, it would be determined by a risk management system or a module for calculating the value of positions. However, in NaivePortfolio you can “naively” send orders right after receiving signals without any risk management.

The method handles the opening of long and short positions, as well as the exit from them based on the current number and a specific financial instrument. Then the corresponding OrderEvent object is OrderEvent :

 # portfolio.py def generate_naive_order(self, signal): """   OrderEvent           . : signal -   SignalEvent. """ order = None symbol = signal.symbol direction = signal.signal_type strength = signal.strength mkt_quantity = floor(100 * strength) cur_quantity = self.current_positions[symbol] order_type = 'MKT' if direction == 'LONG' and cur_quantity == 0: order = OrderEvent(symbol, order_type, mkt_quantity, 'BUY') if direction == 'SHORT' and cur_quantity == 0: order = OrderEvent(symbol, order_type, mkt_quantity, 'SELL') if direction == 'EXIT' and cur_quantity > 0: order = OrderEvent(symbol, order_type, abs(cur_quantity), 'SELL') if direction == 'EXIT' and cur_quantity < 0: order = OrderEvent(symbol, order_type, abs(cur_quantity), 'BUY') return order 

The update_signal method simply calls the method described above and adds the generated order to the event queue.

 # portfolio.py def update_signal(self, event): """   SignalEvent        . """ if event.type == 'SIGNAL': order_event = self.generate_naive_order(event) self.events.put(order_event) 

The final method in NaivePortfolio is to generate a capital curve. It creates a stream of information about profits, which is useful for calculating the performance of the strategy, then the curve is normalized on a percentage basis. The initial account size is set to wound 1.0:

 # portfolio.py def create_equity_curve_dataframe(self): """  pandas DataFrame    all_holdings. """ curve = pd.DataFrame(self.all_holdings) curve.set_index('datetime', inplace=True) curve['returns'] = curve['total'].pct_change() curve['equity_curve'] = (1.0+curve['returns']).cumprod() self.equity_curve = curve 

The Portfolio object is the most complex aspect of the entire event-oriented back tester. Despite the complexity, the processing of positions here is implemented at a very simple level.

In the next article we will look at the last part of an event-oriented historical testing system - the ExecutionHandler object, which uses OrderEvent objects to create a FillEvent from them.

To be continued…

PS Earlier in our blog on Habré we have already considered the various stages of the development of trading systems. ITinvest and our partners conduct online courses on this topic.