Financial Telegram-bot in 30 minutes with Market Data API

Usually in articles about fintech they write about how the exchanges work, which process huge amounts of data at huge speeds, how ingenious traders and quanta use sophisticated algorithms to earn (or lose, sometimes) billions of dollars, or about the work of the blockchain , provided with complex mathematical calculations. All this creates the impression that the level of entry into FINTECH development is prohibitively high. And partly it is true - the requirements for developers of high-loaded financial applications are strict and specific.

But everyone started out small, and we believe that any interested person is able to create an application in the financial sphere. Let's try to develop our own small application that will become useful for users in half an hour.



From the reviews of modern technologies of interaction with users, you can see how quickly all sorts of bots and helpers who understand requests in natural language are gaining popularity. Let's support this trend and create a simple Telegram bot that can tell the user something about the market on demand.
')

Data access

Let's start with a simple situation: the application will use current and historical data about the auction, and not send its own bids to the exchange. These data (so-called market data , or stock information) can be obtained from a number of companies for a relatively small fee or for free. With the submission of applications, everything is much more complicated (at least - more expensive), and we will look at this process in more detail in future articles.

What are the technologies for obtaining stock data? Their list is not too long: this is FIX-protocol (implementations may vary slightly from supplier to supplier), FAST , ITCH and several variants of binary and HTTP APIs (for example, CQG, EXANTE or MOEX). However, universalization is not so fundamental here: the set of data provided can vary greatly, and in any case, the integration will have to deal with the characteristics of a particular supplier.


We will use the recently appeared EXANTE Market Data API : you can start developing with it simply and quickly, registration in the system does not require additional confirmation, and access to the data is free. While the API is working in Tech Preview mode, but access is open to all comers.

Functionality

Having determined the range of possibilities, you need to decide what exactly our chat bot will do. There are a lot of options: from displaying currency rates to analytics of trading strategies for a particular stock instrument. So far, we will not go into the details of financial algorithms and try to do something useful, but at the same time quite simple.

One of the most understandable financial instruments is the shares of companies trading on stock exchanges. We will work with them, having chosen the US stock market for simplicity, since it is easiest to get fundamental data from it, and trading there is the most active.

What interests a novice investor? Of course, the choice of a portfolio of shares, by investing in which, he will be able to make a profit. There are many ways to choose stocks: you can read reviews , you can focus on the portfolios of the best investors, like Warren Buffett or Bill Ekman , and you can use analytical methods. One of the generally accepted and most common methods is the company's valuation using the P / E metric ( price / earnings ratio ). P / E is calculated as the ratio of the company's current share price to the Earning Per Share (EPS, earnings per share) indicator.

Thus, our chat bot will help the investor decide whether to include shares of a certain US stock market company in his portfolio, based on the current price / earnings ratio. A high P / E relative to other companies in this industry will show that stocks have growth potential. Low, on the contrary, will make it clear that in the future the company may face problems.

Architecture

So, as the main source of stock information, select EXANTE Market Data API (MD API). To obtain fundamental information - information about the general state of finances of a company - we will use the open source data datatables.org , which can be worked through YQL (Yahoo! Query Language).

To implement the bot itself, take Python 3, and in order to run it as fast as possible, let's apply a framework that supports all the necessary Telegram: python-telegram-bot methods.

To work with Telegram we will use a polling of new messages from the server, since in the prototype we do not count on a large amount of traffic.

We will think in advance that the application could work not only with one client. To do this, we will process requests in separate threads. To synchronize and run threads, we use the built-in capabilities of the python-telegram-bot framework and synchronization primitives available in Python.

All selected external services are accessible via HTTP, so we will use the well-known Requests module to work with them.

Surely many investors will be interested in the same stocks that are known to us, so we will add a caching layer in order to use resources more efficiently.

MD API requires authorization of requests using JSON Web Token , to generate tokens we will take the PyJWT library.

API connection

To get started with the MD API, you need to register on the EXANTE website for developers .

After registering on the portal, a dashboard becomes available with access data and application management. Create an application for our bot there:



We'll get the bot as described in the Telegram documentation via correspondence with the BotFather robot :

Implementation

To begin with, we will teach the bot to process the received requests. From each message, we will try to isolate the stock tickers and issue information on them so that the dialogue could look like this:

- Hello, robot, today I heard about AAPL in the news, it seems that this is some kind of fruit company, I think to invest money there, what do you say?
- Shares of AAPL (Apple Inc., NASDAQ Exchange) have a current P / E rating of 14, a share price of $ 117.06
- Thanks, but what about NVDA and GOOG?
- NVDA (Nvidia Corp., NASDAQ): P / E 69, priced at $ 105.7
GOOG (Alphabet Inc., NASDAQ): P / E 29, priced at $ 796.42

Initialize the bot and create message handlers:

# -*- coding:utf-8 -*- import re from sys import path from configparser import ConfigParser from telegram import ParseMode, Emoji from telegram.ext import Updater, CommandHandler, MessageHandler, Filters config = ConfigParser() config.read_file(open('config.ini')) # Create telegram poller with token from settings up = Updater(token=config['Api']['token']) dispatcher = up.dispatcher # Welcome message def start(bot, update): msg = "Hello {user_name}! I'm {bot_name}. Ask me about stocks!" # Send the message bot.send_message(chat_id=update.message.chat_id, text=msg.format( user_name=update.message.from_user.first_name, bot_name=bot.name)) def process(bot, update): msg = "I will try to show info on {tickers}" tickers = re.findall(r'[AZ]{1,4}', update.message.text) bot.send_message(chat_id=update.message.chat_id, text=msg.format(tickers=", ".join(tickers))) def main(): # Add handlers to dispatcher dispatcher.add_handler(CommandHandler("start", start)) dispatcher.add_handler(MessageHandler(Filters.text, process)) # Start the program up.start_polling() up.idle() if __name__ == '__main__': main() 

Now our bot already knows how to allocate stock tickers, but can’t do anything else with them.

Let's write an interface for working with the Market Data API and generating tokens. We use documentation and authorization guide .

 import jwt # token expiration time in seconds EXPIRATION = 3600 class MDApiConnector(): token = (None, None) algo = "HS256" def __init__(self, client_id, app_id, key): self.client_id = client_id self.app_id = app_id self.key = key def __get_token(self): now = datetime.now() # if there is token and it's not expired yet if self.token[0] and (now - self.token[1]).total_seconds() < EXPIRATION: return self.token[0] claims = { "iss": self.client_id, "sub": self.app_id, "aud": ["symbols", "ohlc"], # NB: only allowed scopes can be accessed "iat": int(now.timestamp()), "exp": int(now.timestamp()) + EXPIRATION } new_token = str(jwt.encode(claims, self.key, self.algo), 'utf-8') self.token = (new_token, now) return new_token 

The full code of all modules is available in the repository: github.com/exante/telegram-bot-with-md-api

Add a separate stream that will periodically request bulk data on shares:

 class DataStorage(Thread): def __init__(self, connector): super().__init__() self.connector = connector self.stocks = {} def run(self): while True: timeout = 15 * 60 # 15 minutes try: self.stocks = connector.get_stocks() except Exception as e: logger.error(e) timeout = 30 # re-read in case of exception time.sleep(timeout) 

The API method for obtaining a list of US stocks may look like this:

 def get_stocks(self): stocks = self.__request("/types/STOCK") return {x['ticker']: {"id": x["id"], "exchange": x["exchange"], "description": x["description"]} for x in stocks if x.get("country") == "US"} 

After starting this stream and accessing it from the message handler, the bot will be able to display more useful data (P / E is still a stub here):


Add the Earning Per Share request, for this we will do a small wrapper over YQL with caching (in the near future we will be able to replace this call with a similar one from the MD API), which will request the value of “EarningsShare” for the selected action.

Now we can output the resulting EPS figure:


Last but not least: get current stock price. For greater performance, we should subscribe to the stream of updates with prices, but for the prototype, you can choose a simpler way: request the last day candle - this is the name of the price chart element popular among traders.


An example of a candlestick graph of the ratio of the DJI index and gold prices by year

A “candle” is built for a specific period (for example, a day or an hour) and in one figure it combines four figures: the price at the beginning of the period, the maximum and minimum price for the period, and the price at the end of the period. The abbreviation OHLC, denoting such a candle, is decoded as Open-High-Low-Close. The Close price of the most recent candle will correspond to the current stock price.

The method to get the last candle may look like this:

  def get_last_ohlc_bar(self, symbolId): # NB: we use internal symbolId, not ticker # 86400 (sec) - day duration ohlc = self.__request("/ohlc/%s/86400" % symbolId, {"size": 1}) return ohlc[0] 

Putting all the calls together, we get the following code for processing a single ticker:

  stock = storage.stocks.get(ticker) eps = fundamendal_api.request(ticker).get('EarningsShare') price = api.get_last_ohlc_bar(stock['id']) ratio = Decimal("%.4f" % price['close']) / Decimal(eps) msg = "{ticker} ({name}, {exchange}): EPS {eps}, P/E {ratio},  ${price} \n".format( ticker = ticker, name = stock['description'], exchange = stock['exchange'], ratio = "%.2f" % ratio, price = price['close'], eps = eps ) 

And now our bot has become really useful! He can talk about the current situation on the stock market and even recommend something:

Project development

The current project can be found at github.com/exante/telegram-bot-with-md-api

Further development is possible in many areas. For example, you can use the data stream about the current stock price from the MD API ( /md/1.0/feed ) and not ask for the price each time from the “candles”, but simply take it from the internal cache where it will fall when the stream is updated.

You can add monitoring and analytics to the bot (for example, via botan.io ), as well as deploy it on some cloud hosting, like Heroku or Google App Engine.

The bot can be made more “alive” by adding more answers, and also teach it to display price change charts to give the investor more information for analysis. You can add any other metrics for stock pricing, keep a portfolio in the robot data to keep the investor informed of all changes, and extend the functionality — for example, to the Russian stock market.

Conclusion

By connecting the EXANTE Market Data API and using the open fundamental information, in a short time we developed a functional robot that will help the user quickly assess the situation on the market. In the process, we learned about some ways to evaluate stocks in the market and the terminology used in stock trading.

In addition, we considered the possibilities of development, and even such a small robot has a lot of them. There are many more ways to use market data - and many users who are interested in your financial applications.

In February, EXANTE will conduct a hackathon dedicated to working with market data using the EXANTE Market Data API. The authors of the best chat bots and applications will receive prizes, and now is the time to prepare :) We’ll write more about the event a little later.

What API do you use? What would you like to do with market data?