What is this article about?

This article is a brief story about how you can successfully integrate a Telegram bot and an external service with the help of available tools (Firefox) and Python.

The material will be interesting to those who have heard about Telegram bots, but do not know how to approach them and what tasks can be solved with their help. Knowledge of Python is assumed.

Picture to attract attention:
')

( link to the original )

TL; DR

From the article you will learn:

1. How can I use the browser to find out which request is sent to the server when a button is clicked?


2. How easy is it to send a request to the server using Python?


3. How to write a bot in Python?

Problem

There is a single transport card "Strelka" , which allows you to save a lot on trips from Moscow to the region. The reason for the savings is that the fare when paying in cash is 20-30% higher than when paying by the mentioned transport card.

You cannot find out the balance on the account when making a payment. Probably due to the reader’s lack of constant communication with processing servers. Http://strelkacard.ru and mobile applications for popular platforms are provided for information on the status of the account.

After updating the phone to Android 6, something unpleasant happened: the official application became stable to fly out at launch. Quite a few decent users left a corresponding message on the application page in Google Play. But things are there.


The need to obtain information about the balance usually appears at the time of payment of the trip. It’s not very convenient to do this from a mobile device through the site, even when using a personal account.

The question arose: "Is it possible to find out the balance of the card without unnecessary gestures?"

We investigate interfaces

There are at least two places where you can get information about the card balance:

1. Mobile app
2. Official site

Find out what requests the application generates can be, but not trivial. It is much easier to use the familiar browser and its web developer tools. The plugin for Firefox Firebug is very convenient for use.

Go to the page http://strelkacard.ru/ and activate the panel Firebug. Go to the tab Net . If you now enter the card number and request a balance, you can see the request generated to receive this data:



Right click on this request and select the 'Copy as cURL' . This action will lead to the fact that the clipboard will be the command to call the console utility curl with the parameters that allow you to send exactly the same request, which was generated by the browser.

Example:

 curl 'http://strelkacard.ru/api/cards/status/?cardnum=12345678901&cardtypeid=3ae427a1-0f17-4524-acb1-a3f50090a8f3' -H 'Accept: application/json, text/plain, */*' -H 'Accept-Encoding: gzip, deflate' -H 'Accept-Language: en-US,en;q=0.5' -H 'Connection: keep-alive' -H 'Host: strelkacard.ru' -H 'Referer: http://strelkacard.ru/' -H 'User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:44.0) Gecko/20100101 Firefox/44.0' 

If you enter this command without changes in the console, you will receive the expected response in JSON format :

 {"cardactive":false,"balance":100100,"baserate":3000,"cardblocked":false,"numoftrips":0} 

Let us postpone the question of analyzing this answer for later, and now we will try to understand what arguments are really needed to get the correct answer.

Since these actions are available to the user without authorization, you can most likely remove headers from the query without risking an incorrect answer. Remove all the -H arguments and try sending a simple GET request:

 $ curl 'http://strelkacard.ru/api/cards/status/?cardnum=12345678901&cardtypeid=3ae427a1-0f17-4524-acb1-a3f50090a8f3' {"cardactive":false,"balance":100100,"baserate":3000,"cardblocked":false,"numoftrips":0} 

The answer still meets expectations. Among the parameters transmitted with the request is cardnum , which is responsible for the card number, and cardtypeid , the name of which suggests that it is responsible for the type of card. If the need for the first parameter is not in doubt, then the significance of the card type is questionable. Especially provided that the user does not choose it. Let's try to get rid of him:

 $ curl 'http://strelkacard.ru/api/cards/status/?cardnum=12345678901' {"__all__":[" "]} 

The expectations were not met. This means that the type of card somehow participates in the formation of the answer. A quick overview of js-scripts revealed the fact that this parameter is static, and it can be simply remembered and used later when communicating with the server.

Description of the server response

Let's take a closer look at the server response:

 {"cardactive":false,"balance":100100,"baserate":3000,"cardblocked":false,"numoftrips":0} 

Obviously, the value we need is balance . If at the same time we compare the number with the fact that it was displayed in the browser after the execution of the request, it will become obvious that this value has the dimension of kopecks.

We communicate with the interface from Python

curl certainly good, but it's not enough for us to send a request and receive an answer in the form of text. We need to be able to process this text. The Python language supports the conversion of json into its own structure of objects from the box . It also comes standard with a library for sending HTTP urllib2 . Sending a request using this library looks like this:

 >>> import urllib2 >>> f = urllib2.urlopen('http://strelkacard.ru/api/cards/status/?cardnum=12345678901&cardtypeid=3ae427a1-0f17-4524-acb1-a3f50090a8f3') >>> print(f.read()) {"cardactive":false,"balance":100100,"baserate":3000,"cardblocked":false,"numoftrips":0} 

Looks nice. But we are going to change the cardnum parameter, so it would be great to rise one level of abstraction higher and not form the parameter string on our own. Put the requests library and use it:

 >>> import requests >>> CARD_TYPE_ID = '3ae427a1-0f17-4524-acb1-a3f50090a8f3' >>> card_number='12345678901' >>> payload = {'cardnum':card_number, 'cardtypeid': CARD_TYPE_ID} >>> r = requests.get('http://strelkacard.ru/api/cards/status/', params=payload) >>> print("Get info for card %s: %d %s" % (card_number, r.status_code, r.text)) Get info for card 12345678901: 200 {"cardactive":false,"balance":100100,"baserate":3000,"cardblocked":false,"numoftrips":0} 

The requests library provides a json() method for converting a json-formatted server response to a Python data structure.

 >>> r.json() {u'cardactive': False, u'balance': 100100, u'numoftrips': 0, u'baserate': 3000, u'cardblocked': False} >>> r.json()['balance'] 100100 

Create your own wrapper

We already know how to get the information we need using a couple of lines of Python code. Let's create a function to get information about the card balance. Write the following code in the checker.py file:

 #!/usr/bin/python import logging import requests CARD_TYPE_ID = '3ae427a1-0f17-4524-acb1-a3f50090a8f3' logger = logging.getLogger(__name__) def get_status(card_number): payload = {'cardnum':card_number, 'cardtypeid': CARD_TYPE_ID} r = requests.get('http://strelkacard.ru/api/cards/status/', params=payload) logger.info("Get info for card %s: %d %s" % (card_number, r.status_code, r.text)) if r.status_code == requests.codes.ok: return r.json() raise ValueError("Can't get info about card with number %s" % card_number) def get_balance(card_number): r = get_status(card_number) return r['balance']/100. 

The code is suitable for both Python 2.x and Python 3.x.

Now, in order to obtain data on the balance, two lines are enough for us:

 >>> import checker >>> checker.get_balance('12345678901') 1001.0 

The presented code also uses the logging library. With examples of its use in Russian can be found on Habré .

Here I will only note that in order for the logging messages to be displayed on the screen, you need to do something like:

 logging.basicConfig( format='%(asctime)s - %(name)s - %(levelname)s - %(message)s', level=logging.INFO) 

Wrap there. What's next?

Initially, the task was to conveniently receive information about the balance from a mobile device. If you write a separate application, then it should have the functionality to add a map (input text field + button) and get information about the balance on the added maps (button + output text field). The interface does not contain any special elements.

The required functionality can be implemented using the Telegram API for bots.

Is it difficult to write a bot?

In general, no. We now do not need to bot knew how to conduct small talk. He is only expected to support a limited number of teams.

Details and with pictures about Telegram bots can be found in the official blog: "Telegram Bot Platform" . What is important to us: the mobile application has the ability to suggest which teams are supported by the bot and send the necessary literally in a couple of clicks.

Create a bot

Step 1. We get a token

A detailed description of bots and their capabilities is on the "Bots: An introduction for developers" page. There is also information on how to create your bot. To do this, you need to ask for help from the authority among bots: bot @BotFather .



The main thing that you need to get as a result of communication with @BotFather - token to use the API.

Step 2. We are looking for a suitable wrapper library.

We have already started using Python. And even we are able to send requests to the server. You can implement your methods of interaction with the Telegram API. But if we can do it, then surely someone made it before us. Let's look for a suitable library for working with the Telegram bot API. A search in your favorite search engine for the keywords "python telegram bot" showed that there is a python-telegram-bot wrapper library. Version of the library at the time of publication: 3.3.

Using a wrapper is extremely simple: you need to create methods with the correct signature and register them via the addTelegramCommandHandler method addTelegramCommandHandler the correct object. Examples of use are given in the library description .

Step 3. Implement the logic of the bot

Our bot in its minimum working configuration must support the following commands:

1. add card by its number
2. card removal by its number
3. getting information about the balance on the cards

Next to the checker.py will create the checker.py file strelka_bot.py follows:

 #!/usr/bin/python from telegram import Updater, User import logging import checker TOKEN = 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx' # Enable Logging logging.basicConfig( format='%(asctime)s - %(name)s - %(levelname)s - %(message)s', level=logging.INFO) logger = logging.getLogger(__name__) logger.setLevel(logging.DEBUG) # Dictionary to store users by its id users = {} class UserInfo: def __init__(self, telegram_user): self.user = telegram_user self.cards = {} def add_card(self, card_number): card = CardInfo(card_number) self.cards[card_number] = CardInfo(card_number) class CardInfo: def __init__(self, card_number): self.card_number = card_number def balance(self): logger.info("Getting balance for card %s" % self.card_number) json = checker.get_status(self.card_number) return json['balance']/100. def log_params(method_name, update): logger.debug("Method: %s\nFrom: %s\nchat_id: %d\nText: %s" % (method_name, update.message.from_user, update.message.chat_id, update.message.text)) def help(bot, update): log_params('help', update) bot.sendMessage(update.message.chat_id , text="""Supported commands: /help - Show help /addcard - Add a card to the list of registered cards /removecard - Remove a card to the list of registered cards /getcards - Returns balance for all registered cards""") def add_card(bot, update, args): log_params('add_card', update) if len(args) != 1: bot.sendMessage(update.message.chat_id, text="Usage:\n/addcard 1234567890") return card_number = args[0] telegram_user = update.message.from_user if not users.has_key(telegram_user.id): users[telegram_user.id] = UserInfo(telegram_user) user = users[telegram_user.id] if not user.cards.has_key(card_number): user.add_card(card_number) bot.sendMessage(update.message.chat_id, text="Card %s has been added successfully" % (card_number)) else: bot.sendMessage(update.message.chat_id, text="Card %s has been already added. Do nothing" % (card_number)) def remove_card(bot, update, args): log_params('remove_card', update) if len(args) != 1: bot.sendMessage(update.message.chat_id, text="Usage:\n/removecard 1234567890") return card_number = args[0] telegram_user = update.message.from_user if not users.has_key(telegram_user.id): bot.sendMessage(update.message.chat_id, text="There are no cards saved for you") return user = users[telegram_user.id] if user.cards.has_key(card_number): user.cards.pop(card_number) bot.sendMessage(update.message.chat_id, text="Card %s has been successfully removed" % (card_number)) else: bot.sendMessage(update.message.chat_id, text="Card %s has not been added. Do nothing" % (card_number)) def get_cards(bot, update): log_params('get_cards', update) telegram_user = update.message.from_user if not users.has_key(telegram_user.id) or len(users[telegram_user.id].cards) == 0: bot.sendMessage(update.message.chat_id , text="There are no saved cards for you. Please add a card by typing /addcard CARD_NUMBER") return user = users[telegram_user.id] cards = user.cards response = "" for card in cards.values(): if len(response) != 0: response += '\n' response += "Card %s balance: %.2f"%(card.card_number, card.balance()) bot.sendMessage(update.message.chat_id, text=response) def main(): updater = Updater(TOKEN) # Get the dispatcher to register handlers dp = updater.dispatcher # Add handlers for Telegram messages dp.addTelegramCommandHandler("help", help) dp.addTelegramCommandHandler("addcard", add_card) dp.addTelegramCommandHandler("removecard", remove_card) dp.addTelegramCommandHandler("getcards", get_cards) updater.start_polling() updater.idle() if __name__ == '__main__': main() 

The TOKEN variable TOKEN be placed in the token received from @BotFather .

Step 4. Run the bot

Running the resulting strelka_bot.py brings our bot to life.

 $ python simple_strelka_bot.py 2016-03-06 12:02:11,706 - telegram.dispatcher - INFO - Dispatcher started 2016-03-06 12:02:11,706 - telegram.updater - INFO - Updater thread started 2016-03-06 12:02:26,123 - telegram.bot - INFO - Getting updates: [645839879] 2016-03-06 12:02:26,173 - __main__ - DEBUG - Method: get_cards From: {'username': u'username', 'first_name': u'Firstname', 'last_name': u'Lastname', 'id': 12345678} chat_id: 12345678 Text: /getcards 

Now he will respond to messages and write in the console information about which messages the bot comes in and what errors occur. The bot can be terminated using the Ctrl+C combination, which sends an interrupt signal ( SIGINT ).

After adding the cards of interest to us, information about their balances can be obtained with the help of a single /getcards .



You can also add saving users variable when changing, so as not to lose state when restarting the bot. The shelve module is great for this. Examples of use are listed at the end of the documentation page. Its use should not cause difficulties.

Exploring the capabilities of the python-telegram-bot library

Let's look at the python-telegram-bot library description. There is an interesting JobQueue functionality. It allows you to perform pending and repeatable tasks.

How can I use the ability to perform periodically running tasks? The following possible improvements are seen:

1. Periodically (once a day) send the user a message with information about the current balance on the registered cards.
2. Periodically (several times per hour) check the balance on the cards and report on its change.
   To do this, you need to add to CardInfo field that stores the current balance and its value until the last update. And the method for the update itself: some update() .
3. The previous version, but with the ability to set a threshold value that determines whether it is worth saying something to the user or not to disturb it.

Here you will need to add to the user a field that stores this threshold: threshold . To set the value for this field, you will need a separate command for the bot.

The implementation of option number 3 can be seen in the bot repository on Github .

Summary

Using the requests library, the Telegram bot API and the python-telegram-bot library managed to create a bot that notifies users in time that the balance on their cards has dropped to critically low values. The task of replenishing the card has remained beyond the scope, but the functionality of the "personal account" on the official website allows it to be comfortably solved with a PC.

On Github you can see a more complete bot implementation using the shelve module and JobQueue functionality: strelka_telegram_bot .

UPD

The user itspers in the comments suggested that returning the application "Arrow" to life can be an explicit indication of the required rights in the settings. By the selection method it was established that the rights of 'Location' and 'Phone' are required.

UPD2

Added code for Python 3 based on the prefrontalCortex comment .