Features of Telegram bot development with Google API in Docker
Briefly about the bot : gets a list of YouTube-channels of the user and notifies of new videos with the ability to recall it later.
In the article I will tell about the peculiarities of writing this bot and interaction with Google API. I love brevity, so the article will have little “water”.
Which questions the article will answer:
')
Stack :
You can receive commands from Telegram using Long-polling and Webhook. Judging by the reviews on the Internet, Long-polling stops working after a while - Telegram returns 500 errors, so I decided to do it right away via Webhook.
Webhook is the address to which Telegram will send commands and messages from users, so it must be external, and how then to develop locally?
Here come to the rescue services such as: ngrok and Localtunnel ( link 1 , link 2 ).
Both of these services generate a random level 3 domain. If you want a static one, then you will have to pay in ngrok, but not in Localtunnel.
I needed to build OAuth Callback Url, which was tied to the OAuth 2.0 client ID in the Google API, so it’s more convenient if it is static. For this reason, I used Localtunnel.
Both of these services provide HTTPS with a normal valid certificate, so there will be no problems with Telegram.
Telegram allows you to use self-signed certificates. Instructions are on their official website. But then browsers will not trust him, and the same certificate will be used for OAuth Callback Url, so a valid certificate was needed. Come to the rescue Let's Encrypt .
Generating a certificate is not a problem on the Internet full of instructions. The only thing, as far as I understand, it must be generated on the server where it will be used (correct, if it is not).
At ubunte, I used the letsencrypt package and executed the command.
For Webhook Telegram to work, you need to transfer the CA certificate so that Telegram starts to trust it.
In the case of a self-signed certificate, this must be done and the public key must be transferred.
In the case of Let's Encrypt, you don’t need to transfer anything, but you need to configure HTTPS on the web server correctly.
Let's Encrypt will generate 4 certificates:
It is privkey.pem + fullchain.pem that is needed for HTTPS, if you use HAProxy (most likely you need to configure it for others the same way) for Telegram to start trusting our bot.
You can transfer this certificate via Telegraf as follows:
Send a message with a button is not a problem ( Telegraf Markup & Extra can be used). Difficulties begin with the transfer of data and catching clicking on this button.
According to the InlineKeyboardButton documentation , the maximum data size is only 64 bytes . In most cases, this is enough, just keep in mind when developing your bot.
Next you need to process this data, all callbacks come in one function, so you have to disassemble what type of button you clicked on it. So, along with the data, you also need to pass this type. Begs the router. In Telegraf, this router has already been partially implemented - this is the class Router . You can create it, but you will need to parse the command and parameters yourself ( example ). Parameter separator also need to invent yourself. In my opinion this is the last century, but you can live with it.
The bot needs to get a list of user channels, and for this you need a token. This is how you can get it:
In theory, Google with access_type: “offline” should provide a refresh-token, but it just won't do that. I had to update the list of channels of the user in the background, so googling I found this option: to pass the generate_uthUrl method to pass the option approval_prompt: 'force' . Then Google will request the user offline account access and, if the user agrees, he will give us the necessary refresh-token. With it, we can update the access token at any time and receive the list of channels.
For this, the state option can be passed to the generateAuthUrl method. You can only send a string, so all objects need to be serialized / encoded / encrypted.
According to the transmitted data, you can save the token in the database and then receive it already by the user ID.
There are 2 ways: buy a domain or try register for free.
At first I was looking for a free one, so I’ll share with you a link to one of these 4nmv services - it gives you a level 3 domain for free and allows you to configure any ns record for it. Surely there are other services (share references, please), but this one also suited me.
But then I still bought a .com level 2 domain for 69 rubles in GoDaddy for solidity.
In production, I use Docker and Docker Compose. They allow you to quickly raise and update the bot on different hosting (if necessary).
Developing on Node.js, I wrote errors and debug messages to the console and when I deployed them in docker it was not convenient to watch them.
The Papertrail service can come to the rescue , it allows you to save any messages from anywhere, including in the syslog format.
To transfer all messages from all containers, you can use the gliderlabs / logspout image . It is configured very simply. Here is a docker-compose.yml clipping
The bot was launched at the end of December 2016 and is available as @youtube_subs_watcher_bot
→ Sources on GitHub
In the article I will tell about the peculiarities of writing this bot and interaction with Google API. I love brevity, so the article will have little “water”.
Which questions the article will answer:
')
- Where to get the external address of the site for Webhook
- Where to get the HTTPS certificate how to use it so that Telegram trusted it
- How to transfer data and handle clicks on inline buttons
- How to get perpetual OAuth token for google api
- How to transfer user data via OAuth callback url
- How to get a free domain level 3
Stack :
- Back-end: Node.js + Express.js
- DB: Mongo.js + mongoose
- Package Manager: Yarn (he is really fast)
- Telegram-bot framework: Telegraf
- Production: Docker + Docker Compose + Vscale.io
Features in the development of the bot
You can receive commands from Telegram using Long-polling and Webhook. Judging by the reviews on the Internet, Long-polling stops working after a while - Telegram returns 500 errors, so I decided to do it right away via Webhook.
Need external site address for webhook
Webhook is the address to which Telegram will send commands and messages from users, so it must be external, and how then to develop locally?
Here come to the rescue services such as: ngrok and Localtunnel ( link 1 , link 2 ).
Both of these services generate a random level 3 domain. If you want a static one, then you will have to pay in ngrok, but not in Localtunnel.
I needed to build OAuth Callback Url, which was tied to the OAuth 2.0 client ID in the Google API, so it’s more convenient if it is static. For this reason, I used Localtunnel.
Both of these services provide HTTPS with a normal valid certificate, so there will be no problems with Telegram.
In the production will need HTTPS
Telegram allows you to use self-signed certificates. Instructions are on their official website. But then browsers will not trust him, and the same certificate will be used for OAuth Callback Url, so a valid certificate was needed. Come to the rescue Let's Encrypt .
Generating a certificate is not a problem on the Internet full of instructions. The only thing, as far as I understand, it must be generated on the server where it will be used (correct, if it is not).
At ubunte, I used the letsencrypt package and executed the command.
letsencrypt certonly -n -d domain1.com -d domain2.ru --email admin@domain.ru --standalone --noninteractive --agree-tos What certificate and how to transfer it to Telegram
For Webhook Telegram to work, you need to transfer the CA certificate so that Telegram starts to trust it.
In the case of a self-signed certificate, this must be done and the public key must be transferred.
In the case of Let's Encrypt, you don’t need to transfer anything, but you need to configure HTTPS on the web server correctly.
Let's Encrypt will generate 4 certificates:
- cert.pem - public key
- chain.pem - CA certificate
- fullchain.pem - public key + CA certificate
- privkey.pem - private key
It is privkey.pem + fullchain.pem that is needed for HTTPS, if you use HAProxy (most likely you need to configure it for others the same way) for Telegram to start trusting our bot.
You can transfer this certificate via Telegraf as follows:
let cert = { source: '/path/public.pem' }; app.telegram.setWebhook(config.webHookUrl + '/' + config.webHookSecretPath, cert); Data transfer when pressing inline buttons
Send a message with a button is not a problem ( Telegraf Markup & Extra can be used). Difficulties begin with the transfer of data and catching clicking on this button.
According to the InlineKeyboardButton documentation , the maximum data size is only 64 bytes . In most cases, this is enough, just keep in mind when developing your bot.
Next you need to process this data, all callbacks come in one function, so you have to disassemble what type of button you clicked on it. So, along with the data, you also need to pass this type. Begs the router. In Telegraf, this router has already been partially implemented - this is the class Router . You can create it, but you will need to parse the command and parameters yourself ( example ). Parameter separator also need to invent yourself. In my opinion this is the last century, but you can live with it.
Interaction with Google API
The bot needs to get a list of user channels, and for this you need a token. This is how you can get it:
- Create a project through the Google Developers Console
- We select the necessary API. In my case it is Youtube Data API
- Create an OAuth 2.0 client ID. In the field "Allowed redirection URI" you can specify localhost with the desired port
- As a result, we will be given ClientId and ClientSecret.
- We put Google APIs Node.js Client
- Based on this pair, we generate a link for the user using the googleapis .auth.OAuth2.generateAuthUrl method. About this in detail and with examples is written in the package description.
- After the user follows the link and gives permission, we will receive an access token
Why does the access token live for only 1 hour?
In theory, Google with access_type: “offline” should provide a refresh-token, but it just won't do that. I had to update the list of channels of the user in the background, so googling I found this option: to pass the generate_uthUrl method to pass the option approval_prompt: 'force' . Then Google will request the user offline account access and, if the user agrees, he will give us the necessary refresh-token. With it, we can update the access token at any time and receive the list of channels.
How to transfer user data via callback url?
For this, the state option can be passed to the generateAuthUrl method. You can only send a string, so all objects need to be serialized / encoded / encrypted.
According to the transmitted data, you can save the token in the database and then receive it already by the user ID.
Google will not give a token if the callback url is IP
There are 2 ways: buy a domain or try register for free.
At first I was looking for a free one, so I’ll share with you a link to one of these 4nmv services - it gives you a level 3 domain for free and allows you to configure any ns record for it. Surely there are other services (share references, please), but this one also suited me.
But then I still bought a .com level 2 domain for 69 rubles in GoDaddy for solidity.
Production
In production, I use Docker and Docker Compose. They allow you to quickly raise and update the bot on different hosting (if necessary).
Docker logger
Developing on Node.js, I wrote errors and debug messages to the console and when I deployed them in docker it was not convenient to watch them.
The Papertrail service can come to the rescue , it allows you to save any messages from anywhere, including in the syslog format.
To transfer all messages from all containers, you can use the gliderlabs / logspout image . It is configured very simply. Here is a docker-compose.yml clipping
logger: image: gliderlabs/logspout:latest volumes: - /var/run/docker.sock:/var/run/docker.sock command: 'syslog+tls://logsX.papertrailapp.com:PORT' restart: always How to run Yarn in a Docker container
... RUN curl -o- -L https://yarnpkg.com/install.sh | bash RUN $HOME/.yarn/bin/yarn install ... Result
The bot was launched at the end of December 2016 and is available as @youtube_subs_watcher_bot
→ Sources on GitHub
Source: https://habr.com/ru/post/319016/
All Articles