How to create a chat bot for VKontakte using Python, Django and webhook

Why another article about creating a chat bot?

Maybe I was looking badly, but I could not find a detailed guide to creating a python bot using the Django framework and the webhook approach, which is hosted by a Russian company. Most of the materials talk about the use of the Flask framework and the use of free Heroku and PythonAnywhere hosting sites. The experience of the Habr community helps me, so I decided to spend time writing this article as a sign of gratitude. I will describe the practical experience gained so as to enable everyone who is interested in it to save time and better understand how to make a Python bot using the Django framework on your hosting using the webhook approach.

Why paid hosting?

In my opinion, a viable version of the bot is when it is independent of your local computer and is available 24/7. To do this, you need a hosting on which there is: a web server, a database management system (to develop the bot's capabilities), registration of a domain name, obtaining an SSL certificate for it and technical support for the entire farm. Such services cost money. I pay 138 rubles a month to the hosting for maintaining the infrastructure for bot operation: Python + Django support, MySQL 25GB DBMS, SSH support.
In most of the lessons, I saw that a personal computer is used as a server or free hosting with limited time, etc. In the examples, the bot periodically polls the messenger server for new messages from users. This is an extra load on the messenger servers, so the bot can “ban” for a while. All this in my opinion is not vital for productive use. But for the test and training is quite possible.

What is webhook and why is it?

For a productive, I consider the right decision to use a webhook, that is, an approach in which our bot expects messages from the messenger server, rather than “hammer” it with periodic queries: are there new messages or not. With webhook it will be like this: the user wrote a message, the messenger server sent it to your bot, he received the message, processed it and answered.

Why django?

I decided to make a bot in python, so I connected python support to the hosting. But I did not have a choice of framework - hosting only has Django. They say it runs Instagram, Pinterest, Bitbucket and Mozilla. Perhaps that is why hosting offers exactly it.
')

Why VKontakte, not Telegram or Viber?

To move from simple to complex it was important for me to find the easiest and most understandable way to set up a webhook. VKontakte was the most understandable to me because of the clear help and ease of connecting the webhook in the community control panel in the section “Management - Working with API”. A description of how I set everything up and connected will be next. In the future, I want to make my bot available in Viber. And with Telegram is not on the way yet, since my hosting in Russia, and from Russia Telegram is blocked. To avoid problems with Telegram, you can buy hosting abroad.

How to install webhook for the VKontakte bot?

Domain name https: // . First you need to register the domain name of the site and get an SSL certificate for it.

I did not want to use the root domain for the chat bot, so after registering the domain, I made a subdomain and received an ssl certificate for it.

I made all these manipulations on the website hosting in my personal account.
As a result, I received the address of the site mybot.mysite.ru and ssl-certificate to it.

We get for bot bot access key (token) VKontakte. First I created a closed group, then I went into the "management" group, in the section "Working with API". In the “Access Keys” tab there is a token, and in the “Callback API” tab, the webhook settings.



Install and configure Django . Perhaps to run your script in python Django is not needed, but I do not know how it is possible differently.

Using PuTTY, I connected to the server via SSH, set up and activated a virtual environment.

SSH:

virtualenv-2.7 virtualenv/myEnv . virtualenv/myEnv/bin/activate 

the command in the first line creates a virtual environment, and the command in the second one activates it (note the space after the dot). Version 2.7 is dictated by hosting and may differ in your case. So read the hosting help.

Next, install Django
SSH:

 pip install 'django<2' 

I installed Django version not earlier than the second, since python 2.7 is used on the hosting, and only Django version less than 2 works with it.

And installed the python module to work with the VK API
SSH:

 pip install vk 

FTP:
Created a folder for django-projects in the root directory on the hosting. Called her django.

SSH:
Created a new project.

 cd django/ django-admin.py startproject mybot 

as a result, a folder will be created with the name of the project (in our case, this is “mybot”) in the / django folder. It will contain the initial project files created automatically:
/ django
/ mybot - project folder
/ mybot - module with our project settings
__init__.py
settings.py
urls.py
wsgi.py
manage.py

A project in Django is a group of applications. A Django application is a program that performs the actions laid down by the developer.

SSH:
Created an application.

 cd mybot python manage.py startapp vk_bot 

I moved to the / django / mybot folder and created a new application called “vk_bot”.
A folder with the name of the application has been created in the project folder, containing the application files created automatically:

/ django
/ mybot - project folder
/ mybot - module with our project settings
__init__.py
settings.py
urls.py
wsgi.py
manage.py
/ vk_bot - application folder
__init__.py
admin.py
apps.py
models.py
tests.py
views.py

FTP:
I downloaded all the project files on my laptop to work with the code.
I used the Atom application to work with project files and programming.

Atom:
Edited the project settings in the /django/mybot/mybot/settings.py file

 ... DEBUG = False ... ALLOWED_HOSTS = [ u'mybot.mysite.ru', ] ... 

Atom:
Edited URL address routing settings in the /django/mybot/mybot/urls.py file

 ... urlpatterns = [ url(r'^vk_bot/', include('vk_bot.urls')), ] ... 

FTP:
Created a file /django/mybot/vk_bot/urls.py with the following content:

 from django.conf.urls import url from . import views app_name = 'vk_bot' urlpatterns = [ url(r'^$', views.index, name='index'), ] 

Atom:
Edited the file /django/mybot/vk_bot/views.py - added a function called index to it, which will be executed when an address is requested in the browser

  https://mybot.mysite.ru/vk_bot/ 

views.py

 # -*- coding: utf-8 -*- from __future__ import unicode_literals from django.views.decorators.csrf import csrf_exempt from django.shortcuts import render from django.http import HttpResponse from bot_config import * # import token, confirmation_token and over constants from bot_config.py import json, vk # vk is library from VK """ Using VK Callback API version 5.5 For more ditalies visit https://vk.com/dev/callback_api """ """ From Django documentation (https://docs.djangoproject.com/en/1.11/ref/request-response/) When a page is requested, Django automatically creates an HttpRequest object that contains metadata about the request. Then Django loads the appropriate view, passing the HttpRequest as the first argument to the view function. This argiment is <request> in def index(request): Decorator <@csrf_exempt> marks a view as being exempt from the protection ensured by the Django middleware. For cross site request protection will be used secret key from VK """ @csrf_exempt #exempt index() function from built-in Django protection def index(request): #url: https://mybot.mysite.ru/vk_bot/ if (request.method == "POST"): data = json.loads(request.body)# take POST request from auto-generated variable <request.body> in json format if (data['secret'] == secret_key):# if json request contain secret key and it's equal my secret key if (data['type'] == 'confirmation'):# if VK server request confirmation """ For confirmation my server (webhook) it must return confirmation token, whitch issuing in administration web-panel your public group in vk.com. Using <content_type="text/plain"> in HttpResponse function allows you response only plain text, without any format symbols. Parametr <status=200> response to VK server as VK want. """ # confirmation_token from bot_config.py return HttpResponse(confirmation_token, content_type="text/plain", status=200) if (data['type'] == 'message_new'):# if VK server send a message session = vk.Session() api = vk.API(session, v=5.5) user_id = data['object']['user_id'] # token from bot_config.py api.messages.send(access_token = token, user_id = str(user_id), message = "Hello, I'm bot!") return HttpResponse('ok', content_type="text/plain", status=200) else: return HttpResponse('see you :)') 

In the views.py script, in the index (request) function, I had to disable the protection built into Django CSRF, since I received the error "403 Forbidden". CSRF - Cross Site Request Forgery protection - cross-site request forgery protection. How CSRF works can be read in this article .
To disable protection, I used the @csrf_exempt decorator. But in order to still provide this protection, but in a simpler way I used the secret key, which is prescribed in the group management section on the VKontakte site.

This piece of code is responsible for processing requests from the server, which it will send in order to connect our webhook to handle events. Let's just say, "confirmation" of our webhook'a.

 if (data['type'] == 'confirmation'):# if VK server request confirmation """ For confirmation my server (webhook) it must return confirmation token, whitch issuing in administration web-panel your public group in vk.com. Using <content_type="text/plain"> in HttpResponse function allows you response only plain text, without any format symbols. Parametr <status=200> response to VK server as VK want. """ # confirmation_token from bot_config.py return HttpResponse(confirmation_token, content_type="text/plain", status=200) 

Please note that I keep all the configuration settings in the bot_config.py bot's separate configuration file and therefore connect it at the beginning of the script:

 from bot_config import * # import token, confirmation_token and over constants from bot_config.py 

bot_config.py

 # -*- coding: utf-8 -*- """ Configuration file for VK bot """ # token issued on the VK group web-administration page token = '...' # confirmation token issued on the VK group web-administration page in "Callback API" section confirmation_token = '...' # secret key for cross site request forgery protection. It will be in each VK server request secret_key = '...' 

And in this piece of code it processes the messages of users:

 if (data['type'] == 'message_new'):# if VK server send a message session = vk.Session() api = vk.API(session, v=5.5) user_id = data['object']['user_id'] # token from bot_config.py api.messages.send(access_token = token, user_id = str(user_id), message = "Hello, I'm bot!") return HttpResponse('ok', content_type="text/plain", status=200) 

If something seemed incomprehensible, you can additionally read an article about the first setup of Django .

Magic for the web server . To configure the addressing of requests to the web server, I went to the server in the folder with domains via the FileZilla FTP client and created the folder " mybot.mysite.ru " there, put three files into it, the contents of which were taken from the help on the hosting:

.htaccess

 AddHandler wsgi-script .wsgi RewriteEngine on RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ /django.wsgi/$1 [QSA,PT,L] RewriteCond %{HTTP:X-Forwarded-Protocol} !=https RewriteRule .* https://%{SERVER_NAME}%{REQUEST_URI} [R=301,L] 

django.wsgi

 import os, sys virtual_env = os.path.expanduser('~/virtualenv/myEnv') activate_this = os.path.join(virtual_env, 'bin/activate_this.py') execfile(activate_this, dict(__file__=activate_this)) sys.path.insert(0, os.path.join(os.path.expanduser('~'), 'django/mybot')) os.environ['DJANGO_SETTINGS_MODULE'] = 'mybot.settings' from django.core.wsgi import get_wsgi_application application = get_wsgi_application() 

Here, “myEnv” is the name of the virtual environment you created, “django” is the folder in the root partition of the file system on the hosting, “mybot” is the name of the project that we created using Django.

index.html

We "bind" our webhook to the processing of messages in the created Vkontakte group.
To do this, we will return to the management section of our group on the site VKontakte (see the screenshot above). Enter in the field "Address" our webhook'a address

  https://mybot.mysite.ru/vk_bot/ 

and click "Confirm". If our function index (request) , written in the file /django/mybot/vk_bot/views.py works correctly, i.e. there are no typos or errors in it, then a green check mark will appear, symbolizing that everything is fine.

In order for our webhook to receive messages about new messages from the VKontakte server, in the management section of our group on the VKontakte website, in the “Types of events” tab, we tick off the “incoming messages”.

As a result, our script will receive these messages in json format:

 {"type":"message_new","object":{"id":891,"date":1541599508,"out":0,"user_id":1...1,"read_state":0,"title":"","body":"  "},"group_id":1...4,"secret":"uxSBw"} 

Note that in the json message there is a “secret” field. This is the same secret key that I registered in the group management section on VKontakte, instead of the protection built into Django CSRF, which I had to disable.

How to make the bot smarter and better?

You can create a database with the answers to the user and teach the bot to choose the answer that is closest in meaning to the user's question. I will tell about it in separate article.
It is possible and necessary to program user interaction scripts, so to speak, to support the conversation.

Successes in your creative work!