Good day, Habrahabr! This material is a continuation of the first part , which highlights the tools and opportunities for developing a product on a cloud platform. An example is the current mobile expansion of access to the schedule of couples at the university - Telegram-bot.

Having spent time of the first publication, you can familiarize not only with initial expansion and necessary resources, but also with the review of the application from the author. The information below is concentrated to a greater extent on the description of the technical side and to a lesser extent on the conclusions of the current results.

UPD : the project was successfully refactored using the author's framework for university timetables - “Rutetider” ( article on Habrahabr , GitHub ). Therefore, some points in this article may not coincide, some may not exist or look different, but the concept is preserved.

Technical review of the project

In parallel with the explanations, you can count on screenshots “from the scene of events” for a deeper understanding of the writing process, it is also recommended to pay attention to the full source code ( github.com ) and with sufficient interest - delve into the links provided due to the lack of description of one or another at the discretion of the author, explanations of the components used and provided. If part of the description is not clear, the question can be asked to the author in the comments or dialogs, or take two minutes to interact with the subject area directly, which will allow better orientation in the article and the examples given.
')

University Couples Schedule

At the time of designing the application architecture, there were no other options for obtaining a schedule of lectures other than directly collecting information from the official site (parsing). The API for the platform on which the schedule is located is not available, and there is no opportunity to extract information directly from the database.

The required technical operation ( github.com ) is closely related to useful scheduled work on the server, details of which are located at the end of the section.

The modules used are:

• Beautiful Soup - working with HTML markup data structure;
• Requests - send HTTP requests.

The dataLib.py file contains an array of key-values ​​for querying each academic group and a function to add a dateToDateForm () time range to define the date frames for which a schedule is needed.

groupExample = {'TimeTableForm[date1]': firstDay, 'TimeTableForm[date2]': lastDay, 'TimeTableForm[faculty]':5, 'TimeTableForm[course]':3, 'TimeTableForm[group]':613} 

Next, send a post-request to receive the page object and then transfer it to the soup object, which has a number of methods for convenient operation.

 for formData in dataLib.dateToDateForm(firstDay, lastDay): r = requests.post(config.url, data = formData) soup = BeautifulSoup(r.text, 'html.parser') dataProcessing.dataProcessing(soup) 

The dataProcessing module has the same function in which the necessary data is processed. For example, the variable mainHtmlTable is a base table with a schedule that has its own unique identifier.

 mainHtmlTable = soup.find("table", {"id": "timeTableGroup"}) 

(part of the table for illustrative example, the full version can be "felt" by reference , after selecting the faculty, course and group)

Another example is dates based on the receipt of all elements of a block type that have a text of ten length inside themselves (“12.12.2016” - ten characters).

 divDate = [i.text[:-5] for i in mainHtmlTable.findAll('div') if len(i.get_text()) == 10][:-2] 

Next comes a call to a number of interrelated methods and functions that provide final values ​​for further convenient work.

I use three arrays with the name of faculties, courses and groups, directly by the schedule and dates, which allow us to conveniently process data and perform further operations.

General concept of user interaction

One of the most important qualities of the development is the vision of the final result and the competent construction of the architecture. At any stage of professional development, it is necessary to make great efforts for modularity and compactness of written code, interconnection of system components and simplicity of what is happening.

The work done was no exception, despite the opposite at the starting point of the product, and a certain conceptual design pattern was formed for most operations, including a Handler from the user, a series of subsequent operations (Query) and the generation of source data (Reply ) in reply.


There is nothing unusual in this pattern, and if you can determine such a model of application behavior in advance, you can prevent wasting a lot of time and effort in vain.

Request Processing Example

In the subsequent section of the main file handler ( github.com ) of all requests, you can observe a post-request from the user who made a selection of one of the menu items with the name “Updates” (this text value is available as an attribute of the message object “message” due to duplication of the menu selection to chat), entering user data (first name, last name, chat ID, text value “news” as menu item selection and nickname) using the otherFeature () method of middlewareUserData class ( github.com ). This function records user actions for application usage statistics. After previous processing, a text message with information about the location of the project news is sent as a response by the associated chat ID.

 @bot.message_handler(func=lambda mess: "" == mess.text, content_types=['text']) def handle_text(message): middlewareUserData.otherFeature(message.chat.first_name, message.chat.last_name, message.chat.id, 'news', message.chat.username) bot.send_message(message.chat.id, '       — https://telegram.me/dutbotupdates') 

An illustration of the process is provided below. Pictures associated with database tables are clickable (you can look better), because they are presented in poor quality, but just so as not to lose the integrity of the development picture.

The concept of getting a schedule

The schedule for today or tomorrow is available in two ways. Consider the first option - through the gradual data entry: you must select the faculty, course and group. Let me remind you that the “text” attribute of the “message” object is the textual value for selecting a menu item.

1. Button “Get schedule” - the user sends the basic data set:

 middlewareUserData.getUser(message.chat.first_name, message.chat.last_name, message.chat.id, message.chat.username) 

Therefore, the action is displayed in the database table in this format (the following examples of the database operation at the end of the section):


2. The choice of faculty:

 middlewareUserData.updateFaculty(message.text, str(message.chat.id)) 

3. The choice of the course and group, the script of which is somewhat different from the previous ones, because the display of the list group on the screen occurs after a request to the table with a full schedule, where the entire list of groups is withdrawn to the faculty and course set by the user

 middlewareUserData.updateCourse(message.text[:1], str(message.chat.id)) fucAndCourse = middlewareUserData.getFacultyAndGroup(str(message.chat.id)) groupList = selectData.selectGroup(fucAndCourse[0], fucAndCourse[1]) basicMarkupRows.markRowGroupList(groupList, message) 

4. Button “Today” or “Tomorrow”, the code of which contains the current set of dates (today, tomorrow, today's index), input check from the user (the database is one for all, you need to take the result of a specific user) and the schedule display itself ( github.com ):

 day = selectData.selectDates() lastGroup = middlewareUserData.summaryVerification(str(message.chat.id)) bot.send_message(message.chat.id, "   ({0}):".format(day[1])) message.text = printController.show(lastGroup, day[1]) bot.send_message(message.chat.id, message.text) 

Illustration of the alternate filling of the table to user actions:

Go back to the previous menu.

If you carefully read the previous section, you would definitely pay attention to the most important element of the table - “empty”. The function of returning back is repelled directly from the number of such records:

“Get schedule” - a priority object is formed together with four empty values ​​(the text value is “empty”) because no faculty, course, group and final menu item is selected (for example, a schedule for a specific day or a subscription to a group);

The choice of the faculty is already one “empty” less, the choice of the course is already two.

The script ( github.com ) determines the number of such fields in the row of the table for the user, knowing in advance how much of which method to call. For example, we settled on choosing a course, which means that we made a choice of faculty and we had three “empty” meanings:

 if emptyCount == 3: backButton.cancelOperation(str(message.chat.id), 3) basicMarkupRows.markRowGetFacultyList(message) 

The cancelOperation ( github.com ) method defines the function that clears the field we want to cancel:

 if self.emptyCount == 3: self.cancelFaculty(self.chatid) 

If you make a mistake with the faculty, you need to change it. Under the hood, the situation is the following: by user ID, we change the value of the faculty (for example, “Information Technology”) to an empty value (“empty”) through a simple SQL query “Update”:

 def cancelFaculty(self, chatid): self.chatid = chatid self.cursor.execute("UPDATE statistic SET faculty = (%s) WHERE id IN (SELECT max(id) FROM statistic WHERE chatid = (%s))", ('empty', self.chatid)) self.connection.commit() 

It is enough to change the data in the statistics table, take the “faculty” field, where the table field ID is maximum for a specific user ID (that is, the choice of the last line of the table for the desired user is the current state of the menu and data).

Subscribe to group schedule

The function of arranging access to the group schedule for two days from the first menu is based on one entry of its group into the table, that is, you must select the desired group and confirm the subscription available in the most recent menu ( github.com ).

 if message.text == "   ": middlewareUserData.subscribe(message.chat.first_name, message.chat.last_name, message.chat.id, lastGroup, message.chat.username) message.text = '    {0}.'.format(lastGroup) bot.send_message(message.chat.id, message.text) 

The subscribe () function ( github.com ) inserts familiar data to a separate, unrelated subscription storage table, but the identifier and the group itself are the most important ones.

 def subscribe(self, firstname, lastname, chatid, group, username): * -* self.cursor.execute("INSERT INTO subscribers (firstname, lastname, chatid, groupa, username) VALUES (%s, %s, %s, %s, %s)", (self.firstname, self.lastname, self.chatid, self.group, self.username)) self.connection.commit() 

Scheduled work on the server

APScheduler's easy- to-use library, perfectly compatible with the platform on which the application is located, allows you to schedule a variety of tasks to be performed at a specific time.

My bot needs a daily update of the schedule of pairs of today and tomorrow's dates, which are successfully available through the function in the code below. Since the server is located in a different time zone, I need to take away two hours from the required time (updates occur at 00:01). From Monday to Friday, information is withdrawn for the current week, on Sunday (the index of the day is five) - the next.

 @sched.scheduled_job('cron', day_of_week='mon-sun', hour=22, minute=1) @sched.scheduled_job('cron', day_of_week='mon-fri', hour=22, minute=1) @sched.scheduled_job('cron', day_of_week=5, hour=22, minute=1) 

The source code ( github.com ) is not really operating in combat conditions and is still being finalized, but the concept is defined correctly (fixed at 15.12).

Work with databases

The Heroku platform allows you to directly work with databases from a native console (Linux Ubuntu 16.04) or access via SQL Shell (Windows 7), pre-entering data on the type of passwords and logins.

The bot part of the database is limited to simple SQL queries (for example, select, insert, update), and the need to create and delete tables with the appropriate create and drop commands.

findings

While typing the program code on my knee, I didn’t even imagine that my expectations about the final effect would not come true, fortunately - for the better. But no one who is to some extent involved in the application thinks that the current state of affairs is not deserved. Being engaged in the modernization of the architecture, I have never planned to use the database or introduce additional functions. But in the end, all this was present in the development plan and it was possible not only to bring the project to a serious length, but also to obtain the necessary qualitative result both in general and in personal terms. At some stage, we had to “abandon” the usual pattern of behavior, since it was necessary to get used to different roles in the course of the software development life cycle. In the process, even managed to do social marketing, distributing information about the developed product among students, “sucking”, as it seemed then, at least some online.

Even such an insignificant result — more than 100 unique users daily — not only has a positive effect on the attitude to the author of the resource, who every day solves the problems of those around him, but is also an indisputable motivator to continue to engage in process improvements in all areas related to the subject area.