Flask Mega-Tutorial, Part 7: Unit Testing

This is the seventh article in the series where I describe my experience of writing a Python web application using the Flask mic framework.

The purpose of this guide is to develop a fairly functional microblog application, which I decided to call microblog, in the absence of originality.


')

Brief repetition

In the previous parts of this guide, we gradually developed our microblogging, step by step adding new features. By this time, our application can use the database, can register and authorize users, and also allows them to edit their own data in the profile.

Today we will not add new features. Instead, we will try to make the already written code more sustainable, and also create our own framework, which in the future will help us prevent possible failures.

Search bug

At the end of the previous chapter, I mentioned that there is an error in the code. Now I will tell you what it is and we will see how our application reacts to such things.

The problem is that we have no way to ensure the uniqueness of user nicknames. The initial nickname is automatically selected. If the OpenID provider tells us the nickname, we use it. Otherwise, the first part of the user's email will be used as the nickname. If two users with the same nicknames try to register, only the first will succeed. By allowing users to change their data themselves, we made it worse, because here we also have no way to avoid collisions.

Before we solve these problems, let's take a look at how an application behaves when an error occurs.

Flask Debugging

First of all, create a new database. If you have Linux:

rm app.db ./db_create.py 

Windows:

 del app.db flask/Scripts/python db_create.py 

You will need two OpenID accounts to play the bug, better if these accounts are from different providers, otherwise there may be some difficulties with cookies. So, the procedure is as follows:

• Login using your first account.
• On the profile edit page, change the nickname to 'dup'
• Log out
• Log in with a second account
• On the profile edit page, change the nickname to 'dup'

Oops! We got an exception caused by sqlalchemy. The error text reads:

 sqlalchemy.exc.IntegrityError IntegrityError: (IntegrityError) UNIQUE constraint failed: user.nickname u'UPDATE user SET nickname=?, about_me=? WHERE user.id = ?' (u'dup', u'', 2) 

Below is the stack trace of this error, in which you can not only see the source code of each frame, but even execute your own code directly in the browser!

The description of the error is very clear. The nickname we tried to assign to the user is already in the database. Note that in the User model, the nickname field is declared as unique=True , which is why this situation arose.

In addition to this error, we have another problem. If the user's actions lead to an error (this or any other), he will see the description of the error and the whole traceback. This is very convenient during development, but we definitely would not want anyone except us to see it.

All this time, our application worked in otdalki mode. This mode is activated at startup using the debug=True argument passed to the run method. When we run the application on the battle server, you will need to make sure that the debug mode is turned off. For this we need another script (file runp.py ) :

 #!flask/bin/python from app import app app.run(debug = False) 

Now run the application using this script.

 ./runp.py 

Try again to change the nickname of the second account to 'dup'. This time we will not see a detailed error message. Instead, we get an HTTP error code 500 Internal Server Error . Flask generates this page when the debug mode is turned off and an exception occurs. The page looks so-so, but we at least do not disclose unnecessary details about the application.

However, we face two more challenges. First, the default error page 500 looks ugly. The second problem is much more serious. If everything is left as it is, we will never know how and when errors occur, since all errors are silenced now. Fortunately, these problems have very simple solutions.

Native HTTP Error Handlers

Flask allows applications to use their own pages to display errors. As an example, we implement our pages for errors 404 and 500, since they are most often found. For other errors, the process of creating your own pages will be exactly the same.

To declare our own error handler, we need the errorhandler decorator ( app/views.py ) :

 @app.errorhandler(404) def not_found_error(error): return render_template('404.html'), 404 @app.errorhandler(500) def internal_error(error): db.session.rollback() return render_template('500.html'), 500 

I do not think that this code needs clarification. The only expression worth noting here is db.session.rollback() . This function will be called as a result of the exception. If the exception was caused by an error interacting with the database, we need to roll back the current session.

Template for 404 error:

 <!-- extend base layout --> {% extends "base.html" %} {% block content %} <h1>File Not Found</h1> <p><a href="{{url_for('index')}}">Back</a></p> {% endblock %} 

As well as a template for error 500:

 <!-- extend base layout --> {% extends "base.html" %} {% block content %} <h1>An unexpected error has occurred</h1> <p>The administrator has been notified. Sorry for the inconvenience!</p> <p><a href="{{url_for('index')}}">Back</a></p> {% endblock %} 

In both cases, we still use base.html as the parent template, as a result of which both HTTP error messages look the same as the rest of our microblog pages.

Sending error messages to the mail

To solve the second problem, we will configure two application error reporting systems. The first one will send us an email every time an error occurs.

First of all, we need to configure the mail server and the list of administrators of our application. ( config.py ) :

 # mail server settings MAIL_SERVER = 'localhost' MAIL_PORT = 25 MAIL_USERNAME = None MAIL_PASSWORD = None # administrator list ADMINS = ['you@example.com'] 

Of course, you need to change these values.

Flask uses the logging module from the standard Python library, so setting up sending error messages to mail will be quite simple (file app/__init__.py ) :

 from config import basedir, ADMINS, MAIL_SERVER, MAIL_PORT, MAIL_USERNAME, MAIL_PASSWORD if not app.debug: import logging from logging.handlers import SMTPHandler credentials = None if MAIL_USERNAME or MAIL_PASSWORD: credentials = (MAIL_USERNAME, MAIL_PASSWORD) mail_handler = SMTPHandler((MAIL_SERVER, MAIL_PORT), 'no-reply@' + MAIL_SERVER, ADMINS, 'microblog failure', credentials) mail_handler.setLevel(logging.ERROR) app.logger.addHandler(mail_handler) 

We will send these letters only if the debug mode is turned off. It's okay if you do not have a configured mail server. For our purposes, the SMTP debug server, provided by Python, is quite suitable. To start it, type in the console (or on the command line if you are a Windows user):

 python -m smtpd -n -c DebuggingServer localhost:25 

After that, all emails sent by the application will be intercepted and displayed directly in the console. (Note. You can also use an extremely convenient SMTP server that does not require special knowledge to configure - Mailcatcher . This method is much more convenient, but it also requires installing Ruby.)

Write log to file

Receiving error messages in the mail is great, but sometimes it is not enough. In some cases, we will need to get more detailed information than the one that is in the traceback, and then we can use the ability to maintain a log file.

The configuration process is very similar to what we just did for mail (file app/__init__.py ) :

 if not app.debug: import logging from logging.handlers import RotatingFileHandler file_handler = RotatingFileHandler('tmp/microblog.log', 'a', 1 * 1024 * 1024, 10) file_handler.setFormatter(logging.Formatter('%(asctime)s %(levelname)s: %(message)s [in %(pathname)s:%(lineno)d]')) app.logger.setLevel(logging.INFO) file_handler.setLevel(logging.INFO) app.logger.addHandler(file_handler) app.logger.info('microblog startup') 

The log will be saved in the tmp folder under the name microblog.log . We used RotatingFileHandler , which allows us to set a limit on the amount of stored data. In our case, the file size is limited to one megabyte, while the last ten files are saved.

The class logging.Formatter provides the ability to set an arbitrary format of records in the log. Since we want to receive as much detailed information as possible, we will save the message itself, the timestamp , the status of the record, as well as the file name and line number from which the recording was initiated.

To make the log more useful, we reduce the logging level in both app.logger and file_handler , this will allow us to record not only errors, but also other information that may be useful. For example, we will record the launch time of the application. Now every time microblogging is launched without debug mode, this event will be saved to the log.

Currently, we have no need to use, however, if our application runs on a remote web server, diagnostics and debugging will be difficult. That is why you should take care in advance to get the information we need without stopping the server.

Error correction

So, let's finally fix a bug with the same nicknames.

As mentioned earlier, we have two problem areas where there is no duplicate check. The first is in the after_login handler, which is called when the user is authorized in the system and we need to create a new User object. Here is what we can do to get rid of the problem: (file app/views.py ) :

 if user is None: nickname = resp.nickname if nickname is None or nickname == "": nickname = resp.email.split('@')[0] nickname = User.make_unique_nickname(nickname) user = User(nickname = nickname, email = resp.email, role = ROLE_USER) db.session.add(user) db.session.commit() 

Our solution is to instruct the User class to create a unique nickname. Here is how it is implemented ( app/models.py ) :

 class User(db.Model): # ... @staticmethod def make_unique_nickname(nickname): if User.query.filter_by(nickname = nickname).first() == None: return nickname version = 2 while True: new_nickname = nickname + str(version) if User.query.filter_by(nickname = new_nickname).first() == None: break version += 1 return new_nickname # ... 

This method simply adds the counter to the nick until it becomes unique. For example, if the user “miguel” already exists, the method will offer the option “miguel2”, then, if there is such a user, “miguel3” and so on. Pay attention to the staticmethod decorator, we applied it, since this operation is not tied to a specific instance of the class.

The second place where the problem of duplicates is still relevant is the profile editing page. In this case, everything is somewhat complicated by the fact that the user himself chooses his nickname. The best solution in this case will be to check for uniqueness and in case of failure the proposal to choose another nickname. To do this, we need to add another validator for the corresponding field. If the user enters an existing nickname, the form simply does not pass validation. To add your validator, you must app/forms.py validate method (file app/forms.py ) :

 from app.models import User class EditForm(Form): nickname = TextField('nickname', validators = [Required()]) about_me = TextAreaField('about_me', validators = [Length(min = 0, max = 140)]) def __init__(self, original_nickname, *args, **kwargs): Form.__init__(self, *args, **kwargs) self.original_nickname = original_nickname def validate(self): if not Form.validate(self): return False if self.nickname.data == self.original_nickname: return True user = User.query.filter_by(nickname = self.nickname.data).first() if user != None: self.nickname.errors.append('This nickname is already in use. Please choose another one.') return False return True 

The form constructor now accepts a new argument, original_nickname . The validate method uses it to determine if the nickname has changed or not. If it has changed, we check for uniqueness.

We initialize the form with a new argument:

 @app.route('/edit', methods = ['GET', 'POST']) @login_required def edit(): form = EditForm(g.user.nickname) # ... 

We also need to display all errors that occur during form validation, next to the field in which an error was detected. (file app/templates/edit.html ) :

 <td>Your nickname:</td> <td> {{form.nickname(size = 24)}} {% for error in form.errors.nickname %} <br><span style="color: red;">[{{error}}]</span> {% endfor %} </td> 

That's it, the duplicate problem is solved! True, we still have a potential problem with simultaneous access to a database from several threads or processes, but this topic will be covered in one of the future articles.

Now you can try again to change the nickname to the existing one and make sure that the problem is no more.

Testing framework

With the development of the application, it is becoming increasingly difficult to verify that changes in the code will not break the existing functionality. That is why it is so important to automate the testing process.

The traditional approach to testing is quite good. You write tests to test all the capabilities of the application. Periodically, during the development process, you will need to run these tests to make sure everything is still stable. The wider the coverage of the tests, the more you can be sure that everything is in order.

Let's write a small framework for testing using the unittest module ( app/tests.py ) :

 #!flask/bin/python import os import unittest from config import basedir from app import app, db from app.models import User class TestCase(unittest.TestCase): def setUp(self): app.config['TESTING'] = True app.config['CSRF_ENABLED'] = False app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///' + os.path.join(basedir, 'test.db') self.app = app.test_client() db.create_all() def tearDown(self): db.session.remove() db.drop_all() def test_avatar(self): u = User(nickname = 'john', email = 'john@example.com') avatar = u.avatar(128) expected = 'http://www.gravatar.com/avatar/d4c74594d841139328695756648b6bd6' assert avatar[0:len(expected)] == expected def test_make_unique_nickname(self): u = User(nickname = 'john', email = 'john@example.com') db.session.add(u) db.session.commit() nickname = User.make_unique_nickname('john') assert nickname != 'john' u = User(nickname = nickname, email = 'susan@example.com') db.session.add(u) db.session.commit() nickname2 = User.make_unique_nickname('john') assert nickname2 != 'john' assert nickname2 != nickname if __name__ == '__main__': unittest.main() 

The unittest module unittest is beyond the scope of this article. If you are not familiar with this module, for now you can simply assume that our tests are in the TestCase class. The setUp and tearDown have a special meaning; they are performed before and after each test, respectively. In more complex cases, several test groups can be defined, where each group is a subclass of unittest.TestCase , then each group will have its own setUp and tearDown .

In our case, there is no need for any complex actions before and after the test. In setUp , the application configuration changes slightly. For example, we use a separate database for testing. In the tearDown method tearDown we clear the database.

Tests are implemented as methods that should call some function of our application and compares the result of its execution with the intended one. If the results do not match, the test is considered failed.

So, we have two tests in our framework. The first one checks the URL of the avatar that we implemented in the last chapter. The second test checks the make_unique_nickname method we just wrote for the User class. First a user is created with the name 'john'. After it is stored in the database, the method under test should return a nickname other than 'john'. Then we create and save the second user with the proposed nickname and check that another call to make_unique_nickname will return a name that is different from the names of both created users.

Let's start testing:

 ./tests.py 

If any errors are encountered, they will be displayed in the console.

Final words

This concludes our discussion of debugging, bugs and testing. Hope you enjoyed this article. As always, if you have any questions, ask them in the comments.

The current microblog code can be downloaded from the following link:
Download microblog-0.7.zip

As always, the virtual environment and database are missing. The process of their creation is described in previous articles.

See you again!

Miguel