Flask Mega-Tutorial, Part XV: Improving Application Structure

(edition 2018)

Miguel grinberg

There

This is the fifteenth installment of the Flask Mega-Textbook series, in which I am going to restructure the application using a style suitable for larger applications.

Under the spoiler is a list of all articles in this 2018 series.

Note 1: If you are looking for old versions of this course, this is here .

Note 2: If suddenly you would like to speak in support of my (Miguel) work, or simply do not have the patience to wait for the article for a week, I (Miguel Greenberg) offer the full version of this manual (in English) in the form of an electronic book or video. For more information, visit learn.miguelgrinberg.com .

Microblog is already a decent sized app! And I thought it would be time to discuss how the Flask application can grow without becoming a garbage dump or simply too complicated to manage. Flask is a framework that is designed to enable you to organize your project in any way, and as part of this philosophy, it allows you to change or adapt the structure of the application as it grows larger, or as your needs or experience level change.

In this chapter, I’m going to discuss some of the patterns that apply to large applications, and to demonstrate them, I’m going to make some changes to how my Microblog project is structured in order to make the code more maintainable and better organized. But, of course, in the true spirit of Flask, I urge you to accept these changes only as a recommendation, on how to organize your own projects.

GitHub links for this chapter: Browse , Zip , Diff .

Current Restrictions

In the current state of the application there are two main problems. If you look at how the application is structured, you will notice that there are several different subsystems that can be identified, but the code that supports them is all mixed up, without any clear boundaries. Let's look at what these subsystems are:

• The user authentication subsystem, which includes some view functions in app / routes.py , some forms in app / forms.py , some templates in app / templates, and email support in app / email.py .
• The error subsystem, which defines error handlers in app / errors.py and templates in app / templates .
• The main functionality of the application, which includes the display and recording of blog posts, user profiles and subsequent events, as well as live translations of blog posts that are distributed in a large number of application modules and templates.

Thinking about these three subsystems, which I have defined, and how they are structured, you can probably notice the pattern. Until now, the logic of the organization, which I followed, is based on the presence of modules dedicated to the various functions of the application. There is a module for viewing functions, another for web forms, one for errors, one for emails, a directory for HTML templates and so on. Although it is a structure that makes sense for small projects, as soon as a project starts growing, it tends to make some of these modules really big and dirty.

One way to see the problem clearly is to consider how you will start the second project, reusing as much as you can from the first. For example, the part responsible for authenticating the user should work well in other applications. But if you want to use this code as is, you will have to go into several modules and copy / paste the relevant sections into new files in the new project. See how awkward this is? Wouldn't it be better if this project had all the authentication files separated from the rest of the application? The blueprints feature of Flask helps to achieve a more practical organization that simplifies code reuse.

The second problem is not so obvious. A Flask application instance is created as a global variable in app/__init__.py , and then imported by many application modules. Although this in itself is not a problem, using an application as a global variable can complicate some scenarios, in particular those associated with testing. Imagine that you want to test this application in different configurations. Because an application is defined as a global variable, it is actually not possible to create an instance of two applications using different configuration variables. Another situation, which is not ideal, is that all tests use the same application, so the test can make changes to the application that affect another test that is performed later. Ideally, you want all the tests to run on the original application instance.

In fact, you can see all this in the tests.py module. I resort to the configuration change trick after it has been installed in the application to direct tests to use the in-memory database instead of the SQLite database (default) on a disk basis. I have no other way to change the configured database, because by the time the tests were run, the application was already created and configured. For this particular situation, changing the configuration after it has been applied to the application seems to work fine, but in other cases it does not help, and in any case it is a bad practice that can lead to unclear and difficult to find errors.

The best solution would be not to use a global variable for the application, but instead use the application factory function to create the function at run time. This will be a function that accepts a configuration object as an argument and returns an instance of the Flask application configured by these settings. If I could modify the application to work with the application factory function, then writing tests that require special configuration would be easier, because each test could create its own application.

In this chapter, I am going to reorganize the application by introducing the element schema for the three subsystems listed above and the application factory functions. It will not be advisable to show you a detailed list of changes, because there are small changes in almost every file that is part of the application, so I'm going to discuss the steps I took to refactor, and you can download the application with these changes.

Blueprints

In Flask, a project is a logical structure, which is a subset of the application. A project can include elements such as routes, view functions, forms, templates, and static files. If you write your project in a separate Python package, you have a component that encapsulates elements related to a specific function of the application.

The contents of the circuit elements is initially at rest. To associate these elements, you must register the element schema in the application. During registration, all items added to the item map are passed to the application. Thus, it is possible to present the scheme of elements as temporary storage for the functionality of the application, which helps to organize the code.

Blueprint error handling

The first element schema I created was encapsulated to support error handlers. The structure of this concept is as follows:

 app/ errors/ <-- blueprint  __init__.py <-- blueprint  handlers.py <-- error  templates/ errors/ <-- error  404.html 500.html __init__.py <-- blueprint  

In essence, I moved the app / errors.py module to app / errors / handlers.py and the two error templates in app / templates / errors so that they are separate from the other templates. I also had to change the render_template() calls in both error handlers to use the subdirectory of the new error template. After that, I added the blueprint creation to the app/errors/__init__.py and registering the project with app/__init__.py after creating the application instance.

I should note that the schemas of the elements of Flask can be configured on a separate directory for templates or static files. I decided to move the templates to the subdirectory of the application templates directory so that all templates are in the same hierarchy, but if you prefer to have templates belonging to the element scheme inside the element scheme package, this is supported. For example, if you add the template_folder= 'templates' argument to the Blueprint() constructor, you can save the element's schema templates in app / errors / templates .

Creating a layout of elements is much like creating an application. This is done in the __init__.py module of the blueprint package:

app/errors/__init__.py : Errors blueprint.

 from flask import Blueprint bp = Blueprint('errors', __name__) from app.errors import handlers 

The Blueprint class accepts the schema name of the elements, the name of the base module (usually set to __name__ , as in an instance of the Flask application), and several optional arguments that I don’t need in this case. After creating the element schema object, I import the handlers.py module so that error handlers in it are registered in the element schema. This import is down to avoid circular dependencies.

In the handlers.py module , instead of attaching error handlers to an application using the @app.errorhandler decorator, I use the @app.errorhandler blueprint decorator. Although both decorators achieve the same end result, the idea is to try to make the layout of the elements independent of the application so that it is more portable. I also need to change the path to the two error patterns to accommodate the new error subdirectory to which they were moved.

The final step to complete the refactoring of error handlers is to register the element schema in the application:

app/__init__.py : registration of the element schema in the application.

 app = Flask(__name__) # ... from app.errors import bp as errors_bp app.register_blueprint(errors_bp) # ... from app import routes, models # <--     ! 

To register the element map, use the register_blueprint() method of the Flask application instance. When registering a circuit of elements, all presentation functions, templates, static files, error handlers, etc., are connected to the application. I put the import of the schema of elements right above app.register_blueprint() to avoid circular dependencies.

Blueprint authentication

The process of refactoring the functions of authentication of an application into a project is quite similar to the processing of error handlers. Here is the refactoring scheme:

 app/ auth/ <-- blueprint  __init__.py <-- blueprint  email.py <-- authentication emails forms.py <-- authentication forms routes.py <-- authentication routes templates/ auth/ <-- blueprint  login.html register.html reset_password_request.html reset_password.html __init__.py <-- blueprint  

To create this project, I had to transfer all functions related to authentication to the new modules that I created in the project. This includes several browsing features, web forms, and support functions, such as a function that sends password reset tokens by email. I also moved the templates to a subdirectory to separate them from the rest of the application, as it did with error pages.

When determining routes in the scheme of elements, uses the decorator @bp.route instead of @app.route . It also requires changing the syntax used in url_for() to create URLs. For normal browsing functions attached directly to an application, the first argument to url_for() is the name of the view function. If the route is defined in the element scheme, this argument must include the name of the element scheme and the name of the view function, separated by a period. So, for example, I had to replace all url_for('login') url_for('auth.') with url_for('auth.') , and the same for the rest of the view functions.

To register the scheme of auth elements in the application, I used a slightly different format:

app/__init__.py : Register the authentication element scheme in the application.

 # ... from app.auth import bp as auth_bp app.register_blueprint(auth_bp, url_prefix='/auth') # ... 

The register_blueprint() call in this case has an additional argument url_prefix . This is completely optional, but Flask gives you the ability to append the element scheme under the URL prefix, so any routes defined in the element scheme receive this prefix in their URLs. In many cases, this is useful as a kind of "namespace" that separates all routes in the circuit of elements from other routes in the application or other schemes of elements. For authentication, I thought it would be nice to have all the routes starting with /auth , so I added a prefix. Thus, now the login URL will be http: // localhost: 5000 / auth / login . Since I use url_for() to generate a URL, all URLs will automatically include the prefix.

The main scheme of the elements of the application

The third element schema contains the main application logic. To refactor this element schema, the same process is required as for the previous two element schemas. I gave this element scheme the name main , so all url_for() calls that refer to the view functions should have gotten main . prefix. Given that this is the main functionality of the application, I decided to leave the templates in the same places. This is not a problem, because I moved the templates from two other element schemas into subdirectories.

Application Pattern

As I mentioned in the introduction to this chapter, having an application as a global variable leads to some complications, mainly in the form of restrictions for some test scenarios. Before I introduced the element schemas, the application had to be a global variable, because all the presentation functions and error handlers had to be decorated with functions that are in the app , such as @app.route . But now that all the routes and error handlers have been moved to the element schemas, there are far fewer reasons to keep the application global.

So I'm going to add the create_app() function, which creates an instance of the Flask application, and exclude a global variable. The conversion was not trivial, we had to understand several difficulties, but let's first consider the function of the application factory:

app/__init__.py : Application factory function.

 # ... db = SQLAlchemy() migrate = Migrate() login = LoginManager() login.login_view = 'auth.login' login.login_message = _l('Please log in to access this page.') mail = Mail() bootstrap = Bootstrap() moment = Moment() babel = Babel() def create_app(config_class=Config): app = Flask(__name__) app.config.from_object(config_class) db.init_app(app) migrate.init_app(app, db) login.init_app(app) mail.init_app(app) bootstrap.init_app(app) moment.init_app(app) babel.init_app(app) # ...    blueprint registration if not app.debug and not app.testing: # ...    logging setup return app 

You have seen that most of the Flask extensions are initialized by creating an instance of the extension and passing the application as an argument. If the application does not exist as a global variable, there is an alternative mode in which extensions are initialized in two steps. An extension instance is first created in the global scope, as before, but no arguments are passed to it. This creates an instance of the extension that is not attached to the application. During the creation of an application instance in the factory function, you must call the init_app() method in the extension instance in order to bind it to a known application.

Other tasks performed during initialization remain unchanged, but are transferred to the factory function instead of being in the global area. This includes registration of the element schema and logging configuration. Note that I added a not app.testing condition to a condition that decides whether to enable or disable e-mail and file logging so that all these logs are skipped during unit tests. The app.testing flag will be True when performing unit tests due to the fact that the TESTING variable is set to True in the configuration.

So who calls the application factory function? The obvious place to use this feature is the top-level microblog.py script, which is the only module in which the application now exists in the global scope. Another place is in test.py , and I will discuss unit testing in more detail in the next section.

As I mentioned above, most of the links to the application are gone with the introduction of the schema of the elements, but some of them are still present in the code that I have to consider. For example, all app / models.py , app / translate.py and app / main / routes.py applications have links to app.config . Fortunately, the Flask developers tried to simplify the browsing functions to access the application instance without having to import it, as I have done so far. The variable current_app from Flask, is a special “context” variable Flask, which initializes the application before sending the request. You have already seen another context variable before, the g variable, in which I store the current locale. These two, along with current_user Flask-Login and several others that you have not seen, are conditionally “magic” variables, since they work as global variables, but are only available during the processing of a request and only in the stream that processes it.

Replacing the current_app variable in Flask eliminates the need to import an application instance as a global variable. I was able to replace all app.config mentions with current_app.config without any difficulty with a simple search and replace.

The app / email.py module was a bit of a problem, so I had to use a little trick:

app / email.py : Passing an instance of an application to another thread.

 from app import current_app def send_async_email(app, msg): with app.app_context(): mail.send(msg) def send_email(subject, sender, recipients, text_body, html_body): msg = Message(subject, sender=sender, recipients=recipients) msg.body = text_body msg.html = html_body Thread(target=send_async_email, args=(current_app._get_current_object(), msg)).start() 

In the send_email() function, the application instance is passed as an argument to the background thread, which then delivers the email without blocking the main application. current_app send_async_email() , , , current_app — , , . current_app . current_app , current_app proxy object , . , - , current_app . , -, . current_app._get_current_object() , , .

" " — app/cli.py , . current_app , , , , current_app . , , register() , app :

app/cli.py : .

 import os import click def register(app): @app.cli.group() def translate(): """Translation and localization commands.""" pass @translate.command() @click.argument('lang') def init(lang): """Initialize a new language.""" # ... @translate.command() def update(): """Update all languages.""" # ... @translate.command() def compile(): """Compile all languages.""" # ... 

register() microblog.py . microblog.py :

microblog.py : .

 from app import create_app, db, cli from app.models import User, Post app = create_app() cli.register(app) @app.shell_context_processor def make_shell_context(): return {'db': db, 'User': User, 'Post' :Post} 

Unit Testing Improvements

, , , . , , , .

tests.py , , , , . , , .

create_app() . Config config.py , , , . , :

tests.py : .

 from config import Config class TestConfig(Config): TESTING = True SQLALCHEMY_DATABASE_URI = 'sqlite://' 

Config SQLAlchemy SQLite . TESTING True, , , , .

, setUp() tearDown() , , . :

tests.py : .

 class UserModelCase(unittest.TestCase): def setUp(self): self.app = create_app(TestConfig) self.app_context = self.app.app_context() self.app_context.push() db.create_all() def tearDown(self): db.session.remove() db.drop_all() self.app_context.pop() 

self.app , , . db.create_all() , . db , , URI app.config , , , . , db , self.app , ?

. current_app , - , ? , , . , , , current_app . , Python. , python , flask shell , , .

 >>> from flask import current_app >>> current_app.config['SQLALCHEMY_DATABASE_URI'] Traceback (most recent call last): ... RuntimeError: Working outside of application context. >>> from app import create_app >>> app = create_app() >>> app.app_context().push() >>> current_app.config['SQLALCHEMY_DATABASE_URI'] 'sqlite:////home/miguel/microblog/app.db' 

! Flask , current_app g . , . db.create_all() setUp() , db.create_all() current_app.config , , . tearDown() , .

, , Flask. request context , , . , request session Flask, current_user Flask-Login.

Environment variables

, , , , , . , , URL- API Microsoft Translator. , , , , , , .

, , .env . , .

Python, .env , python-dotenv . , :

 (venv) $ pip install python-dotenv 

config.py — , , .env Config , :

config.py : .env .

 import os from dotenv import load_dotenv basedir = os.path.abspath(os.path.dirname(__file__)) load_dotenv(os.path.join(basedir, '.env')) class Config(object): # ... 

, .env , . , .env - . , , .

.env - , FLASK_APP FLASK_DEBUG , , , .

.env , , 25- , API Microsoft Translator :

 SECRET_KEY=a-really-long-and-unique-key-that-nobody-knows MAIL_SERVER=localhost MAIL_PORT=25 MS_TRANSLATOR_KEY=<your-translator-key-here> 

Requirements

Python. - , , , requirements.txt . :

 (venv) $ pip freeze > requirements.txt 

pip freeze , , , requirements.txt . , , , :

 (venv) $ pip install -r requirements.txt 

There