Flask Mega-Tutorial, Part XVIII: Deploying to Heroku

(edition 2018)

Miguel grinberg

---

There

This is the eighteenth part of a series of Flask Mega-textbooks, in which I am going to deploy a microblog on the Heroku cloud platform.

Under the spoiler is a list of all articles in this 2018 series.

Note 1: If you are looking for old versions of this course, this is here .

Note 2: If suddenly you would like to speak in support of my (Miguel) work, or simply do not have the patience to wait for the article for a week, I (Miguel Greenberg) offer the full version of this manual (in English) in the form of an electronic book or video. For more information, visit learn.miguelgrinberg.com .

In the previous article, I showed you the “traditional” way of hosting a Python application, and presented you with two real examples of deployments on Linux servers. If you are not on the "TYU" system with Linux, then you probably thought that the amount of effort spent on such a deployment was slightly more than expected, and certainly there should be an easier way.

In this chapter, I’ll show you a completely different approach in which you rely on a third-party cloud hosting provider to perform most administrative tasks, which allows you to spend more time working on the application itself.

Many cloud hosting providers offer a managed platform on which to run web applications. All you need to provide for your application to be deployed on these platforms is the actual application, because the hardware, operating system, scripting language interpreters, database, etc. managed by the service. This type of service is called Platform as a Service or PaaS.

Sounds too good to be true, right?

I will deploy Microblog to Heroku , a popular cloud service that is also very convenient for Python applications. I chose Heroku not only because it is popular, but also because it has a free level of service that will allow you to follow me and complete the deployment without spending any money.

GitHub links for this chapter: Browse , Zip , Diff .

Hosting on Heroku

Heroku, one of the first cloud platforms, appeared in June 2007 and initially supported only the Ruby programming language, but currently the list of supported languages ​​also includes Java, Node.js, Scala, Clojure, Go, PHP and of course Python.

Deploying a web application to Heroku is done using the git version control tool, so the application must reside in the git repository. Heroku searches for a file called Procfile in the root directory of the application for instructions on how to start the application. For Python projects, Heroku will need a file requirements.txt file with a list of all module dependencies that need to be installed. After the application is uploaded to Heroku servers via git, you basically did everything, and you just need to wait a few seconds for the application to appear on the network. It's really that simple.

Heroku offers you to choose the level of service based on the required computing power and time required for your application, as your user base grows you will need to buy more computing units, which Heroku calls "dynos".

Ready to try Heroku? Let's start!

Create a Heroku account

Before you can complete a deployment in Heroku, you need to get an account. So visit heroku.com and create a free account. After you register and log in to Heroku, you will have access to a dashboard where all your applications will be listed.

Installing Heroku CLI

Heroku provides a command line for interacting with a service called Heroku CLI , available for Windows, Mac OS X and Linux. The documentation contains installation instructions for all supported platforms. Install it on your system if you plan to deploy an application to test the service.

The first thing you need to do after installing the CLI is to log into your Heroku account:

 $ heroku login 

Heroku CLI will ask you to enter your email address and account password. Your authenticated status will be saved in subsequent commands.

Git setup

The git tool is the basis for deploying applications to Heroku, so you should install it on your system if you do not already have it. If you do not have a package available for your operating system, you can visit the git website to download the installer.

There are many reasons to use git for your projects. If you are planning to deploy to Heroku, then you have another one, because to deploy to Heroku, your application must be in the git repository. If you are going to do a test deployment for Microblog, you can clone the application from GitHub:

 $ git clone https://github.com/miguelgrinberg/microblog $ cd microblog $ git checkout v0.18 

The git checkout command selects a specific fixation point for an application in its history corresponding to this chapter.

If you prefer to work with your own code, rather than mine, you can convert your own project to the git repository by running git init . in the top level directory (note the point after init , which is the instruction for git that you want to create a repository in the current directory).

Creating Heroku app

To register a new application in Heroku, use the apps:create command from the root directory, passing the name of the application as the only argument:

 $ heroku apps:create flask-microblog Creating flask-microblog... done http://flask-microblog.herokuapp.com/ | https://git.heroku.com/flask-microblog.git 

Heroku requires applications to have a unique name. The flask-microblog name I used above will not be available to you, because I already use it, so you will need to choose another one for your deployment.

At the output of this command, we get the URL that Heroku assigned to the application, as well as its git repository. Your local git repository will already have an external connection setting ( remote ), called heroku . You can verify that it exists with the git remote command:

 $ git remote -v heroku https://git.heroku.com/flask-microblog.git (fetch) heroku https://git.heroku.com/flask-microblog.git (push) 

Depending on how you created your git repository, the output of the above command may also include another remote source named origin .

Ephemeral file system

The Heroku platform is different from other deployment platforms in that it implements an ephemeral file system running on a virtual platform. What does it mean? This means that at any time Heroku may reset the virtual server, which will result in your server returning to a clean state. It cannot be assumed that any data that you save in the file system will be saved, and in fact, Heroku very often recycles servers.

Working under these conditions creates some problems for my application, which uses several files:

• The SQLite database engine component by default writes data to a file on disk.
• Application logs are written to the file system.
• Compiled language translation stores are also stored in local files.

The following sections will address these three areas.

Working with the Heroku Postgres database

To solve the first problem, I will switch to a different database engine. In Chapter 17, you saw how I use a MySQL database to increase the reliability of Ubuntu deployments. Heroku has its own database suggestion based on Postgres, so I'm going to switch to it to avoid file-based SQLite storage problems.

Databases for Heroku applications are supplied with the same CLI Heroku. In this case, I'm going to create a database at the free (free) level:

 $ heroku addons:add heroku-postgresql:hobby-dev Creating heroku-postgresql:hobby-dev on flask-microblog... free Database has been created and is available ! This database is empty. If upgrading, you can transfer ! data from another database with pg:copy Created postgresql-parallel-56076 as DATABASE_URL Use heroku addons:docs heroku-postgresql to view documentation 

The URL of the newly created database is stored in the environment variable DATABASE_UR L, which will be available when the application starts. This is very convenient, because the application is already looking for the database URL in this variable.

Logging to stdout

Heroku expects applications to do input / output directly to stdout . Everything that the application prints in the standard output is saved and returned when using the heroku logs command. So I'm going to add a configuration variable that indicates whether I need to keep records in stdout or in a file, as I did before. Here is the configuration change:

config.py : Logging option to stdout.

 class Config(object): # ... LOG_TO_STDOUT = os.environ.get('LOG_TO_STDOUT') 

Then, in the application factory function, I can check this configuration to understand how to configure the application logger:

app/__init__.py : Log to stdout or to file.

 def create_app(config_class=Config): # ... if not app.debug and not app.testing: # ... if app.config['LOG_TO_STDOUT']: stream_handler = logging.StreamHandler() stream_handler.setLevel(logging.INFO) app.logger.addHandler(stream_handler) else: if not os.path.exists('logs'): os.mkdir('logs') file_handler = RotatingFileHandler('logs/microblog.log', maxBytes=10240, backupCount=10) file_handler.setFormatter(logging.Formatter( '%(asctime)s %(levelname)s: %(message)s ' '[in %(pathname)s:%(lineno)d]')) file_handler.setLevel(logging.INFO) app.logger.addHandler(file_handler) app.logger.setLevel(logging.INFO) app.logger.info('Microblog startup') return app 

So now I need to define the environment variable LOG_TO_STDOUT if the application is running in Heroku, but not in other configurations. Heroku CLI makes this easy because it provides the ability to set environment variables to be used at runtime:

 $ heroku config:set LOG_TO_STDOUT=1 Setting LOG_TO_STDOUT and restarting flask-microblog... done, v4 LOG_TO_STDOUT: 1 

Compiled translations

The third aspect of microblog based on local files is compiled language translation of files. The easiest option to ensure that these files never disappear from the ephemeral file system is to add the compiled language files to the Git repository so that they become part of the initial state of the application after it is deployed to Heroku.

A more elegant option, in my opinion, is to include the flask translate compile command on the start up command provided by Heroku, so that each time the server restarts, these files will be compiled again. I'm going to go this route, since I know that my startup procedure will require more than one command anyway, since I still need to start the database migration. So now I will put this problem aside and come back to it later when I write Procfile .

Elasticsearch Hosting

Elasticsearch is one of many services that can be added to the Heroku project, but unlike Postgres, this is not a service provided by Heroku, but a third party who collaborates with Heroku to provide add-ons. Today (when I write these lines) there are three different providers of the integrated elasticsearch service.

Before setting up Elasticsearch, keep in mind that Heroku requires your account to have an associated credit card before installing any third-party add-on, even if you stay within their free levels. If you prefer not to provide your Heroku credit card details, skip this section. You can deploy the application, but the search function will not work.

From the Elasticsearch options that are available as add-ons, I decided to try SearchBox , which comes with a free starting plan. To add a SearchBox to your account, you must run the following command when logging in to Heroku:

 $ heroku addons:create searchbox:starter 

This command will deploy the Elasticsearch service and place the connection URL for the service in the SEARCHBOX_URL environment variable associated with the application. Let me remind you once again that this command will not be executed unless you add your credit card to your Heroku account.

If you recall from Chapter 16 , my application looks for the Elasticsearch connection URL in the ELASTICSEARCH_URL variable, so I need to add this variable and set it to the connection URL assigned by SearchBox:

 $ heroku config:get SEARCHBOX_URL <your-elasticsearch-url> $ heroku config:set ELASTICSEARCH_URL=<your-elasticsearch-url> 

Here, I first asked Heroku to print the SEARCHBOX_URL value, and then I added a new environment variable named ELASTICSEARCH_URL set to the same value.

Requirements update

Heroku expects the dependencies to be in the requirements.txt file, just as I defined in Chapter 15 . But to run the application on Heroku, I need to add two new dependencies to this file.

Heroku does not provide its own web server. Instead, he expects the application to start his own web server at the port number specified in the $PORT environment variable. Since the Flask Web Server is not reliable enough for use in work, I’m going to use gunicorn again, a server recommended by Heroku for Python applications.

The application will also connect to the Postgres database, and for this SQLAlchemy requires the installation of the psycopg2 package.

Both gunicorn and psycopg2 need to be added to the requirements.txt file.

Profile

Heroku needs to know how to run the application, and for this it uses a file called Procfile in the root directory of the application. The format of this file is simple, each line contains the process name, a colon, and then the command that starts the process. The most common type of application that runs on Heroku is the web application, and for this type of application the process name must be web . Below you can see the Procfile for Microblog:

Procfile: Heroku Procfile.

 web: flask db upgrade; flask translate compile; gunicorn microblog:app 

Here I have defined a command to launch a web application as a sequence of three commands. First, I launch the database migration update, then compile language translations and, finally, start the server.

Since the first two subcommands are based on the flask command, I need to add the FLASK_APP environment FLASK_APP :

 $ heroku config:set FLASK_APP=microblog.py Setting FLASK_APP and restarting flask-microblog... done, v4 FLASK_APP: microblog.py 

The gunicorn team gunicorn simpler than what I used to deploy Ubuntu, because this server has very good integration with the Heroku environment. For example, the $PORT environment variable is taken into account by default, and instead of using the -w option to specify the number of processes, heroku recommends adding the WEB_CONCURRENCY variable that gunicorn uses when -w not provided, which gives you the flexibility to manage the number of processes unnecessarily modify the procfile.

Deploying the application

All the preparatory steps have been completed, so it's time to launch the deployment. To download an application to Heroku servers for deployment, use the git push command. This is similar to sending changes in the local git repository to GitHub or another remote git server.

And now I have reached the most interesting part, where I send the application to my hosting account Heroku. It's actually quite simple, I just have to use git to push the application to the main branch of the Heroku git repository. There are several options for how to do this, depending on how you created your git repository. If you use my code v0.18 , then you need to create a branch based on this tag, and send it to the remote master branch, as follows:

 $ git checkout -b deploy $ git push heroku deploy:master 

If you still work with your own repository, then your code is most likely already in the main branch, so you first need to make sure that your changes are fixed:

 $ git commit -a -m "heroku deployment changes" 

Then you can start the deployment as follows:

 $ git push heroku master 

Regardless of how you send a branch, you should see the following output from Heroku:

 $ git push heroku deploy:master Counting objects: 247, done. Delta compression using up to 8 threads. Compressing objects: 100% (238/238), done. Writing objects: 100% (247/247), 53.26 KiB | 3.80 MiB/s, done. Total 247 (delta 136), reused 3 (delta 0) remote: Compressing source files... done. remote: Building source: remote: remote: -----> Python app detected remote: -----> Installing python-3.6.2 remote: -----> Installing pip remote: -----> Installing requirements with pip ... remote: remote: -----> Discovering process types remote: Procfile declares types -> web remote: remote: -----> Compressing... remote: Done: 57M remote: -----> Launching... remote: Released v5 remote: https://flask-microblog.herokuapp.com/ deployed to Heroku remote: remote: Verifying deploy... done. To https://git.heroku.com/flask-microblog.git * [new branch] deploy -> master 

The heroku label we used in the git push command is a remote repository that was automatically added by HEROKU CLI when creating the application. The deploy:master argument means that I move the code from the local repository referenced by the deploy branch to the main branch of the master repository Heroku. When you work with your own projects, you will probably push changes through the git push heroku master , which will push (push) your local master branch. Because of how this project is structured, I send a branch that is not master , but the destination branch on the Heroku side must always be master , since this is the only branch that Heroku accepts for deployment.

And now the application should be deployed to the URL specified in the output of the command that created the application. In my case, the URL was https://flask-microblog.herokuapp.com , so this is what I need to enter in order to access the application.

If you want to see log entries for a running application, use the heroku logs command. This can be useful if for some reason the application does not start. If there were any errors, they will be in the logs.

Deploying Application Updates

To deploy a new version of the application, you just need to run the git push command with the new code. This will repeat the deployment process, disable the old deployment, and replace it with new code. The commands in Procfile will run again as part of the new deployment, so any new migrations or database transfers will be updated during the process.

There