Deploy web application with Fabric

Applications grow, become more difficult. The number of manipulations required for their deployment and updating is growing.

• The same repetitive actions take a significant amount of time.
• About some actions are forgotten, others are confused
• Human factor, lightheadedness and short-sightedness

In this article, I will talk about how to turn an exciting and, in some places, unpredictable adventure into a simple, routine and boring operation with a predictable result.

Our goals

• Perform operations faster, incl. parallel on multiple servers
• Have a well-tested procedure.
• Have an easy-to-use tool with which you can perform any action with an application.

Install Fabric

We will be using fabric . This set is an excellent universal tool for automating system administration tasks. He is well represented in this article .
')
So, let's assume that we already have a workstation with Linux and python (2.5-2.7). Install the necessary tools:

# pip install fabric # pip install fabtools 

Little reasoning

We will interact with the server via ssh. Therefore, I assume that you have already generated the key, determined its use in ~ / .ssh / config and put the public part on the server. For the convenience of managing users on the server, we create a personal account for each of the deployment engineers, who, in the course of work, can execute commands from the project user or turn into it.

What will our simple deployment project consist of:

 myapp_deploy/ fabfile.py #  ,   . sudoers/ devel_sudo #  /etc/sudoers    - config/ config-production.php #  

We will deploy the simplest service that resides on a single server. For this service we will define a set of parameters defining its specificity. Such an approach allows you to logically divide the application into modules and deploy each of them separately.

We write a script

 from fabric.api import env, run, sudo, local, put, settings import fabtools def production(): """Defines production environment""" env.hosts = ['192.168.1.2'] #  env.shell = 'bash -c' #       env.use_ssh_config = True #   ssh- env.project_user = "myappuser" #,        env.sudo_user = env.project_user env.base_dir = "/apps" #    env.domain_name = "myapp.example.com" #FQDN     env.domain_path = "%(base_dir)s/%(domain_name)s" % { 'base_dir':env.base_dir, 'domain_name':env.domain_name } #       myapp env.current_path = "%(domain_path)s/current" % { 'domain_path':env.domain_path } #    myapp env.releases_path = "%(domain_path)s/releases" % { 'domain_path':env.domain_path } #     env.git_clone = "git@git.example.com:myapp.git" #,     . env.config_file = "config/config-production.php" #     

Now let's deal directly with the functionality. First we need to perform the initial configuration of the server. We will carry out this operation only if it is necessary to change some settings in the system. The function call is best assigned to a system administrator who has full privileges in the system.

 def permissions(): """Set proper permissions for release""" sudo("chown -R %(project_user)s:%(project_user)s %(domain_path)s" % { 'domain_path':env.domain_path, 'project_user':env.sudo_user }) def setup(): """Prepares one or more servers for deployment""" with settings(sudo_user='root'): sudo("mkdir -p %(domain_path)s/releases" % { 'domain_path':env.domain_path }) sudo("mkdir -p %(base_dir)s/logs/" % { 'base_dir':env.base_dir }) permissions() put("sudoers/devel_sudo", "/tmp/devel_sudo") sudo("chown root:root /tmp/devel_sudo") sudo("chmod 0440 /tmp/devel_sudo") sudo("mv /tmp/devel_sudo /etc/sudoers.d/devel_sudo") 

Next, we decide on the storage releases on the server. This is what the directory structure will look like:

 /apps myapp.example.com/ current #    releases/ #  release0 release1 ... releaseN #   

The release index we will determine with the help of timestamp. Accordingly, we need to find the latest release and the penultimate release (in case we want to roll back) and the path to them.

 def releases(): """List a releases made""" env.releases = sorted(sudo('ls -x %(releases_path)s' % { 'releases_path':env.releases_path }).split()) if len(env.releases) >= 1: env.current_revision = env.releases[-1] env.current_release = "%(releases_path)s/%(current_revision)s" % { 'releases_path':env.releases_path, 'current_revision':env.current_revision } if len(env.releases) > 1: env.previous_revision = env.releases[-2] env.previous_release = "%(releases_path)s/%(previous_revision)s" % { 'releases_path':env.releases_path, 'previous_revision':env.previous_revision } 

Deploy is often connected and requires restarting some services. Despite the fact that we have the simplest project, we will need to reload php-fpm to avoid the known problem .

 def restart(): """Restarts your application services""" with settings(sudo_user='root',use_shell=False): sudo("/etc/init.d/php5-fpm reload") 

Now we need to somehow pick up the code from the repository. We will clone the latest versions of files from the git repository.

 def checkout(): """Checkout code to the remote servers""" env.timestamp = run("/bin/date +%s") env.current_release = "%(releases_path)s/%(timestamp)s" % { 'releases_path':env.releases_path, 'timestamp':env.timestamp } sudo("cd %(releases_path)s; git clone -q -b master --depth 1 %(git_clone)s %(current_release)s" % { 'releases_path':env.releases_path, 'git_clone':env.git_clone, 'current_release':env.current_release }) 

Copy configuration files to the release directory

 def copy_config(): """Copy custom config to the remote servers""" if not env.has_key('releases'): #  ,       releases() put("%s" % env.config_file, "/tmp/config.php") sudo("cp /tmp/config.php %(current_release)s/config/" % { 'current_release':env.current_release }) run("rm /tmp/config.php") 

With the help of simlink we make the latest release relevant

 def symlink(): """Updates the symlink to the most recently deployed version""" if not env.has_key('current_release'): releases() sudo("ln -nfs %(current_release)s %(current_path)s" % { 'current_release':env.current_release, 'current_path':env.current_path }) 

Putting our whole deployment procedure into one

 def deploy(): """Deploys your project. This calls 'checkout','copy_config','migration','symlink','restart','cleanup'""" checkout() copy_config() symlink() restart() 

Let's think about the need to clean releases.

 def cleanup(): """Clean up old releases""" if not env.has_key('releases'): releases() if len(env.releases) > 10: directories = env.releases directories.reverse() del directories[:10] env.directories = ' '.join([ "%(releases_path)s/%(release)s" % { 'releases_path':env.releases_path, 'release':release } for release in directories ]) sudo("rm -rf %(directories)s" % { 'directories':env.directories }) 

If after the release something went wrong, we need the ability to quickly roll back to the previous version.

 def rollback_code(): """Rolls back to the previously deployed version""" if not env.has_key('releases'): releases() if env.has_key('previous_release'): sudo("ln -nfs %(previous_release)s %(current_path)s && rm -rf %(current_release)s" % { 'current_release':env.current_release, 'previous_release':env.previous_release, 'current_path':env.current_path }) else: print "no releases older then current" sys.exit(1) def rollback(): """Rolls back to a previous version and restarts""" rollback_code() restart() 

Great, enough python for today.

Using

You can get a list of all that our script can do as follows:

 $ fab --list Available commands: checkout Checkout code to the remote servers cleanup Clean up old releases copy_config Copy custom config to the remote servers deploy Deploys your project. This calls 'checkout','copy_config','migration','symlink','restart','cleanup' permissions Set proper permissions for release production Defines production environment releases List a releases made restart Restarts your application services rollback Rolls back to a previous version and restarts rollback_code Rolls back to the previously deployed version setup Prepares one or more servers for deployment symlink Updates the symlink to the most recently deployed version 

Great, let's deploy to the server.

 $ fab production deploy [192.168.1.2] Executing task 'deploy' [192.168.1.2] run: /bin/date +%s [192.168.1.2] out: 1431941361 [192.168.1.2] out: [192.168.1.2] sudo: cd /apps/myapp.example.com/releases; git clone -q -b master --depth 1 git@git.example.com:myapp.git /apps/myapp.example.com/releases/1431941361 [192.168.1.2] sudo: ls -x /apps/myapp.example.com/releases [192.168.1.2] out: 1431940514 1431940525 1431940537 1431940547 1431940558 1431940568 1431940578 1431940589 1431940599 1431941361 [192.168.1.2] out: [192.168.1.2] put: config/config-production.php -> /tmp/config.php [192.168.1.2] sudo: cp /tmp/config.php /apps/myapp.example.com/releases/1431941361/config/ [192.168.1.2] run: rm /tmp/config.php [192.168.1.2] sudo: ln -nfs /apps/myapp.example.com/releases/1431941361 /apps/myapp.example.com/current [192.168.1.2] sudo: /etc/init.d/php-fpm reload [192.168.1.2] out: Reloading php-fpm: [18-May-2015 05:29:29] NOTICE: configuration file /etc/php-fpm.conf test is successful [192.168.1.2] out: [192.168.1.2] out: [ OK ] [192.168.1.2] out: [192.168.1.2] out: Done. Disconnecting from 192.168.1.2... done. 

As a result, we got an excellent tool that with minor modifications can be used in a large number of projects with different specifics.