Is there life after the code?

What happens to the code after it is written? In many areas of software development, his life is just beginning. For example, in web development, an application runs somewhere on the server. So, after writing the code, the task is to integrate it into the application and deliver it to the final machine. This is the process we will discuss today.

This text is intended for a wide circle of developers and is designed for those who are not familiar with the code layout process. Also this text can be useful to those who build a deployment system and are in search of ideas.

The article is written on the basis of materials of the internal seminar of the Aori company, and tells about the principles of deployment using the example of the process that was built with us.

Where does the deployment begin

In the most general approximation after writing the code, there is one single task - to send the written somewhere, where it will be launched. That is, roughly speaking, upload it via ftp or ssh.
NB If you upload the update site about cats by ftp, this is also deployment.
')
Today we do not have cats, but there is Linux, a console and we upload everything via ssh. This means that the deployment process will be as follows:

1. Pack project to archive
2. Send it via ssh to a remote machine
3. Unpack it in the folder where the web server is installed.

There is a nuance

When we unpack the project on a remote server, we actually overwrite the contents of the folder with a new one. In an amicable way, you should first clear the folder. However, then the service during the calculations will be unavailable.

Besides, what will we do if we roll out the project with an error? We will have to roll back the local version, re-perform all the manipulations with the archive upload, and there is a high probability of screwing up in a hurry.

There is a very simple solution to these problems - the calculation takes place every time into a new folder, to which the symlink of the same name switches every time:

1. On the remote machine in the home folder of the user, on behalf of whom we work on ssh, creates a directory, for example, versions. With each build, we create a subdirectory in versions, and unpack the project there.
2. After that we create a symlink to this folder by the name, say, current. Our web server looks at this symlink. Thus, switching a project is simply switching a symlink.

ln -nsf /home/project/versions/{0} /home/project/current 

It is convenient to call the subdirectories versions by the sequence number of the calculation procedure.

And one more thing

Most likely, configs for the remote machine need their own. This means that before packing the project it is necessary to roll the battle configs.

Well, if an error occurred during the calculation of one of the steps, the calculation must be stopped.

It turns out that in order to lay out the project on a remote machine, the minimum necessary steps will be as follows:

1. Roll up battle configs
2. Pack project
3. Upload project to remote machine
4. Unzip to new folder
5. Switch simlink

If any of the steps fail, stop.

Automation

It is clear that you do not have to perform these tasks manually, we need some kind of automation, something like a bash script, which we will run for display.
NB Basically, a bash script will do as well.

In Aori, we use Fabric for the final deployment. This is a python library that allows you to solve the problems listed above using very simple tools.

Of the amenities I would like to mention the so-called roles - named groups of machines that allow you to perform some deployment points not on all machines, but only on some. Let's say we have three roles in the battle environment: statics, web application and scripts. Accordingly, warming up the cache on machines with statics is not needed, and you need to update the crontab only on scripting machines.

Here is a piece of our factory script for an example:

 @parallel @roles('web', 'script', 'static') def switch(id): run('ln -nsf /home/project/versions/{0} /home/project/current'.format(id)) @roles('script') def crontab(id): run('crontab /home/project/versions/{0}/build/crontab'.format(id)) @parallel @roles('web', 'script', 'static') def clear(id): with cd('/home/project/versions'): run("ls -1 | grep -E '^[0-9]+$' | sort -n | head -n -3 | xargs -rt rm -rf") def deploy(id): execute(prepare) execute(upload) execute(upload_static, id) execute(build, id) execute(migrate, id) execute(switch, id) execute(crontab, id) execute(clear, id) 

Do not forget about git

If you use branches, you at least have a master for the ready-to-send production code, and development, in which everything is committed, is tested and poured into the master as it stabilizes.

In this case, development is also laid out somewhere, and it needs its own separate factory script.

And about unit tests

Before sending the code, you need to run the tests somewhere, because it is senseless to post a knowingly broken code.

So, the list of actions for the calculation of the code increases.

1. Set branch for calculations.
2. Set the build number for this thread
3. Pull changes to this thread from the common repository
4. Run unit tests
5. Roll configs for this branch
6. Pack project
7. Pour to the remote machine corresponding to this thread
8. Unzip to new folder
9. Switch simlink

If at any of the stages an error occurred, stop.

Well, once we do everything in the mind, then

1. Developers should not have ssh access on production machines
2. Developers should not have access to production configs
3. The deployment branch should roll out immediately after the commit so that the test application is always available for testing.
4. If an error occurs during the display, the letter should fall into the mail of the developers.

It seems that we need another tool, because to carry out all these actions with our hands, even with a set of factories, is unthinkable.

Another tool

The tool that will help us is called Continuous Integration Server and, in fact, deals with the fact that on command, on schedule or on external event, it extracts the fresh code of the specified repository, and then executes the listed set of commands. We use Jenkins as a tool for CI.

It is important to understand that the continuous integration server itself of the above tasks is able to do only the increment counter of assemblies and send a letter in case of failure. To run unit tests, roll up configs and other useful things you need another tool that performs advanced functions of the bash script.

Separate build scripts

If for deployment we took factories, then for the tasks of assembling a project we use Phing . Firstly, it is convenient, secondly, jenkins integrates well with it, and, in the third, it has happened historically.

Here are examples of the tasks our Fing solves:
phing build - project build
phing gerrit-phpcs - check for code standards
phing copy-production-configs - get production-machine configs from a closed repository
phing write-version - save to the timestamp file of the current build.

But the full path of our code from the development branch to the test server:

Fing works first. All actions occur in the jenkins local folder:

1. Make checkout branches
2. We look at the changes between the last successful build and this one (the sha of the last successful build, as well as many other useful information, can be obtained via jenkins api and jenkins variables ).
3. If the changes concern only js-applications, we do not run unit tests.
4. If not, create an empty test database with a unique name. This allows you to run assemblies in parallel.
5. We roll migration. The structure of the database and their changes we keep in the form of migrations
6. We run tests
7. Remove the base
8. We roll configs
9. We write the timestamp of the current assembly. This is needed to invalidate the cache.
10. Run the factories

The factory is responsible for sending the code to the end machines. The sequence of actions is as follows:

1. We pack the project
2. We spread on cars
3. Starting migrations
4. Switch the symlink
5. We update crontab
6. Remove old versions except last three

Codream

In conclusion, I would like to talk about solutions for code inspections.

In the simplest form, a review can be carried out over the shoulder of your employee and give comments orally. You can also watch the developer commit, and write comments on the ticket. In principle, in some cases this is sufficient, but there are specialized tools that allow to simplify the inspection process and combine it with automatic checks.

We use Gerrit in our development.

The main advantage of Gerrit is that it integrates closely with git, becoming, in fact, a git-server. After that, through Gerrit's web interface, you can not only perform the actual review, but also manage the rights to git. Thus, ordinary developers do not have the right to direct push to development or master, they can only send changesets for review by running the script git review
NB , , - , . , , submit , git review.

-. , : " , " . - .


, - . , github . , , , - , , , .


Jenkins - Wikipedia
Jenkins + Python -
Gerrit - Wikipedia
Gerrit -
Fabric -
Fabric
Phing -
Phing
Continuous Integration - Wikipedia

git review
NB , , - , . , , submit , git review.

-. , : " , " . - .


, - . , github . , , , - , , , .


Jenkins - Wikipedia
Jenkins + Python -
Gerrit - Wikipedia
Gerrit -
Fabric -
Fabric
Phing -
Phing
Continuous Integration - Wikipedia

git review
NB , , - , . , , submit , git review.

-. , : " , " . - .


, - . , github . , , , - , , , .


Jenkins - Wikipedia
Jenkins + Python -
Gerrit - Wikipedia
Gerrit -
Fabric -
Fabric
Phing -
Phing
Continuous Integration - Wikipedia

git review
NB , , - , . , , submit , git review.

-. , : " , " . - .


, - . , github . , , , - , , , .


Jenkins - Wikipedia
Jenkins + Python -
Gerrit - Wikipedia
Gerrit -
Fabric -
Fabric
Phing -
Phing
Continuous Integration - Wikipedia

git review
NB , , - , . , , submit , git review.

-. , : " , " . - .


, - . , github . , , , - , , , .


Jenkins - Wikipedia
Jenkins + Python -
Gerrit - Wikipedia
Gerrit -
Fabric -
Fabric
Phing -
Phing
Continuous Integration - Wikipedia