Automation in web development with Vagrant and SaltStack

Web developers need to configure the server. Usually under each project. Since I learned to do it without the help of Google, the setting has ceased to be something interesting, and I wanted to automate it as much as possible.
The first idea was to create a virtual machine and copy it if necessary. It became easier to configure similar configurations, but every time something changed in the configuration, you had to go to the server and configure it. I wanted more.
Through trial and error, I came to a bunch of Vagrant + SaltStack , where Vagrant takes over the isolation of environments, and SaltStack - configuration management.

Vagrant.

Vagrant is a virtual machine manager with customization via Vagrantfile , which allows you to collect a machine configuration ready for launch at one point. He can not only raise virtual machines, but also deal with their "software". Security delegates to systems that are designed for this .
The package format for .box environment in Vagrant is .box . There is a service for the exchange of boxes - https://vagrantcloud.com/ . At the moment it is in beta, but, with the exception of "broken" links to the boxes, I have not seen errors.

Vagrantfile

Creating a Vagrantfile is easy. Details are in the documentation , but I will list what is needed in the example.
The server will run on debian without SaltStack pre-installed components. On https://vagrantcloud.com/ you can find .box .

 config.vm.box = "mokote/debian-7" 

I use a private network within which each virtual machine will receive its own IP . This allows you to run multiple machines at the same time and have access to them.

 config.vm.network "private_network", ip: "192.168.56.107" 

Virtual machine settings:

 config.vm.provider "virtualbox" do |v| v.name = "demostand" v.memory = 1024 v.customize ["modifyvm", :id, "--natdnshostresolver1", "on"] v.customize ["setextradata", :id, "--VBoxInternal2/SharedFoldersEnableSymlinksCreate/v-root", "1"] end 

Synchronize the directories with the SaltStack settings and the project that will be deployed on the newly created machine. For the project, I used nfs as a synchronization method. It is faster than VirtualBox shared folders , and is configured with just one parameter :nfs => true , but requires root access.

 config.vm.synced_folder "salt/roots/", "/srv/" config.vm.synced_folder "~/Development/web/demostand", "/var/www/demostand", id: "vagrant-root", :nfs => true 

It remains to configure the provision of the host configuration SaltStack .
salt.minion_config - the file with which the server configuration begins.
salt.run_highstate = true - whether to run the software at the start of the machine.
salt.pillar - pillar data storage.

 config.vm.provision :salt do |salt| salt.minion_config = 'salt/minion' salt.run_highstate = true salt.pillar({ "database" => { "withUser" => true, "name" => "demostand", "password" => "fm2QTqimWUrk" } }) salt.pillar({"projectName" => "demostand"}) end 

Vagrantfile can find the full Vagrantfile here: https://github.com/ligser/salt-vagrant-demostand/blob/master/Vagrantfile .

Saltstack

SaltStack allows you to create a master -> minion and initiate the minion provision of fresh configs by the master. I did not have such a need, so the provision will be performed without the participation of the master .
The configuration description for SaltStack is a set of statements (states, in SaltStack terminology) that must be satisfied in order for the minion to minion considered configured successfully.
I will describe the features of SaltStack , which are useful to me.
')

salt / minion

First, find the salt/minion file that is listed in the Vagrantfile . This is the file from which to start the machine configuration.
Its content is extremely concise: file_client: local . Here it is stated that the configuration is stored locally on minion , by default, in the /srv/salt directory, into which we set up synchronization.

States

The full manual on the states is on the project website: http://docs.saltstack.com/en/latest/topics/tutorials/index.html#states .
States are stored in .sls files. Usually .sls consists of state descriptions in YAML format.

 apache: # ID pkg: #  - installed #  

Calling one function can be abbreviated:

 apache: pkg.installed 

Also, if the ID does not match the name of the state object, you can specify the name.

 apache_pkg: pkg: - name: apache - installed 

pkg is the state of the package manager. installed is one of the most frequently used functions of this state, implying that a package with this name must be installed on the system.
service - the state of services. With it, you can start or stop services. Status stopping apache2 :

 apache2: service.dead 

file - the state of the file system.
Usually, the managed function is used for working with filesystems.
If the ID same as the file name, we do not need to specify the name parameter.
source - the place where the contents of the file come from. salt:// - pointer to /srv/salt , or relative to Vagrantfile : salt/roots/salt .
By default, files are not jinja templates, so pillar data cannot be used inside them. To make a file a template, you need to specify the template: jinja in the parameters. After that, the file will be processed by the template engine. jinja used in the .sls files themselves.
SaltStack supports state dependencies.
watch_in indicates in which services the file is used, so that when it changes, restart them.
require maintained by the pkg state and requires installation of a specific package.

 /etc/nginx/sites-available/default: file.managed: - source: salt://nginx/default - template: jinja - user: root - group: root - mode: 644 - watch_in: - service: nginx 

Data

There are two systems for storing data in SaltStack .
pillar - data that should not be in config files. In Vagrantfile I put 2 keys in pillar : database - database parameters and projectName - project name.
grains - static information that will be loaded on minion when it is launched.
I practically did not use grains , so I was not interested in the intricacies of this system, if you need more information, it is available on the SaltStack website: http://docs.saltstack.com/en/latest/topics/targeting/grains.html .
By default, any .sls file is processed by the template engine, so you can use the following constructs in it:
{{ }} - output.
{% %} - conditions, or cycles.

 {{ pillar['database']['name'] }}: {% if (pillar['database']['withUser']) %} mysql_user.present: - host: localhost - password: {{ pillar['database']['password'] }} - require: - service: mysql - pkg: python-mysqldb {% endif %} 

In the example above, from pillar value of [database][name] recorded in the Vagrantfile and assigned as the ID for the state mysql_user.present .
The value of ['database']['password'] will be used as a password.

Top.sls

top.sls is the file from which SaltStack will start reading states.
It contains a list of include .sls files for this configuration.

Ready configuration

The finished configuration of Vagrant and SaltStack is on GitHub . It allows you to deploy a server with PHP , nginx and mysql one command:

 vagrant up --provision 

When deploying multiple images at the same time, you need to remember to change the IP in a private network.

Conclusion

Automation allows you to reduce the amount of routine work to minor edits. The deployment of normal servers has been reduced to a copy-paste of typical settings and the launch of vagrant up . What it used to take several hours earlier now takes minutes, most of which the server puts packages on.