Preparing Servers with Chef Solo

In this article I want to talk about using Chef Solo both to prepare the developer’s environment (dev servers) and to prepare combat servers. On Habré, there have already been several articles ( Rapid Deployment of the Development Environment and the Development Environment Using Vagrant and Chef ) on deploying dev servers using Vagrant and Chef Solo. I want to show how we use Chef Solo in our small company.

Our web-project requires a rather complex environment, as it uses a multi-server architecture. Therefore, it was vital for us to automate the preparation of such an environment. To solve this problem, we use Vagrant and Chef Solo.


Vagrant is an open-source tool for creating a virtual environment (using VirtualBox, VMware, AWS, etc.). In other words, this is a program for easy management of virtual machines. Among other things, Vagrant supports several configuration management systems (Chef, Puppet, Ansible). Configuration refers to the state of the server (in this case, the virtual machine), namely: the set of installed packages, the list of running services, the contents of configuration files, the composition of users, etc. That is, Vagrant is able to start the process of bringing the machine to the desired state immediately after launch.
')
Vagrant is perfect for creating a developer environment. It is enough for the developer to make a clone of the Vagrant project, and execute the “vagrant up” command. Everything else is done automatically: downloading the image of a virtual machine (if it is not on the developer's machine), launching the virtual machine with configured network parameters and shared directories, launching the configuration management system. This provides a single environment for all developers, which is very important for large teams or complex projects.

Chef is a server configuration management system. Chef has a client-server architecture. The Chef server stores information about connected Chef clients and sets of “recipes” for their “preparation” (that is, to bring them to the required state). You have to pay for using a Chef server (up to 5 clients can be connected for free).

Why was Chef chosen, and not another configuration management system (seriously only looked at Puppet):

• large community;
• many ready-made recipes;
• recipes are written in ruby;
• predictable order of recipes (unlike Puppet);
• supported in Vagrant.

Chef Solo is an open-source version of the Chef client that allows you to use recipes without a Chef server (recipes must be physically located on the same machine). Chef Solo is free, but has several limitations compared to client-server Chef. For example, there is no possibility to use the search for servers in terms of the recipes.

Why was chosen Chef Solo, but not client-server Chef:

• free of charge (we have more than 5 servers, but not thousands yet);
• safe (data about our servers are not stored on foreign servers);
• easy (working with local files instead of console commands to the Chef server);
• there is still the possibility of an easy transition to the Chef server (if there are too many servers).

Chef Solo allowed to fully automate the installation and configuration of the project on the new server. Deploying the combat server for our application usually takes about 10-15 minutes (depending on the complexity of the recipes).

To manage the servers is the next project . Virtual machines are started from the catalog of this project using Vagrant, and real servers are being prepared. In this project, one virtual server (demo) has already been added, on which the latest version of nginx is installed.

Project structure:

• cookbooks / : a directory of other recipes that have been loaded from the outside and are not editable. More details .
• data_bags / : directory for storing additional data available from recipes. More details .
• environments / : environment directory. An environment is a specific set of settings used for a server that is deployed with an indication of this environment. This is where development / production settings are shared. To be able to use such environments, Chef Solo version is not lower than 11.6.0 and Vagrant version is not lower than 1.3.0. More details .
• nodes / : files with server attributes. For each server, there should be a file with a name equal to its IP address, containing the values ​​of server attributes, as well as a list of roles and recipes to apply to the server. More details .
• roles / : roles directory. Role means a specific set of settings used for the server to which this role is assigned. The server can be assigned several roles. More details .
• site-cookbooks / : directory of their recipes. Contains recipes written with his own hands to provide specific requirements (for example, deploy work projects). This directory is specified in the “solo.rb” and “Vagrantfile” files as an additional storage for recipes.
• deploy.sh : script to prepare the combat server (details below).
• install.sh : a script to install Chef Solo and launch it on the production server (details below).
• solo.rb : Chef Solo configuration file.
• Vagrantfile : Vagrant settings file. It lists all virtual servers.

To deploy a new virtual server for development, you must:

1. In “Vagrantfile”, associate the server name (for example, “frontend”) with its IP address (for example, “10.2.2.100”).
2. In the “nodes” directory create a JSON file of new server attributes (for example, “10.2.2.100.json”).
3. Start the virtual machine with the “vagrant up” command with the machine name (for example, “vagrant up frontend”). When you first start, Chef Solo will run, which will execute all the recipes specified in the corresponding attribute JSON file. In this case, the development environment will be used (this is indicated in the Vagrantfile).

To raise a new combat server, you must:

1. In the “nodes” directory create a JSON file of the attributes of the new server (for example, “23.144.12.15.json”).
2. Run the “deploy.sh” script indicating the user and server IP (for example, “./deploy.sh root@23.144.12.15”). The “deploy.sh” script internally provides an SSH connection to the server, so all the additional parameters specified during the launch of this script will be used as parameters of the “ssh” command (this is important when entering by key). Under Windows, we run this script in Git Bash, since for its execution we need some * nix-commands that are not in “cmd.exe”.

The “deploy.sh” script performs the following actions:

1. connects to the server via SSH;
2. archives the entire project directory and sends the archive via SSH to the server;
3. unpacks the archive on the server in the "~ / chef" directory;
4. runs the install script “install.sh”.

The “install.sh” script performs the following actions:

1. Checks if Chef Solo is installed and, if necessary, installs it. On servers, we use Ubuntu, so for other systems it may be necessary to fix the commands to install Chef.
2. launches Chef Solo with the configuration file “solo.rb” and JSON-file of server settings. In this case, the “production” environment will be used (this is indicated in “solo.rb”).

In addition to the initial preparation of the server, it is often necessary to update its configuration. For example, you need to update some packages, post a new version of the application, or apply the latest changes to the recipes. For a virtual server, this operation is performed by the “vagrant provision # machine_name #” command. For the combat server, you need to re-launch "./deploy.sh root @ # ip_server #".

The “deploy.sh” and “install.sh” scripts are written based on the article Chef Solo tutorial: Managing a single server with Chef .

Instead of “deploy.sh”, you can look at knife-solo , but it has several disadvantages compared with the proposed method:

• Chef will have to be installed on the machine from which the server is being prepared;
• Windows requires cygwin + rsync (non-trivial setting);
• Chef is installed on the server through the Opscode Installer, which has problems installing some gem packages (this problem is solved by specifying your installation script).

So, with the help of simple devices, you can relatively easily deploy virtual servers for development and combat servers for work projects. I hope my article will be useful to someone.