Migrating to Chef Server 11

As part of our EPAM Private Cloud, we initially used Chef Server 10 for autoconfiguring virtual machines.
The list of supported roles has exceeded 60 and included both simple and rather complex cluster solutions.
And when the number of server clients increased to 750, we noticed a significant decrease in performance.
It was not expedient to increase the power of the virtual machine on which Chef Server 10 is installed, it was not so small (2x Intel® Xeon® CPU L5640 @ 2.27GHz and 8Gb of RAM).
Manipulations with tuning chef-solr and chef-expander also did not give the desired performance gain.

That's when it was decided to migrate from Chef 10 to Chef 11.
The site opscode.com says that it is faster, easier to scale and easier to configure and manage.

So, first a brief installation guide for Chef Server 11 for Ubuntu12.04.

#    wget -O chef-server-11.deb https://opscode-omnitruck-release.s3.amazonaws.com/ubuntu/12.04/x86_64/chef-server_11.0.6-1.ubuntu.12.04_amd64.deb sudo dpkg -i chef-server-11.deb #   sudo chef-server-ctl reconfigure #,     sudo chef-server-ctl test 

When the server is installed and running, you need to create a user with administrator rights.
Create a .chef directory in the user's home folder and copy the necessary keys.

 mkdir ~/.chef cp /etc/chef-server/admin.pem ~/.chef cp /etc/chef-server/chef-validator.pem ~/.chef 

Run the configuration file setting for knife.

 knife configure -i #   -  cat ~/.chef/khife.rb log_level :info log_location STDOUT node_name 'chefuser' client_key '/home/chefuser/.chef/chefuser.pem' validation_client_name 'chef-validator' validation_key '/home/chefuser/.chef/chef-validator.pem' chef_server_url 'https://192.168.0.1' syntax_check_cache_path '/home/chefuser/.chef/syntax_check_cache' 

Transfer data from Chef 10.

First you need to install knife-essentials to expand the functionality of utility knife

 /opt/chef-server/embedded/bin/gem install knife-essentials 

Create a directory that we will use as a transit point. And in it we will create a folder for the knife configuration files.

 mkdir -p ~/transfer/.chef 

Create separate knife configuration files for Chef 10 and Chef 11


It remains to copy webui.pem from Chef Server 10 to /home/chefuser/transfer/.chef/chef-webui.pem and
/home/chefuser/.chef/chefuser.pem in /home/chefuser/transfer/.chef/admin.pem.

When it is ready, you can try to download information about nodes, clients, roles, etc.

 /opt/chef-server/embedded/bin/knife download -c .chef/knife-chef10.rb / #    /nodes /clients /roles /cookbooks  ..   . ls transfer/ clients cookbooks data_bags environments nodes roles users 

In the same way, you can download information about an individual node or client.
For example:

 /opt/chef-server/embedded/bin/knife download -c .chef/knife-chef10.rb /nodes/server1.json 

Before you start uploading data to Chef Server 11, I recommend checking that you do not have the same client names or nodes on both servers. Otherwise, information about them may be erased.
')
We download data as follows.

 /opt/chef-server/embedded/bin/knife upload -c .chef/knife-chef11.rb / 

You can also upload by one node, client, etc.

Switching customers from Chef 10 to Chef 11.

This is perhaps the most interesting stage in migration.
If you used a DNS name as chef_server_url , then there is no problem. You just need to reassign it to the new server.
But if you used, as in our case, the name of the virtual machine, which is unique and can not be assigned to another machine, you will have to think.
I decided to create this simple Kukbook, the purpose of which was to change the value of chef_server_url to the desired one and make a restart chef-client.
If anyone is interested in such a implementation, the kukbook will be posted on GitHub.

Results : After migrating to Chef 11, we received a performance increase by several times. The former server was constantly “on the verge of a swap” and could process a simple request for a list of roles for up to 2 minutes. On the current server with the same number of clients, 3 GB of RAM is free and requests are processed within a few seconds.

If anyone has any questions - write in the comments, I will answer with pleasure.