We stamp windows: automated deployment of Windows virtual machines to Hyper-V using Vagrant (part 1)
Sooner or later, any growing IT company faces the challenge of optimizing resources, which from the point of view of system administration necessarily implies maximum automation of all processes. This is necessary for many reasons: reduction of time costs, minimization of the human factor, increase due to this scalability and reliability of the systems as a whole.
Our company Wild Apricot is developing software (SaaS) that runs on ASP and .NET, and uses MS SQL Server as its database. Therefore, the lion’s share of the infrastructure is running under Windows, mainly Server 2012 R2. Accordingly, the sysop department periodically has to raise new machines with it, and sometimes deploy entire farms, both on a live environment and on a test one. Currently, we have about 20 hypervisors and, accordingly, more than a hundred different machines, of which the absolute majority are VMs with Windows. Now it is planned to install another half-dozen developer environments, each of which consists of a dozen components that are desirable to isolate from each other. To solve such problems, we conceived the automation of the entire process. I made the description of the whole way quite detailed, so the article expanded and so I decided to divide it into three parts. In this part I will try to touch upon general questions, explain the choice of tools and tell you how to prepare the environment for further work.
Historically, all new VMs are under the control of the Microsoft Hyper-V hypervisor.
I had the task of automating the creation of new virtual machines. It is necessary to quickly raise on the hypervisor an operating system with all the necessary software and settings. All this is essentially the first step towards a full-fledged continuous deployment system with a configuration management system, which we plan to implement. This is much beyond the scope of this article, but there is enough material on this topic, and I am going to continue the story about all the nuances of working in the world of Microsoft products in the next article if this material turns out to be interesting and useful. As a result, the task was successfully solved using a fairly traditional set of tools, many of which have long been known in the world of open source software, but it was not so easy to use them fully in the Windows world because of the abundance of various kinds of pitfalls. As a result, we stopped at the following “tulkite”:
In this article we will talk about the first three points of the list.
So what do they do and why were they chosen?
Using this tool allows you to get all the magic of Continuous Delivery in one bottle for a familiar system.
')
Well, let's get down to the most interesting, to the applied technical part of the deployment.
First you need to prepare the host - the system on which virtual machines will be deployed. If they are needed for local tests and everything will spin on the same computer where you are sitting - everything is simple and transparent. However, it often happens that virtual users are needed as a result in a completely different place - in a data center on a live environment, on a neighboring server or on a grandmother's netbook. The fact is that out of the box, Vagrant scripts do all operations on local storage. In principle, no one bothers to first deploy it, set up-play, and then migrate to a distant hypervisor, but it turns out that it takes a decent amount of time even when the server is in the same network segment with you. In the case when you need to deploy through a VPN in a data center on the other side of the earth, this can take more than an hour for each instance, and you may need to take extra steps: export-import in my case, plus reconfiguration of the network, if virtual switches are called differently.
Therefore, for myself, I chose the best solution for raising Vagrant on all hypervisors, where its frequent use is planned. This speeds up the process (even for one machine - because the box weighs significantly less than the VHD file of the deployed image), but adds extra software to the server. Fortunately, Chocolatey and Vagrant do not require a GUI, so you can easily install them even on the free Hyper-V Server.
This task is generally rather trivial:
In general, if you do not like to clog servers with unnecessary programs or prefer to always use the latest versions (it should be understood that, despite the fact that the latest version is usually in the repository - it does not appear there instantly) - you can completely do without Chocolatier by installing Vagrant manually from the distribution. It comes in the form of an msi-package, so there should be no problems with the installation from the console. But personally, I prefer the first option not only because I love chocolate - I just used to put all the software at all, even on a home laptop.
At this moment we are faced with the first pitfall (I have already warned that the path will be thorny?). The fact is that our vagabond (and that is how “vagrant” is translated) from version to version puts itself in different directories, and in the latest release for the moment, it again began to be installed directly into the root of the system disk in the C: \ HashiCorp \ Vagrant folder . Everything would be fine, but he periodically forgets to prescribe the path to his folder in the environment variable, so the system may not find it, if not enter the full path to the binary. It is treated with a simple command in Powershell:
For the command line, use the setx command with the / M key. For example, if you need to change the location of the folder where it will store the boxes (by default it stores them on its folder on drive C, which might not really like it when there is a bit of space on the system drive):
This completes the preparation of the host. How to make a basic box for deployment and how to work with it further - I will tell in the following sections ( part 2 ). There will also be a separate chapter with a list of useful materials and links.
Our company Wild Apricot is developing software (SaaS) that runs on ASP and .NET, and uses MS SQL Server as its database. Therefore, the lion’s share of the infrastructure is running under Windows, mainly Server 2012 R2. Accordingly, the sysop department periodically has to raise new machines with it, and sometimes deploy entire farms, both on a live environment and on a test one. Currently, we have about 20 hypervisors and, accordingly, more than a hundred different machines, of which the absolute majority are VMs with Windows. Now it is planned to install another half-dozen developer environments, each of which consists of a dozen components that are desirable to isolate from each other. To solve such problems, we conceived the automation of the entire process. I made the description of the whole way quite detailed, so the article expanded and so I decided to divide it into three parts. In this part I will try to touch upon general questions, explain the choice of tools and tell you how to prepare the environment for further work.
Why Windows, why Hyper-V?
Historically, all new VMs are under the control of the Microsoft Hyper-V hypervisor.
There are several reasons for this:
- In recent years, it has developed quite strongly and, in general, has sufficient functionality for our tasks. The most important thing is that he began to support Vagrant in human terms, which I will discuss in more detail a bit later. There is also support for many Linux distributions, which allows, with some reservations, to host a zoo of different machines on Hyper-V, in case several OSes are used (and is someone wrong?);
- It is convenient to manage with the help of native tools (MMC tools and Powershell), built-in both server and client operating systems from Microsoft;
- The role of Hyper-V is very easy to add to the server, and in general the installation is available to a wider range of administrators (rather than, say, in the case of KVM or ESXi), there is a lot of documentation, guides, virtual labs and so on, much of this is available in Russian language. This makes it easy to install it not only to administrators, but also to developers or testers (who also often need to quickly raise the environment for their own purposes), which expands the range of possible applications;
Migration of physical machines to virtual is very easy and fast with the help of a simple utility Disk2VHD - it uses the snapshots mechanism and due to this it allows you to make a VHD image of even a working system running in a live environment. This helped a lot when we virtualized our infrastructure. In the same way we migrated VMWare hosts. - Microsoft Hyper-V Server does not need to buy, it is free . For us, this was not very relevant, but this is a great opportunity to “play around” with it without any additional costs.
- From Hyper-V, later, if you wish, you can easily migrate to the Azure cloud platform, while continuing to manage it using the same tools.
Goals
I had the task of automating the creation of new virtual machines. It is necessary to quickly raise on the hypervisor an operating system with all the necessary software and settings. All this is essentially the first step towards a full-fledged continuous deployment system with a configuration management system, which we plan to implement. This is much beyond the scope of this article, but there is enough material on this topic, and I am going to continue the story about all the nuances of working in the world of Microsoft products in the next article if this material turns out to be interesting and useful. As a result, the task was successfully solved using a fairly traditional set of tools, many of which have long been known in the world of open source software, but it was not so easy to use them fully in the Windows world because of the abundance of various kinds of pitfalls. As a result, we stopped at the following “tulkite”:
- Chocolatey;
- Vagrant;
- Boxstarter;
- Chef.
In this article we will talk about the first three points of the list.
Why Vagrant and Chef, Chocolatey and Boxstarter?
So what do they do and why were they chosen?
- Chocolatey is a package manager for Windows. Of course, Microsoft announced that Windows Server 10 will have its own, integrated package manager, but until its introduction into our production is still far and so far quite successfully, you can use chocolatier. It allows you to automate the installation of the necessary software along with all dependencies. For this, NuGet packages are used, which can be quite successfully created by yourself, which is very convenient when you need to install your own server software.
- Vagrant allows you to create and use a sample box of a virtual machine to subsequently automatically deploy one (or several or several dozen) virtual machines from it, configure the necessary parameters, and also perform the so-called provisioning — the process of installing and configuring the necessary software. For example, IIS and Chef-client can be installed there, latest Windows updates can be installed, folders can be customized (so that file extensions and hidden files can be seen), ports on the firewall can be opened, user accounts can be configured, and so on - all without direct participation. administrator, with automatic restarts if necessary, according to the previously described scenario.
Of the analogues immediately comes to mind System Center, which also allows you to create a template and roll out of it several virtual machines. However, for many reasons, I preferred Vagrant - it is lighter, does not require a heavy central server, and the disk of the new virtual machine does not become dependent on the disk of the original template. In addition, Vagrant works from the usual command line, which makes it easy to use on hypervisors without a GUI.
The article will have quite a lot of references to various pitfalls - I want to immediately say that the latest version at the moment is considered - 1.7.2, respectively, in subsequent versions (I hope) many problems will be fixed and unnecessary crutches will fall off. But for sure they will add new bugs, which will allow me to make a whole series of articles. - Boxstarter is a set of useful scripts that make it possible to greatly simplify the configuration of both finished machines and those just raised. It works quite closely with Chocolatey and allows you to configure the final system with a single command, downloading the script from a website or network folder. In addition, under it there are ready-made recipes for Chef from the author.
- Chef is used to centrally manage configurations in the broadest sense, including both initial initialization and keeping them up to date. Instead, you can use other systems, but I settled on it because of its prevalence, powerful community and active development: for example, quite recently a new, significantly improved version of the Kupbook community for IIS appeared. And, of course, influenced by the fact that with him already had a little experience before.
Using this tool allows you to get all the magic of Continuous Delivery in one bottle for a familiar system.
')
The main stages of deployment
Well, let's get down to the most interesting, to the applied technical part of the deployment.
Install Hyper-V and Management Software
First you need to prepare the host - the system on which virtual machines will be deployed. If they are needed for local tests and everything will spin on the same computer where you are sitting - everything is simple and transparent. However, it often happens that virtual users are needed as a result in a completely different place - in a data center on a live environment, on a neighboring server or on a grandmother's netbook. The fact is that out of the box, Vagrant scripts do all operations on local storage. In principle, no one bothers to first deploy it, set up-play, and then migrate to a distant hypervisor, but it turns out that it takes a decent amount of time even when the server is in the same network segment with you. In the case when you need to deploy through a VPN in a data center on the other side of the earth, this can take more than an hour for each instance, and you may need to take extra steps: export-import in my case, plus reconfiguration of the network, if virtual switches are called differently.
Therefore, for myself, I chose the best solution for raising Vagrant on all hypervisors, where its frequent use is planned. This speeds up the process (even for one machine - because the box weighs significantly less than the VHD file of the deployed image), but adds extra software to the server. Fortunately, Chocolatey and Vagrant do not require a GUI, so you can easily install them even on the free Hyper-V Server.
This task is generally rather trivial:
- Installing the Hyper-V role in Windows 8 or Server 2012 requires a reboot and is done through the Server Manager for GUI lovers (there is a link in the materials at the end of the article) or using PowerShell with administrative privileges:
Install-WindowsFeature –Name Hyper-V -IncludeManagementTools
- Chocolatier is put from Powershell with one command:
iex ((new-object net.webclient). DownloadString ('https://chocolatey.org/install.ps1'))
- Well, Vagrant rises already from Chocolatey:
choco install vagrant
In general, if you do not like to clog servers with unnecessary programs or prefer to always use the latest versions (it should be understood that, despite the fact that the latest version is usually in the repository - it does not appear there instantly) - you can completely do without Chocolatier by installing Vagrant manually from the distribution. It comes in the form of an msi-package, so there should be no problems with the installation from the console. But personally, I prefer the first option not only because I love chocolate - I just used to put all the software at all, even on a home laptop.
At this moment we are faced with the first pitfall (I have already warned that the path will be thorny?). The fact is that our vagabond (and that is how “vagrant” is translated) from version to version puts itself in different directories, and in the latest release for the moment, it again began to be installed directly into the root of the system disk in the C: \ HashiCorp \ Vagrant folder . Everything would be fine, but he periodically forgets to prescribe the path to his folder in the environment variable, so the system may not find it, if not enter the full path to the binary. It is treated with a simple command in Powershell:
$ env: Path + = ”; C: \ HashiCorp \ Vagrant \ bin”
For the command line, use the setx command with the / M key. For example, if you need to change the location of the folder where it will store the boxes (by default it stores them on its folder on drive C, which might not really like it when there is a bit of space on the system drive):
setx VAGRANT_HOME "X: / your / path" / M
This completes the preparation of the host. How to make a basic box for deployment and how to work with it further - I will tell in the following sections ( part 2 ). There will also be a separate chapter with a list of useful materials and links.
Source: https://habr.com/ru/post/251435/
All Articles