How to automate the creation of virtual machines? We tell in detail
Creating a new virtual machine is a time-consuming routine. And the more infrastructure and organization, the more procedures associated with this process. We automated this process using PowerShell.
Welcome under the cat, if you are interested.
')
Programmers do not like to do double work, system administrators, too.
Below is an example of the automation of one of our customers.
We wanted to make sure that any engineer or project manager could create a new virtual machine with minimal effort and in the shortest possible time. Our customer has an ITSM system, in this example it is ServiceNow, we created the corresponding web form in the service catalog. To “order” a new machine, the manager needs to fill in the fields and confirm the “order”, after that the process chain is started, and at the output we get a ready-to-use machine.
So let's consider what the manager needs to define in order to create a new virtual machine:
VM Description: virtual machine description
Here we need some explanations. PowerShell 5.1 is actively used in our solution, so for now Windows-only, in the future we will try to add support for Unix-machines and switch to PowerShell Core.
OS , operating system. There are no particular obstacles to using Windows 2008 (R2), but we use 2012R2 or 2016.
VM Size , the size of the virtual machine. For each, this can be defined differently, in this example, Small 1CPU-4Gb Ram, Medium 2CPU-8Gb, Large 4-16.
VM Storage , Disk 0 (C: \) has a fixed size that you cannot change, only the Fast / Slow storage selector is available. “Fast” - this could be a Storage Tier with SSD, and “Slow” is storage on “regular” HDDs (of course - SAN). Disk1 (Disk2 and on) also has a selector for the type of Storage, as well as fields for entering the desired size in gigabytes, Letter for the partition and cluster size (which is important for SQL Server).
Trust , we determine that the machine must be Domain-joined or not, with access from the Public Network or not.
Type , type of machine. Almost every car can be defined as front-end or back-end applications or other in all other cases. Based on the selected type, we will be able to further determine the most appropriate subnet for the machine.
Environment , in the customer's infrastructure there are two data centers: Primary (Production) and Secondary (Dev / test), DC are interconnected by a fast communication channel and provide fault tolerance. By convention, all virtual machines in Primary DC have an IP address starting at 10.230, and in Secondary DC at 10.231.
(SLA) Service Level Agreement , this parameter affects the quality of service of this machine.
Applications . We added the ability to install and configure SQL Server. You must select edition, instance name and collation. It is also possible to set up a Web Server role and more.
Now we need to determine how to store the selected values. We decided that the most convenient format is a JSON file. As I said earlier, ITSM ServiceNow is used in the customer’s environment; the manager, after he has selected all the necessary values, clicks the “order” button and after that ServiceNow transfers all the parameters to our PowerShell script (to the back-end ServiceNow), which will create the JSON file. It looks like this:
In the body of the CreateConfiguration .ps1 script:
At the end, export our object to a JSON file:
Approximate sample of configuration:
You may have noticed that the web form was missing the name of the virtual machine and the IP address. We get these values ​​automatically as follows:
The name of the machine , ITSM ServiceNow has a special section: CMDB (Configuration Management Data Base), this database stores all records of existing virtual machines, their status, support team, and so on. We have created about 200 backup records with the status Allocated. To get the name for the virtual machine, we make a REST request to the CMDB and get the first “free” record and change its status from Allocated to Pending install.
IP address and VLAN , we deployed IPAM on our network - this is a built-in feature in Windows Server 2016 that allows you to manage IP addresses on your network. It is not necessary to use all the capabilities of IPAM (DHCP, DNS, AD), but to use it only as a database of IP addresses with a potential extension of functionality. The script that creates the JSON file makes a request to IPAM for the first free IP address in the subnet. A VLAN subnet (x / 24 subnet) is determined based on the selected SLA, Environment, Trust, and Type values.
The configuration file is ready, all fields are in place, you can create a machine. The question is "how to store credentials for all of our scripts?". We use the CredentialManager package. This package works with the built-in Windows Credential Manager API for storing passwords. Example of creating a password:
The password will be readable within this machine and account.
We have a server that stores all configurations with GIT, now we can reliably track all changes in configurations: who, what, where, and when.
The scheduled task is configured on this server: check the configuration folder and write all changes to the Windows Event Log.
After 15 minutes, the scheduled task will write to the Windows EventLog that a new configuration file has been detected.
It's time to check out this configuration. First of all, we need to make sure that the file has the correct formatting:
If everything is good, it's time to start building the machine and run the BuildVM.ps1 script.
In BuildVM.ps1, we verify that the configuration file has a description of all the characteristics of the virtual machine: size, env, sla, type, storage, ram, network.
Be sure to check if there is a machine with the same name in the infrastructure (CheckVM.ps1).
Connect via VMWare PowerShell CLI to our vSphere:
Check if there is a machine with the same name in the infrastructure
And disconnect:
Make sure the machine is also not available via WinRM
If $ VM and $ ping are empty, then you can create a new machine. (We handle situations when the machine is already created in ESXi manually or this machine is in another data center.)
Create a virtual machine
Find the corresponding SLR cluster:
Check that we have enough space on the Datastore:
And enough memory:
We take our template
And create a new virtual machine
It is important to connect the network interface to a subnet with DHCP enabled.
We start the virtual machine
And save the description of the machine, so that you can then determine the machine at the VMWare level.
The machine has started and now we can find out the received MAC address:
Save this value to our JSON file
Here is the time to commit to our Git, that the machine is created and has its own unique MAC.
The machine starts to initialize (after sysprep), tune the hardware and initial configuration.
Let's wait for our WinRM machine to be available with the EstablishConnection.ps1 script.
First, find out what IP the machine received from DHCP:
Now let's wait until the machine is available via WinRM:
The machine is ready to drive.
Desired State Configuration
To configure the desired configuration, we use the PowerShell part - DSC (Desired State Configuration). The network has a configured DSC Pull Server: dscpull.testdomain.eu.
Below is the configuration of our DSC Pull Server. Good article on setting up DSC Pull.
It is available at: https://dscpull.testdomain.eu:8080
His endpoint: https://dscpull.testdomain.eu:8080/PSDSCPullserver.svc
PowerShell 5.1 must be installed on all client pull servers
If non-PowerShell 5.1 is installed:
Install PowerShell 5.1:
A PKI server is also deployed in our network. This is a condition for securely encrypting credentials stored in DSC mof files (Mof files are the “language” in which the Pull Server and its clients communicate). When a client tries to register with the Pull Server, you must specify the Thumbprint certificate and later the Pull Server will use this certificate to encrypt passwords. Below we look at how this works.
We import Root CA to our new machine:
For further work, we need a pair of RSA-keys. We will generate a self-signed certificate and will temporarily work with it.
Now we can register on the Pull Server:
Send the first configuration to our machine
The server is automatically renamed and rebooted. Now we can perform the Join Domain.
Here’s what our mof file looks like:
DSC encrypted credentials from the service account with Domain Admin rights: testdomain.eu \\ service_DomainJoin_001 with a self-signed certificate. The DSC Client decrypts its credentials with its Private Key and applies all configuration modules with the specified domain credentials. In this case, performs a Domain Join to the specified organization unit.
This module adds dscstaging to local administrators for further configuration.
After the reboot, we can log into the machine with domain credentials.
We are waiting for the server to receive a certificate from our PKI (we have configured auto enrollment) and in the future we will work with the certificate issued by our PKI.
Now you will register again on the Pull Server with the updated thumbprint.
Everything, the domain-joined machine, and we can use it as it is convenient for us.
Installing SQL Server
The JSON file describes the requirements for MS SQL Server, and we also use DSC to install and configure SQL Server. Here is the configuration:
Where $ MSSQL is defined:
$ MSSQL.InstanceName - all this is indicated in our Json file. Applying this configuration will install MS SQL Server with all updates in the Updates folder and restart the server if necessary.
The machine is ready.
There are several APIs available in Service-Now. We use Rest API.
To get the list of machines with Allocated status, use the following query:
instance.service-now.com/cmdb_ci_server_list.do?sysparm_query=install_status=16 ^ u_subtype = ^ ORDERBYname
In PowerShell, it looks like this:
The first array object is the hostname we need.
If the machine is ready, you can change the status of the machine in Service-Now, for this the UpdateCI.ps1 script:
To get the table and records, REST API GET requests are used, to change the PUT / POST request record, in the body of which the fields to be changed.
We have created a convenient tool with a graphical tool similar to Azure Portal, which allows you to manage on-premises infrastructure as convenient as possible for us and our customer.
PS 12/24/2018. It all seems outdated? Time to use Azure DevOps. In the next article I will tell you how to do all this using the Azure DevOps pipelines
Welcome under the cat, if you are interested.
')
Programmers do not like to do double work, system administrators, too.
Below is an example of the automation of one of our customers.
We wanted to make sure that any engineer or project manager could create a new virtual machine with minimal effort and in the shortest possible time. Our customer has an ITSM system, in this example it is ServiceNow, we created the corresponding web form in the service catalog. To “order” a new machine, the manager needs to fill in the fields and confirm the “order”, after that the process chain is started, and at the output we get a ready-to-use machine.
So let's consider what the manager needs to define in order to create a new virtual machine:
VM Description: virtual machine description
Here we need some explanations. PowerShell 5.1 is actively used in our solution, so for now Windows-only, in the future we will try to add support for Unix-machines and switch to PowerShell Core.
OS , operating system. There are no particular obstacles to using Windows 2008 (R2), but we use 2012R2 or 2016.
VM Size , the size of the virtual machine. For each, this can be defined differently, in this example, Small 1CPU-4Gb Ram, Medium 2CPU-8Gb, Large 4-16.
VM Storage , Disk 0 (C: \) has a fixed size that you cannot change, only the Fast / Slow storage selector is available. “Fast” - this could be a Storage Tier with SSD, and “Slow” is storage on “regular” HDDs (of course - SAN). Disk1 (Disk2 and on) also has a selector for the type of Storage, as well as fields for entering the desired size in gigabytes, Letter for the partition and cluster size (which is important for SQL Server).
Trust , we determine that the machine must be Domain-joined or not, with access from the Public Network or not.
Type , type of machine. Almost every car can be defined as front-end or back-end applications or other in all other cases. Based on the selected type, we will be able to further determine the most appropriate subnet for the machine.
Environment , in the customer's infrastructure there are two data centers: Primary (Production) and Secondary (Dev / test), DC are interconnected by a fast communication channel and provide fault tolerance. By convention, all virtual machines in Primary DC have an IP address starting at 10.230, and in Secondary DC at 10.231.
(SLA) Service Level Agreement , this parameter affects the quality of service of this machine.
Applications . We added the ability to install and configure SQL Server. You must select edition, instance name and collation. It is also possible to set up a Web Server role and more.
Now we need to determine how to store the selected values. We decided that the most convenient format is a JSON file. As I said earlier, ITSM ServiceNow is used in the customer’s environment; the manager, after he has selected all the necessary values, clicks the “order” button and after that ServiceNow transfers all the parameters to our PowerShell script (to the back-end ServiceNow), which will create the JSON file. It looks like this:
.\CreateConfiguration.ps1 -SecurityZone trusted -VMDescription "VM for CRM System" -Requestor "evgeniy.vpro" -OSVersion 2k16 -OSEdition Standard -BuildNewVM -VMEnvironment Prod -VMServiceLevel GOLD -VMSize Medium -Disk0Tier Fast -Disk1Size 50 -Disk1Tier Eco -Disk1Letter D -MSSQLServer -MSSQLInstanceName "Instance1" -SQLCollation Latin1_General_CI_AS -SQLEdition Standard -Disk2Size 35 -Disk3Size 65 In the body of the CreateConfiguration .ps1 script:
# PowerShell- $config = [ordered]@{} # . $config.SecurityZone=$SecurityZone At the end, export our object to a JSON file:
$ServerConfig = New-Object –TypeName PSObject $config ConvertTo-Json -InputObject $ServerConfig -Depth 100 | Out-File "C:\Configs\TargetNodes\Build\$($Hostname.ToLower()).json" -Force Approximate sample of configuration:
{ "Hostname": "dsctest552", "SecurityZone": "trusted", "Domain": "testdomain", "Requestor": "evgeniy.vpro", "VM": { "Size": "Medium", "Environment": "Prod", "SLA": "GOLD", "DbEngine": "MSSQL", "RAM": 8, "Storage": [ { "Id": 0, "Tier": "Fast", "Size": "100", "Allocation": 4, "Letter": "C" }, { "Id": 1, "Tier": "Eco", "Size": 50, "Label": "Data", "Allocation": 64, "Letter": "D" }, { "Id": 2, "Tier": "Fast", "Size": 35, "Label": "Data", "Allocation": 64, "Letter": "E" }, { "Id": 3, "Tier": "Fast", "Size": 65, "Label": "Data", "Allocation": 64, "Letter": "F" } ] }, "Network": { "MAC": "", "IP": "10.230.168.50", "Gateway": "10.230.168.1", "VLAN": “VLAN168” }, "OS": { "Version": "2k16", "Edition": "Standard", "Administrators": [ "LocaAdmin", "testdomain\\ Security-LocalAdmins" ] }, "OU": "OU=Servers,OU=Staging,DC=testdomain", "Applications": [ { "Application": "Microsoft SQL Server 2016", "InstanceName": "vd", "Collation": "Latin1_General_CI_AS", "Edition": "Standard", "Features": "SQLENGINE", "Folders": { "DataRoot": "E:\\MSSQL", "UserDB": "E:\\MSSQL\\MSSQL11.vd\\MSSQL\\Data", "UserLog": "E:\\MSSQL\\MSSQL11.vd\\MSSQL\\Log", "TempDB": "D:\\MSSQL\\MSSQL11.vd\\MSSQL\\TempDB", "TempDBLog": "D:\\MSSQL\\MSSQL11.vd\\MSSQL\\TempDB", "Backup": "E:\\MSSQL\\MSSQL11.vd\\MSSQL\\Backup" }, "MaxMemory": 2147483647 } ], "Description": "VM for CRM", "Certificate": { "File": null, "Thumbprint": null }, "Version": 0 } You may have noticed that the web form was missing the name of the virtual machine and the IP address. We get these values ​​automatically as follows:
The name of the machine , ITSM ServiceNow has a special section: CMDB (Configuration Management Data Base), this database stores all records of existing virtual machines, their status, support team, and so on. We have created about 200 backup records with the status Allocated. To get the name for the virtual machine, we make a REST request to the CMDB and get the first “free” record and change its status from Allocated to Pending install.
IP address and VLAN , we deployed IPAM on our network - this is a built-in feature in Windows Server 2016 that allows you to manage IP addresses on your network. It is not necessary to use all the capabilities of IPAM (DHCP, DNS, AD), but to use it only as a database of IP addresses with a potential extension of functionality. The script that creates the JSON file makes a request to IPAM for the first free IP address in the subnet. A VLAN subnet (x / 24 subnet) is determined based on the selected SLA, Environment, Trust, and Type values.
The configuration file is ready, all fields are in place, you can create a machine. The question is "how to store credentials for all of our scripts?". We use the CredentialManager package. This package works with the built-in Windows Credential Manager API for storing passwords. Example of creating a password:
New-StoredCredential -Target "ESXi" -UserName "testdomain.eu\vmwareadm" -Password "veryultraP@ssw00rd." -Type Generic -Persist LocalMachine The password will be readable within this machine and account.
$ESXiAdmin = Get-StoredCredential -Type Generic -Target ESXi We have a server that stores all configurations with GIT, now we can reliably track all changes in configurations: who, what, where, and when.
The scheduled task is configured on this server: check the configuration folder and write all changes to the Windows Event Log.
After 15 minutes, the scheduled task will write to the Windows EventLog that a new configuration file has been detected.
It's time to check out this configuration. First of all, we need to make sure that the file has the correct formatting:
$Configuration=(Get-Content -Raw $File | Out-String | ConvertFrom-Json) If everything is good, it's time to start building the machine and run the BuildVM.ps1 script.
In BuildVM.ps1, we verify that the configuration file has a description of all the characteristics of the virtual machine: size, env, sla, type, storage, ram, network.
Be sure to check if there is a machine with the same name in the infrastructure (CheckVM.ps1).
Connect via VMWare PowerShell CLI to our vSphere:
$VmWareAdmin = Get-StoredCredential -Type Generic -Target ESXi Connect-VIServer -Server "vSphereSrv" -Credential $VmWareAdmin | Out-Null Check if there is a machine with the same name in the infrastructure
$VM=Get-VM $server -ErrorAction SilentlyContinue And disconnect:
Disconnect-VIServer * -Force -Confirm:$false Make sure the machine is also not available via WinRM
$ping=Test-NetConnection -ComputerName $Configuration.Hostname -CommonTCPPort WINRM -InformationLevel Quiet -ErrorAction SilentlyContinue If $ VM and $ ping are empty, then you can create a new machine. (We handle situations when the machine is already created in ESXi manually or this machine is in another data center.)
A few words about the car. This is a prepared virtual machine image that was finalized by sysprep and converted to a template in our vSphere. The image has a local administrator with a known password, this account does not crash after sysprep, which will allow us to access each machine from this template, and later we will be able to replace this password for security reasons.
Create a virtual machine
Find the corresponding SLR cluster:
$Cluster=Get-Cluster -Name $Configuration.VM.SLA Check that we have enough space on the Datastore:
$DatastoreCluster = Get-DatastoreCluster |Where-Object {$_.Name -like $Datastore1Name} $Datastore1 = Get-Datastore -Location $DatastoreCluster |sort -Property "FreeSpaceGB" |select -Last 1 IF ($Datastore1.FreeSpaceGB -le "200"){ Write-Host -foreground red "STOP: Not enough datastore capacity for DISK" $vdisk.Id Break } And enough memory:
$VMHost = Get-VMHost -Location $Cluster |sort -Property "MemoryUsageGB" |select -First 1 IF ($VMHost.MemoryUsageGB -le "20"){ Write-Host -foreground red "STOP: No enough ESXi host capacity" Break } We take our template
$VMTemplate = Get-Template -Name 'Win2016_Std_x64_Template' And create a new virtual machine
New-VM -Name $Configuration.Hostname.ToUpper() -VMHost $VMHost -ResourcePool $ResourcePool -Datastore $Datastore -Template $VMTemplate -Location "AutoDeployed VMs" It is important to connect the network interface to a subnet with DHCP enabled.
We start the virtual machine
Start-VM $VM And save the description of the machine, so that you can then determine the machine at the VMWare level.
Set-Annotation -Entity $VM -CustomAttribute "Change request" -Value $Configuration.Request -Confirm:$false Set-VM $VM -Notes $Configuration.Description -Confirm:$false The machine has started and now we can find out the received MAC address:
$vMAC = (($VM | Get-NetworkAdapter | Select-Object -Property "MacAddress").MacAddress).Replace(':','') Save this value to our JSON file
$Configuration.Network.MAC=$VMAC ConvertTo-Json -InputObject $Configuration -Depth 100 | Out-File "C:\Configs\TargetNodes\Build\$Hostname.json" -Force Here is the time to commit to our Git, that the machine is created and has its own unique MAC.
The machine starts to initialize (after sysprep), tune the hardware and initial configuration.
Let's wait for our WinRM machine to be available with the EstablishConnection.ps1 script.
First, find out what IP the machine received from DHCP:
# $MAC = $vMAC while($isOnline -ne $true){ if((Get-DhcpServerv4Lease -ClientId $MAC -ScopeId $StagingDHCPScope -ComputerName $DHCPServer -ErrorAction Ignore).IPAddress.IPAddressToString){ $tempIP=(Get-DhcpServerv4Lease -ClientId $MAC -ScopeId $StagingDHCPScope -ComputerName $DHCPServer).IPAddress.IPAddressToString break } else{ if($isOnline -ne $true){ Write-Host "`r$i`t" -NoNewline $i++ } } } Now let's wait until the machine is available via WinRM:
$LocalAdmin = Get-StoredCredential -Type Generic -Target LocalAdmin $i=0 $isOnline=$false while($isOnline -ne $true){ if(Invoke-Command -ComputerName $tempIP -ScriptBlock{ Get-ItemProperty -Path "Registry::\HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Component Based Servicing" } -Credential $LocalAdmin -ErrorAction SilentlyContinue){ $isOnline=$true break } else{ if($isOnline -ne $true){ Write-Host "`r$i" -NoNewline $i++ Start-Sleep -Seconds 1 } } } The machine is ready to drive.
Desired State Configuration
To configure the desired configuration, we use the PowerShell part - DSC (Desired State Configuration). The network has a configured DSC Pull Server: dscpull.testdomain.eu.
Below is the configuration of our DSC Pull Server. Good article on setting up DSC Pull.
Node $NodeName { WindowsFeature DSCServiceFeature { Ensure = "Present" Name = "DSC-Service" } xDscWebService PSDSCPullServer { Ensure = "Present" EndpointName = "PSDSCPullServer" Port = 8080 PhysicalPath = "$env:SystemDrive\inetpub\PSDSCPullServer" CertificateThumbPrint = $certificateThumbPrint ModulePath = "$env:PROGRAMFILES\WindowsPowerShell\DscService\Modules" ConfigurationPath = "$env:PROGRAMFILES\WindowsPowerShell\DscService\Configuration" State = "Started" DependsOn = "[WindowsFeature]DSCServiceFeature" RegistrationKeyPath = "$env:PROGRAMFILES\WindowsPowerShell\DscService" AcceptSelfSignedCertificates = $true UseSecurityBestPractices = $true } File RegistrationKeyFile { Ensure = 'Present' Type = 'File' DestinationPath = "$env:ProgramFiles\WindowsPowerShell\DscService\RegistrationKeys.txt" Contents = $RegistrationKey } } It is available at: https://dscpull.testdomain.eu:8080
His endpoint: https://dscpull.testdomain.eu:8080/PSDSCPullserver.svc
PowerShell 5.1 must be installed on all client pull servers
If non-PowerShell 5.1 is installed:
$PSVersionTable.PSVersion.Major –lt 5 Install PowerShell 5.1:
Write-Host "Download PowerShell 5.1" Invoke-Command -ComputerName $Node -ScriptBlock { [System.Net.ServicePointManager]::SecurityProtocol=[System.Net.SecurityProtocolType]::Tls12;Invoke-WebRequest -Uri "https://dscpull.testdomain.eu:8080/Files/Updates/WMF.msu" -OutFile C:\TEMP\WMF.MSU } Write-Host "Extract PowerShell 5.1" Invoke-Command -ComputerName $Node -ScriptBlock {Start-Process -FilePath 'wusa.exe' -ArgumentList "C:\temp\WMF.msu /extract:C:\temp\" -Wait -PassThru } Write-Host "Apply PowerShell 5.1" Invoke-Command -ComputerName $Node -ScriptBlock {Start-Process -FilePath 'dism.exe' -ArgumentList "/online /add-package /PackagePath:C:\temp\WindowsBlue-KB3191564-x64.cab /Quiet" -Wait -PassThru } Write-Host "PowerShell 5.1 has been installed" A PKI server is also deployed in our network. This is a condition for securely encrypting credentials stored in DSC mof files (Mof files are the “language” in which the Pull Server and its clients communicate). When a client tries to register with the Pull Server, you must specify the Thumbprint certificate and later the Pull Server will use this certificate to encrypt passwords. Below we look at how this works.
We import Root CA to our new machine:
Invoke-Command -ComputerName $server -ScriptBlock{ $PKI="-----BEGIN CERTIFICATE----- MIIF2TCCA8GgAwIBAgIQSPIjcff9rotNdxbg3+ygqDANBgkqhkiG9w0BAQUFADAe **************************************************************** znafMvVx0B4tGEz2PFss/FviGdC3RohBHG0rF5jO50J4nS/3cGGm+HGdn1w/tZd0 a0FWpn9VCOSmXM2It+tSW1f4nZVt6T2kr1ZlTxkDhT7HMSGsrX/XJswzCkDGe3dE qrVVjNUkhVTaeeBWdujB5J6mcx7YkNsAUhODiS9Cf7FnYnxLFA72M0pijI48P5F0 ShM9HWAAUIrLkv13ug== -----END CERTIFICATE-----" $PKI | Out-File RootCA.cer Import-Certificate RootCA.cer -CertStoreLocation Cert:\LocalMachine\Root | select Thumbprint | Out-Null } -Credential $LocalAdmin | Out-Null For further work, we need a pair of RSA-keys. We will generate a self-signed certificate and will temporarily work with it.
Now we can register on the Pull Server:
$DscHostFQDN = [System.Net.Dns]::GetHostEntry([string]$env:computername).HostName $DscPullServerURL = "https://$($DscHostFQDN):8080/PSDSCPullserver.svc" $DscWebConfigChildPath = '\inetpub\psdscpullserver\web.config' $DscWebConfigPath = Join-Path -Path $env:SystemDrive -ChildPath $DscWebConfigChildPath $DscWebConfigXML = [xml](Get-Content $DscWebConfigPath) $DscRegKeyName = 'RegistrationKeys.txt' $DscRegKeyXMLNode = "//appSettings/add[@key = 'RegistrationKeyPath']" $DscRegKeyParentPath = ($DscWebConfigXML.SelectNodes($DscRegKeyXMLNode)).value $DscRegKeyPath = Join-Path -Path $DscRegKeyParentPath -ChildPath $DscRegKeyName $DscRegKey = Get-Content $DscRegKeyPath [DSCLocalConfigurationManager()] configuration RegisterOnPull { Node $Node { Settings { ConfigurationModeFrequencyMins = 1440 CertificateID = $Thumbprint RefreshMode ='Pull' RefreshFrequencyMins = 1440 RebootNodeIfNeeded = $true ConfigurationMode ='ApplyAndAutoCorrect' AllowModuleOverwrite = $true DebugMode = 'None' StatusRetentionTimeInDays = 1 } ConfigurationRepositoryWeb $([string]$env:computername) { ServerURL = $DscPullServerURL RegistrationKey = $DscRegKey CertificateID = $Thumbprint ConfigurationNames = @("$hostx") } } } RegisterOnPull -OutputPath $MetaConfigsStorage Set-DscLocalConfigurationManager -ComputerName $Node -Path $MetaConfigsStorage -Verbose -Force -Credential $LocalAdmin Send the first configuration to our machine
Configuration Rename { param ( [Parameter()] [System.String[]] $Node, $hostname ) Import-DscResource -ModuleName xComputerManagement Import-DscResource –ModuleName PSDesiredStateConfiguration Node $Node { xComputer JoinDomain { Name = $hostname } } } Rename -Node $Node -OutputPath $DscConfigPath -hostname $hostname New-DscChecksum $DscConfigPath -Force Invoke-Command -ComputerName $Node -ScriptBlock{Update-DscConfiguration -Verbose -Wait } -Credential $LocalAdmin -Verbose The server is automatically renamed and rebooted. Now we can perform the Join Domain.
Configuration JoinAD { param ( [Parameter()] [System.String[]] $Node, [Parameter(Mandatory = $true)] [ValidateNotNullorEmpty()] [System.Management.Automation.PSCredential] $DomainAdmin, $hostname, $domain ) Import-DscResource -ModuleName xComputerManagement Import-DscResource –ModuleName PSDesiredStateConfiguration Node $Node { xComputer JoinDomain { Name = $hostname DomainName = $domain Credential = $DomainAdmin JoinOU = "OU=Servers,OU=Staging,DC=testdomain,DC=eu" } GroupSet LocalAdmins { GroupName = @( 'Administrators') Ensure = 'Present' MembersToInclude = @( 'testdomain-eu\dscstaging' ) } } } $cd = @{ AllNodes = @( @{ NodeName = $Node PSDscAllowPlainTextPassword = $false PSDscAllowDomainUser=$true Certificatefile = $CertFile Thumbprint = $Certificate.ToString() } ) } JoinAD -Node $Node -OutputPath $DscConfigPath -DomainAdmin $DomainAdmin -hostname $hostname -ConfigurationData $cd -domain $domain New-DscChecksum $DscConfigPath -Force Invoke-Command -ComputerName $Node -ScriptBlock{Update-DscConfiguration -Verbose -Wait } -Credential $LocalAdmin -Verbose Here’s what our mof file looks like:
instance of MSFT_Credential as $MSFT_Credential1ref { Password = "-----BEGIN CMS-----\nMIIBsgYJKoZIhvcNAQcDoIIBozCCAZ8CAQAxggFKMIIBRgIBADAuMBoxGDAWBgNVBAMMD1dJTi1H\nNFFKTFFQME4xNQIQOQN77pxew75HU6l7GPn99TANBgkqhkiG9w0BAQcwAASCAQAlhFf7Zs2gJbJEnc1DEK2yWbKcO+BEyD2cr6vKHdn\nQ9TrjvbysEOvYjT15o6MccwkMEwGCSqGSIb3DQEHATAdBglghkgBZQMEASoEEEdKJT+GX4IkPezR\nwYncyQiAIAFKxwJocH4ufRsq9L2Ipkp+VQCx2ljlwif6ac4X/PqG\n-----END CMS-----"; UserName = "testdomain.eu\\service_DomainJoin_001"; }; instance of MSFT_xComputer as $MSFT_xComputer1ref { ResourceID = "[xComputer]JoinDomain"; Credential = $MSFT_Credential1ref; DomainName = "testdomain.eu"; SourceInfo = "C:\\Program Files\\WindowsPowerShell\\Scripts\\JoinAD.ps1::34::9::xComputer"; Name = "dsctest51"; JoinOU = "OU=Servers,OU=Staging,DC=testdomain,DC=eu"; ModuleName = "xComputerManagement"; ModuleVersion = "4.1.0.0"; ConfigurationName = "JoinAD"; };nMIIBsgYJKoZIhvcNAQcDoIIBozCCAZ8CAQAxggFKMIIBRgIBADAuMBoxGDAWBgNVBAMMD1dJTi1H \ nNFFKTFFQME4xNQIQOQN77pxew75HU6l7GPn99TANBgkqhkiG9w0BAQcwAASCAQAlhFf7Zs2gJbJEnc1DEK2yWbKcO + BEyD2cr6vKHdn \ nQ9TrjvbysEOvYjT15o6MccwkMEwGCSqGSIb3DQEHATAdBglghkgBZQMEASoEEEdKJT + GX4IkPezR \ nwYncyQiAIAFKxwJocH4ufRsq9L2Ipkp + VQCx2ljlwif6ac4X / PqG \ n ----- END CMS -----";instance of MSFT_Credential as $MSFT_Credential1ref { Password = "-----BEGIN CMS-----\nMIIBsgYJKoZIhvcNAQcDoIIBozCCAZ8CAQAxggFKMIIBRgIBADAuMBoxGDAWBgNVBAMMD1dJTi1H\nNFFKTFFQME4xNQIQOQN77pxew75HU6l7GPn99TANBgkqhkiG9w0BAQcwAASCAQAlhFf7Zs2gJbJEnc1DEK2yWbKcO+BEyD2cr6vKHdn\nQ9TrjvbysEOvYjT15o6MccwkMEwGCSqGSIb3DQEHATAdBglghkgBZQMEASoEEEdKJT+GX4IkPezR\nwYncyQiAIAFKxwJocH4ufRsq9L2Ipkp+VQCx2ljlwif6ac4X/PqG\n-----END CMS-----"; UserName = "testdomain.eu\\service_DomainJoin_001"; }; instance of MSFT_xComputer as $MSFT_xComputer1ref { ResourceID = "[xComputer]JoinDomain"; Credential = $MSFT_Credential1ref; DomainName = "testdomain.eu"; SourceInfo = "C:\\Program Files\\WindowsPowerShell\\Scripts\\JoinAD.ps1::34::9::xComputer"; Name = "dsctest51"; JoinOU = "OU=Servers,OU=Staging,DC=testdomain,DC=eu"; ModuleName = "xComputerManagement"; ModuleVersion = "4.1.0.0"; ConfigurationName = "JoinAD"; };
DSC encrypted credentials from the service account with Domain Admin rights: testdomain.eu \\ service_DomainJoin_001 with a self-signed certificate. The DSC Client decrypts its credentials with its Private Key and applies all configuration modules with the specified domain credentials. In this case, performs a Domain Join to the specified organization unit.
GroupSet LocalAdmins { GroupName = @( 'Administrators') Ensure = 'Present' MembersToInclude = @( testdomain-eu\dscstaging' ) } This module adds dscstaging to local administrators for further configuration.
After the reboot, we can log into the machine with domain credentials.
We are waiting for the server to receive a certificate from our PKI (we have configured auto enrollment) and in the future we will work with the certificate issued by our PKI.
$vmcert=Invoke-Command -ComputerName $server -ScriptBlock{ return Get-ChildItem -Path cert:\LocalMachine\My | where {$_.EnhancedKeyUsageList.FriendlyName -eq "Document Encryption"-and $_.Issuer -eq "CN=TestDomain Issuing CA, DC=testdomain, DC=eu"} } -ErrorAction Ignore Now you will register again on the Pull Server with the updated thumbprint.
Everything, the domain-joined machine, and we can use it as it is convenient for us.
Installing SQL Server
The JSON file describes the requirements for MS SQL Server, and we also use DSC to install and configure SQL Server. Here is the configuration:
Configuration $Node{ WindowsFeature "NetFramework35"{ Name = "NET-Framework-Core" Ensure = "Present" Source = "\\$DscHostFQDN\Files\Updates" } WindowsFeature "NetFramework45"{ Name = "NET-Framework-45-Core" Ensure= "Present" } SqlSetup "MSSQL2012NamedInstance"{ InstanceName = $MSSQL.InstanceName Features = $MSSQL.Features ProductKey = $ProductKey SQLCollation = $MSSQL.Collation SQLSysAdminAccounts = @('testdomain-EU\SQLAdmins',' testdomain-EU\Backup') InstallSharedDir = "C:\Program Files\Microsoft SQL Server" InstallSharedWOWDir = "C:\Program Files (x86)\Microsoft SQL Server" InstallSQLDataDir = $MSSQL.DataRoot SQLUserDBDir = $MSSQL.UserDBDir SQLUserDBLogDir = $MSSQL.UserLogDir SQLTempDBDir = $MSSQL.TempDBDir SQLTempDBLogDir = $MSSQL.TempDBLogDir SQLBackupDir = $MSSQL.BackupDir SourcePath = $SQLSource SAPwd = $SA SecurityMode = 'SQL' UpdateSource = ".\Updates" Action = "Install" ForceReboot = $True SQLSvcAccount = $SqlServiceCredential AgtSvcAccount = $SqlServiceCredential ISSvcAccount = $SqlServiceCredential BrowserSvcStartupType = "Automatic" DependsOn = '[WindowsFeature]NetFramework35', '[WindowsFeature]NetFramework45' } Where $ MSSQL is defined:
$MSSQL=$Configuration.Applications | where {$_.Application -eq "Microsoft SQL Server 2012"} $ MSSQL.InstanceName - all this is indicated in our Json file. Applying this configuration will install MS SQL Server with all updates in the Updates folder and restart the server if necessary.
The machine is ready.
Service-Now
There are several APIs available in Service-Now. We use Rest API.
To get the list of machines with Allocated status, use the following query:
instance.service-now.com/cmdb_ci_server_list.do?sysparm_query=install_status=16 ^ u_subtype = ^ ORDERBYname
In PowerShell, it looks like this:
$url="https://instance.service-now.com/api/now/table/cmdb_ci_server?sysparm_query=install_status=16^u_subtype=^ORDERBYname" $uri= new-object System.Uri("https://instance.service-now.com/") # $credentials = (Get-StoredCredential -Type Generic -Target DSC).GetNetworkCredential() $credentials = new-object System.Net.NetworkCredential $credentials.UserName, $credentials.SecurePassword Add-Type -AssemblyName System.Net.Http $handler = New-Object System.Net.Http.HttpClientHandler $handler.CookieContainer = New-Object System.Net.CookieContainer $handler.UseCookies=$true $handler.Credentials=$credentials $HttpClient = New-Object System.Net.Http.HttpClient($handler) $HttpClient.BaseAddress= $uri $Header = New-Object System.Net.Http.Headers.MediaTypeWithQualityHeaderValue("application/json") $HttpClient.DefaultRequestHeaders.Accept.Clear() $HttpClient.DefaultRequestHeaders.Accept.Add($Header); $response=$HttpClient.GetAsync($url) $respStream=$response.Result.Content.ReadAsStringAsync() $Servers = $respStream.Result | ConvertFrom-Json # Configuration Items $ServersCI=$Servers.result The first array object is the hostname we need.
If the machine is ready, you can change the status of the machine in Service-Now, for this the UpdateCI.ps1 script:
param( $CI, [ValidateSet("Allocated","In use","Pending install")] $NewStatus='In use' ) $url="https://instance.service-now.com/api/now/table/cmdb_ci_server?sysparm_query=name=$CI" $uri= new-object System.Uri("https://instance.service-now.com/") $credentials = (Get-StoredCredential -Type Generic -Target DSC).GetNetworkCredential() $credentials = new-object System.Net.NetworkCredential $credentials.UserName, $credentials.SecurePassword Add-Type -AssemblyName System.Net.Http $handler = New-Object System.Net.Http.HttpClientHandler $handler.CookieContainer = New-Object System.Net.CookieContainer $handler.UseCookies=$true $handler.Credentials=$credentials $HttpClient = New-Object System.Net.Http.HttpClient($handler) $HttpClient.BaseAddress= $uri $Header = New-Object System.Net.Http.Headers.MediaTypeWithQualityHeaderValue("application/json") $HttpClient.DefaultRequestHeaders.Accept.Clear() $HttpClient.DefaultRequestHeaders.Accept.Add($Header); $response=$HttpClient.GetAsync($url) $respStream=$response.Result.Content.ReadAsStringAsync() $Servers = $respStream.Result | ConvertFrom-Json $ServerCI=$Servers.result[0] $update=@{} if($NewStatus -eq "In use"){ $update.install_status=1 } if($NewStatus -eq "Pending install"){ $update.install_status=4 } $stringcontent = New-Object System.Net.Http.StringContent((ConvertTo-Json -InputObject $update -Depth 100),[System.Text.Encoding]::UTF8, "application/json"); $result=$HttpClient.PutAsync("https://instance.service-now.com/api/now/table/cmdb_ci_server/$($ServerCI.sys_id)", $stringcontent) To get the table and records, REST API GET requests are used, to change the PUT / POST request record, in the body of which the fields to be changed.
We have created a convenient tool with a graphical tool similar to Azure Portal, which allows you to manage on-premises infrastructure as convenient as possible for us and our customer.
PS 12/24/2018. It all seems outdated? Time to use Azure DevOps. In the next article I will tell you how to do all this using the Azure DevOps pipelines
Source: https://habr.com/ru/post/425129/
All Articles