Multistage backup storage for the smallest

More recently, the planet swept the waves of WannaCry and its clones. And the problem itself cryptographers facing system administrators for over 10 years. Sooner or later - but all the implemented and implemented measures to protect against cryptographers do not help, and yet there is a user who opens the letter, the attachment and receives the full “bouquet”. Also a lot of "pleasant and exciting" hours gets the system administrator.



And then everyone clearly begins to understand that backup copies are needed (many, different, in different places). Those. Rule 3-2-1, invented and described by Peter Krogh, is highly desirable to fulfill. This article is an example that helps to make real the implementation of this rule on the "knee" - without buying expensive equipment (in conditions of austerity).

So - the conditions of the problem being solved:

• There is a small virtualization environment from VMware (a couple of ESXi servers, vCenter, the cheapest license package — the initial Kit — in general, this is not important for this article. Similarly, the article will work for Hyper-V);



• There are about a dozen virtual machines whose contents you do not want to lose in the event of an automated Ransomware script running;

')

• There is a backup system from Veeam (free edition, backups are made using PowerShell and Task Schedule).

Tasks:

• Make backup copies of servers once a day at night;



• Copy copies (copy to NAS server with FreeBSD + ZFS). By the way, snapshots are also made on ZFS, which are automatically deleted according to the set schedule (zfSnap + Cron);



• Have an offline copy of backups on removable media.

Implementation:

Since the main server, which makes backup copies of running virtual machines, is managed by the Windows Server operating system (due to the fact that Veeam Backup is currently running only on the basis of this OS), it was decided to use PowerShell to accomplish the tasks.

Solution of the synchronization of backups between the main server (Windows) and the NAS server (FreeBSD):

To solve the problem, a script was required that would run through Task Scheduler and synchronize directory A with network resource B accessible via the SMB protocol. At first I tried using robocopy - but the tests showed a very low speed of the resulting script. And I decided to implement this script on another tool.



A five-minute search and 10 minutes of tests showed a very viable and ready-made solution: powershell-synchronizing-a-folder



The script was chic:



• Works with both local drives and network resources;

• Allows you to exclude certain files from the task;

• Allows you to synchronize files on a given pattern

• It operates at maximum speed (i.e., how much iron and the network can issue — synchronization at such a speed and passes, unlike robocopy).



As a result, a stack of tasks appeared on the main server in the Task Schedule:

powershell.exe "C:\Scripts\syncfolder.ps1 -SourceFolder:G:\Backups\WEBAPPS -TargetFolder:\\192.168.0.232\backups$\WEBAPPS" 

And the task of synchronizing backups after completing the tasks Veeam Backup was solved (2 copies with a delta in time).

Solving the problem of creating offline backups:

The idea is simple:



• We connect to the server with Veeam Backup an external USB 3.0 hard drive with 2 TB



• Most of the time we keep it offline (and this is how we protect ourselves from automated Ransomware);



• When the script runs, it transfers the disk to Online, makes a directory with the current date, copies the current backup copies to it, and when it is completed, returns the disk to Offline again.



Implementation:



The starting point is the command: Get-Disk - we need to understand what disks we have in the system and whether we can see the external USB-drive:

 PS C:\Windows\system32> Get-Disk Number Friendly Name OperationalStatus Total Size Partition Style ------ ------------- ----------------- ---------- --------------- 1 WDC WD30PURX-64P6ZY0 Online 2.73 TB GPT 0 WDC WD10EZEX-60M2NA0 Online 931.51 GB GPT 2 WD Elements 25A3 USB Device Offline 1.82 TB GPT 

Now we need to put a link to the USB disk in a variable. To identify it, it is suggested to use the attribute “Friendly Name”. If you prefer to use other attributes, print the full list (get-disk | select *). Or look at the list of available properties and methods (get-disk | get-member).



Total first part of the script:

 # Find USB disk by FriendlyName $mybackupdisk = get-disk | where {$_.FriendlyName -like 'WD Elements 25A3 USB Device'} 

Next, you need to transfer the disk from Offline to Online, and also make sure that the disk is in Read-Write mode (sometimes, for an unclear reason, the disk became Online Read-Only after switching to Online. To determine the disk number, use the Number property ($ mybackupdisk.Number ).



We get a piece:

 # Make disk Online Set-Disk -Number $mybackupdisk.Number -IsOffline $False Start-Sleep -s 5 # Make disk Writeable (some times it ReadOnly after online - shit happens...) Set-Disk –Number $mybackupdisk.Number -IsReadonly $False Start-Sleep -s 5 

To identify the drive letter, let's make the following trick - hang a label (name) on the USB disk: VMUSBBACKUPS (either via Disk Manager or with the Set-Volume command).



Then, using the Get-Volume command, we determine the letter of the connected USB disk (after transferring it to Online):

 # Find Disk Volume $usbvolumename = Get-Volume | where {$_.FileSystemLabel -like 'VMUSBBACKUPS'} 

And actually copying the necessary data to disk:



Create a directory with the current date in the name:

 $date = Get-Date $newbackupfolder = $date.ToString("yyyy-MM-dd") # Full Backup Fath $createdirfullpath = $usbvolumename.DriveLetter + ":\" + $newbackupfolder # Create Backup Directory New-Item -ItemType directory -Path $createdirfullpath -Force -Confirm:$false Start-Sleep -s 2 

Copy backups:

 # Source Backup Dir (with backups) $sourcebackup = "F:\Backups\VCENTER\" # Copy to USB from Disk Copy-Item $sourcebackup -Destination $createdirfullpath -Recurse Start-Sleep -s 5 

Another option is when we need not to create new directories and copies every time — and rewrite files with new versions — then we use the previously found script to synchronize directory A with B:

 # Sync from HDD to USB: C:\Scripts\syncfolder.ps1 -SourceFolder:F:\Backups\ -TargetFolder:$usbvolumename.DriveLetter:\VMs\ Start-Sleep -s 5 

In any case, when you finish copying or synchronizing, it is highly desirable to reset the operation cache (from RAM to HDD / USB) with the command:

 # Write USB Disk Cache before offline Write-VolumeCache $usbvolumename.DriveLetter Start-Sleep -s 5 

And do not forget to transfer the disk from Online to Offline again:

 # Place USB to Offline Set-Disk -Number $mybackupdisk.Number -IsOffline $True 

Results:

• Received backups in three locations (Windows server, FreeBSD server, USB disk);

• Two types of storage (in balloons and on disk);

• One media of a different type - cooled. You can even have a couple of disks - and just 1 or 2 times a month to change their places (one in the safe). Since the USB-drive in offline mode 95% of the time - it can always be safely pulled out of the server.

My stats:

• this scheme has been working for 6 months without failures;

• the amount of synchronized data (compressed and deduplicated backups - from 500 to 700 GB);

• Sync time to USB drive - 1 hour 20 minutes on average (once a week on weekends).



Full scripts can be downloaded from Google Disk: BackupExamples