Python + vshadow + robocopy - directory synchronization with a remote machine

Not so long ago, I had the task of copying a database from a remote branch in order to receive a full copy of it at our office at our office. The idea of ​​a daily copy. The first thing that came to mind is the rsync server to which you are copying, and from there, the copy is used further. A quick search brought a less adequate instruction to the Bole right there .

The problem is that the client machine is Win system. And the receiving party also uses a similar OS. Rsync server on Linux is an extra layer between the two machines. It is possible to install Rsync on a Win server, but for other distracted reasons, it is unnecessary. As a result, I came across my own directory synchronization script and combining it with creating a shadow copy created a new solution.

The resulting script is completely here . The article just give the main milestones and the resulting functionality.

The script creates a shadow copy of the disk. Copies the specified directories to a remote resource using smb. Calculate the differences between the source and the received copy. Generates a report on the copying and received differences, and then sends by mail a report to the specified recipients.
')
As in the above article, first download the Volume Shadow Copy Service SDK 7.2 from the Microsoft website. We install on the machine and pull out the utilities we need.

We will need vshadow.exe and vshadow.pdb files from this SDK. Attention - they are different for 64-bit and 32-bit systems.

After installation they can be found here:

64-bit -% PROGRAMFILES% \ Microsoft \ VSSSDK72 \ TestApps \ vshadow \ bin \ obj-chk \ amd64
32-bit -% PROGRAMFILES% \ Microsoft \ VSSSDK72 \ TestApps \ vshadow \ bin \ release-xp

We put in a separate directory. I usually create the c: \ Windows \! Script directory for different tasks and there are already subdirectories in it. This is a personal matter for everyone. The resulting two files and the script can be copied to the target machine and customized script for their tasks. But the Python required to run the script will have to be installed on each client machine.

The script body itself begins with line 137. To set up copying targets and other related facilities, variables need to be defined.

#################### ##  ## #################### work_dir = 'C:\Windows\!Script\Backup' #   .        vshadow.exe    source_disk = 'c:' destin_disk = 'o:' param_script = 'vs_generated.cmd' #     cwd_list = [['o:\\Windows\\!Script\\Backup', '\\\\master\\apps\\temp\\viv\\backup'],['o:\\ocs-ng', '\\\\master\\apps\\temp\\viv\\ocs-ng']] #   from_addr = 'admin@typa.ru' #    tech_addr = ['user1@typa.ru', 'user1@typa.ru'] #   

In this case, this is the working directory in which the script is located, the target disk with a copy, the name of the disk to which the created shadow copy will be mounted, the name of the file in which the vshadow.exe utility stores the values ​​of its variables. Next, the cwd_list variable contains a list of directories that need to be copied and where to copy. A variable contains a list of two element lists. Please note that for escaping a slash in paths in Python, it is necessary to put it down twice, and to indicate a link to a network path, even four.

And completes the definition of the sending email settings variable. A variable containing the address of the sender of the letter and a variable containing the list of recipients.

Actually, defining variables in this section is all that is required for successful work. Next, we hang the execution of the script in the Task Scheduler, not forgetting to specify the execution with administrator rights. Without admin rights the creation of a shadow copy does not occur. If we copy to a network resource in a domain, it is advisable to have a specially created account in the domain for this. Accordingly, we set the rights to the target directory in the domain for this account under our preferences. Note that the robocopy utility has many options for its launch. In this case, the source directory and the remote directory are synchronized completely, up to ntfs rights, auditing and the owner of the directory. In general, for a network directory, you will still have to tinker with the task of the initial rights ntfs on the shared network resource. If you do not need all this information, you can simply remove the extra keys in the copy invocation.

Well, in the end I will quickly go through the main milestones in the script.

From line 169 perform the creation of a shadow copy.

 #       #          if not(path.exists(getcwd() + '\\vshadow.exe')) and not(path.isfile(getcwd() + '\\vshadow.exe')) : exit(1) proc = Popen('vshadow.exe -nw -p -script=' + param_script + ' ' + source_disk, shell=True, stdout=PIPE) proc.wait() out = proc.stdout.readlines() 

Pay attention to the call parameter -script = '+ param_script, as a result of the work of vshadow.exe, the values ​​of variables are laid out in this file. We are only interested in them with SHADOW_ID_1, which contains the identifier of the created shadow copy. Actually, then we are committed to reading the resulting file and pulling out the value of the identifier for further work. And after we found the identifier, we connect a shadow copy.

 proc = Popen('vshadow.exe -el=' + SHADOW_ID + ',' + destin_disk, shell=True, stdout=PIPE) 

Then everything is routine. Enumerate the copy list. We set on him robocopy. We read logs we form the report. At the end, we send a report to all interested. Delete the shadow copy and the file containing the variables from vshadow.exe

PS By the way, if the definition file remains at the start of this script, vshadow will not be able to rewrite it and the script will try to connect the old shadow copy by its id. In theory, in the future it will not hurt to check the presence of this file also at the beginning of work and delete it if necessary.

PPS Please note that you are copying directories. If you want to copy the entire disk, you need to slightly change the sync procedure. When copying a log is kept and the name of the log file is obtained from the name of the source directory. If you just specify the drive, the name, the file log will form something like 'o: .txt' and the script will fall out with an error. In this case, it is better to add a check that all characters in the resulting file name are valid and delete others.