Simple homemade data backup 2 (Python + DropBox)

New version of the script for backup ( original ). Or another version ...



The logic of working with file lists has not changed. Only the script parameter has changed - now this is the name of the archive without an extension, the script itself will add .tar.gz .



New:

• Archive creation moved from external archiver inside the script. A tar.gz archive is created.
• All paths are now system-dependent, i.e. the script should work on Linux too

In addition to the pleasant new, an unpleasant minus poked in - the script began to eat a lot of memory.

')

Make a compressed archive using the tarfile module failed. The archive is created and compressed, but for some reason the paths of the archive are moving down.



Those. you need to create an archive d:\dropbox\my dropbox\backup\backup.tar.gz , it appears there. But if you open this gz archive, the tar archive will be deep inside the dropbox\my dropbox\backup\backup.tar . Those. the full path to the data is: d:\dropbox\my dropbox\backup\backup.tar.gz\dropbox\my dropbox\backup\backup.tar\... In principle, you can live with this, but ugly ...



Therefore, I did it differently - first a tar archive is created without compression with all the files, then it is compressed. It is in this place of the function that compresses the archive, it, the archive, is transferred entirely, i.e. the entire file weighing several hundred MB is sucked into the memory.



It would be necessary to dig in this direction and find the best solution ...

Copy Source | Copy HTML

1. import sys
2. import os
3. import os .path
4. import tarfile
5. import time
6. import gzip
8. if __name__ == "__main__" :
9. if ( len ( sys .argv) < 2 ):
10. print ( "Usage: backup.py <output archive filename>" )
11. print ( "Eg: \" backup.py d: \\ dropbox \\ my dropbox \\ backup \ "" )
12. exit ( 1 )
14. RootFoldersFile = "tobackup.lst"
15. IgnoreFoldersFile = "ignore.lst"
16. ExtraFile = "extra.lst"
17. OutputArchive = sys .argv [ 1 ]
18. OutputArchiveTar = OutputArchive + ".tar"
19. OutputArchiveTarGz = OutputArchive + ".tar.gz"
21. # Only mandatory file is RootFoldersFile, check that it exists
22. if ( not os .path.isfile (RootFoldersFile)):
23. print ( "% s doesn't exist" % RootFoldersFile)
24. exit ( 1 )
26. # Read root folders from file
27. rootFolders = [i.strip () for i in open (RootFoldersFile, "r" ). readlines ()]
29. # Read list of folders that need to be igonred (if specified)
30. ignoreList = []
31. if ( os .path.isfile (IgnoreFoldersFile)):
32. ignoreList = [i.strip (). lower () for i in open (IgnoreFoldersFile, "r" ). readlines ()]
34. # Delete old archive if exists
35. if ( os .path.isfile (OutputArchiveTar)):
36. os .unlink (OutputArchiveTar)
38. if ( os .path.isfile (OutputArchiveTarGz)):
39. os .unlink (OutputArchiveTarGz)
41. filesCount = 0
43. # Open output archive for writing (gzip it later)
44. try :
45. tar = tarfile . open (OutputArchiveTar, "w" )
46. except IOError as err:
47. print (err)
48. exit ( 1 )
50. lastTime = time . time ()
51. for rootFolder in rootFolders:
52. for root, dirs, files in os .walk (rootFolder):
53. for file in files:
54. tar.add ( os .path.join (root, file))
55. filesCount + = 1
57. # Show number of processed files each second
58. if ( time . time () - lastTime> = 1 ):
59. print ( "% d files ..." % (filesCount))
60. lastTime = time . time ()
62. # Skip ignored folders
63. for dir in dirs:
64. if ( dir .lower () in ignoreList or ( os .path.join (root, dir) .lower () in ignoreList)):
65. dirs.remove (dir)
67. # Add some files from extra files list
68. if ( os .path.isfile (ExtraFile)):
69. for file in [i.strip () for i in open (ExtraFile, "r" ) .readlines ()]:
70. tar.add (file)
72. tar.close ()
74. print ( "Compressing ..." )
76. # Gzip output archive
77. try :
78. gz = gzip open (OutputArchiveTarGz, "wb" )
79. except IOError as err:
80. print (err)
81. exit ( 1 )
83. gz.write ( open (OutputArchiveTar, "rb" ) .read ())
84. gz.close ()
86. os .unlink (OutputArchiveTar)
88. print ( "Output:% s,% d file (s),% d bytes" % (OutputArchiveTarGz, filesCount, os .path.getsize (OutputArchiveTarGz)))
89. print ( "Done" )