Cleaning Dropbox backup storage by cron
I had a chance to configure Akeeba Backup Pro for remote backup storage in Dropbox. And in the course of the process, it turned out that Akeeba can only litter the very same Dropbox, but it will have to be manually cleaned up after it. But manually - not comme il faut, and archives on gigabyte with small. Therefore, you need to somehow get rid of obsolete hands.
So, given - full backups upload to the " full " folder every three hours. Mysql database - in the folder " mysql " every half hour. So the owner of the site wants, he paid for this Dropbox Pro business.
It is necessary - to delete all the old full archives, leaving one per day (and so that was!), And all the backups of Mysql, except for today.
I'll warn you right away - I have been using CentOS recently, less than a year. It is installed on the virtual machine as a test web server (LAMP), I am on the console once a month, and wrote “batch file” as good old MS-DOS 6.22, because everything described below was born in just a couple of hours from scratch and does not shine grace.
')
First of all, I was puzzled by the question - what to get to Dropbox from the console? After a brief search on the Internet, a wonderful bash script, Dropbox_Uploader , was discovered , which, as it turned out, can not only upload, but also other vital commands, including list and delete . That's just bad luck - I didn’t understand right away if he was trained to delete “according to the list”, so he designed it as a trite cycle (besides, it’s easier to debug by the piece).
Installing dropbox_uploader is simple - download it with the command
we give everyone rights to read and execute
then run
configure access to Dropbox by following the prompts on the screen (create a new application on the Dropbox website and feed the APP key and APP secret to the script) and transfer the script to / usr / bin / so that it is accessible to everyone, and most importantly - the crown.
Dropbox_uploder at the command list full (recall, " full " for me - the name of the folder) gives something like
We create our own clean_dropbox.sh script immediately in / usr / bin (why not?), With the same rights 755, and in it:
1. Create a list of unnecessary archives
in which the resulting list of * .jpa files from the full folder is filtered cut and several grep and saved to a file.
Cut leaves only file names ( -d '' - separator space, -f4 - 4th field). Yes, the fourth, because the line starts with a space.
The first grep discards today's archives, the second excludes those created within an hour after midnight, and the third removes the line with the name of the folder "/ full" ... from the top of the list.
2. The resulting file is read into the variable (array of strings)
3. Cycle through the array and delete each file.
Given that the script will work in the crown, just in case, we write the output to the log file.
At this, the processing of complete archives can be considered complete, the mysql archives remain.
It's all on the groove. A list of files with simplified grep - we don’t need old backups at all, and we can wait a stack for the day, since their sizes are quite acceptable.
Similarly, delete unnecessary in the loop
and finally we removethe list of files
That's all, now it’s enough to create a crown
Voila!
I understand that for the guru of Linux my post is almost useless, but Habr is read by such ignoramuses like me. I hope someone will come in handy.
So, given - full backups upload to the " full " folder every three hours. Mysql database - in the folder " mysql " every half hour. So the owner of the site wants, he paid for this Dropbox Pro business.
It is necessary - to delete all the old full archives, leaving one per day (and so that was!), And all the backups of Mysql, except for today.
I'll warn you right away - I have been using CentOS recently, less than a year. It is installed on the virtual machine as a test web server (LAMP), I am on the console once a month, and wrote “batch file” as good old MS-DOS 6.22, because everything described below was born in just a couple of hours from scratch and does not shine grace.
')
First of all, I was puzzled by the question - what to get to Dropbox from the console? After a brief search on the Internet, a wonderful bash script, Dropbox_Uploader , was discovered , which, as it turned out, can not only upload, but also other vital commands, including list and delete . That's just bad luck - I didn’t understand right away if he was trained to delete “according to the list”, so he designed it as a trite cycle (besides, it’s easier to debug by the piece).
Installing dropbox_uploader is simple - download it with the command
curl "https://raw.githubusercontent.com/andreafabrizi/Dropbox-Uploader/master/dropbox_uploader.sh" -o dropbox_uploader.sh we give everyone rights to read and execute
chmod +rx dropbox_uploader.sh then run
./dropbox_uploader.sh configure access to Dropbox by following the prompts on the screen (create a new application on the Dropbox website and feed the APP key and APP secret to the script) and transfer the script to / usr / bin / so that it is accessible to everyone, and most importantly - the crown.
Dropbox_uploder at the command list full (recall, " full " for me - the name of the folder) gives something like
> Listing "/full"... DONE [F] 1218610223 site-sitename.com-20150908-000001.jpa [F] 1218610223 site-sitename.com-20150908-030001.jpa [F] 1218610223 site-sitename.com-20150908-060001.jpa [F] 1218610223 site-sitename.com-20150908-090001.jpa [F] 1218610223 site-sitename.com-20150908-120001.jpa [F] 1218610223 site-sitename.com-20150908-150001.jpa [F] 1218610223 site-sitename.com-20150908-180001.jpa [F] 1218610223 site-sitename.com-20150908-210001.jpa ... .. ... [F] 1218610223 site-sitename.com-20150915-150001.jpa [F] 1218610223 site-sitename.com-20150915-180001.jpa [F] 1218610223 site-sitename.com-20150915-210001.jpa We create our own clean_dropbox.sh script immediately in / usr / bin (why not?), With the same rights 755, and in it:
1. Create a list of unnecessary archives
dropbox_uploader.sh list full *.jpa | cut -d ' ' -f4 | grep -v $(date +%Y%m%d) | grep -v -e '-00' | grep -v 'full' > todel.txt in which the resulting list of * .jpa files from the full folder is filtered cut and several grep and saved to a file.
Cut leaves only file names ( -d '' - separator space, -f4 - 4th field). Yes, the fourth, because the line starts with a space.
The first grep discards today's archives, the second excludes those created within an hour after midnight, and the third removes the line with the name of the folder "/ full" ... from the top of the list.
2. The resulting file is read into the variable (array of strings)
files=$(<todel.txt) 3. Cycle through the array and delete each file.
for file in $files do dropbox_uploader.sh delete /full/"$file" >> /var/log/cleanup.log done Given that the script will work in the crown, just in case, we write the output to the log file.
At this, the processing of complete archives can be considered complete, the mysql archives remain.
It's all on the groove. A list of files with simplified grep - we don’t need old backups at all, and we can wait a stack for the day, since their sizes are quite acceptable.
dropbox_uploader.sh list mysql *.sql | cut -d ' ' -f4 | grep -v $(date +%Y%m%d) | grep -v 'mysql' > todel.txt Similarly, delete unnecessary in the loop
dropbox_uploader.sh delete /mysql/"$file" >> /var/log/cleanup.log and finally we remove
rm todel.txt That's all, now it’s enough to create a crown
50 23 * * * clean_dropbox.sh
Voila!
I understand that for the guru of Linux my post is almost useless, but Habr is read by such ignoramuses like me. I hope someone will come in handy.
Source: https://habr.com/ru/post/266993/
All Articles