Backup on amazon S3 for beginners

I have long wanted to organize the duplication of key personal files on S3, and finally I got together. We couldn’t find a ready-made solution with an intelligent article in Russian, so I had to remember English, since it turned out that everything was not easy, but very simple.

This short material focuses on “very-very small” Linux users and serves to show how easy and simple it is to back up to Amazon S3 servers.


')
It is understood that you already have a working account on S3 (if not, you are here or here ) and minimal knowledge of the console and any text editor.

Let me briefly say that Amazon S3 is an unlimited storage for your data on the Internet. The service is paid, but with small volumes (on the order of several hundred megabytes) it costs just a few rubles (maximum - tens of rubles) per month.

We also believe that you make fresh backups (and old ones, respectively, are deleted), and they are stored in / home / backup.

Our task: every night (day, evening), after all new backups have been made, copy them to S3 and also remove old ones from there - those that are no longer in / home / backup (we clean old backups, right?).

After a brief search on the Internet, I came across a great Ruby s3sync program that was able to do everything that was required.

First install Ruby:

Fedora: yum install ruby

Debian: sudo apt-get install ruby-full
Centos: yum install -y ruby
(Thanks mikhailov )

Next, download and unpack s3sync:

wget s3.amazonaws.com/ServEdge_pub/s3sync/s3sync.tar.gz
tar vxzf s3sync.tar.gz

Create a bash script, which we will run on cron:

vim go-s3sync-backup.sh

#/bin/sh
export AWS_ACCESS_KEY_ID=[ AWS_ACCESS_KEY_ID]
export AWS_SECRET_ACCESS_KEY=[ AWS_SECRET_ACCESS_KEY]
/home/user/s3sync/s3sync.rb --delete -r /home/backup/ [ S3]:my-backup

chmod +x go-s3sync-backup.sh

The result of the script will be the folder my-backup in the specified basket on S3 with your backups.

The first two parameters we specify are the data for authorization on S3 - you can also specify them in the XML file, but it is simpler to put them into environment variables.

• --delete deletes files that are not in the local folder from S3
• -r turns on recursive folder traversal.

Here are some more useful keys:

• --progress allows you to see the progress of the operation
• -n allows you to see what the script is planning to do without performing the actions themselves (some kind of debugging mode)

We add our script to cron (for example, let it work every day at 7 am)

0 7 * * * /home/user/go-s3sync-backup.sh

To clear your conscience, you can test the script with handles and once again check the access rights to the used basket (folder) on S3 - and you can sleep with peace of mind - your data is in a safe place.

Thanks to all who read, any comments and corrections are welcome!