Making backups: how to stop worrying about your data

When the hosting went into oblivion, it was too late to worry. The site died during active promotion, on Friday, it was impossible to get through to technical support, the brothers in misfortune shrugged their shoulders. We were lucky: one of the developers had an old copy of the site, we restored it and the project was saved. But no one wanted to risk more.



That was five years ago. Then, for the first time, we seriously dealt with the issues of “insurance” of our data, and tried many solutions in working on various projects.



Now that we have got our own solution to the problem, I want to summarize a little the experience we have gained. I hope that our stuffed bumps will help you to reduce suffering a bit and find the best backup service for you.

Custom scripts

')

Pros: low cost, flexibility.

Cons: variable reliability, no guarantees.

To suit: non-standard projects, system administrators.



At first, we decided not to spend money on serious solutions and write scripts to recover data on our own. Well, or almost independently: good, on guthub'e there were a lot of ready-made options. Five sleepless nights, a lot of coffee mugs - and I am now a guru of bash, rsync, and other technologies. When writing your script, it is important to determine where you back up your data and remember to save your databases. This solution has obvious advantages: flexibility - you decide what opportunities you need and how to implement them; learning - in the process of writing you can get acquainted with any new programming language, technology or API. In addition, the cost of the final product is determined only by you.



But the cost of a mistake here is quite high, since it can result in the loss of a part of the project. It is good that we understood this at the stage of information recovery testing. Saving on backups is useless.

Business solutions

Pros: high reliability, technical support.

Cons: high price, closed code and infrastructure.

To suit: medium and large businesses.



Unfortunately, we did not manage to try these solutions in practice - web studios and startups in which I worked could not allocate the necessary budget for the purchase of these solutions.



These are icebreakers in the world of backups. They serve large companies and they have been on the market for many years. If you have a problem, you will definitely come to the aid of technical support in the shortest possible time. I'm talking about Ashampoo, Acronis, Symantec, Handy and others. These solutions bribe their very high reliability, technical support and experience in the market. The only thing that stops small businesses and freelancers from using is the high price for reliability (from $ 1000 per year). There are also questions to the closed code of these systems - it is not clear what is hidden in these black boxes.

Hoster solutions

Pros: low price, quick initial setting.

Cons: low confidence, low flexibility.

To suit: simple sites.



But this path saved me quite recently. One of the very old customers forgot to extend the hosting and his account with all his data was deleted in a month. The story ended with a happy endom: within a day a new account was registered and the site was restored from the backup of the hoster.



Technical implementation is different for hosters, usually it all comes down to one tick when creating a virtual machine. It is not always clear where the hoster stores the backup of your data, because it can be a server in the next rack. It also raises the question of how often it makes backups and how many previous copies it stores. The recovery procedure is incomprehensible, how well and how quickly we will be able to access the necessary information. Carefully clarify this information with the hoster. If you cannot find the answers in the documentation, be sure to ask for support.

Consumer solutions

Pros: low price.

Cons: low reliability, no settings, low speed upload.

To suit: small news portals.



Dropbox, google drive, box.com, crashplan and others are gaining popularity recently, including for use on the server. After testing this option six months ago, I can confidently say that at the moment this raw solution is not suitable for use in live projects. Usually, a short script is added to them (for example, rsync every evening in a folder) and it all seems to work. The main advantage is the low price, since the competition for consumer solutions is big, they have to be dumped. Ideal for backing up media files. But there are a lot of minuses: starting from a low download speed on servers (20 kb / s) and ending with the fact that console clients are very raw, they constantly “fall off”.

Three steps to be calm!

Pros: ease of installation, ease of tincture, reliability.

Cons: lack of flexibility.

To suit: freelancers, indie developers, startups, web studios.



And now, that moment came when, when we next set up a backup for a new project, we said: “Enough to endure it!” And decided to implement a full-fledged service that will help us find peace of mind in three steps. To install on a client server, you need only one team, all backup management (selection of directories and databases, scheduling and tracking) occurs via a convenient web interface. There is also a minus: in some situations, the flexibility of setting up a backup may not be enough. We think that we will be useful to freelancers, indie developers, startups and small businesses (for example, web studios) who want to be sure about their data.

Customer

We develop the client in Python - the language is supported “out of the box” in all server operating systems. The source code will be published in PyPI and on github.com so that every interested developer can check us out. The client will receive the settings from our service, compress the necessary directories and immediately upload them to Amazon S3. All data exchange takes place via https.

Web interface

We develop the web interface using modern frameworks AngularJS and Django. Using one control panel, you can configure multiple servers at once. For each server, you select the necessary directories for backup, specify the connection settings to the database, select the database. Schedule backups configured flexibly. The control panel displays a log of backups with status, size and execution time.

Amazon S3

One of the most popular data storage services. For each user, we create a private and isolated bucket, which makes the probability of “leakage” of backups impossible. All backups are stored on Amazon S3 in encrypted form.

Feedback

Now the project is in the process of active development of the first version of the product. We are very interested in your wishes, advice, impressions, any feedback.