VPS Search - Autumn Update
Autumn came, so I had a desire to report on the work done over the summer on the project poiskvps.ru
I think that almost every webmaster or server owner faced a problem when a backup copy of the data was needed, but for some reason it did not appear. On ordinary shared hosting, it is accepted that the hosts themselves make backup copies and in case of anything they are ready to provide them to users. However, when renting a virtual or dedicated server, the situation is exactly the opposite: the owner must independently monitor the safety of data and perform backups on their own. Sometimes hosters offer “boxed” solutions, when together in the server they give the user the opportunity to rent a place to store backups. In my opinion, this is not the best idea, since backups need to be stored at least in another data center, and preferably in several. For example, you can remember the fire in the hosting.ua data center - no one is immune from such a confluence of events, but such a case shows that storing backup copies in the same data center is not the best idea. Also, in my opinion, keeping backups with the hoster is also not the best idea, even if backups are stored far from the main server. Recently, there are not rare cases when a hoster can simply disappear or all equipment is taken from him.
')
The service of renting a space for backup works less than six months, but recently there have been significant changes. About them and about the service today I want to tell you in detail.
Initially, the idea was to make a simple and convenient server for storing backups, at the same time I wanted the price per gigabyte to be not exorbitant. For this, a fairly well-known data center was chosen - hetzner.de - and until recently, it was only possible to buy a place in this data center in Germany at a very “tasty” price: 10 Gb costs 30 rubles, 50 Gb - 100 rubles and more in proportion.
However, a few days ago, it became possible to rent a storage besides Germany on servers in Russia, in St. Petersburg, DC Selectel. This may be relevant for those users who want to make backups at a very good speed. The traffic is unlimited, the connection speed is 100 Mbit / s, there are no artificial speed limits. The price is similar to accommodation in Germany. Plus, until the end of September, a permanent 10% discount is offered for any amount.
For the user, the repository is a login-password pair with which you can manipulate files using the FTP protocol. The service does not offer any other means of information exchange, since the most common control panels support FTP upload, and the service, first of all, is focused on simplicity.
On the technical side, each server is equipped with the Debian OS and the ISPmanager control panel. Using the API of this control panel, all actions are performed with a user account: activation, changing the password, getting information about the occupied space and getting a list of files, setting restrictions on IP addresses.
Service activation can be divided into two logical parts:
1. Generate login and password and create a user with the necessary restrictions on the server.
A small explanation of why the login and password are generated, instead of the usual proposals to come up with them yourself.
As the login, the result of the phi-function uniqid () is used, respectively, at any given time this value is unique, it is valid for creating a new user in the system, and it is difficult to pick it up. Also, as the server address, a login.backup.poiskvps.ru type construction is always used, respectively, the login must also be a valid subdomain. In order not to “load” the user with unnecessary information, it was decided to use just such an approach, otherwise the user would have to select the correct login for quite a long time to satisfy all the requirements. All the same, this data is used only when setting up and forget about them.
The password is generated by the service and is not requested from the user due to the fact that the activation of the service may be delayed: that is, the user ordered the service, but paid for it in a day. And in order not to keep the password in clear form for the subsequent activation of the repository, the password is automatically generated immediately after the payment and sent to the mail. Naturally, nothing prevents the user from immediately changing the password. Using AJAX on the site makes it quite simple.
2. Creating the required subdomain to activate the server address of the form login.backup.poiskvps.ru. If you need to move a user account from one server to another (for example, to more intelligently manage the occupied space), the best way is to give each user its own subdomain. If for some reason the user account moves to another server, then it will be enough to change one entry in the DNS, and the user will not notice anything. The update time of the DNS records is set to be rather short, so by the time of receiving the letter with the DNS login and password, the records already have time to update, and the user has successfully logged into the server.
Immediately after activating the service and receiving the generated password, any paranoiac will want to change this password. Everything is pretty well thought out: you can change the password directly from the control panel, and the password change is deprived of annoying functions, for example, you do not need to type the password twice, and only one limit is set to “quality”: the password must be 8 characters or more. That is, no notifications “The password should contain a large vowel letter in any even position” the user will never see.
Well, as a bonus, the password change takes place without reloading the page, so a beautiful picture will indicate for about a second that the data is being exchanged, and as soon as a response is received from the destination server that the password has been changed, the user will be notified immediately.
Most often, a backup is set up once and forgot about it until it is time to use a backup. You will agree that it will be very sad to know that there are no backup copies in the last three months, as it turned out that all the available space is occupied. Of course, it is not in our power to envisage, let alone prevent, all cases of “corruption” of backups, but the issue with occupied space can be brought to perfection.
The principle of operation is as follows: the user indicates the percentage of occupied space, after filling which he needs to send a notification that there is not enough space left.
The service has one of the “background” processes, which receives information from all servers once a minute and updates the statistics on free and occupied space. Using this function, you can just see how much was occupied before the update and how much it turned out to be occupied after, and if the threshold specified by the user is overcome, you need to send a notification to the user.
In theory, this is a good plan: a clear notification without delay, and the user immediately finds out that there is little space left. However, the realities add a bit of negativity: most often, the control panels back up as follows: first, they upload a “fresh” version of the backup, and only after that they delete the oldest one. Because of this, at the time of uploading to the storage, free space may run out, and the user will receive a notification each time, although after the file has been downloaded, the free space will reappear. To avoid such cases, the following mechanism is used: as soon as the threshold specified by the user is crossed, a flag is put in the database, and after an hour the occupied space is re-checked: if there is still little space, the user receives a notification, and if the place is free, we are in vain did not bother the user and everyone is happy.
In order to increase security on the service, it is possible to set a restriction on IP addresses. If the server supports the IPv6 protocol, then it is possible to set a restriction on addresses of this type.
Initially, the restriction was planned to be done through a common server firewall, but this would have caused a lot of inconvenience, for example, mandatory access restriction for all users. However, at the moment the restriction is valid for a separate account and does not affect the “neighbors”, so if the user does not want (for example, he does not have a permanent IP address), then the restrictions can be set. Just like the password, the update occurs via AJAX without reloading the page.
Another almost unique feature of the service is viewing the list of files and folders directly from the storage control panel. In order not to connect to the server each time via an FTP client, users have the opportunity to view a list of files and folders in the form of a tree right through the control panel. The principle of operation is as follows: once every 15 minutes all servers are polled, and for each account a map of files and folders is created, in which the name, date of change and size are indicated. Since different users may have a different number of folders, it may take several minutes to crawl all the vaults and compile their maps. Therefore, the information is updated 4 times per hour. However, there are plans for the future to make a button for a “manual” update.
I will be glad to answer questions and listen to comments. On request in lichku or at leonid@poiskvps.ru I can provide test services for checking the data transfer rate.
PS I thank the hosting company VDSina.ru for supporting the project.
I think that almost every webmaster or server owner faced a problem when a backup copy of the data was needed, but for some reason it did not appear. On ordinary shared hosting, it is accepted that the hosts themselves make backup copies and in case of anything they are ready to provide them to users. However, when renting a virtual or dedicated server, the situation is exactly the opposite: the owner must independently monitor the safety of data and perform backups on their own. Sometimes hosters offer “boxed” solutions, when together in the server they give the user the opportunity to rent a place to store backups. In my opinion, this is not the best idea, since backups need to be stored at least in another data center, and preferably in several. For example, you can remember the fire in the hosting.ua data center - no one is immune from such a confluence of events, but such a case shows that storing backup copies in the same data center is not the best idea. Also, in my opinion, keeping backups with the hoster is also not the best idea, even if backups are stored far from the main server. Recently, there are not rare cases when a hoster can simply disappear or all equipment is taken from him.
')
Backup Rental Service
- Service activation
- Change Password
- Notifications about exceeding the occupied space threshold
- Access Restriction by IP Addresses
- View a list of files and folders
The service of renting a space for backup works less than six months, but recently there have been significant changes. About them and about the service today I want to tell you in detail.
Initially, the idea was to make a simple and convenient server for storing backups, at the same time I wanted the price per gigabyte to be not exorbitant. For this, a fairly well-known data center was chosen - hetzner.de - and until recently, it was only possible to buy a place in this data center in Germany at a very “tasty” price: 10 Gb costs 30 rubles, 50 Gb - 100 rubles and more in proportion.
However, a few days ago, it became possible to rent a storage besides Germany on servers in Russia, in St. Petersburg, DC Selectel. This may be relevant for those users who want to make backups at a very good speed. The traffic is unlimited, the connection speed is 100 Mbit / s, there are no artificial speed limits. The price is similar to accommodation in Germany. Plus, until the end of September, a permanent 10% discount is offered for any amount.
For the user, the repository is a login-password pair with which you can manipulate files using the FTP protocol. The service does not offer any other means of information exchange, since the most common control panels support FTP upload, and the service, first of all, is focused on simplicity.
On the technical side, each server is equipped with the Debian OS and the ISPmanager control panel. Using the API of this control panel, all actions are performed with a user account: activation, changing the password, getting information about the occupied space and getting a list of files, setting restrictions on IP addresses.
Service activation
Service activation can be divided into two logical parts:
1. Generate login and password and create a user with the necessary restrictions on the server.
A small explanation of why the login and password are generated, instead of the usual proposals to come up with them yourself.
As the login, the result of the phi-function uniqid () is used, respectively, at any given time this value is unique, it is valid for creating a new user in the system, and it is difficult to pick it up. Also, as the server address, a login.backup.poiskvps.ru type construction is always used, respectively, the login must also be a valid subdomain. In order not to “load” the user with unnecessary information, it was decided to use just such an approach, otherwise the user would have to select the correct login for quite a long time to satisfy all the requirements. All the same, this data is used only when setting up and forget about them.
The password is generated by the service and is not requested from the user due to the fact that the activation of the service may be delayed: that is, the user ordered the service, but paid for it in a day. And in order not to keep the password in clear form for the subsequent activation of the repository, the password is automatically generated immediately after the payment and sent to the mail. Naturally, nothing prevents the user from immediately changing the password. Using AJAX on the site makes it quite simple.
2. Creating the required subdomain to activate the server address of the form login.backup.poiskvps.ru. If you need to move a user account from one server to another (for example, to more intelligently manage the occupied space), the best way is to give each user its own subdomain. If for some reason the user account moves to another server, then it will be enough to change one entry in the DNS, and the user will not notice anything. The update time of the DNS records is set to be rather short, so by the time of receiving the letter with the DNS login and password, the records already have time to update, and the user has successfully logged into the server.
Change Password
Immediately after activating the service and receiving the generated password, any paranoiac will want to change this password. Everything is pretty well thought out: you can change the password directly from the control panel, and the password change is deprived of annoying functions, for example, you do not need to type the password twice, and only one limit is set to “quality”: the password must be 8 characters or more. That is, no notifications “The password should contain a large vowel letter in any even position” the user will never see.
Well, as a bonus, the password change takes place without reloading the page, so a beautiful picture will indicate for about a second that the data is being exchanged, and as soon as a response is received from the destination server that the password has been changed, the user will be notified immediately.
Notifications about exceeding the occupied space threshold
Most often, a backup is set up once and forgot about it until it is time to use a backup. You will agree that it will be very sad to know that there are no backup copies in the last three months, as it turned out that all the available space is occupied. Of course, it is not in our power to envisage, let alone prevent, all cases of “corruption” of backups, but the issue with occupied space can be brought to perfection.
The principle of operation is as follows: the user indicates the percentage of occupied space, after filling which he needs to send a notification that there is not enough space left.
The service has one of the “background” processes, which receives information from all servers once a minute and updates the statistics on free and occupied space. Using this function, you can just see how much was occupied before the update and how much it turned out to be occupied after, and if the threshold specified by the user is overcome, you need to send a notification to the user.
In theory, this is a good plan: a clear notification without delay, and the user immediately finds out that there is little space left. However, the realities add a bit of negativity: most often, the control panels back up as follows: first, they upload a “fresh” version of the backup, and only after that they delete the oldest one. Because of this, at the time of uploading to the storage, free space may run out, and the user will receive a notification each time, although after the file has been downloaded, the free space will reappear. To avoid such cases, the following mechanism is used: as soon as the threshold specified by the user is crossed, a flag is put in the database, and after an hour the occupied space is re-checked: if there is still little space, the user receives a notification, and if the place is free, we are in vain did not bother the user and everyone is happy.
Access Restriction by IP Addresses
In order to increase security on the service, it is possible to set a restriction on IP addresses. If the server supports the IPv6 protocol, then it is possible to set a restriction on addresses of this type.
Initially, the restriction was planned to be done through a common server firewall, but this would have caused a lot of inconvenience, for example, mandatory access restriction for all users. However, at the moment the restriction is valid for a separate account and does not affect the “neighbors”, so if the user does not want (for example, he does not have a permanent IP address), then the restrictions can be set. Just like the password, the update occurs via AJAX without reloading the page.
View a list of files and folders
Another almost unique feature of the service is viewing the list of files and folders directly from the storage control panel. In order not to connect to the server each time via an FTP client, users have the opportunity to view a list of files and folders in the form of a tree right through the control panel. The principle of operation is as follows: once every 15 minutes all servers are polled, and for each account a map of files and folders is created, in which the name, date of change and size are indicated. Since different users may have a different number of folders, it may take several minutes to crawl all the vaults and compile their maps. Therefore, the information is updated 4 times per hour. However, there are plans for the future to make a button for a “manual” update.
I will be glad to answer questions and listen to comments. On request in lichku or at leonid@poiskvps.ru I can provide test services for checking the data transfer rate.
PS I thank the hosting company VDSina.ru for supporting the project.
Source: https://habr.com/ru/post/235431/
All Articles