New cloud storage feature: creating links to download files

So, a new feature has appeared in our cloud storage: creating download links. Having created such a link, you can send it to any third-party user so that he uploads his files to you - absolutely free. It is not even necessary for this user to have an account in the repository.

This function can be useful for solving a variety of tasks, for example:

• to provide any users with access to your repository: for example, friends will be able to upload their photos, audio and video records for you, colleagues and partners - various working materials, etc.
• to upload custom content to sites;
• to facilitate file sharing between members of the thematic online communities;
• for quick backup backup.

Prehistory

Recently, I participated in climbing Elbrus. All members of the group, including me, had cameras with them. Many interesting and beautiful photographs were made, and after the hike the question arose: how to make them available to all participants?
')
Someone offered to upload all pictures on VKontakte or on Facebook, someone on Mail.Ru, someone on other file sharing sites ... But all the listed services did not suit us: first, there are restrictions on the size of the downloaded files; secondly, not all members of the group were registered on these services.

I proposed a simple and easy-to-use solution: I created a public container in the Selectel cloud storage, turned on the file listing option. Then he created a link by clicking on which everyone could upload their own photos into the container, as well as download photos taken by other participants. At the same time, none of the group members needed to create an account in the vault. So a new feature of our cloud storage was tested - creating a link to download files by third-party users. My comrades in climbing Elbrus rated it as “excellent”.

Today we present this feature to a wide audience. We hope that all our users will appreciate it.

Link creation

Let's look at the simplest example of how the new function works. Create a special container for downloading files and name it, for example, share. Then open the container menu and select the “File Download Link” option:



The link settings window will open:


In this dialog box, you can set the following parameters (more on some of them will be discussed below):

• select link type;
• enter explanatory text for users to whom you send the download link (the explanatory text will be displayed in the subtitle at the top of the download page);
• set a password for downloading files;
• set limits on the maximum size of files uploaded to the repository;
• limit the duration of the link, as well as the shelf life of downloaded files;
• enable / disable overwriting a file when loading files with the same name into the storage (if this option is enabled, the old file will be deleted and the new one will be saved; if disabled, the old file will not be deleted, and an additional identifier will be added to the new file name).

Now click on the “Create link” button. A dialog box appears with a link that can be copied and sent to any other user. For each container or folder, you can create an unlimited number of download links with various combinations of settings.

Download page

The user who received the link from you and opened it in the browser will see the following page:



The page supports simultaneous parallel loading of three files, and due to this, the loading process is significantly accelerated:



If any files need to be deleted, this can be done within 10 minutes after the download is complete.

Link Settings

Link types

As mentioned above, when creating a link, you need to select its type. We have provided several types of links, adapted for various uses. Consider each type of link in more detail.

Easy download

Suitable for situations where you need to get files from one or more users. Files will be uploaded to the specified container or folder, and you will be able to access them through the web panel or in other ways.

Add load time to file name

If you select this option, the date and time of loading will be added to the name of the file being downloaded. For example, if you upload the file myphoto.jpg, it will be saved as myphoto_20130830_12h43m53s.jpg.
This can be useful when saving backups, as well as in cases where access to a download link is provided to a large number of users, and you need to exclude matches in the names of the downloaded files.

Add a unique identifier in the file name

If you select this option, a unique identifier will be added to the file name. For example, the file myphoto.jpg after downloading will be renamed to myphoto_cmrsm2ks.jpg.
This type of link is suitable for implementing uploading custom content to sites (in this case, it will also be useful to include the option to ignore the original file name): the user’s given name will be rejected, and the new file name will correspond to its unique identifier.

Upload to a unique folder

If you select this type of link, the files will be saved in a separate folder during each download session. For example, the file myphoto.jpg will be saved to a folder named cmrsm2ks. If you, after downloading this file, close the download page, and then close it again and load other files, they will be saved in another folder, the name of which will be generated automatically; a separate folder will also be created for the next download session, etc.

Upload to folder by date

This mode of operation is well suited for loading backups. The downloaded files will be saved to the folder with the current date in the title. For example, the archive mysql.dump.bz2 after downloading will be available as 2013-08-30 / mysql.dump.bz2.

Download to the folder by the hour

This mode is generally similar to the previous one; the only difference is that in addition to the folder name, the time (hour) of loading will be added to the folder name. The mysql.dump.bz2 archive will be available as 2013-08-30_15 / mysql.dump.bz2.

Download with the specified name

This type of link is more suitable for use through the API for applications or sites.
The link will be valid for one download only, and the downloaded file will be renamed after the link itself. For example, if the link looked like xxx.selcdn.ru/container/you_photo.jpg , then after downloading the file will be available in the repository under the name you_photo.jpg, and the link will no longer be valid.

Temporary restrictions

When creating a download link, you can set time limits:


This is a very flexible and useful option. It allows, first, to limit the duration of the link itself. After the expiration date you specify, third-party users will not be able to upload files to the repository. Such an opportunity may be useful in situations where it is necessary to accept documents from users via the Internet by a certain date (for example, when accepting applications for conferences and competitions).

Secondly, it can be used to set the shelf life of downloaded files. You can specify a specific period of time during which the files from the user who received the download link from you will be stored. At the end of this period, all files will be automatically deleted. Such functionality will be useful, for example, for organizing the loading and storage of backup copies.

Additional features

For developers

Link management can be done through an API and is compatible with the Openstack Swift API. Using libraries to work with the Openstack Swift API, you can easily create and customize links to upload files to our repository directly from your own applications and sites.

With the help of download links you can realize the ability to download content from a third-party site to our repository.
Here is a simple example:

File upload form:

<form id="put-file"> <input id="fileToUpload" type="file" /> <input type="submit" /> </form> 

Javascript code:

 (function() { var $putFile = document.getElementById('put-file'); $putFile.onsubmit = function (event) { var req = new XMLHttsourcequest(); event.sourceventDefault() file = document.getElementById('fileToUpload').files[0] req.open("PUT", "https://xxx.selcdn.ru/containet/sendmefile_link"); req.withCredentials = false; /*    */ req.setRequestHeader("X-Filename", file.name); /*          */ TZ_OFFSET = (new Date).getTimezoneOffset(); req.setRequestHeader("X-Sendmefile-TZ-Offset", TZ_OFFSET); req.send(file); req.onreadystatechange = function () { if(req.readyState === 4 && req.status === 200) alert(req.responseText) } return false; } })() 

For system administrators

The function of creating links to downloads can be useful for quickly downloading files from the server to the storage, as well as for setting up a backup.

To upload a file, you must first create a link in the web panel, and then execute the following command in the server management console using the curl utility:

 $ curl -i https://xxx.selcdn.ru/container/upload_link -T path_to/file.name -H "X-Filename: file.name" 

You can also organize the transfer of data through a named pipe (pipe), without saving to the file:

 #      “ ” $ bzip2 bigfile.dat -c | curl -i http://xxx.selcdn.ru/container/upload_link -X PUT -d @- -H "X-Filename: bigfile.dat.bz2" 

Another interesting feature is the multi-threaded download of files from a specific folder; this will be much faster than using our supload utility :

 #    -P $ find path_to_dir/ -type f | xargs -I "{}" -n1 -P 10 curl http://xxx.selcdn.ru/container/upload_link -i -X PUT -H "X-Filename: {}" -T "{}" 

Backup

The new feature greatly simplifies the backup procedure: now it is not necessary to create separate containers for each server or site; no need to create additional users, the number of which is limited. To ensure the download and storage of backups, you just need to create a password-protected link and configure it accordingly.

Creating links to download backups allows you to solve a number of problems that previously required considerable effort.

First, it is sorting the downloaded backups by date and time. In order for all uploaded files to be automatically sorted or renamed according to date and time, simply create a link of the appropriate type.

Secondly, it is automatic removal of backups upon expiration. Shelf life can be set in two ways: set a strictly specified time and default time. The second option allows you to change the retention period during loading.

Backup via download links can be easily integrated into automatic backup systems, which very often support the execution of an external command after creating an archive with a copy of the data.

Afterword

As is often the case with us, the new feature became available to the user several days before it was officially announced. Some of our clients have already tried to “tie” it to their own services, and we have already received feedback from them. In their comments, they made a number of comments and suggestions to us, which will necessarily be taken into account as soon as possible. Very interesting wishes for the future were also expressed, in particular:

• to realize the possibility of generating links for downloading downloaded files in the repository;
• create hashed passwords, which will not create a new password-protected link every time you download custom content from sites.

We invite all of our readers to try the new feature. Link to the test version of the download page: http://23728.selcdn.ru/sendmefile-test/upload (all files will be stored for an hour, the link itself will be available 30 days from the date of publication of the article).

We will welcome any comments, suggestions, suggestions and will definitely take them into account in our future work.
For those who can not comment on posts on Habré, we invite to our blog .