Uploadcare - file storage for sites and applications

Hello! I want to talk about the project, which certainly will be useful to many developers. In a nutshell, explaining why it is needed is quite difficult, but I will try. Uploadcare is a service for applications and websites that makes it easy to get files from users, store them and transfer them over the network.

Anyone who has ever made a form with <input type="file"> knows that there is nothing difficult in this, but there are unpleasant moments arising along the way. Here are just some of them:

- you can not save a form with a file by ajax;
- it is impossible to show the form with the file already selected;
- if you expect a picture, you need to make sure that the picture is loaded;
- the server must be ready to receive a large request body;
- in some frameworks the downloaded file is a source of increased danger;
- convenient loading of several files is rather difficult to implement;
- indication of the boot process is even more difficult;
- on the server disk may run out of space;

Uploadcare is designed to solve these problems. The principle of operation is very simple: you embed a widget in the form, through which users select files. The widget gives you the ID of the file selected by the user. In the future, you work only with this identifier: store it in the database, use it to build links to files.
')


This removes many of the problems listed above. The identifier can be saved by ajax. You can again submit it to the form, and the widget will recognize and show the file to the user. When the file is selected, the widget can tell the user that only pictures can be downloaded. Using the widget, the user can upload multiple files. On Uploadcare, the space does not end unexpectedly, and its increase is not associated with such problems as disk partitioning and server shutdown.

Files uploaded by users are not immediately linked to the account. They are visible through api, but there is no other way to address them. Only after the server confirms with the help of the secret key and the special api method that the file is really needed, it will become public. This usually occurs after the "validation" of the form with the rest of the user data. Accordingly, you can not worry that some user or even an attacker can download what you do not need. To call api methods, you can use ready-made libraries , which are available for most popular languages ​​and frameworks. However, if the application does not assume the presence of a server, there is a way to change this behavior: you can enable the setting in your account and in the widget that allows you to save files without a private key.

But accepting a file is half the trouble. After that, they also need to distribute. For this you need a wide channel. Pictures need to pinch in a different resolution, and even in several. It is also a good idea to have a separate server do this, because the distribution of large files clogs the operating system caches.

For this, Uploadcare has a smart CDN . Like any normal CDN, it can simply give the files by their ID. And can apply to them different operations. For pictures it is: resizing, cropping, coercion to another format, rotation and others. You describe in url what image you want to receive, and CDN gives it.

Of course, before Uploadcare, there were ready-made solutions, for example, for resizing images. Or various libraries that allow you to save files on the "infinite" S3 . But it’s still tiring enough to set them up, and they didn’t solve all the problems. And few people came to the realization of such amenities as downloading files from social networks and cloud services, or to the possibility for the user to cut out a suitable part of the image himself while downloading a file. With Uploadcare, you get it automatically.



The project has as many as two api (except for CDN), which you can use directly if you have specific requirements: the file download api , accessible by the public key, and the rest api , accessible by the secret. This allows you to use Uploadcare not only in the web, where file uploading is common, but also on mobile devices and desktop applications. By the way, for iOS there are already ready libraries.

On the architecture of Uploadcare, you can write a separate article (by the way, can it do that?). But in short, with reliability, everything is in order. Servers work in the Amazon cloud and are easily scaled horizontally. Different nodes are responsible for different functions. Even if the base and the main site fall, the CDN will continue to distribute files and resize the pictures. CloudFlare is responsible for delivering content. User files are stored on S3, and it is possible to connect your own S3 storage for your account, which can be beneficial if you intend to store large files.



If you are interested in the project, but something does not suit you, do not hesitate to ask and offer in the comments.