The best way to upload files to Ruby is with Shrine. Part 2. Loader
This is the second part of the Shrine series of posts. The purpose of this series of articles is to show the advantages of Shrine over existing file loaders.
In a previous post, I talked about what motivated me to create a Shrine. In this article I want to show you the foundation on which Shrine is based - the repository, the loader and the uploaded files.
Shrine “storage” is a plain Ruby object that encapsulates file management in a specific storage service (file system, S3, etc.). The repository should contain the following 5 methods:
Shrine storages are configured directly by passing options to the constructor (borrowed from Refile) and must be registered with
')
Currently, Shrine has file system support, S3 , Fog , Flickr , Cloudary , Transloadit , Uploadcare , Imgix , GridFS, and SQL , so choose.
You can also easily write your own repository, for this there is a manual and a linter that will automatically check whether everything works correctly.
Loaders are subclasses of Shrine, they encapsulate the logic of unloading a specific attachment (as in CarrierWave).
Uploader objects act as wrappers around the repository, they run all the logic for loading, which is common to any repository:
Creating a boot loader instance with the storage option set:
Loaders do not know about the models; They only work on the file that will be uploaded to the input, and return a view of the uploaded file on the output. Since this assumes that the loaders are stateless, this makes their behavior very predictable.
When the file is uploaded through the loader, the
Since this object knows where it was loaded, it can provide many useful methods:
This object is determined only by the hash. Since the repository can be referenced by its set parameter, this hash can now be serialized into JSON and stored in a database column.
Shrine can load any
In addition,
Shrine comes with a small kernel (<500 lines of code) that provides the necessary functionality. Any additional features can be downloaded via plugins . This gives you the flexibility to choose exactly what the Shrine should do for you, and download the code only for the functionality that you use.
Shrine comes with over 35 plugins, and you can easily write your own. The Shrine plugin system is an adaptation of Roda that I wrote about in the past.
In addition, Shrine loaders can be inherited ( unlike CarrierWave ).
Most libraries for downloading files have pretty monstrous dependencies.
Shrine, on the other hand, has only one mandatory but light dependency - Down . Down is a net / http wrapper for downloading files, which improves open-uri and supports streaming, and is used by almost all Shrine repositories.
In addition, Shrine as a whole loads very quickly, because you load the code only for the functionality that you use. For other downloaders, you need to download the code for all the functionality that you may not need. For example, Shrine loads 35 times faster than CarrierWave without loaded plug-ins and 7 times faster with all loaded ( source ) plug-ins.
Every high-level interface should have a good foundation. Thus, no matter what level of abstraction you are working on, you can always understand what is happening. The basis of Shrine consists of the classes
Original: https://twin.imtqy.com/better-file-uploads-with-shrine-uploader/
Articles from the original series in the blog of the author of the library:
In a previous post, I talked about what motivated me to create a Shrine. In this article I want to show you the foundation on which Shrine is based - the repository, the loader and the uploaded files.
Storage
Shrine “storage” is a plain Ruby object that encapsulates file management in a specific storage service (file system, S3, etc.). The repository should contain the following 5 methods:
class MyStorage def upload(io, id, **options) # `io` `id` end def url(id) # URL- `id` end def open(id) # `id` IO end def exists?(id) # , end def delete(id) # end end Shrine storages are configured directly by passing options to the constructor (borrowed from Refile) and must be registered with
Shrine.storages :')
Shrine.storages[:s3] = Shrine::Storage::S3.new( access_key_id: "abc", secret_access_key: "xyz", region: "eu-west-1", bucket: "my-bucket", ) Currently, Shrine has file system support, S3 , Fog , Flickr , Cloudary , Transloadit , Uploadcare , Imgix , GridFS, and SQL , so choose.
You can also easily write your own repository, for this there is a manual and a linter that will automatically check whether everything works correctly.
Loader
Loaders are subclasses of Shrine, they encapsulate the logic of unloading a specific attachment (as in CarrierWave).
class ImageUploader < Shrine # image uploading logic goes here end Uploader objects act as wrappers around the repository, they run all the logic for loading, which is common to any repository:
- Treatment
- Extract Metadata
- Generating location for file
- Download (at this stage the storage is accessed)
- Close the downloaded file
Creating a boot loader instance with the storage option set:
Shrine.storages[:disk] = Shrine::Storage::FileSystem.new(...) uploader = ImageUploader.new(:disk) uploader.upload(image) #=> #<Shrine::UploadedFile> Loaders do not know about the models; They only work on the file that will be uploaded to the input, and return a view of the uploaded file on the output. Since this assumes that the loaders are stateless, this makes their behavior very predictable.
Uploaded files
When the file is uploaded through the loader, the
#upload method returns a Shrine::UploadedFile object. This object is a complete representation of the file that was uploaded to the repository. uploaded_file = uploader.upload(image) #=> #<Shrine::UploadedFile> uploaded_file.id #=> "43ksd9gkafg0dsl.jpg" uploaded_file.storage #=> #<Shrine::Storage::FileSystem> uploaded_file.metadata #=> {...} Since this object knows where it was loaded, it can provide many useful methods:
uploaded_file.url # generates the URL uploaded_file.download # downloads the file to the disk uploaded_file.exists? # asks the storage if file exists uploaded_file.open { |io| ... } # opens the file for reading uploaded_file.delete # deletes the file from the storage This object is determined only by the hash. Since the repository can be referenced by its set parameter, this hash can now be serialized into JSON and stored in a database column.
uploaded_file.data #=> # { # "id" => "df9fk48saflg.jpg", # "storage" => "disk", # "metadata" => {...} # } uploaded_file.to_json #=> '{"id":"df9fk48saflg.jpg","storage":"disk","metadata":{...}}' Shrine::UploadedFile objects Shrine::UploadedFile do not depend on loaders. This is a significant difference from CarrierWave and Paperclip, which have this dependency on the classes CarrierWave::Uploader::Base and Paperclip::Attachment .Abstraction io
Shrine can load any
IO object that responds to #read , #size , #rewind , #eof? And #close (like Refile). Defining this strict interface, each Shrine function now knows that they can only rely on these methods, which means that they will work correctly regardless of whether you load the File , StringIO , ActionDispatch::Http::UploadedFile , Rack types. or deleted files that are downloaded by stream .In addition,
Shrine::UploadedFile is itself an IO object, wrapping any downloaded file under the same unified interface. This makes moving a file from one repository to another really convenient. In addition, it allows you to optimize some downloads by skipping the download process and re-downloading, for example, to use a copy of S3 if both files are from S3, or just send a URL request if the repository supports it. cache = ImageUploader.new(:s3_temporary) cached_file = cache.upload(image) store = ImageUploader.new(:s3_permanent) store.upload(cached_file) #=> performs an S3 COPY request Plugin system
Shrine comes with a small kernel (<500 lines of code) that provides the necessary functionality. Any additional features can be downloaded via plugins . This gives you the flexibility to choose exactly what the Shrine should do for you, and download the code only for the functionality that you use.
# Loads the processing feature from "shrine/plugins/logging.rb" Shrine.plugin :logging, logger: Rails.logger Shrine comes with over 35 plugins, and you can easily write your own. The Shrine plugin system is an adaptation of Roda that I wrote about in the past.
In addition, Shrine loaders can be inherited ( unlike CarrierWave ).
Shrine.plugin :logging # enables logging for all uploaders class ImageUploader < Shrine plugin :backup # stores backups only for this uploader (and its descendants) end Dependencies
Most libraries for downloading files have pretty monstrous dependencies.
CarrierWave
- ActiveSupport - I really don’t want all those monkey patches
- ActiveModel - Why not perform validation without libraries ?
- MIME :: Types - It is better to determine the MIME type from the file contents
Paperclip
- ActiveSupport - Again, I want to be able to not have any monkey patches
- ActiveModel - Okay, anyway, both ActiveModel and ActiveSupport are required for ActiveRecord
- Cocaine - Open3 monstrous library to run the command shell
- MIME :: Types - MIME type substitution detection has problems
- MimeMagic - enough file utility
Refile
- RestClient - A monstrous solution for easy file uploading.
- Sinatra - This is normal, although Roda is a lighter alternative.
- MIME :: Types - It is better to determine the MIME type from the file contents
Shrine, on the other hand, has only one mandatory but light dependency - Down . Down is a net / http wrapper for downloading files, which improves open-uri and supports streaming, and is used by almost all Shrine repositories.
In addition, Shrine as a whole loads very quickly, because you load the code only for the functionality that you use. For other downloaders, you need to download the code for all the functionality that you may not need. For example, Shrine loads 35 times faster than CarrierWave without loaded plug-ins and 7 times faster with all loaded ( source ) plug-ins.
Results
Every high-level interface should have a good foundation. Thus, no matter what level of abstraction you are working on, you can always understand what is happening. The basis of Shrine consists of the classes
Storage , Shrine and Shrine::UploadedFile , each of which has a well-defined interface and duty.Original: https://twin.imtqy.com/better-file-uploads-with-shrine-uploader/
Articles from the original series in the blog of the author of the library:
Source: https://habr.com/ru/post/328788/
All Articles