Minio - what is it

Minio is a simple, fast and AWS S3 compatible object storage. Minio is designed to accommodate unstructured data, such as photos, videotapes, log files, backup copies, as well as images of virtual machines and containers. Its small size allows it to be included in the stack of applications similar to Node.js, Redis, and MySQL. Minio also supports distributed mode (distributed mode), which provides the ability to connect multiple disk objects to a single storage server, including those located on different machines.

Install minio

Server installation

Putting and running Minio is easy

brew install minio/stable/minio minio server ~/data 

As the documentation advises, if the previous version was previously installed without specifying stability, it must be removed and new stable installed

 brew uninstall minio brew install minio/stable/minio 

The server command help will show the basic ways to launch and configure it. I run it locally

 minio server --address localhost:9000 ~/data 

After starting the server, he will write to what address it can be found, as well as a key with a secret password to access it. This is basically enough to get started with Minio, for details see the official documentation.

Client installation and basic commands

With the Minio client, you can work with the repository just like a Unix command line. Put the client and see the commands

 brew install minio/stable/mc mc --help 

from useful:

 mc mb minio/my-uploads-store #   mc policy -r public minio/my-uploads-store #      mc ls minio/my-uploads-store #       

The official reference manual describes everything in some detail.

Nginx

Basic settings

Nginx will do a lot of useful things, for example:

• Hide storage on intranet
• Organize caching
• Organize load balancing for distributed storage
• Well, a single entry point for your applications, which will allow you to change the storage without any serious consequences.

The simplest setup for Nginx to use Minio

 server { listen 80; server_name minio.dev; location / { proxy_set_header Host $http_host; proxy_pass http://localhost:9000; } } 

Now we can access our repository at minio.dev

Caching

With caching Minio in Nginx as described in the article I had problems. As it turned out - not just me. Unscrew as follows:

• We write to the repository via a direct link to the repository past Nginx (or through it but without caching)
• Reading from the repository takes place on another link with caching
• Correspondingly, in our RoR application the link changes - when writing one, when reading - the other with caching

 proxy_cache_path /var/cache/nginx/minio_cache levels=1:2 keys_zone=minio_cache:10m max_size=10m inactive=60m use_temp_path=off; upstream minio_servers { #       Load Balancing server localhost:9000; } server { listen 80; server_name minio.dev; location / { proxy_cache minio_cache; add_header X-Cache-Status $upstream_cache_status; proxy_cache_revalidate on; proxy_cache_use_stale error timeout updating http_500 http_502 http_503 http_504; proxy_cache_lock on; proxy_ignore_headers Set-Cookie; proxy_cache_valid 1m; proxy_set_header Host $http_host; proxy_pass http://minio_servers; } } 

RoR application

General settings

Pictures in Minio are loaded using Shrine (all the details of the work are in good official documentation and articles attached to the documentation, here are just features for Minio). Directly to work with Minio you need to add to the Gemfile

 gem 'shrine-fog' gem 'fog-aws' 

 require 'shrine/storage/fog' require 'fog/aws' require 'image_processing/mini_magick' minio = Fog::Storage.new( provider: 'AWS', aws_access_key_id: '<key>', aws_secret_access_key: '<secret>', region: 'us-east-1', endpoint: 'http://localhost:9000/', #     path_style: true, ) Shrine.storages[:cache] = Shrine::Storage::Fog.new( connection: minio, directory: '<cache-directory>', public: true, ) Shrine.storages[:store] = Shrine::Storage::Fog.new( connection: minio, directory: '<store-directory>', public: true, ) 

It goes without saying that you need to create the appropriate folders through the Minio client and give them full access (see the section above — the Minio client).

 class AvatarUploader < Shrine include ImageProcessing::MiniMagick plugin :activerecord plugin :determine_mime_type, analyzer: :mime_types plugin :logging, logger: Rails.logger plugin :remove_attachment plugin :store_dimensions plugin :remote_url, max_size: 20*1024*1024 plugin :versions plugin :default_url plugin :processing plugin :host_url, host: 'minio.dev', port: '80' #   ,    Attacher.default_url do |options| #  ,   '/images/default.svg' end process(:cache) do |io, context| #  ,         resize_to_fill!(io.to_io, 300, 300) end end 

Actually, most plugins are optional. For details, see the official shrine documentation. Treatment as usual:

 User.find(1).avatar_url 

Link replacement

I had to tinker with the replacement of the link. Shrine has enough different plug-ins that can be used for most tasks. There is a plugin for replacing the host with default_url_options , but in this particular case it did not help - it changes the host and ignores the port. Hm, what to do? And let's write your own Shrine plugin. The code of the plugin is lower, as it is connected - in the previous listing (the 80th port by default does not register in the final link, all the rest will appear. You can not write it, by the way).

 class Shrine module Plugins module HostUrl def self.configure(uploader, options = {}) uploader.opts[:host_url] = (uploader.opts[:host_url] || {}).merge(options) end module FileMethods def url(**options) new_uri( URI.parse(super), uploader.opts[:host_url][:host], uploader.opts[:host_url][:port] ) end private def new_uri(uri, new_host, new_port) URI::HTTP.new( uri.scheme, uri.userinfo, new_host, new_port, uri.registry, uri.path, uri.opaque, uri.query, uri.fragment ).to_s end end end register_plugin(:host_url, HostUrl) end end 

The code for the new plug-in is set in lib / shrine / plugins / host_url.rb

Data migration

In the official instructions , in principle, everything is described in detail. There are, however, a few nuances - well, there if ((attacher = user.avatar_attacher) .stored? for some reason, it returns false and in small detail - you need to add a new storage with the existing old one, make a migration and then cut the old one ...
But in this particular case, everything can be done much easier:

 rsync -zavP <>@<.>:/<___> ~/data/<store-directory> 

Actually, everything out of the box works with the new code.

Other

Yes, when moving to Minio in the next application, I moved this plugin to gem .

Used sources

Enterprise-Grade Cloud Storage with NGINX Plus and Minio and Russian translations