Enterprise-class cloud storage based on NGINX Plus and Minio

This article describes how to configure the reverse proxy server NGINX or NGINX Plus as a load balancer for object storage (object storage) based on Minio .

Designing a storage facility

Almost any application needs storage, but the requirements for this component of the system can vary greatly. Consider, for example, the document repository: it is possible that at the initial stages of work it will not have to handle a large number of read requests, but may later need scaling. Another application (such as, for example, an image gallery) already from the moment of launch should be able to quickly serve a large number of requests and scale as needed.

These subtleties complicate the process of organizing storage. But everything is not so bad: with the advent of object storage (object storage) as the standard way of storing unstructured data (largely due to the need to use HTTP), the process of standardizing the operation of the application with the storage began.

But the question still remains: how to organize sharpened for your application and at the same time flexible storage of objects?

Since working with the object storage implies the use of HTTP servers and clients, it is necessary to select a suitable web server (for example, that does not need an NGINX representation) to serve HTTP traffic. A lightweight and highly scalable object storage server can be used as a backend. For this role, Minio is great. The flexibility of such a system is a key factor in creating a corporate-level service.

With NGINX Plus, administrators can not only configure incoming traffic balancing, but also caching, throttling, SSL / TLS termination, and even traffic filtering based on various parameters. Minio, on the other hand, offers lightweight object storage compatible with Amazon S3 .

Minio is designed to accommodate unstructured data, such as photos, videotapes, log files, backup copies, as well as images of virtual machines and containers. Its small size allows it to be included in the stack of applications similar to Node.js, Redis, and MySQL. Minio also supports distributed mode (distributed mode) , which provides the ability to connect multiple disk objects to a single storage server, including those located on different machines.

In this article, we will look at several scenarios for using NGINX Plus in combination with Minio, which allow you to set up a highly scalable, fault-tolerant and stable storage of enterprise-class objects.

NGINX Plus as a reverse proxy and load balancer

NGINX Plus is primarily known as reverse proxy. But do you need a reverse proxy for Minio? Let's look at several usage scenarios:

• If one or more Minio servers are faced with the NGINX Plus reverse proxy, it is possible to move the Minio server instances to other machines / sites without the need to change the client settings.
• NGINX Plus can balance incoming traffic and distribute it evenly to instances of the distributed Minio server .
• The NGINX Plus proxy server can be part of a fault-tolerant storage facility created using the mirror command from the Minio Client ( mc ).

NGINX Plus performs reverse traffic proxying by sending requests to an internal server, which is set by the proxy_pass directive. In the following fragment of the configuration file, a single Minio instance is running on localhost and is available at http: // localhost: 9000 . All requests to the top-level directory (/) on www.example.com , coming to port 80, are sent to Minio. NGINX Plus explicitly sets the Host header to equal this value in the original request.

 server { listen 80; server_name www.example.com; location / { proxy_set_header Host $http_host; proxy_pass http://localhost:9000; } } 

If there are several Minio servers, it makes sense to configure traffic balancing by listing these servers in the upstream configuration block and specifying this group in the proxy_pass directive directive:

 upstream minio_servers { server minio-server-1:9000; server minio-server-2:9000; } server { listen 80; server_name www.example.com; location / { proxy_set_header Host $http_host; proxy_pass http://minio_servers; } } 

For more information about configuring NGINX or NGINX Plus as a proxy for Minio, see the Minio documentation .

SSL / TLS termination

Recently, HTTPS has become the default protocol for transmitting most of the web traffic, so for Minio it makes sense to deploy an HTTPS server right away. NGINX Plus as an HTTPS server is easy to set up. The first thing you need is an SSL / TLS certificate that you can get and integrate into NGINX Plus using Let's Encrypt.

Next, you should edit the NGINX Plus configuration file, where you need to specify the ssl parameter in the listen directive, which is located in the server block, and then register the files with the server certificate and private key :

 server { listen 80; server_name www.example.com; return 301 https://www.example.com$request_uri; } server { listen 443 ssl; server_name www.example.com; ssl_certificate www.example.com.crt; ssl_certificate_key www.example.com.key; ssl_protocols TLSv1 TLSv1.1 TLSv1.2; ssl_ciphers HIGH:!aNULL:!MD5; location / { proxy_set_header Host $http_host; proxy_pass http://localhost:9000; } } 

For more information on completing SSL / TLS, see the NGINX Plus Admin Guide .

Caching

Object storage servers are usually not very fast, but this does not mean that client requests are processed slowly. When caching is enabled on the NGINX Plus server, it saves frequently requested data and can immediately return it to the client without redirecting the request to an internal server.

Here is how it works. The NGINX Plus server's web cache is located between the client and the Minio server, and a copy of each file requested from the repository is stored in it. If the requested file is in the cache, NGINX immediately returns it without referring to Minio. This reduces the response time for the client and the load on the Minio server.

To configure the NGINX Plus cache to work with Minio, the directives proxy_cache_path and proxy_cache . The proxy_cache_path directive sets the location and configuration of the cache, and proxy_cache activates it. Additional information can be found in A Guide to Caching with NGINX and NGINX Plus .

 proxy_cache_path /path/to/cache levels=1:2 keys_zone=my_cache:10m max_size=10g inactive=60m use_temp_path=off; server { ... location / { proxy_cache my_cache; proxy_set_header Host $http_host; proxy_pass http://localhost:9000; } } 

Throttling

There are cases when, based on business requirements or security concerns, some http clients need to be a little “strangled”. NGINX Plus allows you to limit the available bandwidth, the number of requests and connections.

To limit the bandwidth, use the limit_rate directive. In the following example, the download speed is limited to 200 Kb per second:

 server { ... location /images/ { limit_rate 200k; ... } } 

To limit the number of requests, use the limit_req and limit_req_zone , similar to this example, in which each unique IP address is limited to 10 requests per second with peaks to 20 requests.

 limit_req_zone $binary_remote_addr zone=my_req_limit:10m rate=10r/s; server { ... location /images/ { limit_req zone=my_req_limit burst=20; ... } } 

To limit the number of connections, use the directives limit_conn and limit_conn_zone . In the following example, each unique IP address is limited to five simultaneous connections.

 limit_conn_zone $binary_remote_addr zone=my_conn_limit:10m; server { ... location /images/ { limit_conn my_conn_limit 5; ... } } 

For more information, see the NGINX Plus Admin Guide .

Conclusion

In this article, we have demonstrated several features of NGINX Plus, which allow load balancing (in particular, for the Minio Object Storage Server). A bunch of NGINX Plus and Minio allows you to customize a flexible storage of objects, tailored to the needs of your application.

This article was originally published on the Nginx blog .

You can find us on Slack: slack.minio.io