Domain sharding: Ruby on Rails implementation and results

I recently decided, using the example of one project, to find out how much domain sharding affects the download speed of a site. Let me remind you, the essence of this optimization is that static files are loaded from different domains (which, however, can point to the same server), and this allows you to bypass the browser restriction on the number of simultaneous connections to the same domain. Intuitively, it seems that in the case of a large number of small files, this should significantly speed up the loading of the site as a whole. Check if this is true.

In brief, I will describe the situation: there is a rather long one-page site, in the process of downloading a little more than a hundred requests for static loading (css, js, fonts, images) are made. The site is written using Ruby on Rails 4.1.12, puma-2.15.3 is used as a web server, nginx gives statics and looks at puma. This is all started on the Digital Ocean droplet in the Frankfurt 1 location. And, having such initial data, we need to transfer requests for statics from example.com domains to domains of the form%% {i} .example.com.

First of all, you need to configure the return of statics from these addresses. For this, it is enough to configure the appropriate DNS records (I just had an entry for * .example.com, in my case it was enough), and then change the settings of nginx (in the server_name directive there is a regular schedule that traps hosts of the form assets0.example .com and assets0.example.ru, because in my case the site is accessible from two different addresses):

server { listen 80; server_name ~^assets\d\.(example\.com|example\.ru)$; root /home/deployer/sites/example/current/public; location ~ ^/assets/ { expires 1m; add_header Cache-Control public,max-age=259200; break; } } 

Then it is necessary to change the generation of static paths on the application side. In rails it is elementary : just add a line to config / production.rb

 config.action_controller.asset_host = "assets%d.example.com" 

Then, when generating addresses, the rails will alternate the hosts “assets0.example.com”, ..., “assets3.example.com”. By the way, I wondered why it was 4 addresses, and not 118 (one for each request, so that it would be completely straight parallel-parallel). First, DNS lookup will be performed for each additional host, and placing so many hosts on a page will only slow down the load. Secondly, in addition to the limit on the number of simultaneous requests to a single host, browsers have a limit on the total number of simultaneous requests (I’ll give you a specific limit at the end of the post).
')
Magic rail is, of course, good, but in my case it would not work because of the need to generate different addresses when visiting a site from different domains. However, it is not much more difficult to configure. I also decided to configure the option of enabling / disabling domain sharding on the site without having to change the application code. The easiest thing to do was to use environment variables:

 if ENV['DOMAIN_SHARDING'] == 'enabled' config.action_controller.asset_host = Proc.new { |source, request| if request "assets#{source.hash % 4}.#{request.host_with_port}" end } end 

The definition of the domain sequence number looks like this (source.hash% 4) so ​​that when the page reloads (well, or when the server cache is reset), the link for each file remains unchanged. This will allow more efficient use of the browser cache. It also remains a mystery to me why we need a check for the existence of a request, but that’s what was written on the docks, and I didn’t dig deeper.

I'm running

 $ DOMAIN_SHARDING=enabled rails s -e production 

and it works! Almost. The fonts are broken, and in the browser console, complaints about the Cross-Origin Resource sharing policy.

With the sad thought that everything does not always work out of the box, it is useful to learn reflexively something about “Cross-Origin Resource sharing policy rails fonts”. I saw the mention of the font_assets heme, in the README I found the line “Sets Access-Control-Allow-Origin response headers for font assets” and decided that this is exactly what I need.

My mistake was to think first. Then I would immediately understand that the fonts, like the rest of the statics, are given on the combat server by nginx, who does not know about any gems. In fact, a small quest was released: after connecting the font_assets, everything broke; found why it broke, corrected the source, it worked; made a fork, registered it in a gemfile; updated the production version; I realized that I had to think first; rolled back the version, deleted the fork.

Actually, the correction of the production situation was simple: a small edit of the location section solves the problem:

 location ~ ^/assets/ { expires 1m; add_header Cache-Control public,max-age=259200; add_header Access-Control-Allow-Origin *; add_header Access-Control-Allow-Methods GET; break; } 

In general, this is the actual setup over and I started the measurements.

Measurement results

Measured like this: I opened the Network tab in the Chrome inspector, set the domain: *. Example.com / domain: example.com in the filter, and updated the page. Not the most high-tech way, but allows you to track not only the load time, but also its nature. (The screenshots show only the lower parts of the graphs).

With caching enabled, no sharding	With caching enabled, with sharding
With disabled caching, without sharding	With disabled caching, with sharding
With caching enabled, no sharding, Firefox	With caching enabled, with sharding, Firefox

With the cache turned on, the total time jumped quite a bit around the mean values, but usually with domain sharding, the download took place ≈0.2-0.4 seconds faster. In FF, the end of the download was about the same, but with sharding turned on, most of the files became available earlier. Also on the graphs clearly visible browser restrictions on the number of simultaneous connections: a maximum of 6 to one host, a maximum of 10 as a whole.
With the cache turned off, the picture smoothed out, but all the same, with sharding it was a little faster. I didn’t quite understand why, but when I turned on the speed limit to 750 kB / s without sharding, it worked a little faster.

To summarize : in general, at the time when browsers allowed only two simultaneous connections, domain sharding improved the situation much more, but now its use makes sense.