A set of Ruby libraries for CMS and media publishing site

A set of libraries for the development of CMS media edition is almost no different from any other application. On the example of the application for Ribbon and Sheets, we decided to comment on the choice of each library. The description is made in the discussion format of each heme.

Previous review article: Media Edition Restart: Review

CMS Lenta.ru

Written about the tape was relevant until April 2014.
')

gem 'unicorn' 

foxweb : There should be an explanation why not Puma.
7even : He does not exist.
zaur : This is my conservative approach that does not give guys the freedom to unlock the potential of the Puma web server. I prefer to use the classic model of the web server when several processes are running. And I don’t like the model where handlers compete to process requests in different threads in the classical interpreter. The lack of confidence in the thread-safety of the MRI interpreter and the unwillingness of all to switch to jRuby or Rubinius predetermined the choice.
7even : Ready to go to Rubinus.
foxweb : Well now, yes.
zaur : 7even , just recently had a different opinion.
7even : Rubinius is developing rapidly. A few months ago, he fell on constructions like %i(foo bar) .

 gem 'newrelic_rpm' gem 'newrelic-redis' 

foxweb : Our Rails application and message queue in Redis worked under constant monitoring of the NewRelic service. If finances allow, you should definitely use this tool on all critical projects. NewRelic saves hours and hard work, quiet sleep for developers, and saves some of them from dismissal and even from physical injuries. Before someone from the editorial office resorted with shouts of “Nothing works!”, We already knew what server it was on, for what reason it was not working, who had broken it, who would fix it, and how much of the body would be amputated. Some of the graphics we brought to the HTML-page, which is 24 hours a day, spinning on a large monitor on the wall. Everyone always knew “what we have with the site.” Shoals of one developer sees the whole team. NewRelic minimizes the reaction time "dropped - raised". For online media, this is especially critical.
zaur : When setting up the module, you should not forget to exclude the test and dev environments. It is done in configuration files.

 gem 'rails', '~> 3.2' 

foxweb : This is Rails.
zaur : With this old rails. They did not switch over to new ones, everyone was afraid that “everything will break.”
foxweb : To be more precise, most of the gems at that time were not adapted for Rails 4.

 gem 'faye' 

foxweb : Faye - a simple implementation of WebSockets, pub / sub sharpened on the exchange of messages between different users sitting on the same site. Initially, this is such a convenient chat on web-sockets. In our project, Faye sent / received messages between editors sitting in the admin panel. Someone saved the new text, someone uploaded 10 pictures - received messages. After some time, the server reports that the pictures were generated and published - all those who are currently engaged in this article received messages. All this was very convenient for the editors, increased the level of interaction and accelerated the work of the editors as a whole. It was not necessary to write in Skype “boys, nobody touch this news, I am saving it now!” Or “go to the gallery, I created it!” It was always clear which editors are currently running this particular article. In combination with the journaling and versioning system, as well as the diff comparison service, the collective work of the editorial board on the same materials has become truly convenient. The only thing that we didn’t manage to realize was separation by roles and access levels in the alert system. It would be interesting to make it so that, for example, the editor-in-chief could see everything on a special page, without being “glowing” anywhere, and the trainees would receive messages of a certain type. Although, a lot of work went so that faye could not be used from the outside. Anything can happen. For example, authorization through cookies and tokens stored in Redis was implemented to some extent. Token tied to a specific editor account. Thanks to this, the editors could see the messages as if on behalf of the author - “Vasily Petrov saved the document”. In the basic version, faye doesn’t deal with user authorization and message protection, and is not at all connected with the Rails environment.
ksavelyev : Well, in fact, not very simple, Faye is good for everyone, but does not scale.

 gem 'faye-redis' 

foxweb : A plugin for faye that allows to use as an intermediate repository for pub / sub messages, as the name implies, Redis. We used it only for authorization and storage of tokens (it seems).
zaur : ... and for storing a queue of background processes, such as Resque or Cloudy (self-made video editing).

 gem 'thin' 

foxweb : Thin is the server for Faye.
zaur : Not your favorite Puma.
7even : And here zaur and not confused asynchronous server :)
ksavelyev : Zaur had no choice, otherwise Faye doesn't work;)

 gem 'pg' 

foxweb : Glorious relocation from MySQL to Postgres. We made a separate branch in git, everything was adjusted there, tested and sgorgil. First, we dumped the structure into a SQL file. Naturally, it turned out to be unsuitable for importing directly into Postgres. It was necessary to replace, for example, quotes throughout the file, some things were corrected manually. I think 7even will write a great story about why MySQL in such projects is bad.
7even : There is no hstore in MySQL :) In general, seriously, Postgres is much more popular in the Ruby community and, as a result, it's easier to google the solution to any problem. Plus, custom data types (arrays, json and the same hstore) make life easier. Psql (console client) is much more convenient (if you master it at least a little).
zaur : But ksavelyev believes that it was necessary to completely switch to MongoDB.

 gem 'postgres_ext' gem 'activerecord-postgres-json', github: 'michaelbn/activerecord-postgres-json' gem 'activerecord-postgres-hstore' 

zaur : When migrating from MySQL to PostgreSQL, the rails were the third version. Used these libraries to support PostgreSQL extensions.

 gem 'marionette-rails' 

ksavelyev : Adds the Marionette library to Asset Pipelite. Marionette adds another level of abstraction to Backbon, introduces the concept of a module, an application, and allows you to conveniently work with visual representations of collections. Makes Bekbona almost that Angulyar.
zaur : from Angular repudiated to the last.

 gem 'haml_assets' gem 'dust_assets' gem 'haml_coffee_assets' 

ksavelyev : Adds support for haml / dust templates, allows you to write templates with the specified syntax and use them inside Asset Pipeline; as a result, inside the JS application, we get a variable containing the compiled and ready-to-use templates that were originally written in haml / dust.

 gem 'turbo-sprockets-rails3' 

ksavelyev : Accelerates the execution of rake assets: precompile by compiling and merging only changed assets, contains additional functions for removing old assets.

 gem 'terminal-notifier-guard' 

foxweb : A heme interface to the system pop-up notification utility. During testing on Linux and OS X systems, a status message is displayed - successful, unsuccessful. Convenient: the tests are run by themselves, at this time you can switch to another window, and a pop-up notification will remind you that the testing is complete.

 gem 'guard' gem 'guard-rspec' gem 'guard-zeus' 

foxweb : Spring appeared in Rails 4.1 for background code preloading. In Rails 3, Zeus had to be used for the same purpose. If a developer changes a file, it will immediately be restarted by Zeus and running the tests related to it by RSpec. Guard is known to many as a tool for tracking events related to changes to project files. guard-rspec just runs tests when saving files.
zaur : That one, that the second brought more inconvenience. Start the desired test easier directly with your hands. And so solid excuses for poor ruby ​​performance.
7even : It's not a matter of “poor ruby ​​performance”, but of slow starting a rail application. Naturally, if it started in a couple of seconds, these crutches would not be needed.
foxweb : 470 specs in 14 seconds is fast enough.

 gem 'rspec-rails', '~> 2.14' 

foxweb : It seems that we used to use TestUnit before, but then, with the submission of 7even, we quickly switched to RSpec. This brought the testing process to a new level. We had a full TDD. Then we stopped writing tests, because there was almost no new code, and some code was very difficult or impractical (in time) to cover with tests.
zaur : A controversial decision, the majority won. And this new level is called “nafig tests”.

 gem 'ffaker' 

foxweb : When creating new entities, it was necessary to test and fill with random data (first name, last name, email, position, etc.). In the process of debugging, an ordinary programmer can make 1-3 records, then he gets bored, and the code remains underdeveloped. Actually, with the help of the classes of this heme, you can generate very diverse and, most importantly, plausible information. 7even contributed to it by adding a generator of Russian names and surnames. Thus, it was possible to debug performance on different data sets before deploying to production. The use of this heme is not related to tests. When writing tests, for example, standard rail fixtures were used.

 gem 'quiet_assets' 

foxweb : silent asses?
ksavelyev : Exactly, silent ones. When the rail application is launched locally on the developer's machine in dev mode, the web server gives the static itself. In production, Nginx deals with this, which keeps separate logs for assets. In development, the information that the web server gave the stack of assets is usually not needed by anyone; this heme blocks the junk messages that the static files were given to the browser. If necessary, if there is no such heme, you can always filter logs using grep, for example, the command tail -f development.log | grep SELECT tail -f development.log | grep SELECT will monitor the log for the appearance of SQL queries in them.

 gem 'colorize' 

foxweb : When outputting data to the console (maybe import logs or some rake tasks), it was possible to output color strings to the console using ruby ​​methods, which increased the overall readability of the logs.

 gem 'rails-erd' 

foxweb : At some point, the number of entities and connections between them increased so much that it became difficult to keep them in mind and transfer knowledge to new employees. With the help of this gem it was possible in a couple of actions to generate model diagrams with links and a bunch of different options. The diagram was then printed on a large sheet of A3 and thought a lot.
zaur : Then they threw it away and painted on the blackboard.

 gem 'ruby_parser' 

foxweb : Someone was experimenting with parsing Ruby. Possibly for your own file annotation mechanism.
7even : Not me.

 gem 'capistrano' 

foxweb : With Capistrano, everything was fine until Mina appeared. Capistrano worked slowly, but he knew how to make multi-server deploy. Mina is fast, and her output is much more readable. We didn’t manage to translate CMS and frontend to Mina, but special projects and, for example, Dom.leta.ru deploy or initially with the help of Mina. Multiserver deploy is quite simply “emulated” using a simple loop on the list of hosts (it seems there were a maximum of 4). That is, just 4 deployments for 4 hosts in one run.
zaur : Now for deployment, neither this nor that is used. Switched to the control scheme using ansible. Depla without interruption in the service provides HAProxy.
7even : It became 5 times slower than mines, but zero-downtime.
foxweb : I'll tell you a secret, it’s not even zero-downtime in the current setup;)

 gem 'better_errors' gem 'binding_of_caller' 

ksavelyev : better_errors significantly improves the standard information screen of the 500th error. A call stack browser and an interactive ruby ​​console appear, allowing you to view the status of objects.
foxweb : I do not understand how we used to debug rail applications without a console in the browser.
7even : Tests should be written, but not manually referenced in the browser links.
ksavelyev : If front-end vendors wrote tests in the backend, and backenders would do shadows and integrate fonts into the page, the world would definitely be better.
foxweb : I meant exactly those cases when it is impossible to catch layout errors in a regular console, for example.

 gem 'jazz_hands' 

7even : A set of pry, awesome_print and a few pry buns.
foxweb : Adds beautiful coloring, syntax highlighting and output formatting to a standard rail console to the console. Left in production, because I often had to climb into the console on working servers.

 gem 'active_record-annotate' 

7even : For annotation of models, annotate_models were initially used, but at some point it began to break down on new data types in PostgreSQL - and we decided to write our own implementation.
foxweb : you decided you wrote.
7even : Github .

 gem 'rabl' gem 'jbuilder' gem 'gon' 

zaur : For serialization, we started using Rabl. Awfully slow solution. Gradually began to move to JBuilder. And to transfer JSON to the page at the stage of HTML generation, Gon was used.

 gem 'fast_seeder' 

foxweb : From the same rank as Ffaker, but it can use data from CSV files as a source.

 gem 'oj' 

7even : The fastest JSON parser.
foxweb : Peter is better than Moscow. And the better? Than Moscow.

 gem 'slim' 

foxweb : At first there was Haml, but then it turned out that Slim is not only easier to read and write, but also renders faster. To replace Haml with Slim for the entire project, you can use, for example, haml2slim .

 gem 'russian' 

foxweb : Famous heme for the russification of dates and standard rail messages.

 gem 'whenever', require: false 

foxweb : Chronifier is a library for managing cron-tasks in Ruby. Something we had there once a day with the news this happened.

 gem 'redcarpet' 

foxweb : Markdown to work with Markdown, but no one remembers where we used it.

 gem 'kaminari' 

foxweb : Paginator - breaks up large lists into pages (1,2,3 ... 100).

 gem 'truncate_html' 

ksavelyev : Laziness-mother forces gems to be added from a single function to the gemfile. In this case, it was lazy for us to correctly bite off a piece of HTML with a regular expression.
foxweb : I didn't know about that.

 gem 'devise' gem 'devise-encryptable' 

zaur : A classic of the genre for creating authorization on the site.

 gem 'mini_magick' 

foxweb : Heme interface to ImageMagick.

 gem 'mimemagic' 

foxweb : heme to determine the type of downloadable content.

 gem 'foreman' 

foxweb : A simple process manager for a developer. When working on a large project, you need to run a lot of small services. Foreman launches everything you need with a single command. In our example, these are Rails, Faye, tests, queuing, and image processing services.
7even : Running tests through Foreman made their output so unreadable that I soon cut them out from Procfile.
foxweb : Recalled, it seems, everything got into the general output along with the tests, including the Faye messages and application logs.

 gem 'faraday' 

7even : Faraday is an indispensable library for accessing external network resources. Thanks to the middleware system (similar to Rack), you can set all transformations of both the request and the response once - for example, add an OAuth authorization header to the request before sending it, or parse the JSON response and wrap it in Hashie :: Mash.
foxweb : “External network resources” - commonplace HTTP requests for the old version of the site.
7even : Ok, to any network resources. In this case, the old version of the site, but we used faraday in a variety of situations - there were API and HTML.

 gem 'faraday_middleware' 

7even : There are many useful middleware for faraday in this gem.
foxweb : Is this, for example, JSON?
7even : And, for example, mashify .

 gem 'hashie' 

7even : Hashie - a collection of hash extensions. We used Hashie :: Mash - a variation of the hash, in which accessing elements goes not as accessing a key, but as calling a method of the same name (for example, not user[:name] , but user.name ). Unfortunately, this gem was later abused and used as a substitute for models.
foxweb : The layout was loved by the designers and called it “dot-notation”.
7even : That's what I'm talking about.

 gem 'nokogiri' 

foxweb : A well-known library for parsing XML / HTML. Actively used in the analysis of old archives. With the help of it during the import process, we replaced outdated links, tags of pictures, cleaned out irrelevant code, garbage and many other things. It seems to be even closed unclosed tags.

 gem 'eventmachine' gem 'em-synchrony' gem 'em-hiredis' 

7even : For video processing, a third-party E ******* m service was used, interaction with which went through their REST API. On our side, a demon worked, which through the queue in radish received the path to the video file loaded by the editor, sent it to the server E ****, transferred it to another Redis queue, polled the server every 5 minutes - and when the processing was on the E *** side * was completed, the daemon was pumping out the finished files. Since all these processes were to run in parallel, the daemon was written in EventMachine.
zaur : Asterisks are not painted over.

 gem 'streamio-ffmpeg' 

7even : This wrapper around ffmpeg allowed us to get the width, height, bit rate, duration and other metadata of the video file.
foxweb : This data was later used to display video content on the site pages.

 gem 'ruby-progressbar' 

foxweb : A simple implementation of the progress bar in the console. Used when importing data from the old database to the new format. When more than 500,000 materials are waiting for you in front of you, the progress bar helps you to plan time and meditate in the process. The class is pretty simple. At the input parameters are set such as a finite number of the cycle, all kinds of view settings, etc. When outputting, he himself considers interest, time, turns the counter, moves the strip, in general, does everything that a normal progress bar does in any OS. It is priceless to inform the editor-in-chief that it takes three days to import old data.

 gem 'version' 

foxweb : A trinket gem that increments the version number in the VERSION file on request and puts the corresponding tag on commits in Git. Played and forgotten.

 gem 'environment' 

zaur : One of the bicycles, for distributing settings to different environments, of which there were more than three. Design error, payback.
7even : Sunk into oblivion.

 gem 'settings' 

7even : Both configatron and rails_config were used to store application settings at different times. But somewhere, gems didn’t work outside of the Rails context - which made it impossible for us to access the settings from Faye and the demon on the event machine - and somewhere there were other problems; therefore, its own implementation was written. It is currently available on Github .

 gem 'bluepill' 

7even : Everything is simple: monitoring services.
foxweb : Often it was necessary to restart dropped or hung services directly on production. Then we got tired and we put Bluepill. Like Foreman, only for the server.
zaur : Awful monitoring of services. But others were no better. It was necessary to be able to follow the process that could change the PID. At the same time not many could do it. The web server changed its process at the moment of zero-downtime restart. For example, during deployment, for the smooth operation of the application, they used such wonderful Unicorn functionality.

 gem 'diffy' 

foxweb : Awesomely useful editorial tool. Allows you to view text editing history, as in Wikipedia. Naturally, in order to have something to compare with, a version history must be kept. She already worked with us, and to fasten such a tool was a matter of a couple of hours.The useful effect for the editorial was enormous.
zaur : There was a case when the editor made a typo. The site was corrected on time, but in the RSS on the main page Rambler. News left, of course, the wrong option.
ksegoh : And then the guys learned the difference between Syria and Libya.



zaur : One of the use cases for versioning. You can follow the trainee writing a note. For example, he can consistently write news by paragraph. And can paint a plan in a few sentences, and further expand the text. That is, you can see the train of thought of a person, or reveal a copy-paste.

 gem 'roo' 

foxweb : Gem to extract data from Excel spreadsheets. It was useful during the Olympics, when editors and content providers provided data on competitions and participants in a table. There was neither time nor desire to fiddle with a web interface for working with tables.

 gem 'net-scp' 

foxweb : Also “Olympic” heme, provided the ability to pick up these same tables via SSH from file servers.

 gem 'resque' gem 'resque-web' gem 'resque-pool' 

zaur : . ImageMagick. ( ) Sidekiq ( ). , , , . , . Resque, ImageMagick. .
foxweb : , . , JPEG , , PNG , , ImageMagick.
zaur : . “” . – . , , HTTP …

 gem 'ar_after_transaction' 

zaur : , .
ksavelyev : ? .

API-

 gem 'rails-api' 

7even : API, JSON, , rails-api.

 gem 'active_model_serializers' 

7even : rabl jbuilder JSON rails-api, .

 gem 'awesome_pry' 

7even : jazz_hands Ruby (, , - ), — -, pry awesome_print.

 gem 'ice_nine', require: %w(ice_nine ice_nine/core_ext/object) 

7even : . , #freeze ; #freeze — , . #deep_freeze .

 gem 'highline' 

7even : For self-sufficiency of the application and the ability to test each route without using the front-end, a small console API client was written, in which the highline was useful.

 gem 'rack-cors' 

7even : Support for CORS requests.

 gem 'database_cleaner' 

7even : Since the tests use the database, database_cleaner was used to clear all the tables before each test.
zaur : It also came in handy when they wrote their rake task to reset the database in dev mode. This need arises because pow keeps the application running. And, it means that the database had active connections that did not allow simply to re-create the database.

 gem 'guard-yard' 

7even : A plugin for guard, which starts the generation of YARD documentation from changed files.

 gem 'apiaryio' 

7even : API was decided to use apiary.io to document, and this heme allows you to generate documentation in HTML and view it locally.

 group :development do gem 'puma' end 

foxweb : Puma is here, and in production it's still Unicorn.

 group :production do gem 'unicorn' end 

foxweb : Unicorn is here, and Puma is still in development.

Frontend CMS Vedomosti

 source 'https://rubygems.org' source 'https://rails-assets.org' 

ksavelyev : The source line rails-assets.org began to appear more and more often in projects with a rich front-end. The point of adding another source is that rails-assets can make gems from almost any github repository. In order for the repository to be transformed into a heme it is necessary that it be a valid bower repository. By specifying an additional source, we can add type calls to the gem file gem 'rails-assets-BOWER_PACKAGE_NAME'. All JS / CSS resources of this gem will be available in the Asset Pipeline and then you can use them in your project as if you added the necessary files by hand.

 gem 'puma' gem 'unicorn', group: :production 

zaur : Well, you understand what I think about it.

 gem 'compass-rails' 

ksavelyev : , , . . — SASS/SCSS. , , CSS, , , , . , .

 gem 'rails-assets-angular' gem 'rails-assets-angular-cookies' gem 'rails-assets-angular-animate' gem 'rails-assets-angular-sanitize' gem 'rails-assets-angular-ui-router' gem 'rails-assets-angular-promise-tracker' gem 'rails-assets-angular-loading-bar' gem 'rails-assets-restangular' gem 'rails-assets-angular-contenteditable' gem 'rails-assets-angular-ui-ace' gem 'rails-assets-angular-bootstrap-colorpicker' gem 'rails-assets-ng-sortable' 

ksavelyev : bower-, rails-assets, .

:

• foxweb (@foxweb) — ( !)
• 7even (@7even) —
• ksavelyev (@ksavelyev) —
• ksegoh —
• zaur (@kavkaz) —