[Translation] VMware CloudFoundry: PaaS to Ruby

Platform as a Service (PaaS) is not the most trivial thing to create, deploy and maintain: to begin with, you have to do a lot of work to manage all services from the inside, then you have to think and implement a good interface, and finally you need to sell and skillfully maintain it. It is not surprising that only a few well-off players in the IT market can afford to invest in the creation of such services.

Therefore, it was doubly interesting to see how VMware is deploying the CloudFoundry service, and with open source (honestly, here is a github account )! Full PaaS, which they also propose to use as hosting, is also available for everyone to run in their own company or data center - now you can run “mini Heroku” or “cloud EngineYard” on your own servers! But in the direction of marketing, it is much more interesting to look under the hood of this project, because it is completely organized in Ruby.


')

CloudFoundry: Rails, Sinatra, EventMachine

CloudFoundry provides the ability to run multiple platforms and frameworks (Rails, Sinatra, Grails, node.js), and also supports the much-needed supporting services (MySQL, Redis, RabbitMQ). In other words, the system is modular and very scalable. The service has a whole channel on youtube , where you can get acquainted with the infrastructure in detail.

Rails 3 (CloudController) conducts this orchestra from modules and stores information about all users, used services and applications, as well as controls the services provided. When you start the console (CLI - command line client) on a local machine, you are really accessing CloudController. Interestingly, the rails application itself is an add-on to the Thin web server, uses Ruby 1.9 and asynchronous database drivers — in other words, asynchronous rails are used that solve many performance problems.

Health Manager helps rails - a daemon that imports all ActiveRecord models from CloudController and regularly compares the state of the database with what other demons have. If a discrepancy is found, the health manager notifies CloudController - an easy and effective way to keep distributed information up to date.

The rest of CloudFoundry works on a specific pattern: each service is a ruby ​​daemon that sends requests to CloudController when it first loads, while sending a JSON report on its status, as well as in some other actions. Not surprisingly, EventMachine stands behind all of this and therefore uses Thin and rack endpoints.

map '/healthz' do
run lambda { |env| [200, RACK_TEXT_HDR, Component.updated_healthz] }
end


The router is responsible for parsing incoming requests and redirecting traffic to one of the applications (droplets). To do this, a map of registered URLs and their corresponding applications is used. When a new server is created with an application, the router map is updated and traffic is redirected according to the updated routes. For a small number of applications, a single router is enough. If there are a lot of them, then the traffic is distributed among several routers.

DEA (Droplet Execution Agent) - managing the process of creating new applications. When receiving the appropriate command from CloudController, it runs the necessary scripts and commands and starts the server itself .

There is also a service that controls server access to resources such as MySQL, Redis, RabbitMQ, etc. Its architecture is again very simple: the ruby ​​daemon gateway listens for incoming requests and calls the necessary “start” / “stop” commands and add / delete user commands. Adding a new or custom service is as easy as implementing a Provisioner class.

Connect all the pieces and NATS

Each ruby ​​daemon described above follows a specific pattern: when loading, sends a request to CloudController and reports its status and status via HTTP endpoints. But how do they work with each other? Naturally, with the help of another service on ruby. Through NATS , a lightweight message publishing and subscription system that brings everything together. Each ruby ​​daemon, when first launched, connects to NATS and subscribes to messages necessary for it, and also starts publishing its own notifications.

This architecture allows CloudFoundry to easily add and remove new routers, DEA agents, service controllers, and so on. Nothing prevents you from performing all the actions described above, on a single machine, or on a cluster in your own data center.

So are distributed Ruby systems possible?

Of course yes! Building a distributed system with as many scalable particles as CloudFoundry is a real feat. By choosing ruby ​​as their primary language, VMware expressed a confidence vote on such non-trivial systems. Under the hood of this huge colossus, you can find rails, sinatra, rack, and a lot of EventMachine code. So I definitely advise all rubists to read the source code at their leisure.