Development of the system architecture through the service-resource model

I want to suggest a little discussion on the topic of the service-resource model and ask about the need to develop a tool for using the service-resource model in the design, development and further operation of systems.

Starting position: I develop and exploit with colleagues an online system that serves hundreds of clients. Our system works on several servers, uses several databases, uses message queues, external services for sending SMS and mail. Typical situation? Full

What I want to get?
')
I want to get a more transparent system to cover the whole picture of the sponsored economy, to see bottlenecks, to see the dependencies of some parts of the system on others, to know that ssh on one server is extremely important for “over that little handler” that works at night on another server.

What for? Sometimes it is required to cover the entire economy of your department to get a clear picture, to see the directions of scaling and bottlenecks. In general, think about the architecture of the entire system. You understand that some services (DB, message queues) are used by others (applications), and these services are in turn used by third ones. This is convenient for the architect-developer in the design and development of the system, it is convenient for the operator in operating the system.

Why not ITSM / ITIL? If you look at ITSM / ITIL, you can see there a topic about the service-resource model . It is likely that I am not quite competent to talk about the complexities of ITSM / ITIL and its implementation in companies. Therefore, correct me, please, if I am mistaken somewhere, here and hereinafter.



In ITSM / ITIL there is a service-resource model. But when I start to read the whole kitchen and think about the introduction of any "enterprise" product, for example: InfraManager or OmniTracker , then "my head is wai-wai." (Moreover, these products are completely useless to me, the cost of their purchase, implementation and replacement of the currently used ones will be hoo, and most of the costs will not be on software). Of all ITSM, I need a little bit - just a resource dependency scheme, moreover, ITSM seems to have to cover the whole organization, I just want to use it on a small dedicated area in a narrow area.

Next, I will present a very simplified version of the service-resource model. Why such a simplification? Just until I feel the need to introduce more entities to understand the work of the entire system.

Basics and terms.

The system is a combination of our entire IT economy, which is installed on our servers.

Service - any application, be it a web server or database (hereinafter application = service)

Resource - the opportunity provided by the service to use it from outside the service.

Dependency - the connection of the service with any resource of another service.

Those. the system can be represented as a set of services, some of which provide resources, and the other uses them.

It must be understood that dependence on a resource is not equal to the flow of data. Those. if our application, for example, depends on a message server, then it both receives and sends messages through a message server.

Let's draw a simple concept schema.



For example, we have two hosts, one has a node.js application and a database server, another has PowerDNS server and a database server. Our application on node.js when creating a new account adds the corresponding 3rd level domain, recording entries in the PowerDNS database.

Then you can add and increase the number of services and their dependencies. The principle is clear.

Instruments

I used to draw similar schemes in Gliffy.com . But now increasingly I come across the need to keep this scheme up to date. But in the gliffy links in the diagram, it’s just a line and rectangle connection. Therefore, there is a search for a tool to store the structure of relationships and display these relationships on the diagram.

Here is an example of my scheme in Gliffy

Sorry, please, for the quality.

Deedoo

While the search is underway (and I hope that you, dear readers, tell me in the comments any tools, maybe something similar has already happened on Habré, I haven’t found it), as a prototype my colleague and I sketched a small deedoo application. In it, we store information about hosts, services, their resources and dependencies, as well as build a dependency scheme. The scheme is constructed using the Joint.js library.

The github project: github.com/antirek/deedoo Installation instructions are in the repository. With the help of deedoo the scheme with PowerDNS is drawn above.

Here, for example, how dependencies of a node.js application look in the deedoo web interface


While the Deedoo application, in addition to drawing dependencies, is no longer able to. But there is still an idea to use it as a settings server. After all, if we know that our arbitrary application depends on some resources, let it require them for its work.

Having written something like config = serverConf ('http://config.server.ru') .appID ('super_key') .get (); we can get all the necessary application settings when it starts. In the absence of the required parameters in the loaded config, we simply do not start the application. For example, when transferring a database to another host, we only need to reload to get the settings by the application, right?

What's next?

I don't know yet. On the one hand, I want it more universally, on the other - practical. After all, the goal is to make the development of the system clearer and more transparent. On the one hand, it backs up monitoring, on the other hand, configuration management. Those. to this list of services - I want to see their metrics of work, status, performance. And I would also like to click a button and update the parameters of the database, applications, etc.

So, come to the most key point of this note: what to do next? Options:

a) do it all seriously - implement ITSM / ITIL, choosing a finished product? (maybe there are excellent free & open source solutions? Internet deployment descriptions?)

b) develop your bike? Those. use a service that describes all the dependencies. And new developed applications use loading settings from this service. (I, in general, like the idea of ​​getting any settings from any application from the settings server, adding groups, types - and generally a robust system will turn out, IMHO. Moreover, getting the settings will link these declared dependencies in deedoo with the real applications that run on the servers. But Can it somehow be done differently? Maybe there are already similar systems?)

c) other?

In general, everything turned out a bit messy and mixed, because there is also about the development of an online system and its operation. Plus some mess in the terminology. I shared my thoughts and ideas, I hope for a constructive feedback from the community on the issues of the note.

Upd.

Infrastructure issues worry not only me. Of the interesting projects I can mention terraform.io (in conjunction with consul.io) and weave.works .