The key to all doors in continuous integration - rundeck
With a large number of servers and virtual machines and even more code in constant deployment, inevitably there are problems in administering this huge farm. There are many tools to organize continuous integration. In our list we already have GIT, Jenkins, Chef, Proxmox, Graylog2. Today we will talk about another handy tool for automating routine tasks using scripts - rundeck . This article is not a detailed manual with examples of configs, but rather reflections on the topic.
Rundeck is an open source project that will allow a system administrator to organize a server of scenarios, automating routine and sample work. In fact, this is the key to all doors. It allows you to execute scripts or shell commands on any number of nodes, monitor their execution and deployed to tell the administrator about each action, whether it is successful or not. Rundeck combines all the tools already used and allows you to organize a transparent interaction between them.
')
- distributed execution of commands and scripts with parameters,
- support of several mechanisms for interacting with the node (ssh by default) and the ability to write your own in the form of plug-ins,
- the script can be described in several steps,
- graphical management interface and console utilities for configuration,
- separation of access rights, integration with LDAP / AD,
- history and audit scenarios and commands
- integration with other continuous integration tools - jenkins, chef,
- API,
- the ability to take all the parameters and settings by url from a remote site.
- this is a server and this is java. Here, all the logic, authentication and access rights for users, console management utilities and web management interface (though not very convenient),
- scenarios, audit results and all logs are stored in the database,
- for interaction with nodes it is used ssh on keys. No reverse ssh connection node-server is required.
It works almost everywhere:
- linux
- Windows XP, Server and newer,
- Mac OS X 10.4 and newer.
Installation and configuration is trivial, all configs are stored in files.
Our usage example, integration with chef. Problems and solutions.
A small lyrical digression about chef . Recipes, nodes, environment - it all works in theory with almost no problems. Problems begin in real production. The connectivity of the system components is unexpectedly high, some modules are very dependent on others. In terms of geo-cluster and distributed data centers, many factors must be taken into account. Network speed, node access speed from remote geographical points, time zone. For large audiences, split deploy can be used when a new chip is shown only to certain regions and only at a certain time.
In such a reality, chef ceases to save and harms more. Recipes are getting too much. Places where you can make more and more. Parameters, environments and flags to recipes - more than recipes. We have to get long and voluminous tables, where the scheme of our military actions will be described with a multi-colored felt-tip pen. Where, where and in what time we are apprehended and why. Once we still split the wrong place, which led to the understanding that chef also needs to be automated and that they also need to be managed.
There was a lot of logic in the recipes to overcome the high coherence of the components. The node turns off from upstream, prepares its code and starts to deploy. At some specific step, it is necessary to change the state of another module, for example, to reindex the data for a new code. After reindexing, we immediately extinguish another node and prepare it for split testing. At this time, the first node finishes its deployment and passes the final testing, after which it is included in the upstream. At the same time, the database should already be ready, where a million transactions have been updated. All this must be done almost simultaneously, while almost simultaneously. Chef seems to know how, but this is inconvenient and you have to use too much logic in the recipes. Exchange signals, common attributes, flags and understand at each stage what state we have neighbors and what to do next.
What gives us a rundeck.
1. Create a project
2. Import there all the nodes from the chef with all the recipes. For this there is a ready tool .
3. We create scenarios that will do everything themselves and see that the connectivity has not broken during the deployment. They will turn on, turn off, synchronize, kick the chef and report to the administrator about each step they take.
4. Profit.
To do this, you need to configure key authentication. Understand what and how an unprivileged user should do, configure sudo. Or cram it in chroot. Well, write a bunch of magic scripts in any language. Perl, python, bash, php))) You can also turn on script stop after an unsuccessful step. Scripts can write whatever you like. It all depends on the imagination and the challenges you face.
Rundeck can work in two modes. Tasks can be performed either in parallel on all nodes at once, or sequentially, finished with one - go to the other.
Node-oriented
1. NodeA step # 1
2. "step # 2
3. "step # 3
4. NodeB step # 1
5. "step # 2
6. "step # 3
Step oriented
1. NodeA step # 1
2. NodeB "
3. NodeA step # 2
4. NodeB "
5. NodeA step # 1
6. NodeB "
Rundeck can execute commands and scripts with arguments, and take them in turn from a remote url, like the rest of the settings. Also, using the API, you can manage only those things that are needed without opening the webUI. In addition, it is not very convenient here.
The big advantage of this solution is that the state of the system as a whole and separately at any time is controlled from one point. And can be forcibly changed, also from one point. Without the need for ssh-megaton servers.
Of the minuses - another component of the series is monitoring the monitoring of the monitoring and managing the deployment of the deployment of the deployment system. Complication is almost always bad. It is necessary to clearly understand the scheme of the entire system and components separately. Otherwise there is a chance to write such miracles that even dismissal will not be enough punishment. You also need to clearly and clearly build a security scheme. And allow the user only what needs to be allowed. The key to all the doors is convenient, but it doesn’t seem so that “we are super scientists and have invented a vigorous bomb. But if she gets into the wrong ruuki ... "
Links
- chef
- Rundeck
Rundeck is an open source project that will allow a system administrator to organize a server of scenarios, automating routine and sample work. In fact, this is the key to all doors. It allows you to execute scripts or shell commands on any number of nodes, monitor their execution and deployed to tell the administrator about each action, whether it is successful or not. Rundeck combines all the tools already used and allows you to organize a transparent interaction between them.
')
Features:
- distributed execution of commands and scripts with parameters,
- support of several mechanisms for interacting with the node (ssh by default) and the ability to write your own in the form of plug-ins,
- the script can be described in several steps,
- graphical management interface and console utilities for configuration,
- separation of access rights, integration with LDAP / AD,
- history and audit scenarios and commands
- integration with other continuous integration tools - jenkins, chef,
- API,
- the ability to take all the parameters and settings by url from a remote site.
Architecture:
- this is a server and this is java. Here, all the logic, authentication and access rights for users, console management utilities and web management interface (though not very convenient),
- scenarios, audit results and all logs are stored in the database,
- for interaction with nodes it is used ssh on keys. No reverse ssh connection node-server is required.
It works almost everywhere:
- linux
- Windows XP, Server and newer,
- Mac OS X 10.4 and newer.
Installation and configuration is trivial, all configs are stored in files.
Our usage example, integration with chef. Problems and solutions.
A small lyrical digression about chef . Recipes, nodes, environment - it all works in theory with almost no problems. Problems begin in real production. The connectivity of the system components is unexpectedly high, some modules are very dependent on others. In terms of geo-cluster and distributed data centers, many factors must be taken into account. Network speed, node access speed from remote geographical points, time zone. For large audiences, split deploy can be used when a new chip is shown only to certain regions and only at a certain time.
In such a reality, chef ceases to save and harms more. Recipes are getting too much. Places where you can make more and more. Parameters, environments and flags to recipes - more than recipes. We have to get long and voluminous tables, where the scheme of our military actions will be described with a multi-colored felt-tip pen. Where, where and in what time we are apprehended and why. Once we still split the wrong place, which led to the understanding that chef also needs to be automated and that they also need to be managed.
There was a lot of logic in the recipes to overcome the high coherence of the components. The node turns off from upstream, prepares its code and starts to deploy. At some specific step, it is necessary to change the state of another module, for example, to reindex the data for a new code. After reindexing, we immediately extinguish another node and prepare it for split testing. At this time, the first node finishes its deployment and passes the final testing, after which it is included in the upstream. At the same time, the database should already be ready, where a million transactions have been updated. All this must be done almost simultaneously, while almost simultaneously. Chef seems to know how, but this is inconvenient and you have to use too much logic in the recipes. Exchange signals, common attributes, flags and understand at each stage what state we have neighbors and what to do next.
What gives us a rundeck.
1. Create a project
2. Import there all the nodes from the chef with all the recipes. For this there is a ready tool .
3. We create scenarios that will do everything themselves and see that the connectivity has not broken during the deployment. They will turn on, turn off, synchronize, kick the chef and report to the administrator about each step they take.
4. Profit.
To do this, you need to configure key authentication. Understand what and how an unprivileged user should do, configure sudo. Or cram it in chroot. Well, write a bunch of magic scripts in any language. Perl, python, bash, php))) You can also turn on script stop after an unsuccessful step. Scripts can write whatever you like. It all depends on the imagination and the challenges you face.
Rundeck can work in two modes. Tasks can be performed either in parallel on all nodes at once, or sequentially, finished with one - go to the other.
Node-oriented
1. NodeA step # 1
2. "step # 2
3. "step # 3
4. NodeB step # 1
5. "step # 2
6. "step # 3
Step oriented
1. NodeA step # 1
2. NodeB "
3. NodeA step # 2
4. NodeB "
5. NodeA step # 1
6. NodeB "
Rundeck can execute commands and scripts with arguments, and take them in turn from a remote url, like the rest of the settings. Also, using the API, you can manage only those things that are needed without opening the webUI. In addition, it is not very convenient here.
The big advantage of this solution is that the state of the system as a whole and separately at any time is controlled from one point. And can be forcibly changed, also from one point. Without the need for ssh-megaton servers.
Of the minuses - another component of the series is monitoring the monitoring of the monitoring and managing the deployment of the deployment of the deployment system. Complication is almost always bad. It is necessary to clearly understand the scheme of the entire system and components separately. Otherwise there is a chance to write such miracles that even dismissal will not be enough punishment. You also need to clearly and clearly build a security scheme. And allow the user only what needs to be allowed. The key to all the doors is convenient, but it doesn’t seem so that “we are super scientists and have invented a vigorous bomb. But if she gets into the wrong ruuki ... "
Links
- chef
- Rundeck
Source: https://habr.com/ru/post/162447/
All Articles