Continuous integration with Jabber. Inventing a bicycle?

In any serious project at some point, the question arises of organizing the CI process and, therefore, the question of choosing the means to automate it from a wide range of those currently on the market. It often happens that with all the wealth of choice it is not easy to find a suitable option, and sometimes a ready-made solution is simply too complicated, and there is no desire or opportunity to allocate resources to support it. In addition, I do not want to put all my eggs in one basket and have a single server for building, testing, storing distributions ...

An idea was born in our company - to support the global trend of decentralization and to divide the CI server into several machines, connecting them with each other by a universal protocol that is accessible to both the machine and the person. Looks like a bicycle invention? Not at all.

The XMPP protocol, also known as Jabber, which was the company's internal communication standard, perfectly suited the role of such a means of communication, and therefore, every developer, tester, and any other CI participant had Jabber-enabled IM in his workplace. Accordingly, any ready-made CI solution with Jabber-plug-in support (and almost all continuous integration servers now have one) easily fits into the general scheme.

To implement this idea, we did the following.
')
• The buildserver, testserver, deployserver, and hudson server accounts were registered in the Jabber roster, created for each group for which the “users” data are available for communication, filled these groups, thus defining a list of real users who would have access to one or another functionality.

They wrote a language of communication (for the first time - with a minimal set of commands), which can be expanded to implement new tasks and is easy to learn for the end user. Commands: build, test, deploy, kill, ban, approve, help. For all these teams, in the process of implementing CI, parameters were added that allowed them to significantly expand their functionality.

• Developed a template for the CI element that uses the jabber-net.dll library containing the handlers for incoming commands and sending outgoing commands based on the results of these handlers.

Sample template in C #:

using System; namespace Jabber { public class Jabberclnt : IDisposable { private jabber.client.JabberClient clnt = null; ~Jabberclnt() { Dispose(false); } public void Dispose() { Dispose(true); GC.SuppressFinalize(this); } public void Dispose(bool disposing) { if (clnt != null) { clnt.Close(); clnt.Dispose(); clnt = null; } } public void Init(string user, string pwd) { clnt = new jabber.client.JabberClient (); clnt.PlaintextAuth = true; clnt.Server = "pt.int"; clnt.NetworkHost = "jabber.ptsecurity.ru"; clnt.Port = 5222; clnt.User = user; clnt.Password = pwd; clnt.AutoLogin = false; clnt.OnMessage += clnt_OnMessage; clnt.Connect (); } void clnt_OnMessage (object sender, jabber.protocol.client.Message msg) { if (null == msg.Body) return; Console.WriteLine(string.Format("[{0}] {1}:{2}", DateTime.Now.ToString(), msg.From.User, msg.Body)); //  } public bool SendMessage(string user, string msg) { bool res = true; try { var jabberuser =new jabber.JID(user); var jabberMessage = new jabber.protocol.client.Message(clnt.Document) { Body = msg, To = jabberuser }; clnt.Write(jabberMessage); } catch (System.Exception ex) { res = false; } return res; } } class Program { static void Main(string[] args) { Jabberclnt clnt = new Jabberclnt(); clnt.Init("username", "password"); System.Threading.Thread.Sleep(1000); clnt.SendMessage("buildserver@pt.int", "test"); System.Threading.Thread.Sleep(1000); } } } 

• Raised the Hudson server with im and jabber plugins - to be able to communicate with other participants.

By implementing this idea, we got the following scheme:



In this scheme, each module is independent and performs separate functions without using the resources of other servers and not forcing other participants to wait in the queue. Developers can give commands like BuildServer with the subsequent execution of tests, or immediately directly to TestServer, if they have a need to conduct an additional set of tests after assembly.

As I have already noted, the developer receives, in the form of IM with Jabber (we have Miranda), a ready-made tool for managing and notifying any server - taking into account, of course, the availability of access rights to it, specified in the Jabber roster.

The scheme of interaction of participants in this chain with a full cycle is as follows.
The developer initiates the process.



BuildServer reports success and transfers control to TestServer.



After running the tests, a success message also follows, and control is transferred to the deployment server.



At the end of the cycle, the person who started the process is notified - and the list of interested parties.



However, it is not always necessary to initiate a full cycle: it is often necessary to perform some one action from the chain, and in this case the scheme of divided servers works especially effectively.



So, this scheme helps to solve the following series of questions.

• There is no limit to the choice of ready-made CI in various projects and divisions: they are easily integrated into the overall structure of the company.
• Resource Sharing: The buildserver load does not affect testing and deployment tasks.
• A ready control tool (notification, automation) using the jabber CI library is available from any application, library or script.
• A single command format allows you to see the full picture of what is happening now in CI, and allows the user to fit in any stage, simulating any server and thus maintaining the purity of the process.

We invite you to discuss the advantages and disadvantages of this approach. If you are interested in the details - ask questions: share experiences :)

PS A report on this topic was presented at Application Developer Days ( presentation and video ).