New TFS discovery

What is the first association that occurs when you hear the phrase Microsoft TFS? Something big, clumsy and corporate. This is exactly what happened before the appearance of Visual Studio Team Services and the release of MS TFS 2015. The first is the cloud version of Team Foundation Foundation, which is ahead of the development of the private version by about three months. One of the main innovations of the updated TFS / VSTS has become a new build system. This system allows you to simply write your assembly steps that can do anything from the actual project assembly to the automatic introduction of defects and the sending of notifications. In addition, the new version provides an advanced REST API for manipulating tasks, defects, and virtually any entities in the TFS database.

That is why when we faced the choice of a new development lifecycle management system, we stopped on this new version of MS TFS. We use TFS for a full cycle - planning-development-testing-deployment, and, at first, everything went quite smoothly. With the increasing complexity of the tasks that we set before the assembly system, there were problems. Fortunately, the REST API and its own build steps allowed them to be successfully solved. Next, I will talk about the problems and how we solved them.

How not to go out the window when you need more tests

We needed an assembly that runs autotests. Simply? But the idea was to combine several test runs on different systems into it and display a single test report. The solution is to make an assembly with several test runs. Everything was fine until we started to get out of the time window - the test launches were performed sequentially one after the other. And there was no out-of-box solution for parallelizing assemblies. And a simple idea came - a master assembly:

• runs separate builds on other agents (in parallel)
• waiting for their completion
• picks up their test results if there is

From the implementation of this idea, the Parallel Builds extension was born.

To ensure concurrency, the extension contains 2 assembly steps:

1. Starter - runs the listed assembly definitions. Each build starts with its own settings. This allows you to completely isolate different assemblies, use different agents and different environment variables.
2. Awaiter - waits for running assemblies and collects their test results. In addition, he adds to the Summary page of the current assembly links to the "original" assembly. This is necessary first of all in order to be able to view the console output and the logs of these assemblies in case of problems.

In the simplest case, the master assembly consists of only two steps:

The extension works in both cloud-based VSTS and private TFS. Written in typescript therefore requires agent version 2.0.

Let the defects create a robot - it is iron

Test automation, it is not in the number of autotests, but in the heads. Therefore, after the third consecutive analysis of failed tests in test launches, it was decided to shift this “intellectual” work to the robot. Another extension? Exactly. The idea was as follows:

• take the test run results
• get the failure test name
• get an error message
• create a defect and assign it to the person responsible for this test run

So in the expansion of Parallel Builds appeared step - AutoDefects.

The automatic creation of defects makes it possible to ensure the obligatory reaction to the test failure, track the life cycle and collect statistics on the type of failure - autotest test, media deployment or functional defect of the system under test.

Jenkins does not share the results - correct

We develop in cross-functional teams and the development process allows teams to use their development tools. With one condition - they must be integrated with TFS. Some teams use Jenkins to build for various reasons. The current version of TFS and Jenkins integration allows you to run the build on Jenkins and wait for it to complete. But, unfortunately, does not import the results of the tests in this assembly.

Fortunately, more recently, Microsoft supports the free software movement and publishes some of its development on GitHub. Assembly tasks for TFS among them

And here is the pull request , which adds to JenkinsQueueJob the functionality of importing results from Jenkins. In addition, it allows you to add links (relative tasks in Jenkins) to the Build Summary page in TFS. For example, you can add a link to the artifacts of this assembly, which are stored on Jenkins server or additional reports, for example, Yandex.Allure

The new version of TFS / VSTS allows you to customize yourself for a completely different tasks and no longer looks like the monster that TFS seemed like before. And given that for small teams the use of VSTS for free, it can be a tool not only for corporations, but also for startups.

As always, source code is available on github