Apply Extensible Pull Request Policies to VSTS to Support the Development Process

Often, as part of the Pull Request check, in addition to the code review itself, there is a need to perform a set of routine checks. Some checks may concern PR design. Others are to check the related conditions that form the basis of the change acceptance process.
If routine checks are not automated, a person can start to forget or bypass them. Because routine is boring.

Visual Studio Team Services offers a fairly convenient infrastructure for handling pull requests. These include customizable merge builds policies, the appointment of reviewers, and the rules for merging changes that are accepted. All this is supplemented with a convenient system for discussing and commenting on the code.

External plug-in policies are a powerful tool for expanding the pull request process.

About their creation and use and talk (and see the code)

Extendable Pull Request statuses

PR statuses are flags defined within the context. Context is a unique combination of the name of the context and the genre. The genre is usually one on the application that manipulates the status.

For example:

Status 1: genre = 'my-policy', name = 'check-1'
Status 2: genre = 'my-policy', name = 'check-2'

The status accepts one of the preset values. In order to support the process, we will be interested in two: succeeded and failed.

Statuses can be used when setting up the brunch policy: the selected flags must have the status succeeded in order for the PR to be accepted. The built-in policies that check the number of appruvers, the presence of an attached ticket, and so on are implemented in exactly the same way.

Go to the policy editing branches. Add foreign policy.

If the status has been set at least once, it will be available in the drop-down menu.
At the bottom, you can specify the title for the status, as it will be displayed in the block of the pull request.

The added statuses will be visible on the pull request page in the checks block:

If the status is not used as a policy, its value is displayed on the page in the Status section. If the status is specified as a policy, it will be visible at the top of the block.

Software status management

PR statuses can be manipulated programmatically using the REST API. Thus, it is possible to implement additional PR checks and broadcast their result directly to the change acceptance process.

The new status value is added by the Create method. In addition to the result and context, you must pass the text that the user sees. Additionally, you can send the URL: in this case, the status label on the PR form will become a link and the user can go to the page with the status details.

Calling the method results in creating a new PR status record. Within one context, the last added status value is considered active. Earlier status records are not visible from the UI interface, but they can be obtained using the List method.

For the status of the statuses in the previous picture, the real list of statuses for a pull request may be as follows:

{ "value": [ { "id": 1, "state": "failed", "description": "PR title format", "context": "@{name=check-title; genre=my-pr-policy}", "creationDate": "2018-11-06T18:35:57.0324172Z", "updatedDate": "2018-11-06T18:35:57.0324172Z", "createdBy": "Masked value", "targetUrl": null }, { "id": 2, "state": "failed", "description": "Build for last update", "context": "@{name=check-build; genre=my-pr-policy}", "creationDate": "2018-11-06T18:35:57.5167963Z", "updatedDate": "2018-11-06T18:35:57.5167963Z", "createdBy": "Masked value", "targetUrl": null }, { "id": 3, "state": "succeeded", "description": "No offset from develop", "context": "@{name=check-offset; genre=my-pr-policy}", "creationDate": "2018-11-06T18:35:57.782379Z", "updatedDate": "2018-11-06T18:35:57.782379Z", "createdBy": "Masked value", "targetUrl": null }, { "id": 4, "state": "succeeded", "description": "PR title format", "context": "@{name=check-title; genre=my-pr-policy}", "creationDate": "2018-11-06T18:46:37.2627154Z", "updatedDate": "2018-11-06T18:46:37.2627154Z", "createdBy": "Masked value", "targetUrl": null }, { "id": 5, "state": "succeeded", "description": "Build for last update", "context": "@{name=check-build; genre=my-pr-policy}", "creationDate": "2018-11-06T18:51:33.7920543Z", "updatedDate": "2018-11-06T18:51:33.7920543Z", "createdBy": "Masked value", "targetUrl": null }, { "id": 6, "state": "failed", "description": "PR title format", "context": "@{name=check-title; genre=my-pr-policy}", "creationDate": "2018-11-06T18:53:44.3075889Z", "updatedDate": "2018-11-06T18:53:44.3075889Z", "createdBy": "Masked value", "targetUrl": null }, { "id": 7, "state": "failed", "description": "Title format is not correct", "context": "@{name=check-title; genre=my-pr-policy}", "creationDate": "2018-11-06T19:26:11.3019433Z", "updatedDate": "2018-11-06T19:26:11.3019433Z", "createdBy": "Masked value", "targetUrl": null } ], "count": 7 } 

After viewing the list of current statuses, you can update the selected by the index in the list. To do this, use the Update method.

Finally, status records can be deleted using the Delete method.

A history of PR status changes can be useful for further analysis. Therefore, we use the following method of status change:

• Find the most recent status record of the same context.
• If she has the same result values, descriptions and links that we want to add, we do nothing
• Otherwise, add a new status record.
  This allows you to keep a history of changes, not much inflating it.

 function Set-PullRequestStatus { param( [Parameter (Mandatory = $true)] [string] $pullRequestId, [Parameter (Mandatory = $true)] [string] $state, [Parameter (Mandatory = $true)] [string] $description, [Parameter (Mandatory = $true)] [string] $contextName, [Parameter (Mandatory = $false)] [string] $contextGenre, [Parameter (Mandatory = $false)] [string] $targetUrl, [Parameter (Mandatory = $true)] [object] $context ) $b = @{ state = $state; description = $description; context = @{ name = $contextName; genre = $contextGenre; }; targetUrl = $targetUrl; } $body = ConvertTo-Json $b # # Get current list of statuses # $endpoint = (Get-ProjectBaseURL) + "/_apis/git/repositories/{repositoryId}/pullRequests/{pullRequestId}/statuses?api-version=4.1-preview.1" $res = Get-AzureRequestReqults -URI $endpoint -context ($context + @{pullRequestId = $pullRequestId}) # # Try to find a status for a given context genre and name. Start looking from the last one. If found - check if it has same values. # $i = $res.count $foundSameStatus = $false while ($i -GT 0) { $r = $res.value[$i-1] if (($r.context.name -EQ $contextName) -AND ($r.context.genre -EQ $contextGenre)) { $foundSameStatus = ($r.state -EQ $state) -AND ($r.description -EQ $description) -AND ($r.targetUrl -EQ $targetUrl) break } $i-- } $res = $r # # If same status / values was not found - add new record. # if (-not $foundSameStatus) { $endpoint = (Get-ProjectBaseURL) + "/_apis/git/repositories/{repositoryId}/pullRequests/{pullRequestId}/statuses?api-version=4.1-preview.1" $res = Get-AzureRequestReqults -URI $endpoint -context ($context + @{pullRequestId = $pullRequestId}) -method POST -query $body } return @{status = $res; status_changed = $(-not $foundSameStatus)} } 

Practical use

We have adopted a rather conservative approach to adopting changes in the integration branch. We try to test the change as far as possible on the feature branch.

Here are some tasks that we solve with the help of PR statuses within the framework of customizable policies:

1. All PRy must be decorated in the same style. Therefore, the first control that was created is the design of the PR header. We check it for regular expression matching.
2. The branch of PR should not lag behind the branch where it is being poured in (integration has been performed).
3. If within the framework of PR the database project files are changed, then autotest files must also be present.
4. There is a successful build branch on the last change.
5. Such an assembly was successfully rolled out to a test bench and autotests were completed.

The listed conditions can be checked manually. We did it before. But this is routine, and it has been replaced by an automated process. Now we can supplement and change the set of checks - it will always be integrated into the process, always executed.

An additional huge plus is the politician - they are transparent to everyone, and always in sight - on the PR page. They no longer need to be reminded.
In addition, for policies that have not passed the test, we show a link to our Wiki pages with a description of the policy and the expected actions.

Technical implementation of policy implementation

At the moment, the logic of policies is implemented in the form of powershell scripts. Due to high-level cmdlets and a good data object model, the script code is very compact. And the ability to run the script step-by-step interactively and poking around in variables greatly simplifies the development and debugging process.

By the way, after the release of powershell core - scripts can be run on other platforms (tested on MacOS).

We run scripts in a special release of VSTS on a schedule about once every couple of hours. Maybe we will start to run through the scheduler more often.

This approach, of course, gives a significantly slower response than if you implement the same in the form of the Azure Function and link it to VSTS via a web hook. But for us it was more important to implement the checks and see how it will work in the process (MVP). The speed of the checks is not important yet. This will be the next step.

The implementation of the VSTS REST API libraries used in the checks, plus a small example that implements some of these policies can be viewed in the repository on GitHub . I hope someone will be useful.