Review Board + Mercurial - experience in implementing and automating the process code review

Some time ago in the company where I work in connection with the expansion of the team, it was decided to introduce the code review process. The choice of instrument fell on the Review Board - the product has sufficient functionality, has been actively developed since 2006 and is open source. We use Mercurial as our version control system.

About what tasks the review code for the bundle Review Board + Mercurial - under the cat faced with when organizing the process.

And so, we began with server deployment. Unfortunately, information on server requirements on the RB website could not be found. After getting acquainted with the results of the search on the topic, it became clear that the main parameter in which the RB “rests” is the RAM. Most often, the figure “4GB” was mentioned as a “comfortable volume”. We started with 2, but the RB didn’t “fly up” on them - while viewing diffs, the error “Can not allocate memory” now and again crashed. Raised to 4GB and everything went like clockwork.

Then began the process of setting up and introducing the process of the code review on the docks off. site.
')
Everything went more or less smoothly until we reached the choice of a tool for publishing a review. The choice was between the official tool from the Review Board team - post-review and the plugin for Mercurial
The official tool has a clear advantage - it is supported by Review Board developers, while its main drawback is that it does not have a GUI. The plugin is sharpened specifically for Mercurial, so it has some advantages over post-review, but its main advantage is the integration into TortoiseHg GUI. Since most developers are sitting on TortoiseHg, it was decided to use the plugin.
Acquainted with the dock plugin - everything seems to be smooth. The main version was downloaded (another alternative is offered on the page), which at that moment was still hosted by Google Code . We tried to use it - and it was not there - it was in a very deplorable state - bugs were crawling, and compatibility with the latest versions of Mercurial was not complete. They put an alternative - things went better, but she didn’t have full compatibility with Mercurial> = 2.3 either. But we did not lose hope and by digging in forks, a worker was discovered! But there were problems with it - for some reason, some users constantly dumped a 404 error when trying to post a revision, both from under a virtual machine with Unix, and from under a “real” piece of hardware with Windows. There were still some minor glitches, now I do not remember. At that time, there was no time to deal with this, so they used what they had.

This went on for a good half a year. During this time, we realized that the formed posting revision process takes a lot of time, and some users forgot (and some were killed) to post a review. Therefore, it was decided to automate the process as much as possible.

Rush information about this, the following plan was adopted:

• Clone on the server running 24/7 our repository
• We launch on schedule pull repository
• We hang on the incoming Mercurial-a hook with a script that will automatically post a review for each “tightened” commit

In the process of thinking through the implementation, we realized that we would need the opportunity to post a review on behalf of another user. The plug-in to Mercurial did not know how. In the hope of a miracle, we go to the plug- in page on the Mercury website and ... do not believe our eyes - the plug-in came to life and began to develop! The official repository moved to bitbucket , the plugin now works with the latest version of Mercurial, and also supports posting a review on behalf of another user. Inspired, started to implement.

So, how it is implemented (all actions occur on the server c CentOS 6.3).

1. We're putting our repo in the folder ~ / home / viktor / autoreview / gdam_frontend /
2. Let's get the plugin for RB in the folder ~ / home / Viktor / autoreview / hgreviewboard /
3. In the Review Board, we create a user with the intuitive username review_bot and give him the right to post a review on behalf of another user (“Can submit as user” permission)
4. Go to the configuration file of the repository /home/viktor/gdam_frontend/.hg/hgrc and add the following lines there:
   

   [extensions]
   reviewboard = / home / Viktor / autoreview / hgreviewboard
   
   [reviewboard]
   # Review Board Repository ID, details - www.reviewboard.org/docs/manual/1.7/admin/configuration/repositories
   repoid = 1
   
   # Review Board Review Group, details - www.reviewboard.org/docs/manual/1.7/admin/configuration/review-groups
   target_groups = A5_Reviewers
   
   # Review Board Site, details - www.reviewboard.org/docs/manual/1.7/admin/installation/creating-sites
   server = reviews / revievboard
   user = review_bot
   password = bot_password
   
   [hooks]
   # Mercurial incoming hook, details - hgbook.red-bean.com/read/handling-repository-events-with-hooks.html#sec : hook: incoming
   incoming.autoreview = /home/viktor/autoreview/a5.sh $ HG_NODE

   
   
5. Now we create the script /home/viktor/autoreview/a5.sh hung on incoming hook:
   

   function log { echo '>> '$1 echo `date`' >> ' $1 >> $script_dir/autoreview.log } script_path=`readlink -f "$0"` script_dir=`dirname "$script_path"` revision=$1 log "Processing revision $revision" messages_file='/home/viktor/autoreview/messages.txt' cd /home/viktor/autoreview/gdam_frontend user=`hg log -r $revision | grep 'user:' | sed -r 's/^user:\s{0,}//g'` log "Revision $revision is made by $user " user_found=`grep "^'$user'" $script_dir/users.txt` if [ "$user_found" != "" ] then commit_message=`hg log -r $revision | grep 'summary:' | sed -r 's/summary:\s{1,}//g'` message_found=`grep "$commit_message" "$messages_file"` if [ "$message_found" == "" ] then merge_commit=`echo $commit_message | grep '^\s\{0,\}[M|m]erge'` if [ "$merge_commit" == "" ] then echo $commit_message >> "$messages_file" rb_user=`echo $user_found | sed -r "s/^'$user'\s//g"` log "Found revision board user - $rb_user" review_id=`echo $commit_message | grep -o '[Rr][Bb]-[[:digit:]]\{1,\}' | sed -r 's/^[R|r][B|b]-//g'` if [ "$review_id" != "" ] then log "Updating review $review_id" hg postreview -e $review_id --submit_as $rb_user -p $revision else log "Posting new review for $revision" hg postreview --submit_as $rb_user -p $revision fi else log "Skipping merge revision $revision ($commit_message)" fi else log "Skipping already processed revision $revision ($commit_message)" fi else log "User $user is not listed in users.txt mapping file" fi 

   
   Let's go through the key points of the script:
   
   
   1. Determine the directory where the script is located
      

       script_path=`readlink -f "$0"` script_dir=`dirname "$script_path"` 

   2. We define the hash of the processed commit, it comes as the $ HG_NODE parameter with the incoming hook
      

       revision=$1 

   3. We define a file with commit messages, used to skip commits with the same messages (often, these appear when there are graft-e commits)
      

       messages_file='/home/viktor/autoreview/messages.txt' 

   4. Determine the user who made the commit (output commit info (hg log -r $ revision) + grep + sed to “pull out” the user)
      

       user=`hg log -r $revision | grep 'user:' | sed -r 's/^user:\s{0,}//g'` 

   5. We define the hash of the processed commit, it comes as the $ HG_NODE parameter with the incoming hook
      

       revision=$1 

   6. We determine the line matching the user of the Review Board to the user who made the commit. It is determined by searching in a special file - users.txt, which looks like this:
      

        'Vasya Pupkin <Vasya.Pupkin@example.com>' vasya-p
       'Ivan Petrov <Ivan.Petrov@example.com>' ivan-p 

      For the first line: Vasya Pupkin <Vasya.Pupkin@example.com> is a user registered in commits, vasya-p is the login of the corresponding user in the Review Board
      

       user_found=`grep "^'$user'" $script_dir/users.txt` 

   7. Rip out a commit message
      

       commit_message=`hg log -r $revision | grep 'summary:' | sed -r 's/summary:\s{1,}//g'` 

   8. Check if there is such a message in the file messages_file
      

       message_found=`grep "$commit_message" "$messages_file"`) 

   9. Check if it is merge
      

       merge_commit=`echo $commit_message | grep '^\s\{0,\}[M|m]erge'`) 

   10. We write commit in message_file
       

        echo $commit_message >> "$messages_file" 

   11. We rip the user’s RB login from the match string
       

        rb_user=`echo $user_found | sed -r "s/^'$user'\s//g"` 

   12. We check if it is an update of an existing review - agreement of updates to designate RB - {{id}} in a commit message, where {{id}} is an ID review in RB
       

        review_id=`echo $commit_message | grep -o '[Rr][Bb]-[[:digit:]]\{1,\}' | sed -r 's/^[R|r][B|b]-//g'` 

   13. Post review
       

        hg postreview --submit_as $rb_user -p $revision 

       or update existing
       

        hg postreview -e $review_id --submit_as $rb_user -p $revision 

   
   
6. Create a script that will pull our repository on schedule /home/viktor/autoreview/pull_a5.sh
   

    message_file='/home/viktor/autoreview/messages.txt' echo '' > "$message_file" cd '/home/viktor/autoreview/gdam_frontend' hg pull rm $message_file 

   
   Please note that in this file we create empty messages.txt, which is used in a5.sh to determine if the commit is repeated in the current pull. After running the pool, we delete this file.
   
7. Add the created script to the cron schedule . An example to run every 5 minutes:
   

     0,5,10,15,20,25,30,35,40,45,50,55 * * * * /home/viktor/autoreview/pull_a5.sh 

   in comments suggested a more optimal recording option:

     * / 5 * * * * /home/viktor/autoreview/pull_a5.sh 

At the output we have the following directory structure autoreview:

/ home / Viktor / autoreview
- / gdam_frontend - clone of the repository, for which we organize code review
- / hgreviewboard - Review Board Plugin for Mercurial
- * a5.sh - a script that posts a review of rickshaws.
--autoreview.log - a5.sh script action log
- * pull_a5.sh - the script that executes the pull repository is launched according to a schedule using cron
--users.txt - Mercurial Users Match File to Review Board Users

* means that the file has an "executable" flag.

Moreover, the process of posting a review of rickstavs is automated and takes place "unnoticed" for committers. The only thing that committers need to do is when committing changes for a review of a ticket, a commit message must begin with a RB - {{ID}}, where {{ID}} is an ID of a review of a ticket.

What can be improved?

1. Filter out branch closing commits (by analogy with merzes)
2. Filter backout commits (by analogy with merzes)
3. Improve the verification of the uniqueness of commit messages so that it occurs without using the messages.txt file (there are no ideas how to do this yet)

We are happy to review all comments and comments.
Thank you for reading to the end!