Ruby code coverage analysis

First, I will give a small test project of three classes, analyze its coverage using the SimpleCov gem , and finally, I will think a little about how coverage analysis can benefit the project, and what are the disadvantages of Coverage in Ruby.

Experimental project

As a project for testing, a small story about a boy was taken, who may ask permission to walk with his mother and father.

#      ,     , #     .       ,   #     ,      . class Mother def permit_walk?(child) child.scarf_put_on && child.homework_done end end 

 #     ,    ,       . class Father def permit_walk?(child) child.scarf_put_on end end 

 #     ,     , #   .      ,   . #  , ,      . class Child attr_reader :homework_done, :scarf_put_on def initialize(mother, father) @mother = mother @father = father @homework_done = false @scarf_put_on = false end def do_homework! @homework_done = true end def put_on_scarf! @scarf_put_on = true end def walk_permitted?(whom_to_ask) parent = if whom_to_ask == :mother @mother else @father end parent.permit_walk?(self) end end 

We cover with tests and see coverage

The tests intentionally cover not all scenarios:

 require "simplecov" SimpleCov.start require "rspec" require_relative "../lib/mother" require_relative "../lib/father" require_relative "../lib/child" RSpec.describe Child do let(:child) { Child.new(Mother.new, Father.new) } context "when asking mother without scarf and without homework" do it "isn't permitted to walk" do expect( child.walk_permitted?(:mother) ).to be false end end context "when asking mother with scarf and with homework" do it "is permitted to walk" do child.put_on_scarf! child.do_homework! expect( child.walk_permitted?(:mother) ).to be true end end end 

SimpleCov is actually a monopolist in coverage analysis in the world of Ruby 1.9.3+. It is a handy wrapper over the Coverage module from the standard library.

The connection is reduced to two lines at the beginning of the test file, and it is important that the initialization of SimpleCov be carried out before connecting the project files. Run the tests:

 rspec 

Voilà! The report / report.html report file was generated. You can view it at the link , and here I will leave a couple of screenshots so as not to go far (the general report is used as the title picture).

father.rb

Excerpt from child.rb

Bonuses from analysis coverage

It is immediately clear from the report that the way in which permission is asked from the father is not tested. From here obvious advantage from the analysis of a covering: in the conditions of non-application TDD the report can show that we forgot to test something. If the project is inherited and the difficult way of testing is just beginning, the report will help decide where to most effectively direct the force.

The second possible use is the automatic provision of "quality" commits. The CI server can reject commits that lead to a decrease in total coverage, drastically reducing the likelihood of untested code appearing in the repository.

What does not cover analysis

First, one hundred percent coverage does not provide the absence of bugs. A simple example: if you change the Mother class like this:

 class Mother def permit_walk?(child) # child.scarf_put_on && child.homework_done child.homework_done end end 

class coverage will remain 100%, the tests will still be green, but the logic will obviously be wrong. You can use the mutant gem to automatically detect "missing but needed" tests. I haven’t tried it yet, but judging by the Readme and the number of stars on the github, the library is really useful. However, this is a topic for a separate post, to which I somehow get.

Secondly, in Ruby at the moment it is possible to analyze the coverage only by rows, branch- and condition-coverage is not supported. It is meant that in one-line view

 some_condition ? 1 : 2 some_condition || another_condition return 1 if some_condition 

there are branch points, but even if tests pass through only one possible branch of execution, coverage will show 100%. There was a pull request to Ruby on this topic, but for two years nothing has been heard from the maintainers. It's a pity.

Afterword

I prefer to write tests immediately after writing the code, and coverage serves me as a reminder of methods that have not yet been tested (I often forget to test exception handlers). In general, coverage analysis may well be of some benefit, but a 100% coverage does not necessarily mean that there are enough tests.

Materials used in the article:

• Test project on githaba
• SimpleCov Analysis Report
• Simplecov
• mutant