Topics

Changes to code coverage?

Randall Wood <rhwood@...>
 

PR https://github.com/JMRI/JMRI/pull/7475 makes no changes to any code (it updates a library), and yet Code Climate coverage shows these changes to coverage (screenshot if this works):


I can’t tell if this is a sign that using Code Climate for total coverage is a mistake (I suggested it, so it’s my mistake), or if this reveals some other possibly transient issue with our testing (is some state sometimes being retained between tests)?

Randall

Randall Wood <rhwood@...>
 

Sorry about the first attachment, I let my phone compress it too much.


Randall

On Oct 5, 2019, at 11:59, Randall Wood <rhwood@...> wrote:

PR https://github.com/JMRI/JMRI/pull/7475 makes no changes to any code (it updates a library), and yet Code Climate coverage shows these changes to coverage (screenshot if this works):
<image0.png>


I can’t tell if this is a sign that using Code Climate for total coverage is a mistake (I suggested it, so it’s my mistake), or if this reveals some other possibly transient issue with our testing (is some state sometimes being retained between tests)?

Randall

Paul Bender
 

On Oct 5, 2019, at 11:59 AM, Randall Wood via Groups.Io <rhwood=mac.com@groups.io> wrote:
I can’t tell if this is a sign that using Code Climate for total coverage is a mistake (I suggested it, so it’s my mistake), or if this reveals some other possibly transient issue with our testing (is some state sometimes being retained between tests)?
This isn’t any different from what we see on coveralls.

We have cleaned up a lot of this, but some tests are still leaving threads behind, which is at least a part of the problem.

There are a couple of tests that have wide fluctuations ( 10s of lines ), but most of them are just a line or two.

As far as total coverage is concerned, we currently have coveralls set to fail on more than a 0.25% drop in coverage. Code climate is currently set to fail at a smaller coverage drop, so it will flag things coveralls does not. They should probably be set to the same value.

Paul