"test code coverage" is inaccurate..

[anonymous]

b_set_step_info = True

steps = {
    'recover': {
        'module': 'recover.recover_main',
        'predecessor': None,
        'param': {
#           'param1': '111',
        },
    },
}

# vim: sw=4 ts=4 bg=dark

When I unittest above code, "trace.py" reports that line is 6 and coverage is 100%.

However, bitten extracts different result from this report.(lines of code is 9 and coverage is 67%)

[osimons]

Could you please add the step with commands you use (from recipe). As Bitten supports several tools for python coverage (trace, coverage, figleaf) they may well produce different results - and any mismatch between runner and reporter could naturally also affect results.

However the Bitten unittest command should use trace as default, and when using python:trace reporting you should see same results as in your shell-testing.

[reporter]

<python:distutils command="unittest" options="--xml-output build/test-results.xml --coverage-summary build/test-coverage.txt --coverage-dir build/coverage --test-suite=tests"/>
<python:unittest file="build/test-results.xml"/>
<python:trace summary="build/test-coverage.txt" coverdir="build/coverage" include="src*" />

Unfortunately I use 'trace'..

In my opinion, the comment line(line 8) causes this case.

If I delete it, coverage will be 100%. However, covered line is still different.

P.S. Could you delete the "bitten_result.jpg"? I want to upload another one.

[reporter]

Sorry.. Not the "bitten_result.jpg", please delete the "trace_result.jpg".

[osimons]

Attachment gone - you'll upload another with same name, right?

As for the problem... There is no setup.cfg with a [unittest] section that tells it to use some other coverage tool?

Here is how things work:

• unittest is a Bitten distutils extension, from source:/trunk/bitten/util/testrunner.py​ - there you will see the _run_with_trace() method that is used to create the test results.
• python:trace is from source:/trunk/bitten/build/pythontools.py​ and the trace() function defined there. It parses the results on disk into XML for reporting to master.

Together with the code, you can use options for slave to keep the build results. That should allow you to quite easily verify where things go astray. Typically the -k option for bitten-slave to 'keep files', even more useful when combined with --work-dir=DIR to request where you want the build to happen (easier to locate than inside a temp folder). All coverage information should then be retrievable, and comparing on-disk results with your manual run and the information that ends up on master should provide a good starting point.

[reporter]

"trace_result.jpg" is kept file from bitten-slave's "build/coverage".

• line 9, 10, 11 are not flagged by _run_with_trace()(= trace.py).
• trace() method of pythontools.py doesn't count line 9, 10, 11 because of line 8.
  1. line 8 is not a code line. so prev_hits is 0
  2. line 9 is not flagged and prev_hits is 0, so this is not covered.
  3. line 10, 11 is iterated..

[osimons]

I won't look more into this before 0.6 final is released, so unless someone makes a working patch for this issue, 0.6.1 is the more likely release for getting this fixed.