Opened 15 years ago
Last modified 10 years ago
#512 new defect
Bitten, Cunit code doesn't report the total number of tests correctly
Reported by: | anonymous | Owned by: | |
---|---|---|---|
Priority: | major | Milestone: | 0.6.1 |
Component: | Trac plugin | Version: | 0.6b2 |
Keywords: | Bitten | Cc: | |
Operating System: | BSD |
Description
I am using Cunit keyword in the recipe to submit the XML file of the cunit test output. One of the odd things of the CUNIT report is that it reports multiple CUNIT_RUN_TEST_FAILURE records if a test fails multiple assertions. At least that is my understanding. If that is not the case please let me know and I will try to figure it out why CUNIT on my system behaves the way I described above. Otherwise please continue to read.
Because of the way total is being incremented i.e. increment by 1 for every record of either CUNIT_RUN_TEST_SUCCESS or CUNIT_RUN_TEST_FAILURE, the total depends on the number of failures. I.e. when test passes, the total is incremented by 1. However if a test fails with say 2 failures, the total gets incremented by 2. Therefore the total number tests reported keeps varying based on number of failures. I think the implementation should one of two things.
- Parse all the failure records and print them but increment the failure count only once per CUNIT_RUN_TEST_RECORD and not for every failure record.
- Alternatively, one could use <CUNIT_SUMMARY_RECORD>
<TYPE> Test Cases </TYPE> <TOTAL> [0-9]+ </TOTAL> <RUN> [0-9]+ </RUN> <SUCCEEDED> [0-9]+ </SUCCEEDED> <FAILED> [0-9]+ </FAILED>
</CUNIT_SUMMARY_RECORD>
to set the total number of tests passed.
The XML that I use to report test results to bitten