Opened 15 years ago
Closed 15 years ago
#564 closed defect (fixed)
Another py.test patch: Ignore ignored tests
| Reported by: | Torsten Landschoff <torsten.landschoff@…> | Owned by: | hodgestar |
|---|---|---|---|
| Priority: | minor | Milestone: | 0.6 |
| Component: | Recipe commands | Version: | 0.6b2 |
| Keywords: | Cc: | ||
| Operating System: | BSD |
Description
After #562 went so smooth, here is the next patch :-)
I just see that it does not quite result in what I wanted - the test result does not show these as "ignores" as I would have expected.
Anyway, marking tests as xfail in py.test results in a <skipped/> test result. This is reported as failure by Bitten.
Test case and patch attached.
Attachments (2)
Change History (10)
Changed 15 years ago by Torsten Landschoff <torsten.landschoff@…>
comment:1 Changed 15 years ago by hodgestar
- Owner set to hodgestar
- Status changed from new to assigned
I'll have to confer a bit on what Bitten's general policy is regarding skipped tests but the patch looks good otherwise. Thank you for taking the time to submit them (and keep them coming :).
comment:2 Changed 15 years ago by osimons
Not having looked at details of the patch or its background, I'd think your want to use status 'ignore' instead of 'skipped'. As seen in bitten.report.testing file, a test can be either 'success' or 'ignore' - anything else is considered a failure by the reporting tools. Report table and charting should (!?) accomodate this without any special handling on the tool side.
comment:3 Changed 15 years ago by osimons
Sorry, trying to keep one eye on charting and one eye on table code and failing... Suggestion for use of 'ignore' still stands, as it is one of 4 recognized states: success, failure, error, ignore.
comment:4 Changed 15 years ago by hodgestar
- Milestone changed from 0.6.1 to 0.6
comment:5 Changed 15 years ago by hodgestar
I've added a modified version of the patch that ensure that the py.test statuses are converted to one of success, failure, error or ignore. The failure and error statuses increment the failed counter. I think the patch is ready to commit but I'd like Torsten to confirm that the new patch works for him before I do so.
comment:6 Changed 15 years ago by Torsten Landschoff <torsten.landschoff@…>
Thanks for the updated patch. I installed it and it is in fact better than my patch (as expected).
The test result view now actually shows the xfailed tests as "ignores" and they are counted into the total number of tests. With my patch, ignores would be 0 and the total would not include the skipped tests.
Ship it, if you ask me! :-)
Greetings, Torsten
comment:7 Changed 15 years ago by anatoly techtonik <techtonik@…>
Native regrtest.py from python explicitly says how many tests skipped and displays why they were skipped. Do we have such ability? For example some tests may fail on Windows and should be skipped, but when compatibility is added in the future, they should run ok. The problem is to find skipped tests then.
comment:8 Changed 15 years ago by hodgestar
- Resolution set to fixed
- Status changed from assigned to closed
Do not consider skipping a test a failure.