Ticket #9874 (closed: fixed)
Have doctest output JUnit style output that Jenkins can parse
Reported by: | Martyn Gigg | Owned by: | Martyn Gigg |
---|---|---|---|
Priority: | critical | Milestone: | Release 3.3 |
Component: | Framework | Keywords: | Maintenance |
Cc: | Blocked By: | ||
Blocking: | Tester: | Alex Buts |
Description
That way the failures and timings are more visible and easier to access.
Attachments
Change History
comment:2 Changed 6 years ago by Martyn Gigg
- Status changed from assigned to inprogress
Add doctest module & TestCase, TestSuite & DocTestOutput classes
Refs #9874
Changeset: 7de8a8cb99c06d566289a75044a696055e4da253
comment:3 Changed 6 years ago by Martyn Gigg
Parses documents where all tests passed.
Refs #9874
Changeset: 9ed63acf383cb922f8965e5472e6cbb7ee3600d3
comment:4 Changed 6 years ago by Martyn Gigg
Add unit test for all passed case and fixed problems unit test found.
Refs #9874
Changeset: 87a17c625a5394d125b5da82a9ae0d977b4cf617
comment:5 Changed 6 years ago by Martyn Gigg
Parser and tests for all pass and all failure cases.
Refs #9874
Changeset: 720388979c9b641b73a1c390fae50584fb92de8b
comment:6 Changed 6 years ago by Martyn Gigg
Finish parsing of mixed pass/fail document.
Refs #9874
Changeset: 3f44170dcf0a1461397a028c70439bb3b7f3a737
comment:7 Changed 6 years ago by Martyn Gigg
Write out xunit XML when Sphinx build has finished.
This only happens if the builder type is set to doctest. Refs #9874
Changeset: 75df64452cc79114cceb6ee8990ccbcd532f701c
comment:8 Changed 6 years ago by Martyn Gigg
Add a test for multi-document output.
Refs #9874
Changeset: 90c90438101112dfae2200f14aa9d1fa21db087b
comment:9 Changed 6 years ago by Martyn Gigg
Change xunit output file to match those produced by ctest
Refs #9874
Changeset: 16a3c0f03985009ca1465876985409b4503ac9b3
comment:10 Changed 6 years ago by Martyn Gigg
comment:11 Changed 6 years ago by Martyn Gigg
Activate the mantiddoc.doctest extension
Refs #9874
Changeset: 674756b8f52d3a8425ba5277c7237e94ed0f33e2
comment:12 Changed 6 years ago by Martyn Gigg
comment:13 Changed 6 years ago by Martyn Gigg
Add package information to XML doctest output.
Refs #9874
Changeset: a98b7221c36bbd36988e163dbfa22eeacdb4ccfe
comment:14 Changed 6 years ago by Martyn Gigg
Fix classname for tests part of fail suite.
Refs #9874
Changeset: 1708cb2a7afd1b4d0f2a26f0d3c42597cba71250
comment:15 Changed 6 years ago by Martyn Gigg
comment:16 Changed 6 years ago by Martyn Gigg
Branch: feature/9874_doctest_xunit_output
Tester: A final step to the doctest build has been added to parse the output and convert it to junit-style output. The results can be seen here:
http://builds.mantidproject.org/job/develop_incremental/1277/label=rhel6-build/testReport/docs/
Jenkins seems particularly slow at loading those pages, which is most probably a configuration issue and will be looked at.
There should be a break down of each file and then clicking on an individual file should give a break down of the tests in that file.
comment:17 Changed 6 years ago by Martyn Gigg
- Status changed from inprogress to verify
- Resolution set to fixed
comment:18 Changed 6 years ago by Martyn Gigg
- Status changed from verify to reopened
- Resolution fixed deleted
It seems there is an issue parsing a failure case where the cleanup code passes. This is demonstrated in the attached file.
comment:19 Changed 6 years ago by Martyn Gigg
- Status changed from reopened to inprogress
Fix parser when encountering testcleanup/testsetup failures.
Refs #9874
Changeset: 2e3742c50867832f0b7d286080d3bff7210cf95f
comment:20 Changed 6 years ago by Martyn Gigg
- Status changed from inprogress to verify
- Resolution set to fixed
comment:21 Changed 6 years ago by Martyn Gigg
- Status changed from verify to reopened
- Resolution fixed deleted
comment:22 Changed 6 years ago by Martyn Gigg
- Status changed from reopened to inprogress
Raise exception doctest build-finished handler if there was an error.
Refs #9874
Changeset: 6927464c9a55c8107a62600ed49c30cde8a2d4ba
comment:23 Changed 6 years ago by Martyn Gigg
- Status changed from inprogress to verify
- Resolution set to fixed
comment:24 Changed 6 years ago by Alex Buts
- Status changed from verify to verifying
- Tester set to Alex Buts
comment:25 Changed 6 years ago by Martyn Gigg
- Status changed from verifying to reopened
- Resolution fixed deleted
comment:26 Changed 6 years ago by Martyn Gigg
- Status changed from reopened to inprogress
Fix regex for finding failed test name.
Refs #9874
Changeset: bcd95955feb3a05e3fe680df0939d7e182e5b3cf
comment:27 Changed 6 years ago by Martyn Gigg
- Status changed from inprogress to verify
- Resolution set to fixed
The regex fix should be the final fix.
This can be seen at: http://builds.mantidproject.org/job/doctest_develop/, where build #5 and #6 parsed the errors correctly and when the build passed in #7, the results confirmed that everything passes.
comment:29 Changed 6 years ago by Alex Buts
- Status changed from verifying to closed
Merge remote-tracking branch 'origin/feature/9874_doctest_xunit_output'
Full changeset: 4b13f01765d884c8e36529d2ec5751e7acffdf18
comment:30 Changed 5 years ago by Stuart Campbell
This ticket has been transferred to github issue 10716