Ticket #9898 (closed: fixed)
Slow Unit Tests for Nick Draper
Reported by: | Owen Arnold | Owned by: | Nick Draper |
---|---|---|---|
Priority: | major | Milestone: | Release 3.3 |
Component: | Framework | Keywords: | Maintenance |
Cc: | Blocked By: | ||
Blocking: | #9905 | Tester: | Andrei Savici |
Description (last modified by Owen Arnold) (diff)
We have a number of slow running unit tests which we need to fix as part of the 3.3 maintenance window. We are targeting test suits than execute in > 2 seconds on our clean rhel6 build servers. For IO tests (Loading & Saving) we have applied a more generous threshold of > 5 seconds.
See the full list to see which tests have been assigned to you: http://www.mantidproject.org/images/2/2f/Slow_tests.xlsx_-_Sheet1.pdf
Criteria
- Test suits (each test class instance) should execute in < 1 second as a rough target
- As a corollary to the above, If the test suite contains lots of test methods aim for < 0.1 second per test method
- If for some reason you get the speed up the test below the target using the strategies listed below, you need to justify why when you close the ticket.
Guidelines/instructions to help
- Keep tests readable and improve code readability where possible. Unit tests are documentation.
- Do not delete test methods without good reason. We do not want to reduce test coverage
- Changing the algorithm code to improve speed is OK, but exercise caution. Add additional test coverage if it's not already good enough.
- Take test methods that are slow and involve IO, or are processor intensive and make them into system tests. Integration tests are not Unit tests.
- Most of the speed improvements will probably come from better selection of input data. Caching input data is also a good option.
- Create sub tickets for algorithms or groups of algorithms to help testability if you wish, but mark this ticket as the 'blocked' by each one.
Change History
comment:1 Changed 6 years ago by Owen Arnold
- Status changed from new to assigned
- Owner changed from Owen Arnold to Nick Draper
- Description modified (diff)
- Summary changed from Slow Unit Tests for Owen Arnold (cloned) to Slow Unit Tests for Nick Draper
comment:3 Changed 6 years ago by Nick Draper
- Status changed from assigned to inprogress
re #9898 halved time for ObjectTest
Changeset: 9e1b5074d3274df1ad371df9397ac97da76887e7
comment:4 Changed 6 years ago by Nick Draper
re #9898 ConvertUnitTest, changed event setup, was taking 90% time
Changeset: 295b7045f6eaaedf0294214b3dc5bc396cff9dad
comment:5 Changed 6 years ago by Nick Draper
re #9898 AlignDetectorsTest, changed event setup, was taking 80% time
Changeset: 4ec67d16e0913388ae454abc2b7f477658a85d0c
comment:6 Changed 6 years ago by Nick Draper
- Status changed from inprogress to verify
- Resolution set to fixed
To Test
Tests affected:
- Geometry\ObjectTest
- Algorithms\ConvertUnitTest
- Algorithms\AlignDetectorsTest
- Compare tests times from a recent develop incremental (rhel 6) against http://builds.mantidproject.org/job/develop_clean/label=rhel6-build/240/testReport/AlgorithmsTest/
- Check tests still pass
- Check the code changes seem sensible
comment:7 Changed 6 years ago by Nick Draper
ObjectTest still takes 3-4 seconds, but not 7 or so as before. It does have over 20 test cases, and several of these are solid angle tests some of which include ray tracing.
comment:8 Changed 6 years ago by Andrei Savici
- Status changed from verify to verifying
- Tester set to Andrei Savici
comment:9 Changed 6 years ago by Andrei Savici
- Status changed from verifying to closed
Merge remote-tracking branch 'origin/feature/9898_Faster_unit_tests'
Full changeset: 09997dce8f240c89b4e75fef53a790d61e9fe705
comment:10 Changed 5 years ago by Stuart Campbell
This ticket has been transferred to github issue 10740