Ticket #9902 (closed: fixed)

Opened 6 years ago

Last modified 5 years ago

Slow Unit Tests for Michael Wedel

Reported by: Owen Arnold Owned by: Michael Wedel
Priority: major Milestone: Release 3.3
Component: Framework Keywords: Maintenance
Cc: Blocked By:
Blocking: #9905 Tester: Nick Draper

Description (last modified by Owen Arnold) (diff)

We have a number of slow running unit tests which we need to fix as part of the 3.3 maintenance window. We are targeting test suits than execute in > 2 seconds on our clean rhel6 build servers. For IO tests (Loading & Saving) we have applied a more generous threshold of > 5 seconds.

See the full list to see which tests have been assigned to you: http://www.mantidproject.org/images/2/2f/Slow_tests.xlsx_-_Sheet1.pdf

Criteria

  1. Test suits (each test class instance) should execute in < 1 second as a rough target
  2. As a corollary to the above, If the test suite contains lots of test methods aim for < 0.1 second per test method
  3. If for some reason you get the speed up the test below the target using the strategies listed below, you need to justify why when you close the ticket.

Guidelines/instructions to help

  1. Keep tests readable and improve code readability where possible. Unit tests are documentation.
  2. Do not delete test methods without good reason. We do not want to reduce test coverage
  3. Changing the algorithm code to improve speed is OK, but exercise caution. Add additional test coverage if it's not already good enough.
  4. Take test methods that are slow and involve IO, or are processor intensive and make them into system tests. Integration tests are not Unit tests.
  5. Most of the speed improvements will probably come from better selection of input data. Caching input data is also a good option.
  6. Create sub tickets for algorithms or groups of algorithms to help testability if you wish, but mark this ticket as the 'blocked' by each one.

Change History

comment:1 Changed 6 years ago by Owen Arnold

  • Owner changed from Andrei Savici to Michael Wedel
  • Status changed from new to assigned
  • Description modified (diff)
  • Summary changed from Slow Unit Tests for Andrei Savici (cloned) to Slow Unit Tests for Michael Wedel

comment:2 Changed 6 years ago by Owen Arnold

  • Blocking 9905 added

comment:3 Changed 6 years ago by Michael Wedel

  • Status changed from assigned to inprogress

Refs #9902. Speeding up InvertMDDimTest

The was using a large MDHistoWorkspace, shrinking it by a factor of 10 in each dimension made the test faster without changing what the test was doing.

Changeset: 9c2f6a33c4ed04068074e39a52142028d422f96e

comment:4 Changed 6 years ago by Michael Wedel

Refs #9902. Speeding up ProjectMDTest

The same procedure as for InvertMDDimTest - shrinking the example data without changing what the test does.

Changeset: 3823e3b0d16ba9371ff96c76d219f48b314409ea

comment:5 Changed 6 years ago by Michael Wedel

  • Status changed from inprogress to verify
  • Resolution set to fixed

Making example workspace smaller helped speeding up the tests. It should be well below 1 second now (at least it is on incremental).

Testing information

Make sure the tests still pass.

Last edited 6 years ago by Michael Wedel (previous) (diff)

comment:6 Changed 6 years ago by Nick Draper

  • Status changed from verify to verifying
  • Tester set to Nick Draper

comment:7 Changed 6 years ago by Nick Draper

  • Status changed from verifying to closed

Unit tests pass, and quickly too

comment:8 Changed 6 years ago by Nick Draper

Merge remote-tracking branch 'origin/feature/9902_slow_unit_tests'

Full changeset: c0837b71be0586ce0e5be5ab5cda787ae3209c50

comment:9 Changed 5 years ago by Stuart Campbell

This ticket has been transferred to github issue 10744

Note: See TracTickets for help on using tickets.