Ticket #7750 (closed: fixed)

Opened 7 years ago

Last modified 5 years ago

Add boilerplate code to performance tests having a constructor, where missing

Reported by: Russell Taylor Owned by: Russell Taylor
Priority: major Milestone: Release 3.0
Component: Framework Keywords:
Cc: Blocked By:
Blocking: Tester: Jay Rainey

Description

I noticed that if I ran the CrystalTest executable directly but gave it a non-existent test name it still ran LoadIsawPeaks. Turns out that this was happening in the FilterPeaks performance test constructor, which was missing the magic code to prevent the constructor being called every time.

Fix this and check for other places that this happen with the performance tests.

Change History

comment:1 Changed 7 years ago by Russell Taylor

  • Status changed from new to inprogress

Re #7750. Add boilerplate code to stop constructor running every time.

In particular, the constructor for FilterPeaksTestPerformance was running LoadIsawPeaks every time any Crystal test was run.

Changeset: 0e8571dd97c73ec34dae3cc35950f7d3c079ce08

comment:2 Changed 7 years ago by Russell Taylor

Re #7750. Add boilerplate code for perf. tests with constructors.

None of these other constructors did much, but there's no harm in having this here anyway (someone might make them do something in the future).

Changeset: 281c26be3e1284386db740aaa0db70c37e37e85c

comment:3 Changed 7 years ago by Russell Taylor

  • Status changed from inprogress to verify
  • Resolution set to fixed

Tester: Branch is bugfix/7750_protect_perf_test_constructors

It's probably enough to check the changes quickly and observe that all tests are passing. Another check is to run the CrystalTest executable manually with a non-existent test as argument (e.g. bin/CrystalTest nonTest. It will fail, but only after running LoadIsawPeaks (before the changes). After the changes that algorithm will not run before failure.

Although our Jenkins jobs are claiming that these changes made the Crystal tests take longer to run, that's because those timings only capture the time spent actually in a test method, not any setup time. Somehow, these changes have made something that was outside that region, move inside. Running ctest -R CrystalTest_ manually before and after the changes will prove that they actually got much quicker.

comment:4 Changed 7 years ago by Jay Rainey

  • Status changed from verify to verifying
  • Tester set to Jay Rainey

comment:5 Changed 7 years ago by Jay Rainey

Testing

  • Tested on Ubuntu 12.04
  • When CrystalTest executable manually it no longer performs LoadIsawPeaks.
  • CrystalTest now runs much quicker. (Previously took 93.50 seconds to complete. It now takes 38.80.)

comment:6 Changed 7 years ago by Jay Rainey

  • Status changed from verifying to closed

Merge remote-tracking branch 'origin/bugfix/7750_protect_perf_test_constructors'

comment:7 Changed 5 years ago by Stuart Campbell

This ticket has been transferred to github issue 8595

Note: See TracTickets for help on using tickets.