Ticket #2190 (closed: fixed)

Opened 10 years ago

Last modified 5 years ago

Make a tool for running tests (WxPython or PyQT gui)

Reported by: Janik Zikovsky Owned by: Janik Zikovsky
Priority: minor Milestone: Iteration 27
Component: Tools Keywords:
Cc: Blocked By:
Blocking: Tester: Vickie Lynch

Description

A GUI with:

  • A tree structure of the tests available.
  • Checkbox to select the tests to run.
  • Graphically show test results:
    • Green: All tests are good
    • Red: Some tests failed.
    • Gray: Tests were not run.

Maybe a way to intelligently select which tests to run based on recent SVN changes?

Change History

comment:1 Changed 10 years ago by Janik Zikovsky

  • Status changed from new to accepted

comment:2 Changed 10 years ago by Janik Zikovsky

Other requirements/nice-to-haves:

  • Run the tests in parallel when possible (like runAllTests.py)
  • Recompile the tests (using the fast makes?) whenever the libraries have been changed (like monitorTests.py)

Use pydev and QtDesigner for IDEs.

comment:3 Changed 10 years ago by Janik Zikovsky

  • Priority changed from major to minor

Further notes:

  • Pete is setting things up so that all our testing will be performed using ctest.
    • Call 'ctest -R KernelTest_Statistics' to run all tests matching this expression.
  • Call 'ctest -N' to show all the tests available without running them.
  • Call make FrameworkTests to make all the framework tests.
  • Call make KernelTest (for example) to make the KernelTest only.

comment:4 Changed 10 years ago by Janik Zikovsky

(In [8495]) Refs #2190: TestViewer ui started.

comment:5 Changed 10 years ago by Janik Zikovsky

(In [8517]) Refs #2190, testing tools

comment:6 Changed 10 years ago by Janik Zikovsky

(In [8518]) Refs #2190, testing tool, runs in the background of GUI and in parallel.

comment:7 Changed 10 years ago by Janik Zikovsky

(In [8520]) Refs #2190

comment:8 Changed 10 years ago by Janik Zikovsky

(In [8531]) Refs #2190 - intermediate checkin

comment:9 Changed 10 years ago by Janik Zikovsky

(In [8534]) Refs #2190 - synchronizing results between threads.

comment:10 Changed 10 years ago by Janik Zikovsky

(In [8536]) Refs #2190 - starting to fill out the test tree

comment:11 Changed 10 years ago by Janik Zikovsky

(In [8545]) Refs #2190 - starting to fill out the test tree

comment:12 Changed 10 years ago by Janik Zikovsky

(In [8559]) Refs #2190 - tree has colors

comment:13 Changed 10 years ago by Janik Zikovsky

(In [8561]) Refs #2190 - tree has colors

comment:14 Changed 10 years ago by Janik Zikovsky

(In [8568]) Refs #2190 - can select which tests to run.

comment:15 Changed 10 years ago by Janik Zikovsky

(In [8592]) Refs #2190: Better updating of tree after running.

comment:16 Changed 10 years ago by Janik Zikovsky

(In [8600]) Refs #2190: Better support of selecting test suites

comment:17 Changed 10 years ago by Janik Zikovsky

(In [8603]) Refs #2190: Much nicer updating of project during calculation

comment:18 Changed 10 years ago by Janik Zikovsky

(In [8606]) Refs #2190: Simplified TestResult

comment:19 Changed 10 years ago by Janik Zikovsky

(In [8613]) Refs #2190: GUI settings are beginning to be saved

comment:20 Changed 10 years ago by Janik Zikovsky

(In [8614]) Refs #2190: Simplified Class hierarchy for TreeItem

comment:21 Changed 10 years ago by Janik Zikovsky

(In [8630]) Refs #2190: Beginning to filter by failed/all

comment:22 Changed 10 years ago by Janik Zikovsky

(In [8633]) Refs #2190: Filtering works more, and more quick selection; but timing info disappeared.

comment:23 Changed 10 years ago by Janik Zikovsky

(In [8635]) Refs #2190: Projects are not made in parallel but serially using -j flag.

comment:24 Changed 10 years ago by Janik Zikovsky

(In [8636]) Refs #2190

comment:25 Changed 10 years ago by Janik Zikovsky

(In [8658]) Refs #2190: Starting to show individual test results

comment:26 Changed 10 years ago by Janik Zikovsky

(In [8659]) Refs #2190: Starting to show individual test results

comment:27 Changed 10 years ago by Janik Zikovsky

(In [8666]) Refs #2190: The command line output of make is displayed in stdout (still want to add it to the GUI part too though).

comment:28 Changed 10 years ago by Janik Zikovsky

(In [8680]) Refs #2190: Console output when making tests.

comment:29 Changed 10 years ago by Janik Zikovsky

(In [8682]) Refs #2190: Catch segfaults

comment:30 Changed 10 years ago by Janik Zikovsky

(In [8684]) Refs #2190: Catch segfaults; and count those as failed tests.

comment:31 Changed 10 years ago by Janik Zikovsky

(In [8685]) Refs #2190: Faster updating of test results; window remembers position.

comment:32 Changed 10 years ago by Janik Zikovsky

(In [8688]) Refs #2190: Command-line arguments and default path values.

comment:33 Changed 10 years ago by Janik Zikovsky

(In [8691]) Refs #2190: Select using SVN; sort by time works; console log follows

comment:34 Changed 10 years ago by Janik Zikovsky

(In [8693]) Refs #2190: Better selection of suites

comment:35 Changed 10 years ago by Janik Zikovsky

  • Status changed from accepted to verify
  • Resolution set to fixed

(In [8720]) Fixes #2190: TestViewer now updates single tests when adding or removing them while it is running. Test project summary is updated dynamically while running. Empty files are cleaned up. Build failure only shown for selected tests.

comment:36 Changed 10 years ago by Janik Zikovsky

(In [8739]) Refs #2190: Should fix finding source files.

comment:37 Changed 10 years ago by Janik Zikovsky

(In [8741]) Refs #2190: Minor GUI fixes

comment:38 Changed 10 years ago by Janik Zikovsky

(In [8742]) Refs #2190

comment:39 Changed 10 years ago by Vickie Lynch

  • Status changed from verify to verifying
  • Tester set to Vickie Lynch

comment:40 Changed 10 years ago by Vickie Lynch

  • Status changed from verifying to closed

I had to build Mantid in Mantid/bin for this to work, but it works well.

comment:41 Changed 5 years ago by Stuart Campbell

This ticket has been transferred to github issue ​3037

Note: See TracTickets for help on using tickets.