Ticket #862 (closed: fixed)

Opened 11 years ago

Last modified 5 years ago

Automate testing of non dialog scripts as part of build

Reported by: Nick Draper Owned by: Russell Taylor
Priority: critical Milestone: Iteration 22
Component: Keywords:
Cc: Blocked By:
Blocking: Tester:

Description (last modified by Nick Draper) (diff)

We need to both 1) check the scripts complete without errors 2) validate the results against previously stored files

Change History

comment:1 Changed 11 years ago by Nick Draper

  • Status changed from new to assigned
  • Owner set to Russell Taylor
  • Description modified (diff)
  • Priority changed from major to critical

comment:2 Changed 11 years ago by Russell Taylor

  • Status changed from assigned to accepted

comment:3 Changed 11 years ago by Russell Taylor

(In [3541]) Small changes to the StressTest code to make it work on Windows when there's a space in the path and when the MySQLdb module is not installed. I also had to change the line separator for code embedded in a python call from new line to a semi-colon. Re #862.

comment:4 Changed 11 years ago by Russell Taylor

(In [3542]) Add an algorithm that compares two workspaces for equality. This will be used in Mantid system/stress tests. Re #862.

comment:5 Changed 11 years ago by Russell Taylor

(In [3544]) Give the windows compiler a helping hand so that it can figure things out. Re #862.

comment:6 Changed 11 years ago by Russell Taylor

(In [3557]) Add files for automated testing of analysis scripts. Re #862.

comment:7 Changed 11 years ago by Russell Taylor

(In [3560]) Add checking of certain Sample details. Re #862.

comment:8 Changed 11 years ago by Russell Taylor

(In [3561]) Leave checking of the Sample & logs off by default. Re #862.

comment:9 Changed 11 years ago by Russell Taylor

(In [3562]) Fix tests. Re #862.

comment:10 Changed 11 years ago by Russell Taylor

(In [3581]) Add SANS test script. Re #862.

comment:11 Changed 11 years ago by Russell Taylor

(In [3587]) Update diffraction reference output to account for tiny differences following change to DiffractionFocussing in [3580]. Re #862

comment:12 Changed 11 years ago by Russell Taylor

(In [3612]) Change catch around LoadLog execution to catch anything derived from std::exception instead of just std::runtime_error. It appears to be possible for a Poco exception to go uncaught somewhere in LoadLog, and all Poco exceptions derive from std::exception and never any of the more specific standard library exception classes (Poco has its own runtime exception, for example). Re #862.

comment:13 Changed 11 years ago by Russell Taylor

(In [3614]) Add ability to disable parts of workspace comparison. Re #862.

comment:14 Changed 11 years ago by Russell Taylor

(In [3615]) Add more analysis tests. Re #862.

comment:15 Changed 11 years ago by Russell Taylor

(In [3719]) Test framework writes out Nexus file if mismatch found with reference. Re #862.

comment:16 Changed 11 years ago by Russell Taylor

(In [3731]) Update to SANS analysis test script to omit checking spectramap. Re #862.

comment:17 Changed 11 years ago by Russell Taylor

(In [3782]) Update reference result following changes to CylinderAbsorption. New result differs by < 0.1%. Re #862.

comment:18 Changed 11 years ago by Russell Taylor

Will wrap this up w/c 8th Feb.

comment:19 Changed 11 years ago by Russell Taylor

(In [3896]) Rename test to avoid name clash with script in main scripts directory. Re #862.

comment:20 Changed 11 years ago by Nick Draper

Final tasks remaining:

Perform publication of msi files to download area on success. Also allow multiple msi or other files to accompany the mantid.msi file.

For example the mantid64.msi file or the mac installer

comment:21 Changed 11 years ago by Russell Taylor

(In [3900]) Refinements to system tests. Re #862

comment:22 Changed 11 years ago by Russell Taylor

(In [3901]) Add ability to get svn revision that a MantidPlot executable corresponds to by passing a command line argument (-r or --revision). Re #862.

comment:23 Changed 11 years ago by Russell Taylor

  • Status changed from accepted to testing
  • Resolution set to fixed

comment:24 Changed 11 years ago by Steve Williams

  • Status changed from testing to closed

comment:25 Changed 5 years ago by Stuart Campbell

This ticket has been transferred to github issue 1710

Note: See TracTickets for help on using tickets.