Ticket #6667 (closed: fixed)

Opened 8 years ago

Last modified 5 years ago

Move ISIS indirect reduction into Mantid and write proper system tests for it

Reported by: Alex Buts Owned by: Alex Buts
Priority: major Milestone: Release 2.6
Component: Direct Inelastic Keywords:
Cc: martyn.gigg@… Blocked By:
Blocking: Tester: Martyn Gigg

Description (last modified by Martyn Gigg) (diff)

Mantid currently uses DirectInelastic reduction algorithms which is used for converting data to energy transfer. The integrity of these is verified by the system tests.

These algorithms are the basis for customized reduction, actually used in ISIS excitation group to process the experimental results.

The custom algorithms mainly rely on Mantid algorithm but override different settings from different places and have some parts of the algorithm implemented differently.

1) The custom algorithm are not covered by system tests and can be easily broken by the changes in Mantid core

2) Two (actually three) sets of default parameters plus possibility to set up parameters from the script itself make it difficult to understand what parameter is actually used.

The purpose of this ticket is to bring all ISIS inelastic reduction script into Mantid and verify its integrity by proper system tests.

Isis reduction script should be very thin layer over verified Mantid reduction scripts.

Change History

comment:1 Changed 7 years ago by Alex Buts

  • Milestone changed from Release 2.5 to Release 2.6

comment:2 Changed 7 years ago by Alex Buts

  • Status changed from new to accepted

comment:3 Changed 7 years ago by Alex Buts

refs #6667 Instrument parameters modified to take some properties

from dgreduce

Changeset: 7054b8b689062ba1e9a63ded7b93f83c01eb541d

comment:4 Changed 7 years ago by Alex Buts

refs #6667 first steps in taking all parameters from dgreduce

and moving them into IDF_Parameters.xml

Changeset: 1d3ead08640467109bf7d2dfd8b0763ba92a6743

comment:5 Changed 7 years ago by Alex Buts

refs #6667 Changed ExperimentInfo::populateInstrumentParameters

to understand booleans, though no unit tests found for populateInstrumentParameters to check this stuff works properly

Changeset: a3ad11252080a8c09bfc31d54a756d4dcc071b31

comment:6 Changed 7 years ago by Alex Buts

refs #6667 Exported parameter type from the parameter map

Very ugly done, as python should be able to retrieve the value and type together. But can not find better way at the moment.

Changeset: 6fa81f435d321d7321ad49a03eb8d4581cfb3f67

comment:7 Changed 7 years ago by Alex Buts

refs #6667 Minor changes to export parameter type

and substitution dictionary in DirectEnergyConversion

Changeset: 7cc4636d2a76931ca3b300c525e30d826d29589e

comment:8 Changed 7 years ago by Alex Buts

refs #6667 Failed attempt to build getParameterType test

Changeset: be8737adb1cb6e8d4dcb3bead11faeaef4cbbe2a

comment:9 Changed 7 years ago by Alex Buts

refs #6667 finally correct test verifying the component type name

Changeset: ea809f14f70cfdb57f9513f58346ce107aaef727

comment:10 Changed 7 years ago by Alex Buts

refs #6667 Hopefully working solution with Parameter-based file

where all default keys are obtained from Par.xml. + int parameter is added to xml parcer.

Need tests and routines, accepting parameters from the arguments.

Changeset: fd1045910f9e6cce0b5b025440581f1b42d8ebfb

comment:11 Changed 7 years ago by Alex Buts

refs #6667 more changes to parameters and some unit tests attempts

Changeset: 22d5bcf8e8e26b6cdbd92235b972dc53afc30052

comment:12 Changed 7 years ago by Alex Buts

refs #6667 DirectEnergyConversion accepts external parameters

and have number of unit tests enabled testing this functionality. DAE name property imported from Mantid Facility (for qtiGenie to work)

Changeset: 8e016fb1c6c43027470d44a26f26082f7d9bc5e2

comment:13 Changed 7 years ago by Alex Buts

refs #6667 intermediate checkout

some unit tests run and ready to run system tests

Changeset: 568b5c837ac8767a259ef23248aa35971802f503

comment:14 Changed 7 years ago by Alex Buts

refs #6667 diagnose seems work

Changeset: 3af63f712d6fb3986bb3aeef2f9203d188f64fcd

comment:15 Changed 7 years ago by Alex Buts

refs #6667 Intermediate checkout

Old system test for MAPS runs with modified DirectEnergyConversion

Changeset: 4fe78d10f0085fdf597acfe9ef0a830d9b6af779

comment:16 Changed 7 years ago by Alex Buts

refs #6667 modifying system tests to work with new dgreduce

in ISIS

Changeset: 6553bd5dafdd41ad095d090c54699aedb8396f6b

comment:17 Changed 7 years ago by Alex Buts

refs #6667 Maps system tests run but do not pass

Both old and new tests failing, which is probably due to different defaults

Changeset: c0e66a5e324bac4801fc10da207f3ff7b035a8aa

comment:18 Changed 7 years ago by Alex Buts

refs #6667 Added dgreducer based system test for MAPS

Changeset: a57029385e6dd3a676d8b642e8b043a1ef3b0ea6

comment:19 Changed 7 years ago by Alex Buts

refs #6667 All generally works but constants are missing.

Changeset: c6a8e0e7416b9ed452f40a167fc3e0a99ef10835

comment:20 Changed 7 years ago by Alex Buts

refs #6667 The run data look very close to each other (MAPS)

but not yet identical -- diag seems a bit different

Changeset: 99a8f64f858ef44bf984d95acdd78af3bfb2e74b

comment:21 Changed 7 years ago by Alex Buts

refs #6667 minor changes

mainly related to ongoing work on other branches

Changeset: b20d8cc966ec528dd550cc2db74e2fae013b6815

comment:22 Changed 7 years ago by Alex Buts

refs #6667 Maps reduction works

But there is subtle difference in reduction results

the difference does not have big physical sense, as the relative values of differences are small, but still unclear why it is present at all.

The CheckWorkspacesMatch got possibility to analyze relative error together with the absolute error to understand the difference better LoadMask.cpp in the ISIS part has been changed to report loaded masks on debug level rather then notice as the messages in the log complicate understanding of what is going on.

Changeset: d4cbed63b7fc690844d623c9a6ecb920e3983c88

comment:23 Changed 7 years ago by Alex Buts

refs #6667 Minor changes for niceness

Changeset: a3a3ba23f03582c28abbb04e30064882c8e61783

comment:24 Changed 7 years ago by Alex Buts

refs #6667 Modified parameters in accordance with MAPS

mean real data which MAPS uses -- not the one it has in MAPS parameters file

Changeset: f0e7a7f96c4ed4b10a1878de0af0d82bc47d4eef

comment:25 Changed 7 years ago by Alex Buts

refs #6667 minor changes

Changeset: 58e44c54245486abd8b81722e48b22a0a0b5a9f0

comment:26 Changed 7 years ago by Alex Buts

refs #6667 Intermediate commit -- debug statements

Changeset: bd79a0f926ec6d56aa056e57e866e5cd3534b64e

comment:27 Changed 7 years ago by Alex Buts

refs #6667 Small improvements and attempt to use hardmask only

unsuccessful attempts

Changeset: b8394503dda2351f22edb473afdf6be865a2ef4d

comment:28 Changed 7 years ago by Alex Buts

refs #6667 minor changes

Changeset: ae375bed30bccf83ce6f328c145fb3c38da9e43e

comment:29 Changed 7 years ago by Alex Buts

refs #6667 Modified Save procedure in DirectEnergyConversion

to use different save routines as lambda functions

plus some small debugging statements which will probably be reverted

Changeset: ee8b3ada5f305d9a70415f75835c823dfa269ce1

comment:30 Changed 7 years ago by Alex Buts

refs #6667 Modified getNeutronAtom without exceptions to be really

without exceptions

As it is impossible to start Mantid in Debug mode as it throws thousand exceptions which are then catched, analyzed, reported and one has to wait for 1 min to start trivial program

Changeset: dc6ef632982584d0254bae99d316da082a671f26

comment:31 Changed 7 years ago by Alex Buts

refs #6667 Presumably created hardMaskOnly mode in diagnose

removed couple of small test clauses.

Changeset: d981f12f335e45371eae00ab4d4a3c128602f718

comment:32 Changed 7 years ago by Alex Buts

refs #6667 Better defined overloaded properties

Changeset: 4c28e762e188775b9bae95b73c0b7d331617425f

comment:33 Changed 7 years ago by Alex Buts

refs #6667 Checkpoint.

Trying to reduce monovanadium and it does not work.

Changeset: 96416491a7cf259b4b4035f10a4cd66c341fa02a

comment:34 Changed 7 years ago by Alex Buts

refs #6667 Hard Mask only works

Changeset: 197697037a468856345391d3cc8fa37a36e8647b

comment:35 Changed 7 years ago by Alex Buts

refs #6667 Implemented default relative mono-vanadium integration

range. There are doubts on if it is correct implementation as seems it was used for different energy ranges. But let's do something and see.

Changeset: c2ba791dd1509323c6b92d18fae8b850fb5f4703

comment:36 Changed 7 years ago by Alex Buts

refs #6667 Attempted to implement masks reuse.

Which does not work because of the way xml mask interacts with workspace. left for the time being.

Changeset: e8972f3a51c44f92f175748debb42843ed297569

comment:37 Changed 7 years ago by Alex Buts

refs #6667 Intensities of old and new dgreduce coincide in arb units

Changeset: 5c2c5acf182527ef874aeea5493a485ab618f55d

comment:38 Changed 7 years ago by Alex Buts

refs #6667 Some comments

Changeset: dc00e5a9f099854567ce64f48b164cc19a66f613

comment:39 Changed 7 years ago by Alex Buts

refs #6667 Minor comments and enhancements

Changeset: f081441e33266ba9952fa04455d30eb5cb3b3e88

comment:40 Changed 7 years ago by Alex Buts

refs #6667 Added option to request comparing all data to

CheckWorkspacesMatch

Changeset: 0168f2ef7f9c8cdce8c9aca8e63afcf72db076b9

comment:41 Changed 7 years ago by Alex Buts

refs #6667 Comments and code niceness.

Changeset: d8342668c12453ea87c5f5d864696faed44577d8

comment:42 Changed 7 years ago by Alex Buts

refs #6667 MAPS system test modified to work hopefully right

and produce correct WB cross-section if set up this way (currently it is not and runs against some run data)

Changeset: 6d2de5721e5f066160971a452a5d48241a7d8c5a

comment:43 Changed 7 years ago by Alex Buts

refs #6667 LET parameters file has been created and tried with LET

reduction

Changeset: 6cd4f86f82d9e2a47d1d06011f11788d18f8eb78

comment:44 Changed 7 years ago by Alex Buts

refs #6667 checkpoint Changed set_instrument_name property

working together with calibration file workspace

Changeset: 7a4012635c12279d8f478e4592b9328984460bb7

comment:45 Changed 7 years ago by Alex Buts

refs #6667 Minor checkpoint

Changeset: fa9cc2ee8fcc7f3a87e6e98e4102343647874650

comment:46 Changed 7 years ago by Alex Buts

refs #6667 Small changes allowing to run DirectEnergyConversion for

LET

Changeset: 10064f82ad0eea0329453dc37dc1fc63891d54b1

comment:47 Changed 7 years ago by Alex Buts

refs #6667 Added provisional parameters for LET MERLIN and MARI

the old remaining as back-up

Changeset: a9ff4638ae5b7e4b8a3d58e441a86cb433bd16ba

comment:48 Changed 7 years ago by Alex Buts

refs #6667 Something like working for MARI

but further investigations pending if it is exactly right

Changeset: 2aaf1e64ffcfc46ddfae7f2c93692443b302036b

comment:49 Changed 7 years ago by Alex Buts

refs #6667 Modified MAPS and MARI System tests pass

one more workflow test has been added for MARI

Changeset: 82196cf97d5770f90c2d4b16b599995495eb5b6c

comment:50 Changed 7 years ago by Alex Buts

refs #6667 Changes to MARI reduction to produce previous results

MARI generates the results which are similar to the one, produced by old reducer. Reasons for difference are generally clear.

Changeset: 7ce4293e1934fa9278d842e128643ace1500f78c

comment:51 Changed 7 years ago by Alex Buts

refs #6667 MARI and MAPS system tests pass

modified system tests, compared against vanadium

Changeset: c177bb10e0cc7d3cb07ebae64b18c682a1662f28

comment:52 Changed 7 years ago by Alex Buts

refs #6667 Extracted unit tests for the interfaces

and added them to CMakeLists.txt

Changeset: 5513e3000e67fddeba82a35c759d735e2e76c581

comment:53 Changed 7 years ago by Alex Buts

refs #6667 System tests modified to work for dgreduce.

Changeset: 6d09eeac5ef01b4a3d7a38990d9492e6da350adb

comment:54 Changed 7 years ago by Alex Buts

refs #6667 Correct test if workspace provided to reducer

Changeset: 9b4ab8bdd9b8e662b2cf554d2b164b703dc78d2a

comment:55 Changed 7 years ago by Alex Buts

refs #6667 Modified getNeutronAtom without exceptions to be really

without exceptions

As it is impossible to start Mantid in Debug mode as it throws thousand exceptions which are then catched, analyzed, reported and one has to wait for 1 min to start trivial program

Changeset: 0df38d27c4d410c0320cb96774a08ca3818b16d0

comment:56 Changed 7 years ago by Alex Buts

refs #6667 Presumably created hardMaskOnly mode in diagnose

removed couple of small test clauses.

Changeset: 596aa803ab64cdbe20b274c861f0511efe3c2d11

comment:57 Changed 7 years ago by Alex Buts

refs #6667 Better defined overloaded properties

Changeset: 3711e07c27c001021b4df979b76cbc57e9fb28ab

comment:58 Changed 7 years ago by Alex Buts

refs #6667 Checkpoint.

Trying to reduce monovanadium and it does not work.

Changeset: ad59da8b56ed48fe39cb01d760ba1048c2e5170b

comment:59 Changed 7 years ago by Alex Buts

refs #6667 Hard Mask only works

Changeset: 2ac4bc0c9d3395edd2ed374d21a9cbaaf1fac280

comment:60 Changed 7 years ago by Alex Buts

refs #6667 Implemented default relative mono-vanadium integration

range. There are doubts on if it is correct implementation as seems it was used for different energy ranges. But let's do something and see.

Changeset: 0aa884c9d76a7b9be3ac8bf0edf310eb0bcf318a

comment:61 Changed 7 years ago by Alex Buts

refs #6667 Attempted to implement masks reuse.

Which does not work because of the way xml mask interacts with workspace. left for the time being.

Changeset: 5dc8cbb42a0666239d367b34552a7e5dc850ccfa

comment:62 Changed 7 years ago by Alex Buts

refs #6667 Intensities of old and new dgreduce coincide in arb units

Changeset: 48d23a944b71168accf16470bbf2b24753f80c57

comment:63 Changed 7 years ago by Alex Buts

refs #6667 Some comments

Changeset: 0279359f44107cd3f66ded6b38f379a27f2e0fdd

comment:64 Changed 7 years ago by Alex Buts

refs #6667 Minor comments and enhancements

Changeset: 8fd0ed42de26ea9b93f3416abc243590b4203cb6

comment:65 Changed 7 years ago by Alex Buts

refs #6667 Added option to request comparing all data to

CheckWorkspacesMatch

Changeset: fb1964e487a39ecddf3ea83cc4b8c2754098aaee

comment:66 Changed 7 years ago by Alex Buts

refs #6667 Comments and code niceness.

Changeset: ee45d2074357cab6ba9a9816a90445944903d966

comment:67 Changed 7 years ago by Alex Buts

refs #6667 LET parameters file has been created and tried with LET

reduction

Changeset: 19aa4f9d4dd8f1eeb779fbb9b12404800a085206

comment:68 Changed 7 years ago by Alex Buts

refs #6667 checkpoint Changed set_instrument_name property

working together with calibration file workspace

Changeset: cb45ffeda2308a46bbca75ab1ff4fb7bbed8cb31

comment:69 Changed 7 years ago by Alex Buts

refs #6667 Minor checkpoint

Changeset: eb5063c9234c143a4c0e6bd5a296c162ba48e21a

comment:70 Changed 7 years ago by Alex Buts

refs #6667 Small changes allowing to run DirectEnergyConversion for

LET

Changeset: 34a2a5c15118b143d874c59f8bad534df79a2193

comment:71 Changed 7 years ago by Alex Buts

refs #6667 Added provisional parameters for LET MERLIN and MARI

the old remaining as back-up

Changeset: 4359af7f11cfa632bc15b44ba643191da50c5089

comment:72 Changed 7 years ago by Alex Buts

refs #6667 Something like working for MARI

but further investigations pending if it is exactly right

Changeset: a80b20b51594f6aa7c0a4d075a95da74fdcc3a46

comment:73 Changed 7 years ago by Alex Buts

refs #6667 Changes to MARI reduction to produce previous results

MARI generates the results which are similar to the one, produced by old reducer. Reasons for difference are generally clear.

Changeset: 884e2ba501aec758fc7fd1d9e392526f79910dff

comment:74 Changed 7 years ago by Alex Buts

refs #6667 MARI and MAPS system tests pass

modified system tests, compared against vanadium

Changeset: 1933b51647408634ceb45ce875e505a05e85cacc

comment:75 Changed 7 years ago by Alex Buts

refs #6667 Extracted unit tests for the interfaces

and added them to CMakeLists.txt

Changeset: 49544007feaaac826c11665dcfbab4ed0d395d34

comment:76 Changed 7 years ago by Alex Buts

refs #6667 Correct test if workspace provided to reducer

Changeset: 032aae03cf3c2c25f91776981fcdd62a56e67c49

comment:77 Changed 7 years ago by Alex Buts

refs #6667 Modified getNeutronAtom without exceptions to be really

without exceptions

As it is impossible to start Mantid in Debug mode as it throws thousand exceptions which are then catched, analyzed, reported and one has to wait for 1 min to start trivial program

Changeset: dc6ef632982584d0254bae99d316da082a671f26

comment:78 Changed 7 years ago by Alex Buts

refs #6667 Presumably created hardMaskOnly mode in diagnose

removed couple of small test clauses.

Changeset: d981f12f335e45371eae00ab4d4a3c128602f718

comment:79 Changed 7 years ago by Alex Buts

refs #6667 Better defined overloaded properties

Changeset: 4c28e762e188775b9bae95b73c0b7d331617425f

comment:80 Changed 7 years ago by Alex Buts

refs #6667 Checkpoint.

Trying to reduce monovanadium and it does not work.

Changeset: 96416491a7cf259b4b4035f10a4cd66c341fa02a

comment:81 Changed 7 years ago by Alex Buts

refs #6667 Hard Mask only works

Changeset: 197697037a468856345391d3cc8fa37a36e8647b

comment:82 Changed 7 years ago by Alex Buts

refs #6667 Implemented default relative mono-vanadium integration

range. There are doubts on if it is correct implementation as seems it was used for different energy ranges. But let's do something and see.

Changeset: c2ba791dd1509323c6b92d18fae8b850fb5f4703

comment:83 Changed 7 years ago by Alex Buts

refs #6667 Attempted to implement masks reuse.

Which does not work because of the way xml mask interacts with workspace. left for the time being.

Changeset: e8972f3a51c44f92f175748debb42843ed297569

comment:84 Changed 7 years ago by Alex Buts

refs #6667 Intensities of old and new dgreduce coincide in arb units

Changeset: 5c2c5acf182527ef874aeea5493a485ab618f55d

comment:85 Changed 7 years ago by Alex Buts

refs #6667 Some comments

Changeset: dc00e5a9f099854567ce64f48b164cc19a66f613

comment:86 Changed 7 years ago by Alex Buts

refs #6667 Minor comments and enhancements

Changeset: f081441e33266ba9952fa04455d30eb5cb3b3e88

comment:87 Changed 7 years ago by Alex Buts

refs #6667 Added option to request comparing all data to

CheckWorkspacesMatch

Changeset: 0168f2ef7f9c8cdce8c9aca8e63afcf72db076b9

comment:88 Changed 7 years ago by Alex Buts

refs #6667 Comments and code niceness.

Changeset: d8342668c12453ea87c5f5d864696faed44577d8

comment:89 Changed 7 years ago by Alex Buts

refs #6667 LET parameters file has been created and tried with LET

reduction

Changeset: 6cd4f86f82d9e2a47d1d06011f11788d18f8eb78

comment:90 Changed 7 years ago by Alex Buts

refs #6667 checkpoint Changed set_instrument_name property

working together with calibration file workspace

Changeset: 7a4012635c12279d8f478e4592b9328984460bb7

comment:91 Changed 7 years ago by Alex Buts

refs #6667 Minor checkpoint

Changeset: fa9cc2ee8fcc7f3a87e6e98e4102343647874650

comment:92 Changed 7 years ago by Alex Buts

refs #6667 Small changes allowing to run DirectEnergyConversion for

LET

Changeset: 10064f82ad0eea0329453dc37dc1fc63891d54b1

comment:93 Changed 7 years ago by Alex Buts

refs #6667 Added provisional parameters for LET MERLIN and MARI

the old remaining as back-up

Changeset: a9ff4638ae5b7e4b8a3d58e441a86cb433bd16ba

comment:94 Changed 7 years ago by Alex Buts

refs #6667 Something like working for MARI

but further investigations pending if it is exactly right

Changeset: 2aaf1e64ffcfc46ddfae7f2c93692443b302036b

comment:95 Changed 7 years ago by Alex Buts

refs #6667 Changes to MARI reduction to produce previous results

MARI generates the results which are similar to the one, produced by old reducer. Reasons for difference are generally clear.

Changeset: 7ce4293e1934fa9278d842e128643ace1500f78c

comment:96 Changed 7 years ago by Alex Buts

refs #6667 MARI and MAPS system tests pass

modified system tests, compared against vanadium

Changeset: c177bb10e0cc7d3cb07ebae64b18c682a1662f28

comment:97 Changed 7 years ago by Alex Buts

refs #6667 Extracted unit tests for the interfaces

and added them to CMakeLists.txt

Changeset: 5513e3000e67fddeba82a35c759d735e2e76c581

comment:98 Changed 7 years ago by Alex Buts

refs #6667 Correct test if workspace provided to reducer

Changeset: 9b4ab8bdd9b8e662b2cf554d2b164b703dc78d2a

comment:99 Changed 7 years ago by Alex Buts

refs #6667 Added missing imports to unit tests

Changeset: 77ec783d59f37ba144ec5099496d340fad0f6d50

comment:100 Changed 7 years ago by Alex Buts

refs #6667 Fixing previous error

and trying to guess correct CMakeList for python tests

Changeset: cc70258be3534f8685b521a25b3f54b2de0d1e04

comment:101 Changed 7 years ago by Alex Buts

refs #6667 temporary disabling failing test

Changeset: 92be120d826aaceb78671f723819fd89df2387dd

comment:102 Changed 7 years ago by Alex Buts

refs #6667 Trying to add python library to test to python search path

Changeset: 6257183e3d28c39df8015d53bacc6153ffbaa085

comment:103 Changed 7 years ago by Alex Buts

refs #6667 Another attempt to set search path for python test

Changeset: da8b25469e97f434ca99bf58f21f9b4fc2bb7eb2

comment:104 Changed 7 years ago by Alex Buts

refs #6667 Playing with python search path in CMakeLists.txt

No luck

Changeset: dbeb44a269a58f73876ebfafe34ab08275365454

comment:105 Changed 7 years ago by Alex Buts

refs #6667 last attempt to give proper path to test modules

Changeset: 80903791551cf4a0b546a6d229159dbd5f2c7b39

comment:106 Changed 7 years ago by Alex Buts

refs #6667 All back, does not work

Changeset: 055920832d8ee320a78c904968b6901cbb52088d

comment:107 Changed 7 years ago by Alex Buts

refs #6667 Added missing file needed for MAPS reduction

Changeset: 11ed88d8f68145863b04070a9084a7a2c91c4930

comment:108 Changed 7 years ago by Alex Buts

refs #6667 Fixed DirectInelasticDiagnostic and LoadLotsOfFiles

hopefully

Changeset: 96fb4dc5943268a8b7489454bfa1cd622b032348

comment:109 Changed 7 years ago by Alex Buts

refs #6667 Modified WFA helper to understand bool property

this should fix DGSReduction algorithm

Changeset: 2d4a9344e112b573c80ae7c1af56f92d79a9432b

comment:110 Changed 7 years ago by Alex Buts

refs #6667 Should fix mysterious error in MARI sum test

Changeset: 151c11e017ee3b31777e6e05cdbe9bd88f3a3019

comment:111 Changed 7 years ago by Alex Buts

refs #6667 Some key-words corresponding to Parameters file

Changeset: 763fe727b31e58027dfc48e16e6262c472239615

comment:112 Changed 7 years ago by Alex Buts

refs #6667 guessed last parameters providing difference

in DirectInelasticDiagnostic2

Changeset: d60da64f7f8e132db9ea304044e51ce67cc574d8

comment:113 Changed 7 years ago by Alex Buts

refs #6667 Minor changes

use correct procedure to find non-run type file path

Changeset: d02604fcbbc45ffc37bc9991ff15797d295cb72b

comment:114 Changed 7 years ago by Alex Buts

refs #6667 works correctly with necessary file as run number

also tries to modify SNS reduction to understand new properties.

Changeset: ce9456e354ead12ca0c23cf86b953553909e0fbf

comment:115 Changed 7 years ago by Alex Buts

refs #6667 fixed dgreduce test run separately

Changeset: 9bcd9527ea2c3a60b0157029794fa2cea0f2aafe

comment:116 Changed 7 years ago by Alex Buts

refs #6667 The change should fix python unit tests

hopefully. And the tests were enabled.

Changeset: 9457c592f32ea8a41f3806dd24dc2bfa18032901

comment:117 Changed 7 years ago by Alex Buts

refs #6667 Made unit test comparison almost equal for floating point

As the test fails on Unix otherwise.

Changeset: a01b0cb1f8ae31f09e63bdf81945cda12510c23d

comment:118 Changed 7 years ago by Alex Buts

refs #6667 Apparently more places where almost equal does not work on

Unix

Changeset: bb183539afd3139857ceb4ea634e56c42c3e4a79

comment:119 Changed 7 years ago by Alex Buts

refs #6667 Added new parameter types to API-1

for old DGS reduction to work

Changeset: 19705bdb3a7d48c59b6fe182b46b8087eafc6803

comment:120 Changed 7 years ago by Alex Buts

  • Cc martyn.gigg@… added
  • Status changed from accepted to verify
  • Resolution set to fixed

The ticket branch exists and need merging both in Mantid and SystemTests repositories.

The main task of this ticket is hardly testable.

I took the “trusted” reduction scripts used on ISIS inelastic instruments, extracted the parameters, embedded into these scripts into InstrumentName_Parameters.xml files and put new xml files and remaining shell of the scripts into Mantid. The scripts are built around old DirectEnergyConversion (dgreduce) Mantid script, so I have changed the interface of this script to understand parameters provided in the new way.

Then I’ve verified if the modified Mantid reduction produces almost exactly the same results and the trusted scripts and where it produces different results (MARI the example) me and the instrument scientist understand the reason for the difference and agree about the changes.

System tests run in Mantid were modified to understand new parameters and produce the results on the basis of these parameters. The DirectEnergyConversion scripts were slightly modified to fix some errors but full modification was not event tired from point of view of switching to DGSReduction workflow, developed in SNS.

As all this works, produces reasonable results have been extensively used during last tree weeks, tested and agreed with ISIS instrument scientists, a tester can accept it without big worries.

There are two areas related to the changes in C++/python code where careful tester can try to verify things: 1) The instrument parameter file can now have Boolean and Int parameters in addition to old double and string parameters. The example of changes is in the ticket’s xml files and unit tests are in Python, in DirectEnergyConversionTest.py file. Tester can look on how this code works and if anything can be broken/ improved there. 2) While testing the changes I have added new parameter to CheckWorkspaceMatch algorithm, which allows testing difference in workspaces as relative error. The changes are trivial and I used modified algorithm extensively, but tester can think about ways of verifying this.

comment:121 Changed 7 years ago by Martyn Gigg

  • Status changed from verify to reopened
  • Resolution fixed deleted

There are failing system tests that need to be looked at first.

comment:122 Changed 7 years ago by Alex Buts

refs #6667 Should fix issue when arb units requested files

necessary for abs units only

Changeset: b8cecf234d2187c1da936c7530386a78fe7a537d

comment:123 Changed 7 years ago by Alex Buts

refs #6667 Should fix the issue when abs units files are requested

for arb units run

Changeset: 6d3e1ed171464711d46de21466e1e036a558c232

comment:124 Changed 7 years ago by Alex Buts

  • Status changed from reopened to accepted

comment:125 Changed 7 years ago by Alex Buts

  • Status changed from accepted to verify
  • Resolution set to fixed

comment:126 Changed 7 years ago by Martyn Gigg

  • Status changed from verify to verifying
  • Description modified (diff)
  • Tester set to Martyn Gigg

comment:127 Changed 7 years ago by Martyn Gigg

  • Status changed from verifying to reopened
  • Resolution fixed deleted

There are a few things with the changes to CheckWorkspacesMatch algorithm that we need to change slightly. Adding the new properties will change the order that they are called in Python so I think we need to move:

  • declareProperty("ToleranceRelerr",false, "Treat tolerance as relative error rather then the absolute error");
  • declareProperty("CheckAllData",false, "Usually checking data ends wnen first mismatch occurs. This will force algorithm to check all data and print mismatch to debug log");

to the end after

  • declareProperty("Result","",Direction::Output);

That way the order will stay the same in Python.

Could you also change the name ToleranceRelerr to ToleranceRelErr, so it is more consistent with normal property style. While you are there could you fix the spelling of wnen->when.

Be aware that ticket #7403 removed the "supported.facilties" key from the properties file. I've merged master to the branch feature/6667_ISISReductionInMantid and updated the code in DirectEnergyConversion to use the new features. Make sure you do

  • git checkout feature/6667_ISISReductionInMantid
  • git pull origin feature/6667_ISISReductionInMantid

before continuing with any work

comment:128 Changed 7 years ago by Alex Buts

  • Status changed from reopened to accepted

comment:129 Changed 7 years ago by Martyn Gigg

Add export of Facility* to python.

Needed for using those returned from config.getFacilities. Refs #6667

Changeset: dfc89c4153951e50bd8360b06dca726de748c174

comment:130 Changed 7 years ago by Nick Draper

  • Status changed from accepted to assigned

comment:131 Changed 7 years ago by Alex Buts

refs #6667 Minor modifications to CheckWorkspacesMatch

to address tester's comments.

Changeset: 39e22fdbc0a5ff1c7a2790312d3077b50e42de83

comment:132 Changed 7 years ago by Nick Draper

  • Status changed from assigned to new

comment:133 Changed 7 years ago by Alex Buts

refs #6667 enabled check_background property

It is apparently used by new SNS reduction, in the meaning similar to background property. It is easy to modify DirectEnergyConversion to enable it for current isis reduction to simplify move

Changeset: 0c3a72f8c2817e3bff73dc884b24f838de26de2c

comment:134 Changed 7 years ago by Alex Buts

refs #6667 Fixed unit test affected by check_background changed

Changeset: 9741bfb341668b93202015a94bd2b60c4eb330d2

comment:135 Changed 7 years ago by Alex Buts

  • Status changed from new to verify
  • Resolution set to fixed

comment:136 Changed 7 years ago by Nick Draper

  • Component changed from Mantid to Framework

comment:137 Changed 7 years ago by Nick Draper

  • Component changed from Framework to Direct Inelastic

comment:138 Changed 7 years ago by Nick Draper

  • type changed from task to enhancement

comment:139 Changed 7 years ago by Martyn Gigg

  • Status changed from verify to verifying

comment:140 Changed 7 years ago by Martyn Gigg

  • Status changed from verifying to closed

The component stuff is not ideal but the parameter extraction is in a bit of a mess anyway and this hasn't really made it much worse.

The changes to CheckWorkspacesMatch & Component/ParameterMap should really have been done in another ticket that could have been tested in isolation but they look okay.

The systemtests are passing and the new reference files seem to make sense (visually) anyway and are required because of the changes to default parameters.

comment:141 Changed 7 years ago by Martyn Gigg

Merge remote-tracking branch 'feature/6667_ISISReductionInMantid'

Refs #6667 Conflicts:

Code/Mantid/scripts/CMakeLists.txt Code/Mantid/scripts/Inelastic/DirectEnergyConversion.py

Changeset: c3362412c6241d9929095408b11737fae087cbf0

comment:142 Changed 7 years ago by Martyn Gigg

Merge remote-tracking branch 'feature/6667_ISISReductionInMantid'

Refs #6667 Conflicts:

Code/Mantid/scripts/CMakeLists.txt Code/Mantid/scripts/Inelastic/DirectEnergyConversion.py

comment:143 Changed 7 years ago by Martyn Gigg

Merge remote-tracking branch 'origin/feature/6667_ISISReductionInMantid'

comment:144 Changed 5 years ago by Stuart Campbell

This ticket has been transferred to github issue 7513

Note: See TracTickets for help on using tickets.