Mover run properties to descriptor and move common run functionality to them

[Alex Buts]

Currently run properties in reducer are simple python properties (e.g. sample_run, wb_run monovan_run) with all necessary functionality spread among multiple files (common_functions, DirectEnergyConversion and PropertyManager) which leads to code duplication and ineffective testing.

The purpose of this ticket is to collect all functionality (load, getMonitorWorkspace, sum) together within a RunFileDescriptor and write unit/system tests to ensure this functionality.

Also it would be nice to decrease DirectEnergy conversion size.

[Alex Buts]

Re #10881 Mainly created run descriptor with methods for single file

Changeset: 8748b7cf8894e8eca2d1c8ae6556d69e276fbfef

[Alex Buts]

Re #10881 Descriptor in working state for single file

Changeset: 911d3d8697d1b073902771d54548e1dd13c6fb88

[Alex Buts]

Re #10881 System tests for various ISIS loaders and data formats

and some data with them

Changeset: c702df20ef5c3de63825019b1b9c544feeb20068

[Alex Buts]

Re #10881 Changes and bugfixes to single file loader, found testing

various ISIS data formats

Changeset: 6aa796733214a3651919b15bde5347c25ed235ad

[Alex Buts]

Re #10881 System tests for various ISIS loaders and data formats

and some data with them

Changeset: b282410e49bc6a988949b601c5846d75f17050bb

[Alex Buts]

Re #10881 DirectEnergyConversion runs with run descriptor

Changeset: 4d81e5becacf5fc2b5ef835b529a2bb922a6cab2

[Alex Buts]

Re #10881 System test works for MARI from file

Changeset: 4a55137cc446cbc77ffd19a187e02052833b095b

[Alex Buts]

Re #10881 Removed some duplicate workspace loading

and fixed number of unit tests failing due to changes

Changeset: 3c01e1a9d7a5db9d404ca516990dcade4aa312d2

[Alex Buts]

Re #10881 MARI reduction from workspace works

Changeset: 2a5927e3f992b4bc7002d99f8c28fb35e948cba0

[Alex Buts]

Re #10881 File name parser

Changeset: 05da019ad91cb511e9667daabd77513abec3160c

[Alex Buts]

Re #10881 Struggling with separate monitor workspace

Changeset: f48235e71fc2f0b073dd2d8954fc63f90414eab0

[Alex Buts]

Re #10881 Uncommented diagnostics

Changeset: 068f2991225934505d9d4ac608c8a71e214ad2c6

[Alex Buts]

Re #10881 Fixed WB workspace storage (for MARI at the moment)

Changeset: 4e518b6a873d2aa240cf977a826bd69cd0ed4b78

[Alex Buts]

Re #10881 fixed naming in case of separate monitor ws

and fixed system test for MAPS

Changeset: 3eb300c6fba704d784d119c8680bfe1b7271773c

[Alex Buts]

Re #10881 Minir changes to system tests to reflect changes in reduction interface

Changeset: bb193f6e92fd46cdf65382fed9bcf5ad11026942

[Alex Buts]

Re #10881 Fixed LET system test

Changeset: 928a5ec854fdd3a1521fb3f17249405b4da53090

[Alex Buts]

Re #10881 Minor changes to system test to satisfy new interface

Changeset: 4e43203ce702802fd1ec8b8b2c3cc88e77932418

[Alex Buts]

Re #10881 fixed bleed test (and Merlin reduction works)

Changeset: afb464cacdd13eb50d11d340056af507cfaa4a7b

[Alex Buts]

Re #10881 Fixed diagnostics test

Changeset: 84c2d12bab891eb174e8371f4569b004a9f414d7

[Alex Buts]

Re #10881 Fixed diagnostics test

Changeset: f97fd4ed4dca32bd46237942fb2f92bf35e4793d

[Alex Buts]

Re #10881 Sum workspaces works

Changeset: b1f6a9194d8c90518370c25c86695e6f6df90d9f

[Alex Buts]

Re #10881 Remove legacy code not used any more

Changeset: f8b0b46c7fa585bb3e8d9d7e6af0ff56322d728e

[Alex Buts]

Re #10881 Fixed (probably) normalise to mon-2

Changeset: 875c2b88aad264c7a9c9acb5824a44f3bc7d27f6

[Alex Buts]

Re #10881 Intermediate checkout. Verifying NormalizeToMonitor-2

Changeset: aec09297b443a63bf2e019da2b142a3e696a6d8f

[Alex Buts]

Re #10881 Method to identify monitor-2 integration range

Changeset: 7a632d90fe97ba75a3f782cc1456dea8f26646ce

[Alex Buts]

Re #10881 Method returning used monitors list

Changeset: 4302cb684402cd9e3c32125b812be38fa1be0eab

[Alex Buts]

Re #10881 Unit test for get_used_monitors_list

Changeset: 6714158131e0d514ffe6746a27eb87359889679f

[Alex Buts]

Re #10881 Proper normalise to monitor-2 procedure implementation

Changeset: a520c6f921cd05912278173a3424aca279e9d365

[Alex Buts]

Re #10881 Verified and fixed system tests for mon2 normalization.

Changeset: 1ce4ddcbf6c2e558d618867b91b1003855cfb4b3

[Alex Buts]

Re #10881 Changes to normalize to monitor2 system test

Changeset: 0331a4114deffbefe58b95c7f3060e7b6c6ef013

[Alex Buts]

Re #10881 Minor changes and comments

Changeset: 536a4a77f443a3fe33d691d16698e6551c4e742a

[Martyn Gigg]

The LoadLotsOfFiles.py system test has broken - ​http://builds.mantidproject.org/job/develop_systemtests_rhel6/1002/testReport/junit/SystemTests/LoadLotsOfFiles/LoadLotsOfFiles/. det_LET_cycle12-3.dat needs to be added to the ignore list

[Alex Buts]

Re #10881 Fixing system test (added new det.dat file to list of

exclusions)

Changeset: adc5b28af12607f6ddd84eb1908788d40e8ed8c6

[Alex Buts]

Re #10881 Updated IDF options to include&describe monitor-2

integration range

Changeset: df1905aacb40227d955145ae02b64ea03e3b6bd2

[Alex Buts]

Re #10881 Re-enabled "check_abs_norm_defaults_changed option

which verify if user set up sample mass and sample rmm and unit test for it.

Changeset: 055419846531dd661424a030db6b41c699aee6ce

[Alex Buts]

Re #10881 fixing Typos

Changeset: 143cd4a9ef8afee1c742fb23877339701ad7d047

[Alex Buts]

Re #10881 fixing doctest

Changeset: 804e69a9f735bc20c332df587c7f05b105c3f812

[Alex Buts]

Re #10881 fixing normalise ws with missing monitors option

Changeset: 1ead2a992ea3448fbffa46e0d5039a0a6da1933e

[Alex Buts]

Re #10881 Changes to formatting and some string length-s

Changeset: 690d151e8dcb99667f7b406f4b2f9af507f3d8c9

[Alex Buts]

Re #10881 Fixed system test for LETReductionEvent2014Multirep

Changeset: 18f6477afdb663c85d1dc6a2743e1882015bbee1

[Alex Buts]

Re #10881 Typo in MERLIN_Parameters*.xml

Changeset: dae4e45ca5491519d906555c78b549d2035d4ec2

[Alex Buts]

Re #10881 Another typo in the same files

Changeset: fb1d338ebbe5d1508cb6f41d8da44f3cb2d6cc65

[Alex Buts]

Re #10881 Apparently, C++ diagnostics do not understand integer mon

numbers. Shame. Have to revert this to folat monitor number.

Changeset: 204574df8c3a75233d635fa2391b542fb526deb1

[Alex Buts]

Re #10881 Added one more system test for MARI

(verify logic for vanadium run without monitors) Fixed monitor-2 normalization, which makes separate monitor-2 normalization file unnecessary.

Changeset: 3e26a19b03ce4ffcdd49bc9338eb17714c23ccf1

[Alex Buts]

Re #10881 Fixed comparison of run properties when updated params

from instrument.

Changeset: f1d427a01cd22ec35a0019751dcc709cf60510ec

[Alex Buts]

Re #10881 minor comments

Changeset: 91b4f047b36c4050290d46ba6fc8bcd8d5c3a1da

[Alex Buts]

The changes are both in system tests and Mantid, so both need to be merged to master. Test only after #10836 as it relies on changes made there.

It is the final step of refactoring Direct inelastic using python descriptors. All functionality related to a user run has been moved to the run descriptor, and main reduction code has been rewritten to support this functionality. Duplicated code deleted.

Doing that I have identified and fixed number of minor bugs and inefficiencies, particularly possible bugs related to normalization by monitor 2 and inefficiencies in recalculating and reloading the same wb run multiple times.

Doing so I have ensured that system tests produce the same results as before. (And mon-2 normalization works properly -- produce the same result as all other normalizations)

Number of additional unit and system tests were written, so old and new tests ensure the changes are correct -- tester may be satisfied by this.

If a tester wishes to proceed with code review, he/she should pay attention to data workflow -- if any spurious workspaces appear(left) in Mantid (only result workspace, integrated vanadium, and may be white beam should be left there) and if any unnecessary load operation download data multiple times.(Analyse run history for that) Please note, that monovan is loaded/processed multiple times for multirep mode. This is to address in ticket #10684

Current workflow correctness guaranteed by system tests but tedious tester may try to change data sources (provide a run number instead of workspace or v.v. or include/exclude monitors with workspace when possible) and see if it holds.

[Alex Buts]

Merge branch 'feature/10836_SystemDictionary' into feature/10881_RunDescriptor

Full changeset: ​e6cd6a0340ff42aabfebf08ccc3a787daf6df218

[Ian Bush]

Merge remote-tracking branch 'origin/feature/10881_RunDescriptor'

Full changeset: ​b0be876c5eb07355e4dae614c285c696514aa905

[Ian Bush]

Merge remote-tracking branch 'origin/feature/10881_RunDescriptor'

Full changeset: ​c6d9fe4c4832f3981534fa8ff22572055f0a7c70

[Ian Bush]

There are a lot of changes here, a lot of which are just tidying up. The other changes seem sensible as best I can tell, and the system tests all seem to be passing fine, so accepting this as fixed.

[Alex Buts]

Merge branch 'feature/10881_RunDescriptor' into feature/10958_UpdatedReductionWrapper

Full changeset: ​5343a9627d05193ff739c42bbfef32dfd47371a3

[Alex Buts]

Merge branch 'feature/10881_RunDescriptor' into feature/10958_UpdatedReductionWrapper

Full changeset: ​5343a9627d05193ff739c42bbfef32dfd47371a3

[Stuart Campbell]

This ticket has been transferred to github issue ​11720