Generic load algorithm does not load monitors as(and) LoadISISNexus do not allow monitors loading

[Alex Buts]

Let's take file MER23092.n001 which is histogram mode nexus file from Merlin taken from cycle 2014_2

w1=Load('MER23092.n001',LoadMonitors=1)

fails with strange error (can not find a histogram with appropriate number)

This may be because LoadISISNexus does not allow loading monitors and LoadNexusMonitors should be used to load them or due to other reasons but generic loader should be able to load data and monitors from neuxs file in a way, it does for any other loader.

[Alex Buts]

refs #9989 Crude fix allowing monitors loading when the later are

placed after the detectors. Ignoring different time channels, if any are in the file.

Changeset: 718d34808e6a6326430a308402d5675fc1246b77

[Alex Buts]

refs #9989 new property to load monitors

Changeset: b77cff67a3621caf021209b9ceda85d24e3db4b1

[Alex Buts]

refs #9989 Make LoadRaw monitor loading options transferable

to provide common code for LoadRaw3 and LoadISISNexus

Changeset: 492991c5556949a413eb3aa3b851a342fa397933

[Alex Buts]

refs #9989 Load monitors "Exclude&Separate" works for main workspace

though unit tests are needed and prospective include LoadNexusMonitors for "Separate" option

Changeset: e31757fedad009457665cd2d3c12d27bd61afb94

[Alex Buts]

refs #9989 Unit test for LoadMonitorsSeparately option

and some small niceness to the code, which performs this operation (comments, var names, code duplication).

Changeset: 23863776bab89d3e9133805e1a02907a383eac7e

[Alex Buts]

refs #9989 Fixing unit test

Changeset: f0092432761e177c4f3bcebe8c3fbfaefa44f9f3

[Alex Buts]

refs #9989 Unix compilation error

Changeset: 1597485ebe8b4dade55cded2288ea1c1d2e336fa

[Alex Buts]

refs #9989 doxygen error

Changeset: 294cfddc9fdb82f7bc18d582d062cac0c1923c1f

[Alex Buts]

refs #9989 Preparing LoadRawHelper for use with LoadISISNexus2

+ changes intended to make logic of LoadISISNexus2 simpler

Changeset: 87a745864ca0afef98a5e9e20a833bff8843f629

[Alex Buts]

refs #9989 Include&Exclude monitors seems work fine

(with groups too)

Changeset: da40d52c8c56ccbef7f5973f98891316ad05737d

[Alex Buts]

refs #9989 Doxygen errors and compiler warnings

Changeset: 10c743471f93ded9d13285196ba8fb393e5022db

[Alex Buts]

refs #9989 C++ check and doxygen warnings

Changeset: b87d6c18e408a5dd172e33f941ad30118a9f67bc

[Alex Buts]

refs #9989 Kind of works

Changeset: 3f35bb777a57a6db4ecd8d4aa2249335b4cc5d15

[Alex Buts]

refs #9989 Doxygen errors

Changeset: 8dae714e806ff0c9bbd44ab951e6dc6c87c04237

[Alex Buts]

refs #9989 Should fix SANS2D data loading

Changeset: 862b30f67c561bc5892adc577b7f8c03ac9472a1

[Alex Buts]

refs #9989 Setup monitor workspace attached to data ws

and little code improvements.

Changeset: 1bad2767406af308561cd8c11ffaa6d988cb2e64

[Alex Buts]

The code described in the ticket should work now. This requested changes in the code much larger then I had ever anticipated.

The problem with the generic loader above was that LoadISISNexus2, invoked by it after this command did not understand option "Load Monitors separately"

To fix this ticket I've implemented two and accounted for third thing:

1) An interface (borrowed from load raw), which allows to select separate monitor's loading.

2) A possibility to load monitors which have different time binning wrt the detectors.

3) Sometimes, scientists on instrument mix up wiring table and monitors data go into detectors data. I do not know if monitors data are written into monitors block, but I've tried explicitly identify this situation and reading the monitors data from the detector's table in this case.

LoadISISNexus have now two logically separate operation modes. The old one, which is default and should work as before if LoadMonitors->Included is selected.

The new one, where spectra selection options are largely ignored should be common for LoadISISNexus, LoadRaw and LoadEventNexus. It is necessary for seamless functioning of reduction from common loader's interface and loads monitors together, separately or excludes them from workspace.

I tried but have not implemented LoadMonitors->Separately for group workspace. If somebody needs this option, a separate ticket to do that should be issued.

I tried this loader on range of files, presented at ​ftp://ftp.nd.rl.ac.uk/scratch/abuts/Ticket_9989/ It works fine for me but changes to underlying code are so big, that tester should try to break the loader selecting different files and different mixtures of the interface options.

Special attention should be paid for correspondence between detectors ID-s, spectra numbers and loaded data.

[Alex Buts]

it worth checking if the changes fixed #9499

[Federico M Pouzols]

I tested with the files included in the ftp url given above, and also ome other nexus and non-nexus test and systemtest files (like TOPAZ_3680_5_sec_MDEW.nxs, MAP17186.raw, LOQ48097.raw, etc.). I could not make it fail or crash:

• It seems to work when using different values for Spectrum Min/Max/List (and combining them). The indices and spectra numbers are correct.

• LoadMonitors->Include/Exclude/Separate work well for different files.

• Tests and systemtests pass

• The code is well organized and documented, and new unit tests have been added.

• The (not implemented) loading LoadMonitors->Separately for group workspaces produces an informative message.

I only noticed that the value for LoadMonitors was accepted as an integer before:

w1=Load('MER23092.n001',LoadMonitors=1)

it used to work. But now it will produce an error

home/fedemp/test/build-mantid/bin/mantid/simpleapi.pyc in Load(Filename, **kwargs)
    109             logger.warning("You've passed a property (%s) to Load() that doesn't apply to this file type." % key)
    110             del final_keywords[key]
--> 111     _set_properties(algm, **final_keywords)
    112     algm.execute()
    113 

/home/fedemp/test/build-mantid/bin/mantid/simpleapi.pyc in _set_properties(alg_object, *args, **kwargs)
    530             alg_object.setPropertyValue(key, value.name())
    531         else:
--> 532             alg_object.setProperty(key, value)
    533 
    534 def _create_algorithm_function(algorithm, version, _algm_object):

TypeError: No registered converter was able to produce a C++ rvalue of type std::string from this Python object of type int

This must be related to other ongoing changes that I'm not directly aware of, as it also affects other types of files.

[Alex Buts]

Merge branch 'master' into bugfix/9989_LoadISISNexusAndMonitors

Full changeset: ​93df62213e29d30b7cc8f9a5d95467e48fbf36ed

[Federico Montesino Pouzols]

Merge remote-tracking branch 'origin/bugfix/9989_LoadISISNexusAndMonitors'

Full changeset: ​acfcb3fce4db1ac91c21482a32667d2ead741794

[Stuart Campbell]

This ticket has been transferred to github issue ​10831