SANS: Understand why adding sans2d event files does not seem to be right

[Anders Markvardsen]

When completing #6237 to make the adding of files in the adding tab give the same result as adding of files using Richard python the implementation taken was to add the histogram data stored.

This ticket is to understand why adding sans2d event data appears to give the wrong answer. A guess which may be little truth to it, is that this could be a subble effect caused by the way ISIS stores event data compared to SNS and how the adding algorithm works.

To investigate this convert two event sans2d dataset to histogram data (convert the event data rather than using the histogram data already stored in the nexus files). Then add those together to get result A. Then add the event data and convert these summed data into histogram data, call this result B. What should be the case is that result A = result B regardless of what binning was used etc.

In this ticket test this and then go from there.

[Gesner Passos]

The problem was related to considering the number of monitors as 4, while, the files that were being added had 8 monitors. This caused the Added value to be shifted in relation to the correct value.

So, for simplicity, I've attached the first version of the SANSadd2.py file. Substitute the file from SANSadd2.py with the attached one.

For double check, use this script to generate the acceptable solution:

l = LoadNexus('16183')
r = LoadNexus('16197')
Summed = Plus(l,r)
DeleteWorkspace(l)
DeleteWorkspace(r)

From ISIS Interface, add runs tab, add 16183 and 16197 runs. The result could be SANS2D00016197-add.nxs or SANS2D00016183-add.nxs (depending on the order you added). Load this file.

If you plot the Summed[40] spectrum and SANS2D00016197-add[36] they will match.

Restore the file SANSadd2.py (from the repository, changed in this ticket)

Run again the test, and you will see that Summed[40] matches SANS2D00016197-add[40].

[Gesner Passos]

Old version of SANSadd2.py that has bug.

[Gesner Passos]

Testing loading Event Data directly

[Gesner Passos]

Besides, this ticket also investigated Loading Event Data and comparing with the histogram data already saved at Nexus files (the data that we read using LoadNexus algorithm). You can apply the script checking this ​LoadingEventData. You will see that the difference between the histogram data and the EventData are not relevant. (I found only 2 counts of difference). This means that we can still Load the event data using the LoadNexus without compromising the results, and still getting faster answer.

[Gesner Passos]

Solve the issue of wrong monitors numbers

Now the monitors are taken from the workspace, so, to be robust against changes in these numbers. The spectra are set with the new api, that set all the values using the numpy array. (Faster and easier)

Restore the old Load method, does not skip loading the event files any more.

re #6594

Changeset: 46ca0728179bf8506a594d224b44f408def4d4c8

[Gesner Passos]

So, this ticket should first prove that the loading method trough Load and LoadNexus produces an equivalent dataset (#comment:3 to test it). Another thing, it to explain the bug found at #6237. So, by proving that now we can add two Event Data through its events or histogram data obtaining a equivalent answer we prove that we solved the issue (#comment:2 to test it).

branch name: feature/6594_add_sans2d

[Gesner Passos]

Solve the issue of wrong monitors numbers

Now the monitors are taken from the workspace, so, to be robust against changes in these numbers. The spectra are set with the new api, that set all the values using the numpy array. (Faster and easier)

Restore the old Load method, does not skip loading the event files any more.

re #6594

Changeset: 46ca0728179bf8506a594d224b44f408def4d4c8

[Gesner Passos]

Solve the issue of wrong monitors numbers

Now the monitors are taken from the workspace, so, to be robust against changes in these numbers. The spectra are set with the new api, that set all the values using the numpy array. (Faster and easier)

Restore the old Load method, does not skip loading the event files any more.

re #6594

Changeset: 46ca0728179bf8506a594d224b44f408def4d4c8

[Gesner Passos]

Solve the issue of wrong monitors numbers

Now the monitors are taken from the workspace, so, to be robust against changes in these numbers. The spectra are set with the new api, that set all the values using the numpy array. (Faster and easier)

Restore the old Load method, does not skip loading the event files any more.

re #6594

Changeset: 46ca0728179bf8506a594d224b44f408def4d4c8

[Gesner Passos]

Solve the issue of wrong monitors numbers

Now the monitors are taken from the workspace, so, to be robust against changes in these numbers. The spectra are set with the new api, that set all the values using the numpy array. (Faster and easier)

Restore the old Load method, does not skip loading the event files any more.

re #6594

Changeset: 46ca0728179bf8506a594d224b44f408def4d4c8

[Gesner Passos]

Solve the issue of wrong monitors numbers

Now the monitors are taken from the workspace, so, to be robust against changes in these numbers. The spectra are set with the new api, that set all the values using the numpy array. (Faster and easier)

Restore the old Load method, does not skip loading the event files any more.

re #6594

Changeset: 46ca0728179bf8506a594d224b44f408def4d4c8

[Gesner Passos]

The user reported an error when he tried to sum up: 14259+14246+14236

RuntimeError: Workspace2D::getSpectrum, histogram number 73736 out of range 73736
at line 3 in '<Interface>'
caused by line 121 in 'C:/MantidInstall/scripts/SANS\SANSadd2.py'

To workaround the issue, the ticked #6986 add the workspaces loading only the histogram data, and ignoring that there are event data inside as well. But, it is necessary to correct this. It is also necessary to discuss how we want to save the data, now that slicing events will be available to this release (2.6)

[Gesner Passos]

Solve the issue of adding event data for SANS2D

The problem was a vector overflow. This correct the problem

re #6594

Changeset: d221175003b35a145298a0ca62142ba347b54d73

[Gesner Passos]

Revert "Do not load event data for adding data"

This reverts commit 46dde6dddb9b2fd2a979af6a067c5d48bc5e8250. Commit related to #6986

re #6594

Changeset: e6fb41778b2d7c0ca3c0793be5c0cbe34b2cb15a

[Gesner Passos]

Tester:

• open SANS ISIS Interface
• Select Add Runs
• Add ndxsans2d/Instrument/data/cycle_12_2 to the search path (Manager User Directories)

*Inside the first line add the numbers: 14259, 14246, 14236 clicking ADD after each number

• Finally, click on Sum button.
• It must add all the files, check that loading events appear at the algorithm progress.
• The final result is a histogram data file created at your default output folder.

[Gesner Passos]

feature/6594_add_sans2d_ev

[Jay Rainey]

Testing

• Testing on Ubuntu 12.4.
• Passed using the provided information.
• No longer produces out of range issues.
• Successfully joins event data files and outputs a histogram file as expected.
• Tested with additional variations of adding data files - all successful.

Code review

• Clean and consistent code.
• No visible bugs/problems.

Note: I found a bug when testing this ticket, that was not relevant to the particular code changes above, but is in SANS. This can be seen #7530

[Jay Rainey]

Merge remote-tracking branch 'origin/feature/6594_add_sans2d_ev'

[Stuart Campbell]

This ticket has been transferred to github issue ​7440