SANS: Assess & rectify zero errors in ISIS SANS data

[Stephen King]

ISIS SANS Users have experienced issues least-squares model-fitting reduced I(Q) data output by Mantid because on occasion some data points are assigned an intensity error of zero. (I believe the most likely scenario that causes this is that the reduction procedure has generated a datapoint with I(Q)=0).

After discussion with SANS instrument scientists at other facilities I contend that I(Q) data cannot, ever, have an error of zero, and that any part of Mantid likely to generate such needs to be investigated and subject to appropriate remedial action. What that action should be (ie, what value should be assigned instead of zero) is, perhaps, something that should be consulted on more widely.

Though I raise this ticket on behalf of ISIS SANS users, other ISIS data reduction procedures may be similarly affected.

I am led to believe that SNS SANS reduction procedures do not suffer this defect (but have no first-hand knowledge).

[Nick Draper]

Moved at the end of release 2.2

[Nick Draper]

Moved to milestone 2.4

[Nick Draper]

Moved at the code freeze for release 2.4

[Nick Draper]

Moved to r2.6 at the end of r2.5

[Nick Draper]

Moved to backlog at the code freeze for R2.6

[Anders Markvardsen]

A wider investigation of this has now been done by members of the mantid team.

The main points are that

1. The reason for the zero errors is that data with zero counts are recorded with zero error and that there are good reasons why this make sense in some cases

The conclusion is that a Mantid wide algorithm will not be put in place to overwrite zero errors.

However on an individual group basis, or for a type of detector, algorithms may be created that can optionally adjust zero errors.

What remains in this ticket is to investigate this further.

Note, unfortunately, there is no hard science which tells exactly the best why of adjusting such zero errors (otherwise everyone would do just that!), but there is literature suggesting various possibilities what can be used to adjust such zero errors.

Here I would suggest restricting the discussion to that of looking for an alternative of a zero error in the content of this error being thought of as Gaussian error.

Note for fitting data within mantid work is in progress of adding a Poisson cost function which will gracefully handle the case discussed here. However, when exporting Mantid output into other packages, like sasview, such packages do require gaussian errors, and falls over if zero errors are specified.

[Gesner Passos]

Requirement:

Yes, that when any reduced SANS data (1D or 2D) is written to file, any points with zero errors should have the error replaced by 10^6.

Depending on how you intend to implement this, there may be a small issue here:

So this change should certainly apply to the 1D & 2D RKH format, the 1D CanSAS format, and the NIST 2D format. Whether the data is written by (eg, batch mode)/from (eg, user clicks Save Result) the GUI, or from the user calling up the algorithm directly from the list of Mantid algorithms.

The issue is, it also needs to apply to any 1D data one of our users writes out in CSV format (as some do) - and again, from the GUI or algorithm directly - except, of course, this format is not SANS-specific and we established that there are other instruments who do NOT want zero errors replaced. So this will presumably require an instrument test in the CSV output?

Steve

[Gesner Passos]

We will probably add a step before the save where we will 'translate' the zero-error. Do we really need a flag to go for this behaviour, or this will be the new default action for any new SANS data reduced in Mantid? Gesner

It can be the new default action. Steve

[Nick Draper]

Bulk move to assigned at the introduction of the triage step

[Nick Draper]

Ownership handed over to Peter

[Nick Draper]

It might be worth getting a sample case and working back to understand where the non zero errors are being introduced in the reduction workflow.

[Stuart Campbell]

This ticket has been transferred to github issue ​6457