IntegratePeaksMD does not process results of ConvertToMD properly

[Alex Buts]

The problem is as reported by Dennis below and should be investigated.

First, the good news: The MD workspace from the ConvertToMD algorithm now works with FindPeaksMD. The peaks found are essentially the same, though very slightly less accurate based on testing with one file. That is, 532 of 550 peaks indexed with the MD workspace from ConvertToMD, while 536 of 550 peaks indexed with the MD workspace from ConvertToDiffractionMDWorkspace. I use the same splitting in both cases: split into 2x2x2 boxes, with 20 as the SplitThreshold and 12 for the MaxRecursionDepth. The difference between 532 and 536 peaks indexed may not be significant, and more testing with more files should be done. The UB matrix from each set of peaks was essentially the same, and was correct. The local intensity of the peaks was very different between the sets of peaks.

MD SCD peak integration does not give meaningful results. As a result, I don't think work on this problem is complete yet.

IntegratePeaksMD did not work correctly with the output ConvertToMD. The algorithm completed successfully, but only about 40 out of 550 peaks integrated to non-zero values, whereas virtually all of the peaks should have had non-zero intensities. The 40 non-zero integrated values were way off, for example the first one integrated to 9 instead of 3500.

It will probably take more significant effort to track down what behavior IntegratePeaksMD expects from the MD workspace, that is set up differently by ConvertToMD compared to ConvertToDiffractionMDWorkspace.

[Nick Draper]

Investigate, and then talk to me.

[Nick Draper]

Moved at the code freeze for release 2.4

[Owen Arnold]

Both Dennis and I have been comparing results between the two algorithms. I have found very little difference between the outputs of the two algorithms, and Dennis is finding the same for SC reduction. As Dennis has pointed out, the differences are very small compared to the actual errors associated with the data. It might be that something has been fixed recently to resolve these problems, but both Dennis and I are happy to migrate to ConvertToMD.

I suggest that we could 'gut' ConvertToDiffractionMDWorkspace and wire it directly in to ConvertToMD. Before doing this, we should provide warning to our algorithm users, and to avoid anyone getting upset about slight differences in results. The alternative is to find the source of the small differences, but this could take quite a bit of investigation, without much return.

[Stuart Campbell]

Sorry - only noticed after I took it that you had set Nick to be the tester (probably for a reason) so I am reassigning it back to him.

[Stuart Campbell]

One comment I would have is that it would be nice to be able to have the option to convert to Q (lab) for ConvertToMD.

Or/And have the option NOT to load the goniometer on the load step.

[Owen Arnold]

Stuart, I think that this must be working ok for the LabQ case, because it's being used in Dennis's reduction scripts ​https://github.com/mantidproject/mantid/blob/master/Code/Mantid/scripts/SCD_Reduction/ReduceOneSCD_Run.py

Another way that this could be verified/fixed would be to gut ConvertToDiffractionMD, and wire it up to use ConvertToMD under the hood. That way ConvertToMD would be constrained by all the same test cases as applied to ConvertToDiffractionMD.

[Dennis Mikkelson]

As Stuart mentioned, it would be nice to have an option to choose whether the conversion is to Q(lab) or Q(sample). Line 207 of the script cited by Owen shows that sample coordinates are being used when the conversion is applied to TOPAZ data. When I changed the scripts to use ConvertToMD rather than ConvertToDiffractionMDWorkspace, I had to change the coordinate system for integration to sample coordinates for TOPAZ data. If no goniometer information is available, the conversion must be done to Q(lab). When TOPAZ data is loaded goniometer information is set and ConvertToMD uses it to get Q(sample). Since the choice of Q(lab) or Q(sample) is determined implicitly by what information is available, there could be some confusion when moving from one instrument to another. Perhaps there should be an explicit choice of which coordinates to use. If the user specifies Q(sample) and there is no goniometer information available, the algorithm could either default to Q(lab) and provide a warning, or give an error message and not proceed.

[Stuart Campbell]

This ticket has been transferred to github issue ​6955