MDImage Performance Investigation

[Owen Arnold]

TESTER: DO NOT MERGE TO MASTER

MDHistoWorkspace provides a great interface for the existing MDWork, but also has great potential for Imageing work for the near future.

We are now seeing that finding contiguous blocks of memory for the workspaces is proving very difficult and therefore the usability of this workspace type is limited.

An area of exploration would be to attempt to find some way to breaking the arrays into piecewise contigous chunks. This is effectively what happens with a Workspace2D, which is very efficient in memory. This is an experimental piece of work. If we find that performance is badly negatively effected, then it will not prove to be a suitable solution.

Here's my suggested approach to this problem:

• Identify, or write some performance tests that are heavily dependent on the MDHistoWorkspace performance. ConnectedComponentLabelingTestPerformance and IntegratePeaksUsingClustersPerformanceTest spring to mind.
• Find a limiting case where we exhaust memory. Again, example scripts such as this one ​http://trac.mantidproject.org/mantid/ticket/9360#comment:4 run several times with the Number of Bins argument to BinMD increased to say 800 for all would be a good example.
• Extract the container types in MDHistoWorkspace to a new type MDHistoWorkspaceContainer. It may be a good idea to template this by the element type. Expose the subscript operator[] so that the consumer usage of these containers will still work.
• Initially, Implement the held container in MDHistoWorkspaceContainer as an array, but change it to a deque and see how the tests above fair. Could also try a piecewise continuous array, or a vector with a custom allocator.

[Owen Arnold]

It appears that all Image types, including IMDHistoWorkspace fundamentally expose the raw arrays via a pointer to the first element. This forces the data to be served-up in a SINGLE contiguous block because otherwise the pointer arithmetic would not work. Not sure what to do here. Options are:

• Remove the raw access option, this probably will affect performance, but would make the interface better
• Create an array to return on demand, but this will increase memory usage, not decrease it

[Nick Draper]

After discussion with Owen, this has been converted into an investigation ticket. Therefore CORE removed, converted to task, and NO CODE CHANGES SHOULD BE MERGED TO MASTER.

In a local branch (no published), time the loading, saving and visualization of a dataset, remove the raw access option and do a simplistic repair of the affected functions and perform the timing again.

This should be timeboxed to 2 days maximum.

The results if positive of this may lead to a separate CORE design and implementation ticket.

[Nick Draper]

Moved to Backlog at the code freeze of release 3.2

[Stuart Campbell]

This ticket has been transferred to github issue ​10231