BinToMDHistoWorkspace: Alternate method for few events + lots of boxes

[Janik Zikovsky]

... where you iterate through every event and place them in the proper bin in the output MDHistoWorkspace.

Hard to parallelize!

[Janik Zikovsky]

In [13204]:

Refs #3369: Fix the performance test. LoadSQW debugging output.

[Janik Zikovsky]

In [13203]:

Refs #3364, Refs #3369: Option on BinToMDHistoWorkspace to iterate through all events instead of by bin. This is significantly faster for workspaces with relatively few events. Option is set to true by default so it can work with Paraview by default.

[Janik Zikovsky]

In [13303]:

Refs #3369: Progress reporting in BinToMDHistoWorkspace alternate method.

[Janik Zikovsky]

In [13395]:

Refs #3369: More performance tests for the binning routine.

[Janik Zikovsky]

In [13397]:

Refs #3369: Performance test, start of the use of implicit function in the binning algorithm.

[Janik Zikovsky]

In [13433]:

Refs #3369: Using getBoxes() limited by an implicit function has sped up the iterateEvents way to bin MDEW from 61 msec to 28 msec (large region) or 37 msec to 6.7 msec (small region). This is for a fake workspace with 1 million events, binning into 60^{3 bins.}

[Janik Zikovsky]

In [13462]:

Refs #3369: Parallelized the IterateEvents method of BinToMDHistoWorkspace. 2.5 times faster for the small iron data set; 5.3 times faster for a 200 Million event uniformly spread fake workspace.

[Janik Zikovsky]

In [13692]:

Refs #3369: Added a check to see if a box is completely within a bin, then it does not need to load the data and just uses the cached total. Disabled parallelized other method (not iterated events) for file-backed systems.

[Janik Zikovsky]

I think this is complete for now, and should work well with the file back-end in Iterate events mode. The per-bin method might be retired in the future, as it is only occasionally faster and is sometimes much slower.

[Janik Zikovsky]

In [13711]:

Refs #3369: Try to fix Mac build.

[Janik Zikovsky]

In [13728]:

Refs #3369: Fix slow performance test because I flipped the parallel/not parallel check on file-backed MDEW.

[Owen Arnold]

This does work. I've been using it for some time with the default IterateEvents option set to on.

[Stuart Campbell]

This ticket has been transferred to github issue ​4216