Use matrix workspace to store splitters for fast frequency log

[Wenduo Zhou]

The current way to generate event filter and then filter events does not fit for high-frequency log. Generating and parsing a SplitterWorkspace (i.e., a TableWorkspace) containing near million rows is very costly.

Therefore, an alternative way is to record the splitters in a matrix workspace, and filter events from this workspace.

Enable GenerateEventFilter to output either SplitterWorkspace (as usual) or MatrixWorkspace (new) by user's choice.

[Wenduo Zhou]

Refactored code. Refs #8685.

Changeset: e4511983d4c55f1218a914662b1bc68afb02c021

[Wenduo Zhou]

Implemented matrix workspace output. Refs #8685.

Also added several unit tests for the new feature.

Changeset: e003fdc698650e73c18c27d170d524e9111b1150

[Wenduo Zhou]

Enabled to split events by vector splitter. Refs #8685.

Changeset: a6bd39122ed0e56722891c183425496888fba70c

[Wenduo Zhou]

Enabled to split events using splitter matrix workspace. Refs #8685.

Changeset: c774e4c9def3252796c74963b6bfa9e06d14d877

[Wenduo Zhou]

Added method to split events. Refs #8685.

Added and tested a method to filter/split events with spliters instored in vectors.

Changeset: e3487731d0c138192eca2afde97fb5c3e2f8c8b6

[Nick Draper]

Moved to backlog at the end of Release 3.1

[Wenduo Zhou]

Enabled to generate filters in matrix workspace. Refs #8685.

1. Enabled to generate filters and output them in MatrixWorkspace;
2. Enabled to generate filters in parallel (not finished yet).

Changeset: 565927f0d59e7c36678efcd009998642ef54f229

[Wenduo Zhou]

Enabled to return with debug message. Refs #8685.

Changeset: d3d6754bcb700389fc2974b5aaaeacccdb3010ac

[Wenduo Zhou]

Added new unit test for returning matrix workspace. Refs #8685.

Changeset: 365eae123aef01041b7f0dae187fc398fcc7dd71

[Wenduo Zhou]

Add debug output. Refs #8685.

1. Enabled to output more detailed debug information;
2. Code refactored for parallelization by OpenMP.

Changeset: afa81763e95e6d572e787ef2b376c0cb72af35db

[Wenduo Zhou]

Resolved conflict. Refs #8685.

Changeset: 703e658fc92a5e468fcc535b81c784765c0bc566

[Wenduo Zhou]

Adapted to new FilterEvents splitter workspace property. Refs #8685.

Changeset: 37d033c5fe5ee3719362a9730dcdf3ef4d6e9231

[Wenduo Zhou]

Parallelized with OpenMP. Refs #8685.

Changeset: f256fdbad0eba00941ae868588b511c856eba479

[Martyn Gigg]

There is a cppcheck warning in ​GenerateEventsFilter

[Wenduo Zhou]

Improved performance on speed. Refs #8685.

1. Removed log in debug level in loop.
2. Reduced access to a huge vector.
3. Refactored the operation on DateAndTime and time_duration for speed.

Changeset: da57667c8e7d913cfcd650027795b0b31a0ef6d7

[Martyn Gigg]

There is a compiler warning about an ​unused variable

[Wenduo Zhou]

Cleaned up code for logging and multithread. Refs #8685.

Changeset: 2b81df764254340bdab37e11546973f0553d8094

[Wenduo Zhou]

Cleaned the codes. Refs #8685.

Changeset: 156debd74588bd9f568e15c8092a1a369cc14de8

[Wenduo Zhou]

Fixed a compiler warning. Refs #8685.

Changeset: f1b865f9397969b8f544ae1f4e4500c6a28244d9

[Wenduo Zhou]

Refactored filtering by multiple log values. Refs #8685.

Changeset: 6dba5722f2d5c62f95a193a623d1f57f5381738a

[Wenduo Zhou]

Refactored algorithm to adding new splitter. Refs #8685.

Changeset: aa9ebbb978640601bb0755de754cc9273c98062c

[Wenduo Zhou]

Edit wiki and cleaned the codes. Refs #8685.

Changeset: 4c3523ae86ffbd5941142d1630eb5c4572b6e6aa

[Wenduo Zhou]

For tester

1. Get access of VULCAN run 20002 and/or 20006;
2. Download script verify8685a.py;
3. Before merging the change from this ticket, run script by choosing run number as 20002 or 20006. You will find it takes very long time to generate event filters and even longer time to filter events;
4. Download script verify8685b.py. Compare it to verify8685a.py to see the difference; Be noticed that the option 'GenerateTOFCorrection' in FilterEvents is a new functionality.
5. Merge the change from this ticket and build Mantid;
6. Run verify8685b.py and compare the speed.

[Martyn Gigg]

There are still some ​cppcheck warnings that need to be addressed in GenerateEventsFilter & FilterEventsByLogValuePreNexus

[Wenduo Zhou]

Suppressed a cppcheck error. Refs #8685.

Changeset: 2ba0d4bad9b23c55b4af374fd270258e2c479656

[Wenduo Zhou]

Fixed a Windows compilter error. Refs #8685.

Changeset: 2eceef66b04d5e4941e7f4dbf21c5cd8b2edae9a

[Wenduo Zhou]

Turned on parallel and cleaned code. Refs #8685.

Changeset: 569f2e738e86249dcdefc31139bd7499c30cdcf3

[Wenduo Zhou]

Fixed a doxygen warning. Refs #8685.

Changeset: 26483167ac04c4ddf37a2abee9c9a4dbddd6acc4

[Wenduo Zhou]

Corrected some doxygen warnings. Refs #8685.

Changeset: 282bcc80c9dd68bb1b1392b1fe9e8086754414ee

[Russell Taylor]

There are a bunch of new memory leaks in EventListTest (spotted because I've been squashing the older ones). I can show you where they are.

[Wenduo Zhou]

Removed progress bar in openmp loop. Refs #8685.

Changeset: 8360cba88c28d109eec79f0463e8eb48d5914d76

[Wenduo Zhou]

Fixed some memory leaks. Refs #8685.

Changeset: 66ddc8fe9dbe57f71128f5cca03fa29462df978f

[Russell Taylor]

Best not to close things until the builds have passed....

[Wenduo Zhou]

Modified an error message. Refs #8685.

Changeset: 39f169c6dc6f98d553042989864352ab305eb840

[Andrei Savici]

Could not run the python scripts in the before case. No FastLog and no UseParallelProcessing. Please update testing procedure

[Wenduo Zhou]

Test script is corrected.

[Andrei Savici]

verify7789a.py​ still has a problem in FilterEvents. There is no SplitSampleLogs keyword

[Wenduo Zhou]

Removed an unnecessary warning log. Refs #8685.

Changeset: 46e9acd2dabb0db97145f16ff5fe50b1a14d3000

[Wenduo Zhou]

Removed a progress update within OpenMP loop. Refs #8685.

Changeset: 4f5809aa0f3f550f763ba9137d5b33268c2d6b87

[Andrei Savici]

Refs #8685. Merge 'origin/feature/8685_matrixws_filter'

Conflicts:

Code/Mantid/Framework/Algorithms/src/GenerateEventsFilter.cpp

Changeset: a0e7b5ee47f2e854e5b5c771cd9f0ccba8a8335c

[Stuart Campbell]

This ticket has been transferred to github issue ​9529