LoadEventNexus slow for ISIS event files

[Martyn Gigg]

Part of this is that the ISIS files only have one block of data defined so the parallel loading can't help. On this side can we consider reading the data in chunks as we do with the ISIS raw NeXus files? Probably needs discussion with Freddie.

The other issue here is that the extra work required for redistributing the events at the end of the load is taking the bulk of the time. Can this be parallelised or at least sped up?

A good example file LET00004286.nxs in cycle_11_4 on the ISIS archive.

On Windows I have found the VerySleepy program, ​http://www.codersnotes.com/sleepy, extremely helpful for call graph profiling.

[Nick Draper]

This is waiting on Freddie to start positioning the events in random positions within the bins, when he does so he will tell us the flag to check the file is doing this so we don't have to. -- Instant speed up.

[Nick Draper]

Moved to iteration 33 at iteration 32 code freeze

[Nick Draper]

Nick,

I am now randomising the events within a bin rather than putting them all at the bin centre – this behaviour is indicated by a new field in NXevent_data called "event_time_offset_shift" which will have the value "random" in such cases.

I have re-ended run WISH00020020 with this change in place, so WISH00020020.nxs on the central archive will have the usual all bin-centre events and
olympic\babylon5\scratch\freddie\WISH00020020_with_random_shift.nxs is the example with the random shifts applied Regards,

Freddie

[Roman Tolchenov]

Re #4258. Skip randomization if data already random

Changeset: 74521c9b49d358e00a6aace539ff54b4de4c2480

[Roman Tolchenov]

Re #4258. Skip randomization if data already random

Changeset: 74521c9b49d358e00a6aace539ff54b4de4c2480

[Martyn Gigg]

Verified that the given WISH file drops from 80->6 seconds after after applying this fix.

[Stuart Campbell]

This ticket has been transferred to github issue ​5105