SaveNexusProcessed: Progress reporting is terrible

[Nick Draper]

It jumps instantly to 30% reporting Writing Parameter Map and then waits for a long time before completing.

[Nick Draper]

Pete, can one of your team handle this?

[Nick Draper]

Bulk move of tickets at the end of iteration 28

[Nick Draper]

Accepted and assigned tickets moved at iteration 29 code freeze

[Nick Draper]

Bulk move of tickets to iteration 31 at the iteration 30 code freeze

[Karl Palmen]

A look at the code shows it arbitrarily allocates the first 30% to Opening the file, writing the header and writing the sample and instruments and the remaing 70% to precessing events.

In all the cases I've tried, SaveNexus has been so fast that progress reporting has not mattered. I expect the cases where it has mattered each have a large number of events, which may take 99% or more of the processing time.

[Karl Palmen]

Reduced Progress shown for initial part of SaveNexusProcessed to 5%

This is a quick fix for re #2415. I also rearranged code to make this figure easier to change in future or make dependent on the type of workspace or any of its easily read properties.

Signed-off-by: Karl Palmen <karl.palmen@…>

Changeset: 5af6911e46771f47d250e33352248ddd050f592f

[Anders Markvardsen]

Tested by loading MERLIN data from
isis\inst$\NDXMERLIN\Instrument\data\cycle_11_4 of size about 200MB, and then right-clicking over workspace select 'Save Nexus' (which exe SaveNexusProcessed). For such files the progressing bar jump to 3% where it sits until it eventually jumps to 100% after about 30-40 sec on my pc.

Karl see if there is an easy fix for this. If not move this ticket to the next iteration.

[Nick Draper]

Moved at end of release 2.1

[Karl Palmen]

I expect the problem is that the number of events to be saved is not known in advance and so progress reporting during this phase cannot be realistically organised.

[Karl Palmen]

A look at the code shows that it does have the number of events in advance and even reports progress once per histogram. This is done within a Parallel Interrupt Region and perhaps does not work for this or perhaps for events, there is only one histogram.

[Karl Palmen]

A run of the code with debugger when running such a Merlin file, reveals that the part after the 3% is run by NexusFileIO.cpp, which has no progress reporting whatsoever. This seems to be the cause of the bad progress reporting.

[Karl Palmen]

I'll try a new constructor for NexusFileIO that takes a progess object as argument and then use it if not null.

[Karl Palmen]

Make progress reporting dependent on type of workspace re #2415

Signed-off-by: Karl Palmen <karl.palmen@…>

Changeset: 14ec5a8796a5d74e8ebfd07ac43d9fb258037955

[Karl Palmen]

Get NexusFileIO.cpp to do some of the progress reporting re #2415

Signed-off-by: Karl Palmen <karl.palmen@…>

Changeset: 00d85009445e5825b2cef6d2408889279e6d5458

[Karl Palmen]

Progress reporting improved. It could be improved further in a new ticket.

[Karl Palmen]

For testing, load a MERLIN file from isis\inst$\NDXMERLIN\Instrument\data\cycle_11_4 or similar of size about 200MB, and then right-clicking over workspace select 'Save Nexus' (which exe SaveNexusProcessed). The progress bar should jump a little several times.

[Karl Palmen]

You may also wish to test an event file such as those from LETS to verify sensible progress behaviour then.

[Stuart Campbell]

This ticket has been transferred to github issue ​3262