Ticket #7780 (closed: fixed)
Slow deletion event workspaces with large number of spectra and events
| Reported by: | Martyn Gigg | Owned by: | Martyn Gigg |
|---|---|---|---|
| Priority: | critical | Milestone: | Release 3.0 |
| Component: | Framework | Keywords: | |
| Cc: | Blocked By: | ||
| Blocking: | Tester: | Keith Brown |
Description (last modified by Martyn Gigg) (diff)
To reproduce you'll need a large LET event file, e.g. cycle_13_2/LET00011162.nxs
- Load the file, unchecking LoadMonitors
- Hit delete and it will take a few seconds.
If you load the PG3_2358_event file in systemtests/Data, which has more events than the above LET file, then the workspace is deleted much quicker.
Change History
comment:1 Changed 7 years ago by Martyn Gigg
- Description modified (diff)
- Summary changed from Slow deletion of large event workspaces from ISIS instruments on Windows to Slow deletion event workspaces with large number of spectra and events
comment:2 Changed 7 years ago by Martyn Gigg
The difference between the PG3 file and the LET file is in the number of spectra. The PG3 file has ~110 million events in 24794 spectra whereas the LET file has ~96 million events in 40960 spectra.
The more spectra there are the more vectors of events there are to delete.
comment:4 Changed 7 years ago by Martyn Gigg
Set Precount option LoadEventNexus to true by default.
It prevents the vectors from being over allocated by push_back and means the deletion on Windows is then much quicker. The memory fragmentation should also decrease as the allocation to the correct size is done once per vector.
The performance tests will have to be monitored to check on the overall effect on speed when combined with tcmalloc on linux. On windows, at least, the loading is faster. Refs #7780
Changeset: b1803d69ca5de4c3a1771851092b6a2a21d2cb24
comment:5 Changed 7 years ago by Martyn Gigg
Check for invalid workspace indices on precount/compress...
in LoadEventNexus. Some event files can have bad pixel IDs that have been set to EmptyInt(). Refs #7780
Changeset: 57ef4ed6f8f1b338f8588ddc31e19c653dd9bbe6
comment:6 Changed 7 years ago by Martyn Gigg
- Status changed from inprogress to verify
- Resolution set to fixed
Branch: feature/7780_slow_eventws_deletion_windows
Tester: This should be tested on windows with the LET file mentioned in the description. Precount should now be on by default. When this is the case deleting the workspace created by the loading the given file should drop significantly.
comment:7 Changed 7 years ago by Keith Brown
- Status changed from verify to verifying
- Tester set to Keith Brown
comment:8 Changed 7 years ago by Keith Brown
just tested the time it took before fix using the current release version: 12 seconds
comment:10 Changed 7 years ago by Keith Brown
- Status changed from verifying to closed
Merge remote-tracking branch 'origin/feature/7780_slow_eventws_deletion_windows'
comment:11 Changed 5 years ago by Stuart Campbell
This ticket has been transferred to github issue 8625
