Ticket #3400 (closed: fixed)
ManagedWorkspace forget the spectra/detector lookups
Reported by: | Martyn Gigg | Owned by: | Russell Taylor |
---|---|---|---|
Priority: | blocker | Milestone: | Iteration 30 |
Component: | Mantid | Keywords: | |
Cc: | zikovskyjl@… | Blocked By: | |
Blocking: | Tester: | Martyn Gigg |
Description
When moving to the ISpectrum implementation, the detector IDs moved to be contained within the ISpectrum objects. A managed workspace saves and reads ManagedDataBlocks to a file and this means that when it saves the data block it only knows about the data.
The SpectraDetectorMap was always held in memory and so this was not lost when saving the blocks to disk.
This needs to be fixed urgently as it renders ManagedWorkspaces effectively useless.
Attachments
Change History
comment:4 Changed 9 years ago by Russell Taylor
- Status changed from new to accepted
- Owner set to Russell Taylor
comment:5 Changed 9 years ago by Russell Taylor
- Cc taylorrj@… removed
My first thought was to write out the spectrum number and detector IDs to the file. However, that won't work because we rely on the data relating to a spectrum being of fixed length (to seek to the right point in the file) and the number of detector IDs is not fixed.
The two options I see are:
- Make sure that a managed workspace always keeps a full set (i.e. one per spectrum) of ISpectrum/Histogram1D instances in existance, and just fills and collapses the data vectors as things go back and forth to file. Currently, Histogram1D instances (actually ManagedDataBlock instances, which hold 1 or more Histogram1Ds) are created and destroyed as they enter and leave the mru list.
- Have a ManagedWorkspace hold a separate copy of all the spectrum/detector information and just push it into the ISpectrum/Histogram1D object on its creation.
The second option seems like a bit of a hack (with maintenance issues) but the first seems like a lot more work.
A further issue is that we need to make sure that any calls that in principle go through all the spectra looking for certain detector IDs, for example, don't lead to all the data being churned on and off the disk.
comment:10 Changed 9 years ago by Janik Zikovsky
Also: Make Workspace2D's data[] vector shared with the data in ManagedWorkspace2D for more robustness.
comment:11 Changed 9 years ago by Janik Zikovsky
In [13534]:
comment:12 Changed 9 years ago by Janik Zikovsky
In [13535]:
comment:13 Changed 9 years ago by Mathieu Doucet
In [13556]:
comment:14 Changed 9 years ago by Russell Taylor
Some tests on a lousy win32 box:
Run ConvertUnits on a ManagedRawFileWorkspace in place: Last release ~43s, now perhaps ~47s
Clone a GEM ManagedRawFileWorkspace into a new ManagedWorkspace: Last release ~45s, now ~95s
Tests on a Mac showed similar behaviour.
comment:15 Changed 9 years ago by Martyn Gigg
Last night the system tests took ~5 hours rather than the usual 1.5 hours. I've attached the log. A cursory glance seems to suggest that the loading parts are the worst.
comment:16 Changed 9 years ago by Russell Taylor
Actually, it's MaskDetectorsInShape that accounted for 3.75 hours of the 4.5 total time spent running algorithms.
comment:17 Changed 9 years ago by Russell Taylor
- Status changed from accepted to verify
- Resolution set to fixed
ManagedWorkspaces should now work. A ticket ([3740]) has been created for the next iteration to look into the suggestion that performance has been degraded.
comment:18 Changed 9 years ago by Martyn Gigg
- Status changed from verify to verifying
- Tester set to Martyn Gigg
comment:19 Changed 9 years ago by Martyn Gigg
Tested with ManagedRawFileWorkspace and standard ManagedWorkspace from raw and NeXus files. The detector/spectrum tables look okay. As mentioned above #3740 has been created to examine the performance implact.
comment:21 Changed 5 years ago by Stuart Campbell
This ticket has been transferred to github issue 4247