Ticket #2355 (closed: fixed)

Opened 10 years ago

Last modified 5 years ago

Improve performance of SaveNexus for eventWS

Reported by: Nick Draper Owned by: Janik Zikovsky
Priority: major Milestone: Iteration 28
Component: Mantid Keywords:
Cc: Blocked By:
Blocking: Tester: Michael Whitty

Description

From Garrett:

If I do the following two things,

1) load an event nexus into a workspace
2) right click and press save Nexus

It takes all of 7.57 seconds to read it in then it takes 725 seconds to save it.
I think there is some optimization needed in the save command.

what this means is that if I have say 10 workspaces and save a .mantid file,  it takes forever.

Also the progress bar does not at advance.

Change History

comment:1 Changed 10 years ago by Janik Zikovsky

  • Status changed from new to accepted

comment:2 Changed 10 years ago by Janik Zikovsky

This refs #1525: SaveNexus is not finished for event workspaces.

comment:3 Changed 10 years ago by Janik Zikovsky

(In [9342]) Refs #1525, #2268, #2269, #2355: SaveNexusProcessed and LoadNexusProcessed for event workspaces.

comment:4 Changed 10 years ago by Janik Zikovsky

(In [9369]) Refs #2355: fix occasional segfault in tests.

comment:5 Changed 10 years ago by Janik Zikovsky

(In [9377]) Refs #2355

comment:6 Changed 10 years ago by Janik Zikovsky

  • Status changed from accepted to verify
  • Resolution set to fixed

SaveNexus is significantly faster and there is progress reporting. A file that takes 18 seconds to load takes 88 seconds to save - that it is still fairly slow but that is limited by Nexus writing - see ticket #2342.

comment:7 Changed 10 years ago by Michael Whitty

  • Status changed from verify to verifying
  • Tester set to Michael Whitty

comment:8 Changed 10 years ago by Michael Whitty

  • Status changed from verifying to reopened
  • Resolution fixed deleted

File saved can not be loaded (in [9515])

Steps to reproduce:

  • Load CNCS_7860_event.nxs using LoadEventNexus
  • Right click on workspace, select 'Save Nexus'
  • Try to run either LoadNexus or LoadNexusProcessed, you receive error message "bad allocation".

comment:9 Changed 10 years ago by Janik Zikovsky

Are you using windows. I tried to reproduce the error on Linux but could not. See also ticket #2419.

comment:10 Changed 10 years ago by Michael Whitty

Yes, I am using windows xp 32bit.

comment:11 Changed 10 years ago by Janik Zikovsky

  • Milestone changed from Iteration 27 to Iteration 28

comment:12 Changed 10 years ago by Janik Zikovsky

  • Status changed from reopened to accepted

comment:13 Changed 10 years ago by Janik Zikovsky

(In [10638]) Refs #2355. Some attempts were made at parallelizing writing NXS but HDF does not seem to support that. Compression is the bottleneck.

comment:14 Changed 10 years ago by Janik Zikovsky

Speed comparison depending on the type of compression in NEXUS format. The file is TOPAZ_2170_events, loaded then saved as SaveNexusProcessed.

  • Uncompressed: 38 seconds to write 2.2 GB.
  • LZW compression (default): 308 seconds to write 1.3 GB.
  • SZIP compression: 56 seconds to write 1.7 GB. SZIP requires HDF with SZIP enabled, available from the HDF site but not from repository (license issues).

comment:15 Changed 10 years ago by Janik Zikovsky

(In [10639]) Refs #2355: Compression disabled for eventowrkspaces, for speed.

comment:16 Changed 10 years ago by Janik Zikovsky

  • Status changed from accepted to verify
  • Resolution set to fixed

See #2342 for changes related to SZIP (if any).

comment:17 Changed 9 years ago by Michael Whitty

  • Status changed from verify to verifying

comment:18 Changed 9 years ago by Michael Whitty

  • Status changed from verifying to closed

comment:19 Changed 5 years ago by Stuart Campbell

This ticket has been transferred to github issue 3202

Note: See TracTickets for help on using tickets.