Ticket #1694 (closed: fixed)

Opened 10 years ago

Last modified 5 years ago

SumSpectra is extremely slow on large data sets

Reported by: Martyn Gigg Owned by: Martyn Gigg
Priority: major Milestone: Iteration 25
Component: Mantid Keywords:
Cc: Blocked By:
Blocking: Tester: Michael Whitty

Description

On an 8-core machine with 92Gb of RAM, loading MERLIN and collapsing all spectra into 1 using SumSpectra (excluding monitors) takes ~120 seconds.

This seems very long for a simple summation.

Change History

comment:1 Changed 10 years ago by Martyn Gigg

  • Status changed from new to accepted

comment:2 Changed 10 years ago by Martyn Gigg

(In [6526]) Improve performance of SumSpectra. The hinderance was the spectra remapping using the remap function, this is extremely slow. Changed to clear then add the relevent detectors as it go along. This reduces the time for summing a whole MERLIN data set from ~100 secs to ~1sec! Re #1694

comment:3 Changed 10 years ago by Martyn Gigg

  • Status changed from accepted to verify
  • Resolution set to fixed

comment:4 Changed 10 years ago by Michael Whitty

  • Status changed from verify to verifying
  • Tester set to Michael Whitty

comment:5 Changed 10 years ago by Michael Whitty

  • Status changed from verifying to closed

On my system (dual core 3ghz, 2gb ram) summing the MER02257 raw file into a single spectra took 19.48 seconds in r[7143] compared with 366.33 seconds in r[5988]. Good enough, I say!

comment:6 Changed 5 years ago by Stuart Campbell

This ticket has been transferred to github issue ​2541

Note: See TracTickets for help on using tickets.