SliceMD does not works on file-based workspaces (practically).

[Alex Buts]

To check the ticket one needs large dataset.

I was using ~30Gb Fe200Mev dataset loaded from sqw file using loadSQW algorithm

making base slice (Horace marking:) plain=[1,0,0],[0,1,0] q1=[-10,0.1,10], q2=[-10,0.1,10],q3=[-0.2,0.2], dE=[-30,30] from the file based workspace takes ages. I wait for 5 minutes and cancelled on promise of another 20 minutes while the speed of the cut decreases.

Similar job done over the file loaded in memory completes and takes about 70sec. (all memory is occupied)

Mantid-memory based script: Add target datafile to load algorithm to make the script file-based

LoadSQW(Filename=r'D:/Data/Fe/Aug10_v1/sqw/fe_E200_8K_clear.sqw',OutputWorkspace='Fe400K')
SliceMD(InputWorkspace='Fe400K',AlignedDim0='Q_\zeta,-3.72425,10.1672,100',AlignedDim1='Q_\xi,-8.75388,10.1672,100',AlignedDim2='Q_\eta,-0.2,0.2,1',AlignedDim3='E,-30.5,30.5,1',OutputWorkspace='Slice1',TakeMaxRecursionDepthFromInput='0',MaxRecursionDepth='20')

[Alex Buts]

refs #5873 some modifications to MDBox

allowing default memory allocation. Will see if it does not break something.

Changeset: 34075179c2e89e69256fc4ed9f37e134a2cb2a73

[Alex Buts]

refs #5873 Moved BinMD & SliceMD into MDAlgorithms.

This is MD algorithms after all.

Changeset: 8c9698435bf7cc0eaf9156e94c86af462e941cb1

[Alex Buts]

refs #5873 temporary reverting memory expensive changes

Changeset: 851fe15028504f2408f610480749828bb1f7ea22

[Alex Buts]

refs #5873 fixing VATES tests errors

Changeset: 9fc8a8a189d75beb7493aee3a30ca920bea7782c

[Martyn Gigg]

Remove MDAlgorithms dependencies from VatesAPI Refs #5873

The algorithms are now created via the factory in API.

Changeset: 01c3ba4ded33c2fb692c1334f9565993c41e5180

[Alex Buts]

refs #5873 Fix load large file-based workspaces

cased by default box size changes.

Changeset: 72b734c7facfde9cf73d33064c4f0cab25251b05

[Alex Buts]

refs #5873 Attempt to block putting events into write buffer

when doing SliceMD.cpp The question remains why should one put them into write buffer when doing getConstEvents but will see.

Changeset: 994863034416160d29e72c3672a8165d6db08dea

[Alex Buts]

refs #5873 Disabled write cash on BinMD.cpp

why do you need write cash for getConstEvents anyway?

Changeset: 714d06ae9898c07bd863be7dda93bdccb2ef7d33

[Alex Buts]

refs #5873 this looks like fix it

Disabled the write cache for input workspaces and SliceMD/BinMD algorithms. Now it generates(slices) approximately the same number of events as Horace at approximately the same amount of time. This amount is still different from Horace with similar setting but not fundamentally different, so this is the topic of another ticket.

Changeset: 825bbc038fe6e5992ad3fbe650033471260966f4

[Alex Buts]

Note: Linear binning with large accuracy (e.g. 50x50x50x1 for slice described in the ticket) is very expensive as recursive box structure occupies too much memory)

the coarse binning can be set up as 5x5x5x1 and fine binning is achieved on the internal box structure.

[Alex Buts]

refs #5873 fixing progress bar

Changeset: 32382d0731b7c93d9e009ed32887a150a2505e8b

[Roman Tolchenov]

Tested with a 10G dataset. The performance of SliceMD with a file based workspace is not worse than with a memory based one.

[Alex Buts]

refs #5873 some modifications to MDBox

allowing default memory allocation. Will see if it does not break something.

Changeset: 34075179c2e89e69256fc4ed9f37e134a2cb2a73

[Alex Buts]

refs #5873 Moved BinMD & SliceMD into MDAlgorithms.

This is MD algorithms after all.

Changeset: 8c9698435bf7cc0eaf9156e94c86af462e941cb1

[Alex Buts]

refs #5873 temporary reverting memory expensive changes

Changeset: 851fe15028504f2408f610480749828bb1f7ea22

[Alex Buts]

refs #5873 fixing VATES tests errors

Changeset: 9fc8a8a189d75beb7493aee3a30ca920bea7782c

[Martyn Gigg]

Remove MDAlgorithms dependencies from VatesAPI Refs #5873

The algorithms are now created via the factory in API.

Changeset: 01c3ba4ded33c2fb692c1334f9565993c41e5180

[Alex Buts]

refs #5873 Fix load large file-based workspaces

cased by default box size changes.

Changeset: 72b734c7facfde9cf73d33064c4f0cab25251b05

[Alex Buts]

refs #5873 Attempt to block putting events into write buffer

when doing SliceMD.cpp The question remains why should one put them into write buffer when doing getConstEvents but will see.

Changeset: 994863034416160d29e72c3672a8165d6db08dea

[Alex Buts]

refs #5873 Disabled write cash on BinMD.cpp

why do you need write cash for getConstEvents anyway?

Changeset: 714d06ae9898c07bd863be7dda93bdccb2ef7d33

[Alex Buts]

refs #5873 this looks like fix it

Disabled the write cache for input workspaces and SliceMD/BinMD algorithms. Now it generates(slices) approximately the same number of events as Horace at approximately the same amount of time. This amount is still different from Horace with similar setting but not fundamentally different, so this is the topic of another ticket.

Changeset: 825bbc038fe6e5992ad3fbe650033471260966f4

[Alex Buts]

refs #5873 fixing progress bar

Changeset: 32382d0731b7c93d9e009ed32887a150a2505e8b

[Stuart Campbell]

This ticket has been transferred to github issue ​6719