PlusMD: Incorrect result for file-backed inputs that have had events added but not cached to disk

[Janik Zikovsky]

import time

# Some parameters
ws1Name = 'Workspace_1'
ws2Name = 'Workspace_2'
ws3Name = 'Workspace_3'
plusWSName = 'Plussed_Workspace'

backEndFile1 = ''
backEndFile2 = ''
backEndFile3 = ''

backEndFile1 = 'Backend_' + str(time.time()) + '_1.nxs'
backEndFile2 = 'Backend_' + str(time.time()) + '_2.nxs'
backEndFile3 = 'Backend_' + str(time.time()) + '_3.nxs'

splitInto1 = '5'
splitInto2 = '5'
splitInto3 = '5'

splitThreshold1 = '100'
splitThreshold2 = '100'
splitThreshold3 = '100'

minRecursionDepth1 = '1'
minRecursionDepth2 = '1'
minRecursionDepth3 = '1'
maxRecursionDepth1 = '5'
maxRecursionDepth2 = '5'
maxRecursionDepth3 = '5'

CreateMDWorkspace(OutputWorkspace=ws1Name, Dimensions='3',EventType='MDLeanEvent',Extents='-10,10,-10,10,-10,10',Names='k,l,m',Units='-,-,-',SplitInto=splitInto1,SplitThreshold=splitThreshold1,MinRecursionDepth=minRecursionDepth1,MaxRecursionDepth=maxRecursionDepth1,Filename=backEndFile1,Memory='1')

CreateMDWorkspace(OutputWorkspace=ws2Name, Dimensions='3',EventType='MDLeanEvent',Extents='-10,10,-10,10,-10,10',Names='k,l,m',Units='-,-,-',SplitInto=splitInto2,SplitThreshold=splitThreshold2,MinRecursionDepth=minRecursionDepth2,MaxRecursionDepth=maxRecursionDepth2,Filename=backEndFile2,Memory='1')

CreateMDWorkspace(OutputWorkspace=ws3Name, Dimensions='3',EventType='MDLeanEvent',Extents='-10,10,-10,10,-10,10',Names='k,l,m',Units='-,-,-',SplitInto=splitInto3,SplitThreshold=splitThreshold3,MinRecursionDepth=minRecursionDepth3,MaxRecursionDepth=maxRecursionDepth3,Filename=backEndFile3,Memory='1')

# Insert fake data into Workspace_1
FakeMDEventData(InputWorkspace=ws1Name,PeakParams='1000,5,5,5,2')
#SaveMD(ws1Name, UpdateFileBackEnd=1)

# Insert fake data into Workspace_2
FakeMDEventData(InputWorkspace=ws2Name,PeakParams='1000,-5,-5,-5,2')
#SaveMD(ws2Name, UpdateFileBackEnd=1)

# Insert fake data into Workspace_3, which is basically the same as the data inserted into 1 and 2.
FakeMDEventData(InputWorkspace=ws3Name,PeakParams='1000,5,5,5,2')
FakeMDEventData(InputWorkspace=ws3Name,PeakParams='1000,-5,-5,-5,2')

PlusMD(LHSWorkspace=ws1Name,RHSWorkspace=ws2Name,OutputWorkspace=plusWSName)

If you do NOT use file-backed files, the resulting workspace has 2000 events. If you call SaveMD after FakeMDEvent, the resulting workspace has 2000 events. If you do neither of these, the resulting workspace has 1000 events; WS1 was cloned but its in-memory events were forgotten!

[Janik Zikovsky]

Calling SaveMD if needed before the algo on WS1 would likely solve this.

[Janik Zikovsky]

This occurs because FakeMDEventData adds events to the "current" vector but does not save to disk. The first step in PlusMD is to clone the workspace by copying its file. This is wrong if it has not been cached to disk first.

I will add a flag saying "This workspace is file-backed but contains un-cached events." This will be set at the algorithm level.

[Janik Zikovsky]

Refs #3786 test that fails on my machine but I have a hunch works on build servers. Possible NEXUS version/build issue on my build?

Changeset: 1a896231c14e6995e039d3740b5ce235ed534ed9

[Janik Zikovsky]

Refs #3786 fix test.

Changeset: 6c0af16db5cac058bad36645eafbcb5a8367ff04

[Janik Zikovsky]

Refs #3786 changes towards handling an error in PlusMD.

Changeset: c3fc97358698928b079b791a3ecbdbaa4399d78d

[Janik Zikovsky]

Refs #3786 comment out tests that fail on windows, for now

Changeset: e8b7b219ed8dd118b710c2a85c25e8811e2396a8

[Janik Zikovsky]

Refs #3786 : algorithms that add events to a file-backed MDW mark that.

Changeset: 4f5852c3b64209318a5654d2e4bb57e4075e3ee7

[Janik Zikovsky]

Refs #3786 fixes this problem. CloneMDWorkspace will properly update the file first. comment out failing test, made a ticket for that.

Changeset: d1772b830a032e2c510a0938a670dedeb1760e34

[Janik Zikovsky]

Script above runs as expected. Note to testers: the real underlying issue was in doing CloneMDWorkspace on a MDWS that is file-backed AND was modified but NOT updated with SaveMD.

[Janik Zikovsky]

Refs #3786 : algorithms that add events to a file-backed MDW mark that.

Changeset: 4f5852c3b64209318a5654d2e4bb57e4075e3ee7

[Janik Zikovsky]

Refs #3786 fixes this problem. CloneMDWorkspace will properly update the file first. comment out failing test, made a ticket for that.

Changeset: d1772b830a032e2c510a0938a670dedeb1760e34

[Peter Parker]

Works fine for me now.

[Stuart Campbell]

This ticket has been transferred to github issue ​4633