NOMAD autoreduction with MPI

[Vickie Lynch]

There are two problems with autoreduction on multiple nodes. First this cannot be done with preserveEvents=True since all events may not fit on one node. Second the can and vanadium files are currently read from only one node.

[Vickie Lynch]

Refs #7553 mpi chunks for can and vanadium

Changeset: 37cfccb94eb8be5be19ceb4a64376042a38c4939

[Vickie Lynch]

Refs #7553 preserveEvents=False agrees with True

Changeset: e6148fdfc8cd905debe717c369262e5d8379b99b

[Vickie Lynch]

On my rhel6 machine time for reduction went from 22 min with preserveEvents=True to 16 min with preserveEvents=False. When I used "mpirun -np 2 python" with the MPI build and this script, the time was 11 min and the answer was the same.

import os
import sys
import shutil


sys.path.append(os.environ['MANTIDPATH'])


#sys.path.append("/mantid/Code/mpi-release/bin")
from MantidFramework import mtd
mtd.initialize()
import mantid.simpleapi as api

cal_dir = "/home/vel/workspace/"
cal_file  = os.path.join(cal_dir, "NOM_calibrate_d14526_2013_05_18.cal")
char_file = "/home/vel/workspace/NOM_characterizations.txt" #os.path.join(cal_dir, "NOM_characterizations.txt")
sam_back =     15072
van      =     14728
van_back =     14727

#from mantidsimple import *

eventFileAbs="NOM_14526"
outputDir="/home/vel/workspace"
maxChunkSize=8.0

eventFile = os.path.split(eventFileAbs)[-1]
nexusDir = eventFileAbs.replace(eventFile, '')
runNumber = eventFile.split('_')[1]
configService = mtd.getSettings()
dataSearchPath = configService.getDataSearchDirs()
dataSearchPath.append(nexusDir)
configService.setDataSearchDirs(dataSearchPath)

api.SNSPowderReduction(Instrument="NOM", RunNumber=runNumber, Extension="_event.nxs",
                   MaxChunkSize=maxChunkSize, PreserveEvents=True,PushDataPositive='AddMinimum',
                   CalibrationFile=cal_file, CharacterizationRunsFile=char_file,
                   BackgroundNumber=sam_back, VanadiumNumber=van,
                   VanadiumBackgroundNumber=van_back, RemovePromptPulseWidth=50,
                   ResampleX=-3000, BinInDspace=True, FilterBadPulses=True,
                   SaveAs="gsas and fullprof and pdfgetn", OutputDirectory=outputDir,
                   StripVanadiumPeaks=True,
                   NormalizeByCurrent=True, FinalDataUnits="MomentumTransfer")

[Vickie Lynch]

Refs #7553 mpi version should not gather all events on first core

Changeset: 535b85013a67ccb2128795a78f31098066b7e3ab

[Vickie Lynch]

MPI on 2 nodes not saving events takes 7.5 minutes

[Vickie Lynch]

Refs #7553 adding warning for resetting preserveEvents

Changeset: fb6a7a71423838818fcbc587f1fa368d0a0be649

[Vickie Lynch]

Refs #7553 mpi chunks for can and vanadium (cherry picked from commit 37cfccb94eb8be5be19ceb4a64376042a38c4939)

Changeset: 1c8a597700c1fb5ea329f5ec49a29e12a7a69b56

[Vickie Lynch]

Refs #7553 preserveEvents=False agrees with True (cherry picked from commit e6148fdfc8cd905debe717c369262e5d8379b99b)

Changeset: 78016a05cd03e4358a5713cab223354c8d2420ec

[Vickie Lynch]

Refs #7553 mpi version should not gather all events on first core

Changeset: b07acdaac862ec0bbf3af9101c03cc81a0fc44f2

[Vickie Lynch]

Refs #7553 adding warning for resetting preserveEvents (cherry picked from commit fb6a7a71423838818fcbc587f1fa368d0a0be649)

Changeset: 2c92005bb758ddd4f6dd8e4ce5931a94da44801f

[Stuart Campbell]

This ticket has been transferred to github issue ​8398