Ticket #6543 (closed: fixed)

Opened 8 years ago

Last modified 5 years ago

Modify LoadDetectorInfo algorithm to understand detector.nxs files

Reported by: Alex Buts Owned by: Alex Buts
Priority: major Milestone: Release 2.5
Component: Mantid Keywords:
Cc: Blocked By:
Blocking: Tester: Nick Draper

Description

Detector.dat file describes modified detector's positions which replace Instrument description or raw file's detectors positions with the positions, calculated from a calibration procedure.

Unfortunately, it is an ascii file, which has to be read at each reduction step. This takes substantial time.

In addition to that, there are some cases (e.g. maps) when the detector.dat file is not availible and nxs file with specific format is provided instead.

The purpose of this ticket is to modify LoadDetectorInfo algorithm to understand this kind of nxs files too and modify the detector position/time of flight on the basis of such files.

Change History

comment:1 Changed 8 years ago by Alex Buts

  • Status changed from new to accepted

comment:2 Changed 8 years ago by Alex Buts

refs #6543 Reading old (and wrong) libisis format

Changeset: f0be7db69cdb9a6a3a377680a3d3c4113559fcf3

comment:3 Changed 8 years ago by Alex Buts

refs #6543 Unit tests and some efficiency modifications

Changeset: a0515ce3ab71f661e56616fce51571a0884543a3

comment:4 Changed 8 years ago by Alex Buts

refs #6543 One more unit test and make most protected for testing

methods back private

Changeset: 15d04735a89c581b8f540d95340cd08bc50fa6cd

comment:5 Changed 8 years ago by Alex Buts

refs #6543 replaced Mantid Nexus with 3-rd party nexus

Changeset: a81a51d54617ab1b4f0feac52a21f5e42f4bd6e3

comment:6 Changed 8 years ago by Alex Buts

refs #6543 Modified Nexus header to match new repo structure.

Changeset: 2970b1933b5c4058b8a7c057de1970bb3ccf7448

comment:7 Changed 8 years ago by Alex Buts

refs #6543 One more location of old NeXus file

Changeset: 26f4660c0aedefa26d7259b14ced0b0ad2b2873d

comment:8 Changed 8 years ago by Alex Buts

refs #6543 One windows warning.

Changeset: 01118f554c4b2d69fb4503bf23f426b757dd9969

comment:9 Changed 8 years ago by Alex Buts

  • Status changed from accepted to verify
  • Resolution set to fixed

To test this ticket one needs to check that workspace modified by loadDetectorInfo(det.dat) is equivalent to a workspace, modified by loadDetectorInfo(det.nxs).

det.nxs file can be obtained from det.dat file by python script, available within qtiGenie in user scripts (the function convertDetDataToNexus(det.dat) ) To use qtiGenie from user scripts repository, one need to add path to it to python scripts path and restart Mantid.

A tester can invoke loadDetectorIndfor with det.dat and det.nxs and check if detector positions and the time scales for both workspaces are equivalent.

This is ideal test which I would like to try by myself but has not done for its complexity. A unit tests doing similar comparison on test workspaces so this kind of testing is in a some way already done. An alternative check would be to convert to energy using det.dat and det.nxs files obtained from det.dat. The script which uses det.dat file to do conversion should be obtained from an excitation group instrument scientist and I am not providing it below. The parameters used by this script are in ISIS svn repository https://svn.isis.rl.ac.uk/InstrumentFiles/trunk and the referred data files -- in the data repository. The illiad function referred in the script is also in qtiGenie repository.

The tester may not bother checking libisis.nxs branch, as this is very specific operation, which no general user would be exposed to.

from qtiGenie import *
from PySlice2 import *

inst='MER'
iliad_setup(inst)
ext='.raw'
mapfile='rings_123'
#det_cal_file must be specified if the reduction sends out put to a workpsace
cal_file='det_corr_123.dat'
#cal_file="12669"
#load vanadium file
whitebeamfile="13961"
LoadRaw(Filename=whitebeamfile,OutputWorkspace="wb_wksp",LoadLogFiles="0")
#LoadRaw(Filename=whitebeamfile,OutputWorkspace="wb_wksp",LoadLogFiles="0")
#---------------------------------------------------------------------------------------------------------------
#---------------------------------------------------------------------------------------------------------------

#ei=50
#rebin_params='-70,0.5,48'
ei=60
rebin_params='-45,0.25,43'


runs=[14185]
############## normal reduction####################

#save .spe file
for runfile in runs:
	save_file=inst+str(runfile)+'.spe'
	LoadRaw(Filename=str(runfile),OutputWorkspace="run_wksp",LoadLogFiles="0")
	w1=iliad("wb_wksp","run_wksp",ei,rebin_params,mapfile,det_cal_file=cal_file,norm_method='current')
	SaveSPE('w1',save_file)
print type(w1)


Last edited 8 years ago by Alex Buts (previous) (diff)

comment:10 Changed 8 years ago by Alex Buts

refs #6543 deal with warnings

Changeset: 2ccd3a73b9ea60fcc43e079cc89e8969f5d154dc

comment:11 Changed 8 years ago by Alex Buts

refs #6543 Added OMP around main detector modifications loop

Changeset: d107c378118a33cc6e2428e99d8b51c17e19d9de

comment:12 Changed 8 years ago by Russell Taylor

Re #6543. Move continue statement outside of catch block.

Works around a defect with the Intel compiler, which can get stuck when it encounters continue statements inside catch blocks within a parallelized loop.

Changeset: 22688a08aade0bf8638317a6e8e35447c7c1245d

comment:13 Changed 8 years ago by Russell Taylor

Re #6543. Remove unnecessary guard that fails on the Mac.

And the test that checks it. If the file doesn't exist, it won't be possible to execute the algorithm as the validator on the Filename property will prevent it.

Changeset: c366a6c6fb83129158d0123f29d8a8b59a4162d6

comment:14 Changed 8 years ago by Alex Buts

refs #6543 Trying to fix crash on Mac

Changeset: 32c86e74633e34edb7cbd323de8f53fec6049c06

comment:15 Changed 8 years ago by Alex Buts

refs #6543 resolving conflicts

Changeset: c70c63eb0b0d73d4fb9ac771bfb02acb8f83286b

comment:16 Changed 8 years ago by Alex Buts

refs #6543 disabling OMP for the time being.

Changeset: f07871399662aad089bad42cbc6095e0463d4a8c

comment:17 Changed 8 years ago by Russell Taylor

Re #6543. Remove redundant line.

hFile can never be NULL at that point. If the file cannot be opened the constructor will throw and it will be caught in the algorithm base class.

Changeset: 34fdba33c8057ffebab4f76872944122ad3d12d3

comment:18 Changed 8 years ago by Alex Buts

refs #6543 Attempt to fix and enable OMP again

Changeset: 3b3db5d1cf6b7ddb302f8d86b04cff5819109c41

comment:19 Changed 8 years ago by Alex Buts

refs #6543 technical changes

Changeset: a2a5f686c6cb903de60df081420b2ef4f0c456ca

comment:20 Changed 8 years ago by Alex Buts

refs #6543 -- disabled OMP again

to resolve strange problem with memory leak in DataHandling dll.

Changeset: d589aa3b44bc2d0a903df0524ad4d82ecdf9edba

comment:21 Changed 8 years ago by Alex Buts

refs #6543 enabled OMP again

Changeset: 525c8eb56c1eed4650b53b889c0a0d87377154f7

comment:22 Changed 8 years ago by Alex Buts

refs #6543 Disable OMP

Changeset: c18b61aabdc52027d303b357f16c8ba8771d4b83

comment:23 Changed 8 years ago by Russell Taylor

  • Status changed from verify to reopened
  • Resolution fixed deleted

Despite having been marked as fixed over a week ago, this ticket has fresh commits as recently as yesterday. At a minimum, the 'fixed' assertion needs reaffirming if that really is the case.

comment:24 Changed 8 years ago by Alex Buts

  • Status changed from reopened to accepted

comment:25 Changed 8 years ago by Alex Buts

  • Status changed from accepted to verify
  • Resolution set to fixed

It had some changes related to omp but apparently most expensive data modifying operation is not thread safe so OMP does not have any sense. OMP disabled for the time being and the ticket is fixed as above.

The tester should compare workspaces, modified by loadDetectorInfo("det.dat") file and loadDetectorInfo("det.nxs") file and check that they are equivalent. There are apparently operations which compare workspace.

the det.nxs file can be created from det.dat file by onvertDetDataToNexus("det.dat") from qtiGenie user script.

comment:26 Changed 8 years ago by Nick Draper

  • Status changed from verify to verifying
  • Tester set to Nick Draper

comment:27 Changed 8 years ago by Nick Draper

changes are on Master

comment:28 Changed 8 years ago by Nick Draper

  • Status changed from verifying to closed

Checked by loading and cloneing an LET workspace LoadDectorInfo from a matching .nxs and .dat file, with move detectors on The CheckworkspacesMatch including the instrument. Passed

comment:29 Changed 5 years ago by Stuart Campbell

This ticket has been transferred to github issue 7389

Note: See TracTickets for help on using tickets.