ISIS direct reduction does not remove backgound properly if workspace is not normalized

[Alex Buts]

Merlin script below fails. Its obvious that the reason for that is missing monitor workspace necessary for "on-demand" normalization.

Necessary and easy to fix.

""" Sample MER reduction script, to run on CEPS with MANTID 3.3 (slow CEPS output) """ 
# Two rows necessary to run script outside of the mantid. You need also set up 
# appropriate python path-es
import os,sys
import shutil
os.environ["PATH"] = r"c:/Mantid/Code/builds/br_master/bin/Release;"+os.environ["PATH"]
from mantid import *
from Direct.ReductionWrapper import *
# this exact namd is necessary for web services to work until we implement factory
class MERLINReduction(ReductionWrapper):
#------------------------------------------------------------------------------------#
   @MainProperties
   def def_main_properties(self):
       """ Define main properties used in reduction. These are the property 
           a user usually wants to change
       """ 
       prop = {}

       ei=[50.1, 21.9,12.2] #[150,64,36] # multiple energies provided in the data file
       ebin=[-0.25,0.005,0.85]    #binning of the energy for the spe file. 
       # if energy is specified as a list (even with single value e.g. ei=[81])
       # The numbers are treated as a fraction of ei [from ,step, to ]. If energy is 
       # a number, energy binning assumed to be absolute (e_min, e_step,e_max)
       #
       prop['incident_energy'] = ei
       prop['energy_bins'] = ebin
       #
       # the range of files to reduce. This range ignored when deployed from autoreduction,
       # unless you going to sum these files. 
       # The range of numbers or run number is used when you run reduction from PC.
       prop['sample_run'] = 24207 #range(24191,24193) # 'MER23700.n001'
       prop['wb_run'] = '23684.raw'
       #
       prop['sum_runs'] = False # set to true to sum everything provided to sample_run
       #                        # list
  
       # Absolute units reduction properties. Set prop['monovan_run']=None to do relative units
       prop['monovan_run'] = None # vanadium run in the same configuration as your sample
       #prop['sample_mass'] = 9 # mass of your sample 
       #prop['sample_rmm'] = 496.4 # molecular weight of scatterers in your sample

       return prop
#------------------------------------------------------------------------------------#
   @AdvancedProperties
   def def_advanced_properties(self):
      """  Set up advanced properties, describing reduction.
           These are the properties, usually provided by an instrument 
           scientist
            
           separation between simple and advanced properties depends
           on scientist, experiment and user.   All are necessary for reduction 
           to work properly
      """
      prop = {}
      prop['map_file'] = 'rings_143.map'
      prop['det_cal_file'] = 'det_corr_143.dat' #'det_corrected7.nxs - testing'
      prop['bleed'] = False
      prop['norm_method']='monitor-1'
      prop['detector_van_range']=[40,55]
      prop['background_range'] = [18000,19000] # TOF range for the calculating flat background
      prop['hardmaskOnly']='MER23698.msk' # diag does not work well on MER. At present only use a hard mask RIB has created
      #prop['hard_mask_file'] = "Bjorn_mask.msk"

      prop['check_background']=True
      #prop['ei-mon2-spec']=69641
      #prop[fix_ei]=True
      

      prop['save_format'] = 'nxs' #'nxspe' #nxs,nxspe'
       # if two input files with the same name and  different extension found, what to prefer. 
      prop['data_file_ext']='.nxs' # for MER it may be choice between event and histo mode if 
      # raw file is written in histo, and nxs -- in event mode
      # Absolute units: map file to calculate monovan integrals                                                                      
      prop['monovan_mapfile'] = 'rings_143.map'
      prop['vanadium-mass']=7.85 # check this
      # change this to correct value and verify that motor_log_names refers correct and existing 
      # log name for crystal rotation to write correct psi value into nxspe files
      prop['motor_offset']=None
      #
      prop['monovan_lo_frac']=-0.4
      prop['monovan_hi_frac']= 0.4
      #RAE added
      prop['bleed_maxrate']=0.005
      
      return prop
      #
#------------------------------------------------------------------------------------#
   @iliad
   def reduce(self,input_file=None,output_directory=None):
      """ Method executes reduction over single file

          Overload only if custom reduction is needed or
          special features are requested
      """
      results = ReductionWrapper.reduce(self,input_file,output_directory)
      #
      #run_num = PropertyManager.sample_run.run_number() 
      #energies = self.reducer.prop_man.incident_energy
      #for ind,res in enumerate(results):
      #      ei = energies[ind]
      #      filename = "LET{0}_{1:<3.2f}meV.nxspe".format(run_num ,ei)
      #      file = os.path.join('/stagingarea',filename)
      #      print 'Saving file to {0}'.format(file)#

      #      SaveNexus(res,Filename = file)
      #      t_folder = config['defaultsave.directory']
      #      targ_path = os.path.join(t_folder,filename)
      #      shutil.move(file,targ_path)     
      return results
   #
   #
   def set_custom_output_filename(self):
      """ define custom name of output files if standard one is not satisfactory 
          In addition to that, example of accessing reduction properties 
          Changing them if necessary
      """ 
      def custom_name(prop_man):
          """ sample function which builds filename from 
              incident energy and run number and adds some auxiliary information 
              to it.
          """ 
          # Note -- properties have the same names  as the list of advanced and 
          # main properties
          ei = PropertyManager.incident_energy.get_current()
          # sample run is more then just list of runs, so we use 
          # the formalization below to access its methods
          run_num = PropertyManager.sample_run.run_number()
          name = "MER{0}atEi{1:<3.2f}meV_One2One".format(run_num ,ei)
          return name
       
      # Uncomment this to use custom filename function
      # Note: the properties are stored in prop_man class accessed as
      # below. 
      return lambda : custom_name(self.reducer.prop_man)
      # use this method to use standard file name generating function
      #return None
   #
   def validation_file_place(self):
      """Redefine this to the place, where validation file, used in conjunction with
         'validate_run' property, located. Here it defines the place to this script folder.
          but if this function is disabled, by default it looks for/places it 
          in a default save directory"""
      return os.path.split(os.path.realpath(__file__))[0]
   
   def __init__(self,web_var=None):
       """ sets properties defaults for the instrument with Name"""
       ReductionWrapper.__init__(self,'MER',web_var)
#
if __name__ == "__main__":
#------------------------------------------------------------------------------------#
# SECTION USED TO RUN REDUCTION FROM MANTID SCRIPT WINDOW #
#------------------------------------------------------------------------------------#
##### Here one sets up folders where to find input data and where to save results ####
    # It can be done here or from Mantid GUI:
    #      File->Manage user directory ->Browse to directory
    # Folder where map and mask files are located:
    map_mask_dir = r'c:\Users\wkc26243\Documents\work\InstrumentFiles\merlin' #'/usr/local/mprogs/InstrumentFiles/merlin'
    config.appendDataSearchDir(map_mask_dir)
    # folder where input data can be found
    data_dir = r'd:\Data\Mantid_Testing\15_04_21\data'
    config.appendDataSearchDir(data_dir)
    config.appendDataSearchDir(map_mask_dir)
    # auxiliary folder with results
    #ref_data_dir = 'd:/Data/MantidSystemTests/Data' 
    # Set input search path to values, specified above
    #config.setDataSearchDirs('{0};{1};{2}'.format(data_dir,map_mask_dir,ref_data_dir))
    # use appendDataSearch directory to add more locations to existing Mantid 
    # data search path
    #config.appendDataSearchDir('d:/Data/Mantid_GIT/Test/AutoTestData')
    # folder to save resulting spe/nxspe files.
    config['defaultsave.directory'] = r'd:\Data\Mantid_Testing\15_04_21'

###### Initialize reduction class above and set up reduction properties.        ######
######  Note no web_var in constructor.(will be irrelevant if factory is implemented)
    rd = MERLINReduction()
    # set up advanced and main properties
    rd.def_advanced_properties()
    rd.def_main_properties()
#### uncomment rows below to generate web variables and save then to transfer to   ###
    ## web services.
    run_dir = os.path.dirname(os.path.realpath(__file__))
    file = os.path.join(run_dir,'reduce_vars.py')
    rd.save_web_variables(file)

#### Set up time interval (sec) for reducer to check for input data file.         ####
    #  If data file is not present and this value is 0,reduction fails 
    #  if this value >0 the reduction wait until file appears on the data 
    #  search path checking regularly after time interval specified below.
    rd.wait_for_file = 0  # waiting time interval (in sec)
   
### Define a run number to validate reduction against future changes    #############
    # After reduction works well and all settings are done and verified, 
    # take a run number with good reduced results and build validation
    # for this result. 
    # Then place the validation run together with this reduction script.
    # Next time, the script will run reduction and compare the reduction results against
    # the results obtained earlier.
    #rd.validate_run_number = 21968  # Enabling this property disables normal reduction
    # and forces reduction to reduce run specified here and compares results against
    # validation file, processed earlier or calculate this file if run for the first time.
    #This would ensure that reduction script have not changed,
    #allow to identify the reason for changes if it was changed 
    # and would allow to recover the script,used to produce initial reduction
    #if changes are unacceptable.

####get reduction parameters from properties above, override what you want locally ###
   # and run reduction. Overriding would have form:
   # rd.reducer.prop_man.property_name (from the dictionary above) = new value e.g. 
   # rd.reducer.prop_man.energy_bins = [-40,2,40]
   # or 
   ## rd.reducer.prop_man.sum_runs = False
   # 
    
    rd.run_reduction()

[Alex Buts]

fixed as PR ​https://github.com/mantidproject/mantid/pull/637

A tester may try to run script taking files from archives and map/masks from ​https://svn.isis.rl.ac.uk/InstrumentFiles/trunk/MERLIN but I've done that and it works.

Ideally it would be nice to systest the change, but it will substantially increase build time for everybody with very little gain, as the changes are simple.

I think, a tester can accept changes after code review if existing systests for direct inelastic pass.

[Stuart Campbell]

This ticket has been transferred to github issue ​12459