Ticket #9955 (closed: fixed)
Strange NaN issues with Stitch1D
Reported by: | Owen Arnold | Owned by: | Owen Arnold |
---|---|---|---|
Priority: | blocker | Milestone: | Release 3.2.1 |
Component: | Framework | Keywords: | |
Cc: | Blocked By: | #9956 | |
Blocking: | Tester: | Anders Markvardsen |
Description (last modified by Owen Arnold) (diff)
The following is a script reported by an scientist.
# Load SURF00110504, SURF00110503. SURF00110505 from ICAT RenameWorkspace('SURF00110504', OutputWorkspace='_110504') RenameWorkspace('SURF00110503', OutputWorkspace='_110503') RenameWorkspace('SURF00110505', OutputWorkspace='_110505') CreateSingleValuedWorkspace(OutputWorkspace='__tmp_binary_operation_double') Plus(LHSWorkspace='__tmp_binary_operation_double',RHSWorkspace='_110504',OutputWorkspace='__python_op_tmp0') ConvertUnits(InputWorkspace='__python_op_tmp0',OutputWorkspace='sum_wavelength',Target='Wavelength',AlignBins='1') CropWorkspace(InputWorkspace='sum_wavelength',OutputWorkspace='_monitor_ws',StartWorkspaceIndex='1',EndWorkspaceIndex='1') CloneWorkspace(InputWorkspace='sum_wavelength',OutputWorkspace='_in_rng') CropWorkspace(InputWorkspace='_in_rng',OutputWorkspace='_in_rng',StartWorkspaceIndex='2',EndWorkspaceIndex='2') CropWorkspace(InputWorkspace='_in_rng',OutputWorkspace='_detector_ws',XMin='0.5',XMax='9') CalculateFlatBackground(InputWorkspace='_monitor_ws',OutputWorkspace='_monitor_ws',StartX='7.5999999999999996',EndX='8.5',WorkspaceIndexList='0') RebinToWorkspace(WorkspaceToRebin='_monitor_ws',WorkspaceToMatch='_detector_ws',OutputWorkspace='_I0P') Integration(InputWorkspace='_I0P',OutputWorkspace='_monInt',RangeLower='4',RangeUpper='6') Divide(LHSWorkspace='_detector_ws',RHSWorkspace='_monInt',OutputWorkspace='IvsLam') CreateSingleValuedWorkspace(OutputWorkspace='__tmp_binary_operation_double') Plus(LHSWorkspace='__tmp_binary_operation_double',RHSWorkspace='_110503',OutputWorkspace='__python_op_tmp1') ConvertUnits(InputWorkspace='__python_op_tmp1',OutputWorkspace='sum_wavelength',Target='Wavelength',AlignBins='1') CropWorkspace(InputWorkspace='sum_wavelength',OutputWorkspace='_monitor_ws',StartWorkspaceIndex='1',EndWorkspaceIndex='1') CloneWorkspace(InputWorkspace='sum_wavelength',OutputWorkspace='_in_rng') CropWorkspace(InputWorkspace='_in_rng',OutputWorkspace='_in_rng',StartWorkspaceIndex='2',EndWorkspaceIndex='2') CropWorkspace(InputWorkspace='_in_rng',OutputWorkspace='_detector_ws',XMin='0.5',XMax='9') CalculateFlatBackground(InputWorkspace='_monitor_ws',OutputWorkspace='_monitor_ws',StartX='7.5999999999999996',EndX='8.5',WorkspaceIndexList='0') RebinToWorkspace(WorkspaceToRebin='_monitor_ws',WorkspaceToMatch='_detector_ws',OutputWorkspace='_i0p_trans') Integration(InputWorkspace='_i0p_trans',OutputWorkspace='_mon_int_trans',RangeLower='4',RangeUpper='6') Divide(LHSWorkspace='_detector_ws',RHSWorkspace='_mon_int_trans',OutputWorkspace='transWS') RebinToWorkspace(WorkspaceToRebin='transWS',WorkspaceToMatch='IvsLam',OutputWorkspace='_i_vs_lam_trans') Divide(LHSWorkspace='IvsLam',RHSWorkspace='_i_vs_lam_trans',OutputWorkspace='_i_vs_lam_corrected') RenameWorkspace(InputWorkspace='_i_vs_lam_corrected',OutputWorkspace='IvsLam') MoveInstrumentComponent(Workspace='IvsLam',ComponentName='point-detector',Y='0.031287739139067995',Z='11.43',RelativePosition='0') ConvertUnits(InputWorkspace='IvsLam',OutputWorkspace='IvsQ',Target='MomentumTransfer') RenameWorkspace(InputWorkspace='IvsQ',OutputWorkspace='110504_IvsQ') CreateSingleValuedWorkspace(OutputWorkspace='__tmp_binary_operation_double') Plus(LHSWorkspace='__tmp_binary_operation_double',RHSWorkspace='_110505',OutputWorkspace='__python_op_tmp0') ConvertUnits(InputWorkspace='__python_op_tmp0',OutputWorkspace='sum_wavelength',Target='Wavelength',AlignBins='1') CropWorkspace(InputWorkspace='sum_wavelength',OutputWorkspace='_monitor_ws',StartWorkspaceIndex='1',EndWorkspaceIndex='1') CloneWorkspace(InputWorkspace='sum_wavelength',OutputWorkspace='_in_rng') CropWorkspace(InputWorkspace='_in_rng',OutputWorkspace='_in_rng',StartWorkspaceIndex='2',EndWorkspaceIndex='2') CropWorkspace(InputWorkspace='_in_rng',OutputWorkspace='_detector_ws',XMin='0.5',XMax='9') CalculateFlatBackground(InputWorkspace='_monitor_ws',OutputWorkspace='_monitor_ws',StartX='7.5999999999999996',EndX='8.5',WorkspaceIndexList='0') RebinToWorkspace(WorkspaceToRebin='_monitor_ws',WorkspaceToMatch='_detector_ws',OutputWorkspace='_I0P') Integration(InputWorkspace='_I0P',OutputWorkspace='_monInt',RangeLower='4',RangeUpper='6') Divide(LHSWorkspace='_detector_ws',RHSWorkspace='_monInt',OutputWorkspace='IvsLam') RebinToWorkspace(WorkspaceToRebin='transWS',WorkspaceToMatch='IvsLam',OutputWorkspace='_i_vs_lam_trans') Divide(LHSWorkspace='IvsLam',RHSWorkspace='_i_vs_lam_trans',OutputWorkspace='_i_vs_lam_corrected') RenameWorkspace(InputWorkspace='_i_vs_lam_corrected',OutputWorkspace='IvsLam') MoveInstrumentComponent(Workspace='IvsLam',ComponentName='point-detector',Y='0.058102261279893055',Z='11.43',RelativePosition='0') ConvertUnits(InputWorkspace='IvsLam',OutputWorkspace='IvsQ',Target='MomentumTransfer') RenameWorkspace(InputWorkspace='IvsQ',OutputWorkspace='110505_IvsQ') Rebin(InputWorkspace='110504_IvsQ',OutputWorkspace='110504_IvsQ_binned',Params='0.011,-0.0376509,0.051') Rebin(InputWorkspace='110505_IvsQ',OutputWorkspace='110505_IvsQ_binned',Params='0.02,-0.0376509,0.142471') CloneWorkspace(InputWorkspace='110504_IvsQ_binned',OutputWorkspace='currentSum') Stitch1D(LHSWorkspace='currentSum',RHSWorkspace='110505_IvsQ_binned',OutputWorkspace='currentSum') RenameWorkspace(InputWorkspace='currentSum',OutputWorkspace='110504_50') plotSpectrum('110504_50')
The plot is bad because of the first y-value in 110505, which is a NAN. We should be ignoring this! Anyway, changing
Rebin(InputWorkspace='110505_IvsQ',OutputWorkspace='110505_IvsQ_binned',Params='0.02,-0.0376509,0.142471')
to
Rebin(InputWorkspace='110505_IvsQ',OutputWorkspace='110505_IvsQ_binned',Params='0.0206,-0.0376509,0.142471')
Fixes the issue of the discontinuity at the intersection, but we should really be keeping Nans post processing.
Change History
comment:3 Changed 6 years ago by Owen Arnold
The fix that the instrument scientist have asked for in Stitch1D is as follows:
- After rebinning the data, find the positions of the special values (NaNs, Infs)
- Perform the processing including determining the stitch overlap
- Right at the end of the processing, put the special values back.
- As a corollary to this, we may not need to perform the current call to ReplaceSpecialValues since the rebinned workspaces being integrated will already be missing the special values.
comment:5 Changed 6 years ago by Owen Arnold
refs #9955. No need for replace special values.
Replace special values done as part of finding and recording those indexes.
Changeset: 0c5642237c428bd5ca151ef7435ba6cf8c7882c4
comment:6 Changed 6 years ago by Owen Arnold
refs #9955. Add test and introduce replace functionality.
Changeset: 1355491d395009ea197ae1fe7f7cab181e6baee3
comment:7 Changed 6 years ago by Owen Arnold
refs #9955. Special treatment for errors too
Changeset: 346f9eaa54e593bf89b3cd94243ae670b6e35efe
comment:8 Changed 6 years ago by Owen Arnold
refs #9955. Document and format.
Changeset: 7f5d87212f8baaedf134c648f50fff8434fd9702
comment:9 Changed 6 years ago by Owen Arnold
Tester:
- Check the new documentation
- New unit test covers this functionality
- Running the following should include the perturbation due to the nan, but (a) the scale factors (error) and (value) with log=debug for Stitch1D should be real numbers, the continuation of the curve on both sides of the perturbation should look ok.
SURF00110503=Load('SURF00110503') SURF00110504=Load('SURF00110504') SURF00110505=Load('SURF00110505') RenameWorkspace('SURF00110504', OutputWorkspace='_110504') RenameWorkspace('SURF00110503', OutputWorkspace='_110503') RenameWorkspace('SURF00110505', OutputWorkspace='_110505') CreateSingleValuedWorkspace(OutputWorkspace='__tmp_binary_operation_double') Plus(LHSWorkspace='__tmp_binary_operation_double',RHSWorkspace='_110504',OutputWorkspace='__python_op_tmp0') ConvertUnits(InputWorkspace='__python_op_tmp0',OutputWorkspace='sum_wavelength',Target='Wavelength',AlignBins='1') CropWorkspace(InputWorkspace='sum_wavelength',OutputWorkspace='_monitor_ws',StartWorkspaceIndex='1',EndWorkspaceIndex='1') CloneWorkspace(InputWorkspace='sum_wavelength',OutputWorkspace='_in_rng') CropWorkspace(InputWorkspace='_in_rng',OutputWorkspace='_in_rng',StartWorkspaceIndex='2',EndWorkspaceIndex='2') CropWorkspace(InputWorkspace='_in_rng',OutputWorkspace='_detector_ws',XMin='0.5',XMax='9') CalculateFlatBackground(InputWorkspace='_monitor_ws',OutputWorkspace='_monitor_ws',StartX='7.5999999999999996',EndX='8.5',WorkspaceIndexList='0') RebinToWorkspace(WorkspaceToRebin='_monitor_ws',WorkspaceToMatch='_detector_ws',OutputWorkspace='_I0P') Integration(InputWorkspace='_I0P',OutputWorkspace='_monInt',RangeLower='4',RangeUpper='6') Divide(LHSWorkspace='_detector_ws',RHSWorkspace='_monInt',OutputWorkspace='IvsLam') CreateSingleValuedWorkspace(OutputWorkspace='__tmp_binary_operation_double') Plus(LHSWorkspace='__tmp_binary_operation_double',RHSWorkspace='_110503',OutputWorkspace='__python_op_tmp1') ConvertUnits(InputWorkspace='__python_op_tmp1',OutputWorkspace='sum_wavelength',Target='Wavelength',AlignBins='1') CropWorkspace(InputWorkspace='sum_wavelength',OutputWorkspace='_monitor_ws',StartWorkspaceIndex='1',EndWorkspaceIndex='1') CloneWorkspace(InputWorkspace='sum_wavelength',OutputWorkspace='_in_rng') CropWorkspace(InputWorkspace='_in_rng',OutputWorkspace='_in_rng',StartWorkspaceIndex='2',EndWorkspaceIndex='2') CropWorkspace(InputWorkspace='_in_rng',OutputWorkspace='_detector_ws',XMin='0.5',XMax='9') CalculateFlatBackground(InputWorkspace='_monitor_ws',OutputWorkspace='_monitor_ws',StartX='7.5999999999999996',EndX='8.5',WorkspaceIndexList='0') RebinToWorkspace(WorkspaceToRebin='_monitor_ws',WorkspaceToMatch='_detector_ws',OutputWorkspace='_i0p_trans') Integration(InputWorkspace='_i0p_trans',OutputWorkspace='_mon_int_trans',RangeLower='4',RangeUpper='6') Divide(LHSWorkspace='_detector_ws',RHSWorkspace='_mon_int_trans',OutputWorkspace='transWS') RebinToWorkspace(WorkspaceToRebin='transWS',WorkspaceToMatch='IvsLam',OutputWorkspace='_i_vs_lam_trans') Divide(LHSWorkspace='IvsLam',RHSWorkspace='_i_vs_lam_trans',OutputWorkspace='_i_vs_lam_corrected') RenameWorkspace(InputWorkspace='_i_vs_lam_corrected',OutputWorkspace='IvsLam') MoveInstrumentComponent(Workspace='IvsLam',ComponentName='point-detector',Y='0.031287739139067995',Z='11.43',RelativePosition='0') ConvertUnits(InputWorkspace='IvsLam',OutputWorkspace='IvsQ',Target='MomentumTransfer') RenameWorkspace(InputWorkspace='IvsQ',OutputWorkspace='110504_IvsQ') CreateSingleValuedWorkspace(OutputWorkspace='__tmp_binary_operation_double') Plus(LHSWorkspace='__tmp_binary_operation_double',RHSWorkspace='_110505',OutputWorkspace='__python_op_tmp0') ConvertUnits(InputWorkspace='__python_op_tmp0',OutputWorkspace='sum_wavelength',Target='Wavelength',AlignBins='1') CropWorkspace(InputWorkspace='sum_wavelength',OutputWorkspace='_monitor_ws',StartWorkspaceIndex='1',EndWorkspaceIndex='1') CloneWorkspace(InputWorkspace='sum_wavelength',OutputWorkspace='_in_rng') CropWorkspace(InputWorkspace='_in_rng',OutputWorkspace='_in_rng',StartWorkspaceIndex='2',EndWorkspaceIndex='2') CropWorkspace(InputWorkspace='_in_rng',OutputWorkspace='_detector_ws',XMin='0.5',XMax='9') CalculateFlatBackground(InputWorkspace='_monitor_ws',OutputWorkspace='_monitor_ws',StartX='7.5999999999999996',EndX='8.5',WorkspaceIndexList='0') RebinToWorkspace(WorkspaceToRebin='_monitor_ws',WorkspaceToMatch='_detector_ws',OutputWorkspace='_I0P') Integration(InputWorkspace='_I0P',OutputWorkspace='_monInt',RangeLower='4',RangeUpper='6') Divide(LHSWorkspace='_detector_ws',RHSWorkspace='_monInt',OutputWorkspace='IvsLam') RebinToWorkspace(WorkspaceToRebin='transWS',WorkspaceToMatch='IvsLam',OutputWorkspace='_i_vs_lam_trans') Divide(LHSWorkspace='IvsLam',RHSWorkspace='_i_vs_lam_trans',OutputWorkspace='_i_vs_lam_corrected') RenameWorkspace(InputWorkspace='_i_vs_lam_corrected',OutputWorkspace='IvsLam') MoveInstrumentComponent(Workspace='IvsLam',ComponentName='point-detector',Y='0.058102261279893055',Z='11.43',RelativePosition='0') ConvertUnits(InputWorkspace='IvsLam',OutputWorkspace='IvsQ',Target='MomentumTransfer') RenameWorkspace(InputWorkspace='IvsQ',OutputWorkspace='110505_IvsQ') Rebin(InputWorkspace='110504_IvsQ',OutputWorkspace='110504_IvsQ_binned',Params='0.011,-0.0376509,0.051') Rebin(InputWorkspace='110505_IvsQ',OutputWorkspace='110505_IvsQ_binned',Params='0.02,-0.0376509,0.142471') CloneWorkspace(InputWorkspace='110504_IvsQ_binned',OutputWorkspace='currentSum') Stitch1D(LHSWorkspace='currentSum',RHSWorkspace='110505_IvsQ_binned',OutputWorkspace='currentSum') RenameWorkspace(InputWorkspace='currentSum',OutputWorkspace='110504_50') graph_spec=plotSpectrum('110504_50', 0) l = graph_spec.activeLayer() l.logLogAxes()
comment:11 Changed 6 years ago by Owen Arnold
- Status changed from inprogress to verify
- Resolution set to fixed
comment:12 Changed 6 years ago by Anders Markvardsen
- Status changed from verify to verifying
- Tester set to Anders Markvardsen
comment:13 Changed 6 years ago by Anders Markvardsen
- Status changed from verifying to closed
Merge remote-tracking branch 'origin/feature/9955_nans'
Full changeset: 41410b10f418e639101ebabda460fb916867ff44
comment:14 Changed 6 years ago by Anders Markvardsen
update to doc matches changes made to code and script runs with continuation on both sides
comment:15 Changed 6 years ago by Peter Peterson
- Keywords PatchCandidate removed
- Milestone changed from Release 3.3 to Release 3.2.1
comment:16 Changed 6 years ago by Martyn Gigg
comment:17 Changed 6 years ago by Owen Arnold
Cherry-pick fixes from feature/9955_nans into next
Changes:
refs #9955. No need for replace special values.
Replace special values done as part of finding and recording those indexes. (cherry picked from commit 0c5642237c428bd5ca151ef7435ba6cf8c7882c4)
refs #9955. Add test and introduce replace functionality. (cherry picked from commit 1355491d395009ea197ae1fe7f7cab181e6baee3)
refs #9955. Special treatment for errors too (cherry picked from commit 346f9eaa54e593bf89b3cd94243ae670b6e35efe)
refs #9955. Document and format. (cherry picked from commit 7f5d87212f8baaedf134c648f50fff8434fd9702)
Changeset: e834c2a0c1141e3ee258d6d756bcd3aa2e853203
comment:18 Changed 5 years ago by Stuart Campbell
This ticket has been transferred to github issue 10797