Ticket #10855 (closed: fixed)

Opened 6 years ago

Last modified 5 years ago

Rewrite indirect diffraction reduction algorithm

Reported by: Dan Nixon Owned by: Dan Nixon
Priority: major Milestone: Release 3.5
Component: Indirect Inelastic Keywords:
Cc: Blocked By:
Blocking: #9093, #11431 Tester: Federico Montesino Pouzols

Description (last modified by Dan Nixon) (diff)

This can use functions from the ET algorithm, allowing workflow diagrams for both algorithms.

Change History

comment:1 Changed 6 years ago by Dan Nixon

  • Status changed from new to assigned

comment:2 Changed 6 years ago by Dan Nixon

  • Status changed from assigned to inprogress

Checkpointing work on new algorithm

Gets identical results for IRIS reductions (at least usng as much as has been implemented so far)

Refs #10855

Changeset: 0f20d2a926a5635235c7851512d702874dae431e

comment:3 Changed 6 years ago by Dan Nixon

Added correct rebinning, workspage grouping, moved rst page

Refs #10855

Changeset: 7c7531a80c69c034823c4bdad8a25c2c9c228e55

comment:4 Changed 6 years ago by Dan Nixon

Load the IPF, added skeleton function for spectra grouping

Refs #10855

Changeset: 6014e82d4566fa350b84f94805219acf83e35305

comment:5 Changed 6 years ago by Dan Nixon

Added workspace naming step

Refs #10855

Changeset: ac13c17a519abe936dd3838f3a2cbdba91540fd7

comment:6 Changed 6 years ago by Dan Nixon

Support proper grouping options

Refs #10855

Changeset: 55c7717b609b51ea1aca860a0f649834cb861f53

comment:7 Changed 6 years ago by Dan Nixon

Better handling of grouping, convert to wavelength property

Refs #10855

Changeset: 005af0a8f4d49041804066c5e32414686efbbf6c

comment:8 Changed 6 years ago by Dan Nixon

Handle summing raw files

Refs #10855

Changeset: 00d8feb5f01e187866a44874ec382c255922e227

comment:9 Changed 6 years ago by Dan Nixon

Add original run numbers as sample log

Refs #10855

Changeset: b04023c931702ca14372f07563d1a0251efd68ea

comment:10 Changed 6 years ago by Dan Nixon

Use correct scale factor parameter

Refs #10855

Changeset: 378a8457240acfdbb8a98622e1fecb1b32078a48

comment:11 Changed 6 years ago by Dan Nixon

Swap tests to use new algorithm

Refs #10855

Changeset: 7b8c2bbb54cc1d22576234614301de59a37b5b2c

comment:12 Changed 6 years ago by Dan Nixon

To test:

  • Ensure system tests are passing: IRISDiffractionTest, OSIRISDiffractionTest, TOSCADiffractionTest

comment:13 Changed 6 years ago by Dan Nixon

Updated ref results for changes in algorithm

Previously param files were not loaded so monitor scaling was not done

Refs #10855

Changeset: 4f935edf7617180a542990d98f37ccc2ed06faa3

comment:14 Changed 6 years ago by Dan Nixon

Removed old reduction scripts, switch UI to new algo

Need to add saving

Refs #10855

Changeset: bbe37ec11c3b123d378bf59812acdb1ac5010b35

comment:15 Changed 6 years ago by Dan Nixon

Re add saving to generic reductions

Refs #10855

Changeset: ca008d2133db9276c58ba17590a5d667d365718b

comment:16 Changed 6 years ago by Dan Nixon

Merge branch 'master' into feature/10855_rewrite_indirect_diffraction_reduction

Conflicts:

Code/Mantid/MantidQt/CustomInterfaces/src/Indirect/IndirectDiffractionReduction.cpp

Refs #10855

Changeset: 4344b3f8f1e233e41c8d9efbaa7bf258a5b13a80

comment:17 Changed 6 years ago by Dan Nixon

  • Blocking 11139 added

comment:18 Changed 6 years ago by Dan Nixon

Merge branch 'master' into feature/10855_rewrite_indirect_diffraction_reduction

Refs #10855

Changeset: c8a3e551b0be3bf5c1b38a71c832c56fafd85650

comment:19 Changed 6 years ago by Dan Nixon

  • Blocking 11139 removed

comment:20 Changed 6 years ago by Dan Nixon

Merge branch 'master' into feature/10855_rewrite_indirect_diffraction_reduction

Conflicts:

Code/Mantid/Framework/PythonInterface/plugins/algorithms/WorkflowAlgorithms/MSGDiffractionReduction.py Code/Mantid/scripts/Inelastic/IndirectDiffractionReduction.py

Refs #10855

Changeset: 44fa4c0595039f59bb3f80b4dc3659a63c4747c5

comment:21 Changed 6 years ago by Dan Nixon

Switch system tests to use new algorithm

Refs #10855

Changeset: 462192a06a731685740da4be99bd1e74cdf28123

comment:22 Changed 6 years ago by Dan Nixon

  • Blocking 11431 added

comment:23 Changed 6 years ago by Dan Nixon

  • Milestone changed from Release 3.4 to Release 3.5

comment:24 Changed 5 years ago by Dan Nixon

Merge branch 'master' into feature/10855_rewrite_indirect_diffraction_reduction

Conflicts:

Code/Mantid/Framework/PythonInterface/plugins/algorithms/WorkflowAlgorithms/MSGDiffractionReduction.py

Refs #10855

Changeset: 11748c8c0c1f2cac751d81e3473f7d477ba4c976

comment:25 Changed 5 years ago by Dan Nixon

Merge branch 'master' into feature/10855_rewrite_indirect_diffraction_reduction

Refs #10855

Changeset: f93593a2575c1c275d24f4dac4e5f27c72916272

comment:26 Changed 5 years ago by Dan Nixon

Start moving common functionality to seperate script file

Refs #10855

Changeset: d28eab1a5f10c3f1e4133b651e0164ab8664874e

comment:27 Changed 5 years ago by Dan Nixon

Split all functions into additional file

Refs #10855

Changeset: 4cbc8168fc870f061ee48ea5ab458194d20e53db

comment:28 Changed 5 years ago by Dan Nixon

Refactor diffraction algorithm, all tests now pass

Refs #10855

Changeset: 8d205bd0c55c6dd80ec3dbe8d43e7222978434be

comment:29 Changed 5 years ago by Dan Nixon

Remove unused files Refs #10855

Changeset: fb4df43300cbecbed3ef81b076f21deafccbf64c

comment:30 Changed 5 years ago by Dan Nixon

Move more common code to IndirectReductionCommon.py

Refs #10855

Changeset: f2bbec8e929763627851ffed017c6137e7124bd4

comment:31 Changed 5 years ago by Dan Nixon

Added diagrams for ET and diff workflows

Refs #10855

Changeset: 10e95398dc998d3d257fe2c2c33cc817f43375c1

comment:32 Changed 5 years ago by Dan Nixon

More work on workflow diagrams

Refs #10855

Changeset: 832f4cbb63825c489be772c4111257387ec75bba

comment:33 Changed 5 years ago by Dan Nixon

Update diffraction reduction diagram

Refs #10855

Changeset: c746ce676c0908eb821328ce5ff89a515a64c7f6

comment:34 Changed 5 years ago by Dan Nixon

  • Description modified (diff)

comment:35 Changed 5 years ago by Dan Nixon

  • Status changed from inprogress to verify
  • Resolution set to fixed

This is being verified as pull request #763.

comment:36 Changed 5 years ago by Dan Nixon

Did not account for VESUVIO on diffraction.

comment:37 Changed 5 years ago by Dan Nixon

Support VESUVIO on diffraction properly

Refs #10855

Changeset: 5312298a21f4da4d56bc5350f58c25aa987564bd

comment:38 Changed 5 years ago by Dan Nixon

Fix broken system tests

Refs #10855

Changeset: ac4d05b29a015400e65fbba82508240f7f3ac1c2

comment:39 Changed 5 years ago by Dan Nixon

Fix trivial doctest failure Refs #10855

Changeset: bad9b5f7fb6913c8de81504ad7b542e5d8994165

comment:40 Changed 5 years ago by Dan Nixon

Merge branch 'master' into feature/10855_rewrite_indirect_diffraction_reduction

Conflicts:

Code/Mantid/Framework/PythonInterface/plugins/algorithms/WorkflowAlgorithms/ISISIndirectEnergyTransfer.py

Refs #10855

Changeset: 342140e2620c36d21114d344d206d903e0a3f9e2

comment:41 Changed 5 years ago by Federico Montesino Pouzols

  • Status changed from verify to verifying
  • Tester set to Federico Montesino Pouzols

comment:42 Changed 5 years ago by Federico Montesino Pouzols

All looks fine and the documentation makes perfect sense. But pylint has a different opinion... IndirectReductionCommon.py comes with 42 warnings: http://builds.mantidproject.org/job/pull_requests-pylint/439/violations/file/IndirectReductionCommon.py/, and ISISIndirectDiffractionReduction.py introduces 15. Some of them make sense and seem esay to fix (like an unused import). Then there is also the annoying issue with mantid.simpleapi where from mantid.simpleapi import ... will produce a warning for every function name that you use (https://github.com/mantidproject/mantid/pull/806#issuecomment-106856824). I'm running the system and doc tests locally just in case, but otherwise everything seems fine.

comment:43 Changed 5 years ago by Dan Nixon

Keep Pylint happy Refs #10855

Changeset: 0d577606ac7d8ffb95d3529f40fe7ab8772037e1

comment:44 Changed 5 years ago by Dan Nixon

I'm assuming no-name-in-module was the right thing to disable to fix those errors, I'm slightly reluctant to have to change the use of those algorithms since there is actually nothing wrong with the way it was already done.

Considering I do this in pretty much every Python algorithm I write (as do a fair few other developers) this should just be ignored globally.

comment:45 Changed 5 years ago by Federico Montesino Pouzols

The only (spurious) issue is the failed windows doc-test. I passed them locally and it went fine. The doc-test issues are totally unrelated to this PR.

comment:46 Changed 5 years ago by Dan Nixon

Checkpointing work on new algorithm

Gets identical results for IRIS reductions (at least usng as much as has been implemented so far)

Refs #10855

Changeset: 0f20d2a926a5635235c7851512d702874dae431e

comment:47 Changed 5 years ago by Dan Nixon

Added correct rebinning, workspage grouping, moved rst page

Refs #10855

Changeset: 7c7531a80c69c034823c4bdad8a25c2c9c228e55

comment:48 Changed 5 years ago by Dan Nixon

Load the IPF, added skeleton function for spectra grouping

Refs #10855

Changeset: 6014e82d4566fa350b84f94805219acf83e35305

comment:49 Changed 5 years ago by Dan Nixon

Added workspace naming step

Refs #10855

Changeset: ac13c17a519abe936dd3838f3a2cbdba91540fd7

comment:50 Changed 5 years ago by Dan Nixon

Support proper grouping options

Refs #10855

Changeset: 55c7717b609b51ea1aca860a0f649834cb861f53

comment:51 Changed 5 years ago by Dan Nixon

Better handling of grouping, convert to wavelength property

Refs #10855

Changeset: 005af0a8f4d49041804066c5e32414686efbbf6c

comment:52 Changed 5 years ago by Dan Nixon

Handle summing raw files

Refs #10855

Changeset: 00d8feb5f01e187866a44874ec382c255922e227

comment:53 Changed 5 years ago by Dan Nixon

Add original run numbers as sample log

Refs #10855

Changeset: b04023c931702ca14372f07563d1a0251efd68ea

comment:54 Changed 5 years ago by Dan Nixon

Use correct scale factor parameter

Refs #10855

Changeset: 378a8457240acfdbb8a98622e1fecb1b32078a48

comment:55 Changed 5 years ago by Dan Nixon

Removed old reduction scripts, switch UI to new algo

Need to add saving

Refs #10855

Changeset: bbe37ec11c3b123d378bf59812acdb1ac5010b35

comment:56 Changed 5 years ago by Dan Nixon

Re add saving to generic reductions

Refs #10855

Changeset: ca008d2133db9276c58ba17590a5d667d365718b

comment:57 Changed 5 years ago by Dan Nixon

Merge branch 'master' into feature/10855_rewrite_indirect_diffraction_reduction

Conflicts:

Code/Mantid/MantidQt/CustomInterfaces/src/Indirect/IndirectDiffractionReduction.cpp

Refs #10855

Changeset: 4344b3f8f1e233e41c8d9efbaa7bf258a5b13a80

comment:58 Changed 5 years ago by Dan Nixon

Merge branch 'master' into feature/10855_rewrite_indirect_diffraction_reduction

Refs #10855

Changeset: c8a3e551b0be3bf5c1b38a71c832c56fafd85650

comment:59 Changed 5 years ago by Dan Nixon

Merge branch 'master' into feature/10855_rewrite_indirect_diffraction_reduction

Conflicts:

Code/Mantid/Framework/PythonInterface/plugins/algorithms/WorkflowAlgorithms/MSGDiffractionReduction.py Code/Mantid/scripts/Inelastic/IndirectDiffractionReduction.py

Refs #10855

Changeset: 44fa4c0595039f59bb3f80b4dc3659a63c4747c5

comment:60 Changed 5 years ago by Dan Nixon

Switch system tests to use new algorithm

Refs #10855

Changeset: 462192a06a731685740da4be99bd1e74cdf28123

comment:61 Changed 5 years ago by Dan Nixon

Merge branch 'master' into feature/10855_rewrite_indirect_diffraction_reduction

Conflicts:

Code/Mantid/Framework/PythonInterface/plugins/algorithms/WorkflowAlgorithms/MSGDiffractionReduction.py

Refs #10855

Changeset: 11748c8c0c1f2cac751d81e3473f7d477ba4c976

comment:62 Changed 5 years ago by Dan Nixon

Merge branch 'master' into feature/10855_rewrite_indirect_diffraction_reduction

Refs #10855

Changeset: f93593a2575c1c275d24f4dac4e5f27c72916272

comment:63 Changed 5 years ago by Dan Nixon

Start moving common functionality to seperate script file

Refs #10855

Changeset: d28eab1a5f10c3f1e4133b651e0164ab8664874e

comment:64 Changed 5 years ago by Dan Nixon

Split all functions into additional file

Refs #10855

Changeset: 4cbc8168fc870f061ee48ea5ab458194d20e53db

comment:65 Changed 5 years ago by Dan Nixon

Refactor diffraction algorithm, all tests now pass

Refs #10855

Changeset: 8d205bd0c55c6dd80ec3dbe8d43e7222978434be

comment:66 Changed 5 years ago by Dan Nixon

Remove unused files Refs #10855

Changeset: fb4df43300cbecbed3ef81b076f21deafccbf64c

comment:67 Changed 5 years ago by Dan Nixon

Move more common code to IndirectReductionCommon.py

Refs #10855

Changeset: f2bbec8e929763627851ffed017c6137e7124bd4

comment:68 Changed 5 years ago by Dan Nixon

Added diagrams for ET and diff workflows

Refs #10855

Changeset: 10e95398dc998d3d257fe2c2c33cc817f43375c1

comment:69 Changed 5 years ago by Dan Nixon

More work on workflow diagrams

Refs #10855

Changeset: 832f4cbb63825c489be772c4111257387ec75bba

comment:70 Changed 5 years ago by Dan Nixon

Update diffraction reduction diagram

Refs #10855

Changeset: c746ce676c0908eb821328ce5ff89a515a64c7f6

comment:71 Changed 5 years ago by Dan Nixon

Support VESUVIO on diffraction properly

Refs #10855

Changeset: 5312298a21f4da4d56bc5350f58c25aa987564bd

comment:72 Changed 5 years ago by Dan Nixon

Fix broken system tests

Refs #10855

Changeset: ac4d05b29a015400e65fbba82508240f7f3ac1c2

comment:73 Changed 5 years ago by Dan Nixon

Fix trivial doctest failure Refs #10855

Changeset: bad9b5f7fb6913c8de81504ad7b542e5d8994165

comment:74 Changed 5 years ago by Dan Nixon

Merge branch 'master' into feature/10855_rewrite_indirect_diffraction_reduction

Conflicts:

Code/Mantid/Framework/PythonInterface/plugins/algorithms/WorkflowAlgorithms/ISISIndirectEnergyTransfer.py

Refs #10855

Changeset: 342140e2620c36d21114d344d206d903e0a3f9e2

comment:75 Changed 5 years ago by Dan Nixon

Keep Pylint happy Refs #10855

Changeset: 0d577606ac7d8ffb95d3529f40fe7ab8772037e1

comment:76 Changed 5 years ago by Federico Montesino Pouzols

  • Status changed from verifying to closed

Merge pull request #763 from mantidproject/feature/10855_rewrite_indirect_diffraction_reduction

New ISIS indirect diffraction reduction algorithm

Full changeset: eb7b303011d72d9aac711a2bfaa00c7cbbe89c32

comment:77 Changed 5 years ago by Stuart Campbell

This ticket has been transferred to github issue 11697

Note: See TracTickets for help on using tickets.