Ticket #7019 (closed: fixed)

Opened 7 years ago

Last modified 5 years ago

The RunPythonScript should not have special behaviour for a WorkspaceGroup input

Reported by: Russell Taylor Owned by: Martyn Gigg
Priority: major Milestone: Release 3.0
Component: Python Keywords:
Cc: Blocked By:
Blocking: Tester: Russell Taylor

Description

Basically, Algorithm::checkGroups() needs to be overridden to return false, but it's a Python algorithm so I don't know how to do that.

Change History

comment:1 Changed 7 years ago by Nick Draper

  • Status changed from new to assigned
  • Owner set to Martyn Gigg

comment:2 Changed 7 years ago by Russell Taylor

My thinking has moved on since I wrote this. My feeling is that (unless there's a technical reason why it can't be) RunPythonScript should be converted to a C++ algorithm. If you don't see why that can't be the case, do send the ticket back my way.

comment:3 Changed 7 years ago by Martyn Gigg

I'll have a look. I think Janik tried initially and there was a problem but I'm not sure what it was

comment:4 Changed 7 years ago by Martyn Gigg

  • Milestone changed from Release 2.6 to Release 2.7

Batch move to 2.7

comment:5 Changed 7 years ago by Nick Draper

  • Status changed from assigned to new

comment:6 Changed 7 years ago by Nick Draper

  • Milestone changed from Release 2.7 to Backlog

comment:7 Changed 7 years ago by Martyn Gigg

  • Milestone changed from Backlog to Release 3.0

comment:8 Changed 7 years ago by Martyn Gigg

  • Status changed from new to inprogress

Rewrite RunPythonScript in C++.

This is a core algorithm for the LiveData so needs away from possible user modification and allows custom behaviour for WorkspaceGroups. Refs #7019

Changeset: fb72c2375c708111c4cc7c09abd0d4e88aabc6f4

comment:9 Changed 7 years ago by Martyn Gigg

  • Status changed from inprogress to verify
  • Resolution set to fixed

Branch: feature/7019_cpp_runpythonscript

Tester: This is best tested through the StartLiveData algorithm: http://www.mantidproject.org/StartLiveData. The behaviour should be the same as before with the exception that the algorithm will only be run once for a WorkspaceGroup input rather than once per member.

comment:10 Changed 7 years ago by Russell Taylor

  • Status changed from verify to verifying
  • Tester set to Russell Taylor

I'll grab this for testing as I have a strong interest in making certain all the live stuff is stable. I won't test it until tomorrow though.

comment:11 Changed 7 years ago by Martyn Gigg

There seems to be a problem with one of the tests on fedora 18 that I'm looking into at the moment. If I don't sort it by the end of the day I'll reopen the ticket.

comment:12 Changed 7 years ago by Martyn Gigg

  • Status changed from verifying to reopened
  • Resolution fixed deleted

comment:13 Changed 7 years ago by Martyn Gigg

  • Status changed from reopened to inprogress

Disable a test while it is diagnosed. Refs #7019

Changeset: 0cfb9b2573ff9fabe95317dad2ab12313022f7a5

comment:16 Changed 7 years ago by Martyn Gigg

Switch from Poco->boost regex engine.

There is a bug in Poco::RegularExpression that causes and out of memory error on some regular expressions. The boost version does not suffer from this and actually has cleaner syntax for the eol conversion. Refs #7019

Changeset: 103522339b9e51a828268d30acef5fe70ceced89

comment:17 Changed 7 years ago by Martyn Gigg

Switch out usage of Poco regex for boost in GUI ScriptCode class

Not technically related to this ticket but the regexes were identical so the ScriptCode class would have suffered the same out of memory error. Refs #7019

Changeset: 65b17b38e13df051386d457e53933ea2a4b75c76

comment:18 Changed 7 years ago by Martyn Gigg

  • Status changed from inprogress to verify
  • Resolution set to fixed

comment:19 Changed 7 years ago by Russell Taylor

  • Status changed from verify to verifying

comment:20 Changed 7 years ago by Russell Taylor

  • Status changed from verifying to reopened
  • Resolution fixed deleted

When I try this out using the DGS Reduction GUI (see the to_live_script method in scripts/Interface/reduction_gui/reduction/inelastic/dgs_reduction_script.py), I get the following back the first time MonitorLiveData goes through it (i.e. the second time overall as StartLiveData already ran it):

MonitorLiveData-[Error] MonitorLiveData: UNKNOWN Exception is caught in exec()

This comes out of line 177: boost::python::exec(script.c_str(), globals, locals);, but that's all I could figure out (the exception appears not to be derived from std::exception or boost::exception).

The code that's being run in case you can spot anything wrong with it is:

from mantid.simpleapi import *
def PyExec(input=None,output=None):
  if mtd.doesExist("live_spe"): input.run().addProperty("EnergyRequest",mtd["live_spe"].getRun()["EnergyRequest"],True)
  DgsReduction(SampleInputWorkspace=input,OutputWorkspace=output,UseIncidentEnergyGuess=True,IncidentBeamNormalisation="None",)
  return input,output
input,output = PyExec(input,output)

Note that the "if mtd.doesExist("live_spe"):" test will be true the second time around where it was false the first time.

comment:19 Changed 7 years ago by Martyn Gigg

  • Status changed from reopened to inprogress

Backwards-compatible check for Python none object in C++.

Refs #7019

Changeset: 766c49f0676ce1172cc446163137094efb550b32

comment:20 Changed 7 years ago by Martyn Gigg

Refactor Python error handling code into separate header.

Refs #7019

Changeset: 91b512e51769774e6ab29cfb100ab535e7c2491f

comment:21 Changed 7 years ago by Martyn Gigg

Translate Python errors to C++ exception in RunPythonScript

Refs #7019

Changeset: 0dad518a5808683b3832b471ca138646a8f82d91

comment:22 Changed 7 years ago by Martyn Gigg

Adjust exported WorkspaceCreationHelper module to only uses interfaces

The unit tests will now behave much more like a real situation where the concrete types are not exposed. Refs #7019

Changeset: ec34864fb1ba9b97de2b363cc14ad7cd68ad9eca

comment:23 Changed 7 years ago by Martyn Gigg

Ensure RunPythonScript sees proper workspace types.

Refs #7019

Changeset: 003fb4f55fd9da88daa38db2fa72dc228b3a8328

comment:24 Changed 7 years ago by Martyn Gigg

This went to the wrong ticket:

Update class name as a result of change elsewhere. Refs #7109

Changeset: 4f9825ae34e42260919bb8a626864cd20e80bba9

comment:25 Changed 7 years ago by Martyn Gigg

  • Status changed from inprogress to verify
  • Resolution set to fixed

This should be fixed now and ready to test again.

comment:26 Changed 7 years ago by Martyn Gigg

Only gcc 4.6 accepts typename outside template. Refs #7019

Changeset: f829c2d94371d7f07fe66b9ac9321127a585eb93

comment:27 Changed 7 years ago by Martyn Gigg

Please leave this for Russell to test

comment:28 Changed 7 years ago by Russell Taylor

  • Status changed from verify to verifying

comment:29 Changed 7 years ago by Russell Taylor

  • Status changed from verifying to reopened
  • Resolution fixed deleted

All my testing checked out OK now. The DGS reduction GUI runs OK, as do other live scripts I have. Sensible messages come back if I insert deliberate errors into scripts. The algorithm now only runs once if it gets a group as input.

I did observe some strange behaviour for which tickets #8007 & #8008 have been created, though these are not regressions due to this work.

There is one thing I noticed though - PythonInterface/test/cpp/RunPythonScriptTest.h is not being build and run because it's not included in the CMakeLists. I'm reopening so that can be fixed (some method calls in the test need changing so that it will compile).

comment:30 Changed 7 years ago by Martyn Gigg

  • Status changed from reopened to inprogress

Add C++ RunPythonScript test to CMakeLists...

and fix the test. Refs #7019

Changeset: 53855f98a0b92bf3ed51000f47f48f7286d80b49

comment:31 Changed 7 years ago by Martyn Gigg

  • Status changed from inprogress to verify
  • Resolution set to fixed

I'd been focussing on the RunPythonScript test written in Python as the C++ is basically there so that there is a matching test.

It should be ready to test again now.

comment:32 Changed 7 years ago by Russell Taylor

  • Status changed from verify to verifying

comment:33 Changed 7 years ago by Russell Taylor

  • Status changed from verifying to closed

Merge remote branch 'origin/feature/7019_cpp_runpythonscript'

comment:34 Changed 7 years ago by Russell Taylor

BTW, there is one way in which this behaves differently to before - it won't allow you not to have an output workspace whereas before you could (e.g. your processing script could just be something like print "Hello"). However, I don't think the previous behaviour was correct.

comment:35 Changed 5 years ago by Stuart Campbell

This ticket has been transferred to github issue 7865

Note: See TracTickets for help on using tickets.