Wrap the MaxEnt Fortran routine in F2Py and expose to Mantid

[Peter Parker]

Adrian has supplied me with a Fortran routine called MaxEnt that he currently runs from OpenGENIE. He would like it to be wrapped using F2Py so that it may be called from within Mantid in the form of an algorithm.

I want to get a working copy to Adrian very early on in this release cycle so that he has plenty of time to test and make comments before it makes it into version 3.2.

I will handle the Fortran in a similar way to how it is used in Inelastic Indirect; i.e. keep a copy of the source in the 3rdpartysources repo and keep the compiled .pyd files in the repo. We'll be using the Script Repository to prove this functionality, however, to keep F2Py out of the main repo.

I will make sure that the resulting algorithm is only available on Windows -- the only currently supported F2Py platform on Mantid.

[Nick Draper]

Bulk move of tickets out of triage (new) to assigned at the introduction of the triage state

[Peter Parker]

Refs #8910 - Original code from Adrian.

Adrian has confirmed that this code is not copyrighted.

Changeset: ​0cf0e053b26f95588d1acfd23cb035d3a611644a (3rd Party Sources Repo)

[Peter Parker]

Refs #8910 - Comment out OpenGENIE print calls.

Changeset: ​fd1c71ef5c8e4125a617430ca7fc34e951b1839f (3rd Party Sources Repo)

[Peter Parker]

Refs #8910 - Implement our own top-layer, callable from Mantid.

Basically a replacement for opengenie_maxent in Read.for.

Changeset: ​1b9d546f664b0a27e5326c0e5e8d581efe868c77 (3rd Party Source Repo)

[Peter Parker]

Refs #8910 - Files to enable compilation via F2Py.

These files need to be added to Code/Third_Party/lib/win64/Python27/Lib/distutils/ to make sure F2Py knows about VS2012.

Changeset: ​5f27d923b5b4e2792217328cad289d9e0080b09d (3rd Party Source Repo)

[Peter Parker]

Refs #8910 - Batch file for compilation.

IMPORTANT: This script requires two things to run properly:

1 - The dir of the python installation you want to use. This will have to be the installation you've modified with the msvccompiler.py files.

2 - It needs to be run using the "Intel Composer XE 2013 SP1 Intel(R) 64 Visual Studio 2012" command prompt, which, as you might imagine, needs both Visual Studio 2012 and the Intel Fortran compiler.

32-bit compilations should be possible, as long as 32-bit versions of Mantid's Python and the command prompt are used.

Changeset: ​b68a400212c84a77ba74038dd9a5dc4758fc0d93 (3rd Party Source Repo)

[Peter Parker]

Refs #8910 - Add callback for sending messages to Mantid Results Log.

• Replace all previous instances of the OpenGENIE command, module_print.
• Remove all writes (to 69 and 99).
• Expose callbacks to all the Fortran code, by using intent(callback) and removing the corresponding parameters from the entry-point subroutine.

Changeset: ​3e8433c3e8b07767733ef3b52c985829d7d848a7 (3rd Party Source Repo)

[Peter Parker]

Adrian has requested that the defaults used by the algorithm be displayed. #2480 will handle this functionality, but generically for all algorithms.

[Peter Parker]

Refs #8910 - Simplify and clean up use of fit/fix phase/dead within the Fortran.

Changeset: ​7e0c8527ce38893c9fca5446733710526758d712 (3rd Party Source Repo)

[Peter Parker]

Refs #8910 - Change Fortran to support EMU.

Note that this only involved a straight swap of any occurances of "64" in the code with "96". This was never meant to be a future- proof method of supporting more than the previously-supported 64 detectors.

This change will require testing from the scientists to verify that things are still working as expected.

Changeset: ​cc8191c0ecfefa7241427191d1418ec02a0a25b5 (3rd Party Source Repo)

[Peter Parker]

Refs #8910 - Return chi-squared data from the Fortran.

So that it may be output as a TableWorkspace in Mantid.

Changeset: ​43dd05bd2534afb764a519a369ccbecb788255d7 (3rd Party Source Repo)

[Peter Parker]

Uploaded some items to the Script Repo:

• ​6dd7dc353065f1a9c557475777c04866dba72d84 - The actual MaxEnt Python algorithm.
• ​19d1ba5b0f760e9443edc2ebc2fccf43f2ed739f - Default grouping file for MUSR.
• ​8dddb0dfc9f5c037d25025b1a0a638be9d00f2a7 - Conversion script.
• ​6095e338d1c1787040eb9339bef16b80b92623b6 - The compiled-from-Fortran Python module.
• ​e3074682dbfa6ed4a3f04cd162a1ffe71d493ae3 - File containing phase information.
• ​fffaaffccf35d312cab34584307e8534f590a5cd - File containing deadtime information.

[Peter Parker]

Script Repo:

Changed float column types to double so that saving and loading of the tables would be possible - ​58268d0a8070a250e6f78856264598b80fe8d2d3.

Also added some ​unit tests, which unfortunately have to be run manually.

[Peter Parker]

Adrian would like to see an easier way to reset the .dat files to "default", and I have opened #9558 for this. Apart from that, MaxEnt appears to be working as intended (it was used during the recent Muon Training School), so I'm closing.

Testing

Windows only. ISIS only.

Merging

All the changes were made to the master branch of the 3rd Party Source and Script Repos, so no git commands are necessary.

Installation Instructions

We are providing MaxEnt using the Script Repository rather than as part of Mantid. The steps to get it working are as follows:

• Start an installed or built version Mantid. Go to File → Script Repository and when asked, choose a location for your local copy of the repo.

• Download the “muon” folder (which contains a “MaxEnt” subdirectory) using the green arrow.

• Go to View → Preferences → Mantid → Directories and navigate to the recently-downloaded MaxEnt directory on your local machine. MaxEnt will now be an available algorithm when Mantid starts up.

Running the Unit Tests

You will have to run the unit tests manually, using the version of Python that comes with whatever version of Mantid for which you did the installation steps above. The tests are in scriptrepository/muon/MaxEnt/Tests/TestMaxEnt.py. Note that the tests assume you have access to MUSR run 48033 either via a managed directory or through archive searching.

General Testing

• Play with various inputs to the algorithm, and try to break it. The Fortran will not cope very well with incorrect inputs, so the idea is that validation done by the Python will catch any problematic inputs and stop the algorithm gracefully before the Fortran has a chance to crash.

The algorithm does need .dat and grouping files as inputs. The default values and unit tests should help you out here, but if it is not clear then please ask.

• Code review MaxEnt.py. You are welcome to review the Fortran, but please be aware that this was only slightly modified by me from the original code provided by Adrian, and it was beyond the scope of this ticket to make changes to it (barring what was absolutely necessary).

[Roman Tolchenov]

Algorithm crashed with the message:

AlgorithmManager:: Unable to create algorithm MaxEnt 'MaxEnt' object has no attribute 'setOptionalMessage'
  at line 123 in '/repo/muon/MaxEnt/MaxEnt.py'

[Peter Parker]

This was due to the recent change in how we document algorithms, a problem we unfortunately didn't catch since the unit tests are not run automatically.

I pushed a fix to bring this algorithm in line with the change. (​fe4c1664188443161f4680bacada870a70599363.)

After a discussion with Martyn I've opened #9727, to address the problem of automated testing for the script repo at a later date.

[Peter Parker]

Just realised it would be good to make the change backward compatible for those who are still using older versions of Mantid. (​c9855764d293b034abb667b6d4a9ea595e9ab008.)

[Roman Tolchenov]

Use hasattr(...) instead of getattr(...) on line 132 of MaxEnt.py

[Peter Parker]

Made the following changes:

• ​987569aac39aa3e4096d244b27dc8c8a701d1525.
• ​ecb8452e67c5a8f1f74e9ead3674daf372b4ce9d.

[Stuart Campbell]

This ticket has been transferred to github issue ​9753