Load algorithm with operations do not copies to python correctly

[Alex Buts]

if one loads number of workspaces into Mantid using new "add workspaces" and then tries to generate python script from the final workspace, the script is completely wrong.

e.g.

after running script

Load('MAR11001+11015',OutputWorkspace='wsMDS')

, obtaining wsMDS history and copying this history to python, one gets script:

Load(Filename='MAR11001.RAW',OutputWorkspace='__MAR11001.RAW_temp')
RenameWorkspace(InputWorkspace='__MAR11001.RAW_temp',OutputWorkspace='__11001_11015')
RenameWorkspace(InputWorkspace='__11001_11015',OutputWorkspace='wsMDS')

This script generates workspace with the name 'wsMDS', but the workspace is not the sum of two workspaces, but in fact workspace MAR11001.raw

[Peter Parker]

Refs #5212, #5441 - Better parsing when multifile loading. Hist Fixed.

Allow filenames specified with the format <<short><operator>>...<short> where:

<short> = <dir><inst><underscore><runs><ext> <operator> = "+" or "," <runs> = lists and/or ranges of (possibly added) runs e.g. "12,13+14,15:20".

See header file of MultipleFileProperty for more info on syntax.

This enables workspaces loaded through the algorithm to have a history that can be used to reproduce the workspace properly.

MWRunFiles widget of LoadDialog now remembers the exact text a user typed - in this way any user who types "INST0-50", clicks "Run" and reopens the dialog is not met by a huge string containing a list of all 51 fully resolved files, but rather their original input.

Changed the vector<vector<string>> templated versions of the toString and toValue functions to accept customised delimiters. This enables us to add an option for the user to turn off multifile parsing if they are crazy enough to want to load files with a plus or a comma in their path.

Changed the inner workings of Load for multi files. No longer calling sub algorithms with setChild or alwaysStoreInADS. All workspaces are essentially created outside the ADS and managed by the algorithm, right up until the OutputWorkspaces are set.

Tests added where appropriate, but the majority of parsing cases are already covered by MultiFileNameParser.

Changeset: 5b78e0e5d0e8d64f7d93b668d676148a029d66e5

[Peter Parker]

Refs #5212 - Fix non-win builds.

Changeset: a9c8a59af7b75f15edd0a5e9662a38921d4f44d5

[Peter Parker]

Refs #5212 - Reinstate multifile stuff, with fixed unit tests.

This reverts commit 4fe9cd9d3f49ef676201c4e0a85342233caeb3e9, and fixes tests.

Changeset: 7739c3d5336a7c2607df31940c39c4ea1cfa40e5

[Peter Parker]

Refs #5212 - Uncomment tests.

Changeset: 5cc29bc67f1a35b3fa2f713934d29bef104656a8

[Peter Parker]

Refs #5212 - Fix unit test.

Changeset: 1caf15f2d9c71152e7f401a7732a4e347fe5b10f

[Peter Parker]

Refs #5212 - This time...

Changeset: 6744bccb52e7bfc61a0eeb66ca07f6c664f11091

[Peter Parker]

Refs #5212 - Once more.

Really should check this, but my PC is compiles than a potato.

Changeset: 86a6131c51105dbf337b5097fb8ed715350d6c27

[Peter Parker]

Refs #5212, #5441 - Better parsing when multifile loading. Hist Fixed.

Allow filenames specified with the format <<short><operator>>...<short> where:

<short> = <dir><inst><underscore><runs><ext> <operator> = "+" or "," <runs> = lists and/or ranges of (possibly added) runs e.g. "12,13+14,15:20".

See header file of MultipleFileProperty for more info on syntax.

This enables workspaces loaded through the algorithm to have a history that can be used to reproduce the workspace properly.

MWRunFiles widget of LoadDialog now remembers the exact text a user typed - in this way any user who types "INST0-50", clicks "Run" and reopens the dialog is not met by a huge string containing a list of all 51 fully resolved files, but rather their original input.

Changed the vector<vector<string>> templated versions of the toString and toValue functions to accept customised delimiters. This enables us to add an option for the user to turn off multifile parsing if they are crazy enough to want to load files with a plus or a comma in their path.

Changed the inner workings of Load for multi files. No longer calling sub algorithms with setChild or alwaysStoreInADS. All workspaces are essentially created outside the ADS and managed by the algorithm, right up until the OutputWorkspaces are set.

Tests added where appropriate, but the majority of parsing cases are already covered by MultiFileNameParser.

Changeset: 5b78e0e5d0e8d64f7d93b668d676148a029d66e5

[Peter Parker]

Refs #5212 - Fix non-win builds.

Changeset: a9c8a59af7b75f15edd0a5e9662a38921d4f44d5

[Peter Parker]

Refs #5212 - Reinstate multifile stuff, with fixed unit tests.

This reverts commit 4fe9cd9d3f49ef676201c4e0a85342233caeb3e9, and fixes tests.

Changeset: 7739c3d5336a7c2607df31940c39c4ea1cfa40e5

[Peter Parker]

Refs #5212 - Uncomment tests.

Changeset: 5cc29bc67f1a35b3fa2f713934d29bef104656a8

[Peter Parker]

Refs #5212 - Fix unit test.

Changeset: 1caf15f2d9c71152e7f401a7732a4e347fe5b10f

[Peter Parker]

Refs #5212 - This time...

Changeset: 6744bccb52e7bfc61a0eeb66ca07f6c664f11091

[Peter Parker]

Refs #5212 - Once more.

Really should check this, but my PC is compiles than a potato.

Changeset: 86a6131c51105dbf337b5097fb8ed715350d6c27

[Peter Parker]

To test:

1. Load some files in a similar manner to the ticket description. You may want to use some of the more complicated syntax (see the Load page of the wiki).
2. Copy the workspace history of one of the resulting workspaces to the clipboard and then paste it into the Python console window.
3. Delete the workspaces in your workspace list.
4. Run the script and make sure the same workspaces are loaded as per 1.

[Stuart Campbell]

I was having some problems that when I reran the history script it generated a workspace group rather than a single workspace. But I dont have time today to fully investigate, so I'm returning it to the pool for someone else to verify.

[Peter Parker]

Stuart - are you loading a range of files (which end up in the same group workspace), getting the history of one of those workspaces, re-running it, and then getting the same range of files loaded into a group?

If so, this is actually the expected (but perhaps not ideal?) result of running the algorithm.

[Owen Arnold]

There are two issues here.

Firstly Stuart is right, the history for a single workspace should apply to just that workspace. Secondly, the algorithm is not interpreting the script generated from the history propertly. Try this:

Load(r'LOQ48094+48097,48098.raw',OutputWorkspace='test')

Gives a history like this:

Load(Filename=r'C:\Users\spu92482\Documents\mantid\mantid\Test\AutoTestData\LOQ48094.raw+C:\Users\spu92482\Documents\mantid\mantid\Test\AutoTestData\LOQ48097.raw,C:\Users\spu92482\Documents\mantid\mantid\Test\AutoTestData\LOQ48098.raw',OutputWorkspace='test')

Which does not generate the same output as the original script.

[Peter Parker]

Refs #5212 - Fix the second of the two MultipleFileProperty issues.

Ensure that input into a MultipleFileProperty properly survies a "round trip", i.e. what you put in is what you get out, and that using the output again, as input into another MultipleFileProperty gives the same results.

Proper functionality in a large number of use cases is assured by vastly improved coverage of unit tests, which was lacking previously.

Simplify parts of the MultipleFileProperty class, including the filename generation functors I introduced that weren't doing anybody any favours.

Changeset: 5d7d14094f6d2c2ad08c3ebd1f32ae9c96feb872

[Peter Parker]

Refs #5212 - Fix non-windows builds.

Changeset: 0533bf3a0beb53d3252ddc0a25de5188901d081d

[Peter Parker]

Refs #5212 - Fix non-win builds and run MultiFile tests statically.

Changeset: c24c0614e84a690c5241c685eb88b5cd3449ac73

[Peter Parker]

Refs #5212 - Fix problem highlighted by Mac build.

Changeset: 600482c6dd3589f7a3abfb86fcf5d7aff4b0e5f3

[Peter Parker]

Refs #5212 - Hopefully final commit to fix builds.

Changeset: e8b61bc19a528c6b558827126a2453b9cb57cff9

[Peter Parker]

Refs #5212 - Fix LoadLotsOfFiles system test.

Add unit test to cover the case where a file that has no extension is loaded.

Changeset: ec319e0a6e275cf2a196c97d1bf31e0f2c852532

[Peter Parker]

Refs #5212 - Add further unit test.

Changeset: beaaa12852fe9f3ca6054deefa44c239e9b0d32a

[Peter Parker]

Remove unit test.

Revert "Refs #5212 - Add further unit test."

This reverts commit beaaa12852fe9f3ca6054deefa44c239e9b0d32a.

Changeset: 769d93533c7419bd6dbd69c75bd00270ccda8261

[Peter Parker]

Closing.

The second of the two issues raised by Owen in comment 22 has now been resolved - a "round trip" using the Python history is now possible with each of the available multifile loading syntaxes. I am confident that the increased number of unit tests cover this functionality.

Taking the advice of Martyn and opening a separate ticket (#5788) for the first issue raised. This is a more general problem to do with how GroupWorkspaces are managed in terms of their AlgorithmHistory. (For example: load some multi-period data with LoadRaw, copy the history of one of those periods to the clipboard and then use it to reload the data. All periods will have been loaded.)

[Peter Parker]

Refs #5212 - Fix the second of the two MultipleFileProperty issues.

Ensure that input into a MultipleFileProperty properly survies a "round trip", i.e. what you put in is what you get out, and that using the output again, as input into another MultipleFileProperty gives the same results.

Proper functionality in a large number of use cases is assured by vastly improved coverage of unit tests, which was lacking previously.

Simplify parts of the MultipleFileProperty class, including the filename generation functors I introduced that weren't doing anybody any favours.

Changeset: 5d7d14094f6d2c2ad08c3ebd1f32ae9c96feb872

[Peter Parker]

Refs #5212 - Fix non-windows builds.

Changeset: 0533bf3a0beb53d3252ddc0a25de5188901d081d

[Peter Parker]

Refs #5212 - Fix non-win builds and run MultiFile tests statically.

Changeset: c24c0614e84a690c5241c685eb88b5cd3449ac73

[Peter Parker]

Refs #5212 - Fix problem highlighted by Mac build.

Changeset: 600482c6dd3589f7a3abfb86fcf5d7aff4b0e5f3

[Peter Parker]

Refs #5212 - Hopefully final commit to fix builds.

Changeset: e8b61bc19a528c6b558827126a2453b9cb57cff9

[Peter Parker]

Refs #5212 - Fix LoadLotsOfFiles system test.

Add unit test to cover the case where a file that has no extension is loaded.

Changeset: ec319e0a6e275cf2a196c97d1bf31e0f2c852532

[Peter Parker]

Refs #5212 - Add further unit test.

Changeset: beaaa12852fe9f3ca6054deefa44c239e9b0d32a

[Peter Parker]

Remove unit test.

Revert "Refs #5212 - Add further unit test."

This reverts commit beaaa12852fe9f3ca6054deefa44c239e9b0d32a.

Changeset: 769d93533c7419bd6dbd69c75bd00270ccda8261

[Nick Draper]

TESTED WITH

Load(r'LOQ48094+48097,48098.raw',OutputWorkspace='test')

extracted python

Load(Filename='C:\Mantid\Test\AutoTestData\LOQ48094.raw+C:\Mantid\Test\AutoTestData\LOQ48097.raw,C:\Mantid\Test\AutoTestData\LOQ48098.raw',OutputWorkspace='test')

Also then performed a plus on the two workspaces on the group and extracted the python from that. It correctly identified that the two load algorithms were in fact the same and only created one.

[Stuart Campbell]

This ticket has been transferred to github issue ​6058