[ICAT] Publish algorithm

[Jay Rainey]

The goal is to allow the users of Mantid to publish their modified data to ICAT.

[Jay Rainey]

Removed unused method and forward declaration. Refs #7636.

Changeset: 28f3c5325466aef1135092f035ac2c10f74d4231

[Jay Rainey]

Create the method outline for the uploadURL. Refs #7636.

Changeset: ed4db5a5d0884037d2887a7c5f968765bb3053f3

[Jay Rainey]

Generate upload URL in ICat4. Refs #7636.

• In order to publish (upload/PUT) we need to know the dataset the user wants to PUT that file to. This is "calculated" in the getDatasetIdFromFileName method, which uses the related filename (extracted from path or workspace in the algorithm) & obtains the related datasetID.

Changeset: 110546238a4693f5172874d9384c5c6022fc9db1

[Jay Rainey]

Added CatalogPublish algorithm. Refs #7636.

Changeset: cc3a7eccfdb7991dd771e6a8cc849cd91f3c9f3b

[Jay Rainey]

Corrected indent in cmakelist. Refs #7636.

Changeset: 095226c24539ffaad0bc79b4d2717f7d28d7db35

[Jay Rainey]

Added correct ICAT call to method. Refs #7636.

Changeset: 96686ca348b6e92e92856e0a9e94b2218156d733

[Martyn Gigg]

There is a compiler warning that needs addressing: ​http://download.mantidproject.org/jenkins/view/Develop%20Clean%20Builds%20and%20Tests/job/is_clean_rhel6_develop/292/warnings20Result/

[Jay Rainey]

Fix compiler warning. Refs #7636.

Changeset: ff6edb140d78ad629308c1f29c9aa3e040766f9b

[Jay Rainey]

Removed unused exception. Refs #7636.

Changeset: 70fd22cd19e614cedf45355916220b39cda48244

[Jay Rainey]

Published a commit to test it on windows. The changes were for testing purposes, and won't be in the branch when tested.

[Jay Rainey]

Set encoding type. Upload now works correctly. Refs #7636.

• Added IsDataFile method to detect if the file being uploaded is datafile, in order to set the correct mode.

Changeset: dc0628641b8e1602d57f03b27bc1de131bcfedcb

[Jay Rainey]

Merge remote-tracking branch 'origin/master' into feature/7636_icat_publish. Refs #7636.

Conflicts:

• Code/Mantid/Framework/ICat/inc/MantidICat/ICat4/ICat4Catalog.h

Changeset: e28c983e7beb1afee3408ce8d14924a56efe9ceb

[Jay Rainey]

Helper method to obtain file name from a given path. Refs #7636.

Changeset: 08f41c17ae873fb50cd96cb54f13dac9910e0996

[Jay Rainey]

Added workspace property. Refs #7636.

• Removed unused headers.
• Forgot to remove extractFileName code from exec.

Changeset: 4caa02b08d6c619504ff9a1da54925fef13b3692

[Jay Rainey]

Encapsulate publish functionality. Refs #7636.

Changeset: a1a97ea8e4390cbbb92a9680e694e8040c8e4d9e

[Jay Rainey]

Upload of datafile works correctly. Refs #7636.

Changeset: e0912e3fd456e6ef0eab18aa87f6182390480d0b

[Jay Rainey]

Let saveNexus take care of workspace groups. Refs #7636.

Changeset: 494cbe294739e18a8fd89f88bbf92c5e391af871

[Jay Rainey]

Merge remote-tracking branch 'origin/master' into feature/7636_icat_publish. Refs #7636.

Conflicts:

• Code/Mantid/Framework/ICat/inc/MantidICat/ICat4/ICat4Catalog.h

Changeset: 9cf9ba0c9b69bc52b3f00ebe77ff648245f6f25f

[Jay Rainey]

Added setICATProxySettings to publish method. Refs #7636.

Changeset: 798963a3c6201b2990d2b097313da004da3ff7af

[Jay Rainey]

Add publishing of workspace support. Refs #7636.

• If user has selected a workspace, we will save it to their default directory then upload that file to the IDS.

Changeset: 53779aed414375639c154bd046e390ffd32fecd8

[Jay Rainey]

Add const to ICatalog and related classes. Refs #7636.

Changeset: c4b7daccd3a9c929c94bf83247a1797afb0c325d

[Jay Rainey]

Publish workspace history if workspace was selected. Refs #7636.

Changeset: c56101c3410154bec80360c236028269af7015f0

[Jay Rainey]

Remove hard-coded download URL. Refs #7636.

Changeset: cf52c9a36892dd7aa7e14d1f4f4714fab6de3bae

[Nick Draper]

I ran into a few problems testing this on windows:

1. The parameters of the algorithm need to be improved / harmonised [filepath -> FileName CreateFileName -> NameInCatalog? (default use existing filename)]
2. I was refused to upload an ILL file, we should add a InvestigationNumber property (defualt extract from file).
3. Trying to search for a file I uploaded failed, the instrument field drop down was empty, no instruments to choose from, and entering the uploaded filename and searching gave the response "You have not input any terms to search for."

[Jay Rainey]

Use SSLManager to make use of same context. Refs #7636.

Changeset: 5cd084e7682f0f5883c401433cd6a7910e8fd3fd

[Jay Rainey]

Harmonise publish algorithm parameters. Refs #7636.

Changeset: 0f29f76421f2dfc2a3a951ae97ffef64705172d6

[Jay Rainey]

Remove bad merge code from develop. Refs #7636.

Changeset: dd1b033adf220fe351151a7e557d4eaf5160a29a

[Jay Rainey]

Allow user to upload any file if they specify inves num. Refs #7636.

Changeset: 8c06f260008b0aa52f7a050052d8540ecb769f12

[Jay Rainey]

Use filename including extension. Refs #7636.

Changeset: fc2fefe23373cd64b5a83e00b6511b066bd48a37

[Jay Rainey]

Remove bad merge code from develop. Refs #7636.

Changeset: 38d87cdaffe603cf9574048a31ffbb9a59a5c703

[Jay Rainey]

Fixing a "hang" when the user selects a file (with an invalid filename) & does not specify the investigation number.

[Russell Taylor]

There's an MSVC compiler warning concerning the use of Poco::SharedPtr that needs addressing.

[Jay Rainey]

Use the correct datafile name for generated py script. Refs #7636.

• Use the datafile name to obtain the dataset ID unless the user wants to provide the specific run number.

Changeset: c73f45a8dfbe5cdc325046b07ec03b652fc204f1

[Jay Rainey]

Improved error message. Refs #7636.

Changeset: 6be9870ca8fd79dd9d2ddbb2947fe4a89bf4ad9a

[Jay Rainey]

Handle errors returned by the IDS. Refs #7636.

• It's not possible to throw a runtime_error as poco catches it, then throws another I/O error instead! This is related to a bug in POCO 1.4 (​http://sourceforge.net/p/poco/bugs/403/). Instead, I have logged the message and informed the user of the possible problems.
• This "error handling" will be improved on the GUI, but needs to exist as is for when user's write scripts against this algorithm.

Changeset: 277579a8fa2f03277237bde09d63743320efd557

[Jay Rainey]

User must be logged into ICAT to publish. Refs #7636.

Changeset: decfc706be40942f9138ecd777cba15f84714d07

[Jay Rainey]

Use raw pointer instead of shared. Refs #7636.

Changeset: 4f753192a00462ad12b94ed879bfb5ee9f4f2c84

[Jay Rainey]

Note

• You can only publish data to investigations of which you are an investigator on.
• This ticket needs to be tested on multiple platforms.
• You can only test this ticket if you have results in "my data". (E.g. you are an investigator in investigations.)

To test

Change Facilities.xml

You will need to change the soapendpoint and externaldownload in Facilities.xml to:

  soapendpoint     -> "https://icatdev.isis.cclrc.ac.uk/ICATService/ICAT"
  externaldownload -> "https://isisoxfordvmsrv.isis.cclrc.ac.uk/ids/"

Publishing a file (with a valid name)

1. Log into the catalog.
2. From the Algorithms menu input CatalogPublish and press enter.
3. Select a file to upload that has a valid name (E.G. It must have the instrument name follow by a valid rb number that is in your investigations). For example, GEM62839.txt. To obtain a valid filename you could search My data' in the ICAT GUI, select an investigation then download a file.
4. Verify that the file has been uploaded to the IDS by searching for the file in the archives. I recommend entering the name of the file in the Datafile name: field and selecting the correct instrument for a quicker search response.

Publishing a workspace (single and grouped)

1. Load a workspace into Mantid.
2. Log into the catalog.
3. Select the CatalogPublish algorithm as before.
4. This time select the workspace from the dropdown (the filepath should be empty, otherwise you will get an error).
5. Click Run. The file should now have been uploaded to the IDS.
6. Verify that the file has been uploaded by finding it in the archives. (Perhaps also loading it into Mantid & comparing it against the workspace you uploaded.)
7. A feature of the publishing algorithm is to also upload the history of the workspace in a .py file. This should also be verified. (It will have the same name as you have named the file, but with a .py extension.)

Note: You must test uploading of both a single AND a grouped workspace.

Testing the publishing parameters

1. You should provide a name to save the file in the catalog via the NameInCatalog box. By default, the filename of the file you are uploading is used.
2. Attempt to publish with an invalid name, investigation ID, and both.
3. Try to break it!

In addition, verify that there are no compiler warnings (specifically on Windows), that the code is clean, and that I have addressed Nick's comment.

[Jay Rainey]

Include necessary header & use shared pointer. Refs #7636.

Changeset: 2a4a3303c1a4546946b2b0a2badec10249a53d16

[Jay Rainey]

Better error handling. Refs #7636.

Changeset: c9cc8157b7f6872c0cfa06a31a99b205ab1a3052

[Roman Tolchenov]

Merge remote-tracking branch 'origin/feature/7636_icat_publish'

Full changeset: ​1e501ca0726ffbe9314b6fbd66d28829a6705b3d

[Jay Rainey]

Testing publishing on windows. Refs #7636.

Changeset: 3ffc02a234a5846be6901f43669a7d5a130d10a5

[Jay Rainey]

Merge branch 'feature/7636_icat_publish'. Refs #7636.

Conflicts:

• Code/Mantid/Framework/ICat/src/ICat4/ICat4Catalog.cpp
• Code/Mantid/instrument/Facilities.xml

Changeset: 10f8dbf60fbdd44136c2b03e006b3eb1c339e168

[Jay Rainey]

Use SSLManager to make use of same context. Refs #7636.

Changeset: b61d40675911ec274c22c39709333fd860dd9d55

[Jay Rainey]

Remove bad merge code from develop. Refs #7636.

Changeset: 38d87cdaffe603cf9574048a31ffbb9a59a5c703

[Stuart Campbell]

This ticket has been transferred to github issue ​8481