Ticket #7769 (closed: fixed)

Opened 7 years ago

Last modified 5 years ago

Use the new web API for remote jobs

Reported by: Ross Miller Owned by: Ross Miller
Priority: major Milestone: Release 3.0
Component: Framework Keywords:
Cc: Blocked By:
Blocking: #7907 Tester: Vickie Lynch

Description

Update the framework to use the new and improved web API for submitting remote jobs.

Change History

comment:1 Changed 7 years ago by Ross Miller

  • Status changed from new to inprogress

Start moving to new web API

First changes to support the new web api:

  • RemoteJobManager hierachy has been collapsed into a single (and much simpler) class that now lives in MantidKernel
  • RemoteJobManagerFactory is gone (no longer needed)
  • Added a new 'Authenticate' algorithm
  • Disabled the other remote algorithms because they haven't been updated yet.

Refs #7769

Changeset: b69ceb46a484b56d653ee8873479be048ddb1b27

comment:2 Changed 7 years ago by Ross Miller

Modified the start and stop transaction algorithms

Updated StartRemoteTransaction and StopRemoteTransaction to use the new web API. Also updated the cookie handling code in RemoteJobManager

Refs #7769

Changeset: 85edbebfb3e968f2d6e658db203af36ccb35af9a

comment:3 Changed 7 years ago by Ross Miller

Changes to the HTTP POST function

Couple of minor changes to make POST requests work properly: 1) Need a blank line before the start of the data in multi-part mime data 2) Separate file data from other data (by using a second PostDataMap reference) because it needs to be handled a little differently in the mime encoding

Refs #7769

Changeset: b4cc9b09fda447fbe0be4d00f9622dbcb54dc742

comment:4 Changed 7 years ago by Ross Miller

Convert the upload and download algorithms to the new web API

Refs #7769

Changeset: 1ead42e4f12bc5e8bd9b3cf3f2097243063cc005

comment:5 Changed 7 years ago by Ross Miller

Update the Doxygen comments for the upload algorithm

Refs #7769

Changeset: 21a785e70f626b90d6a7550e7f5463e63b0aac08

comment:6 Changed 7 years ago by Ross Miller

Add the Authenticate algorithm

Refs #7769

Changeset: 3f3e37d8341088180c417458dc165bda6e9963f7

comment:7 Changed 7 years ago by Ross Miller

Finish converting remote algorithms to new API

Finished converting the remote algorithms to the new API. Note: The query and submit algorithms haven't been tested yet because they must be tested against Fermi and that can only be done from the analysis machines. Checking the code in now so that I can pull it down to an analysis machine and try it.

Refs #7769

Changeset: 838c4c44fafa1acd3b6870ea9c93b9ecc9934877

comment:8 Changed 7 years ago by Ross Miller

Fix a typo in a log message

Refs #7769

Changeset: 47b89c10054307ec7764ea9f9ed98e27e094c62f

comment:9 Changed 7 years ago by Ross Miller

Add code to the JSONValue calls to convert...

...bools & numbers to strings.

Refs #7769

Changeset: d115dc3ec7bec72c9649d9c8972022d54d861559

comment:10 Changed 7 years ago by Ross Miller

Fix typo in SubmitRemoteJob algorithm

Refs #7769

Changeset: 0d3340ecc82dfb7a14cc04d6dcfaf9ab3ff20e5f

comment:11 Changed 7 years ago by Ross Miller

Fix improper handling of return stream for QueryRemoteJob

Refs #7769

Changeset: 78a05f33b1bc69a5774be9ae3bda63824174145f

comment:12 Changed 7 years ago by Ross Miller

SubmitRemoteJob should check for HTTP_CREATED, not HTTP_OK

Refs #7769

Changeset: 6f39d565b500ed1397b09f478a062b954540aa23

comment:13 Changed 7 years ago by Ross Miller

Adding algorithm to abort remote jobs

Refs #7769

Changeset: 95e2ee06667165ac75279135aba8faabb6070c50

comment:14 Changed 7 years ago by Ross Miller

Remove Framework/Remote

Moved the SimpleJSON code over to Framework/RemoteAlgorithms and then remove everything else under Framework/Remote. Update the various CMakeLists.txt files.

Refs #7769

Changeset: 444d31d2ebe2db7c2184cb6c13a8c4dc21060b15

comment:15 Changed 7 years ago by Ross Miller

Update the compute resource tag in Facilities.xml

The new web API doesn't need as much detail in Facilities.xml

Refs #7769

Changeset: c0ee1b548b4adb4efb35780f64ac5be0245f2b22

comment:16 Changed 7 years ago by Ross Miller

Merge branch 'feature/7769_NewRemoteAPI' into develop

Conflicts:

Code/Mantid/Framework/Remote/src/RemoteJobManager.cpp

Refs #7769

Changeset: 4544bd6786d0cbdd29f221576ed8968a1053605b

comment:17 Changed 7 years ago by Ross Miller

Hack to work around problem with winsock.h

It looks like the Poco headers are trying to include both winsock.h and winsock2.h (possibly because windows.h includes winsock.h by default). Even if this works (and I won't know until I push it into the develop branch), it's still a lousy solution.

Refs #7769

Changeset: e0e3bfe3c59c4b4b6faf62d23e146a72e15e5ba8

comment:18 Changed 7 years ago by Ross Miller

Attempt #2 to hack around the winsock.h problem

Refs #7769

Changeset: 2e9177e333917a3f6e9ec4074687c1f72d9be748

comment:19 Changed 7 years ago by Ross Miller

Last attempt to hack around the winsock issue before I give up

Refs #7769

Changeset: 82bfbf5324e42a3f9463b08ec372c99d3432c5a6

comment:20 Changed 7 years ago by Ross Miller

Yet another winsock hack

I think the previous hack almost worked - looks like it fixed the problem for one cpp file anyway. Moving my hack over to the header file that both RemoteJobManager and FacilityInfo include...

Refs #7769

Changeset: 928d9b74ffa9b16e52029151a9b3ad168c3279f6

comment:21 Changed 7 years ago by Mathieu Doucet

Re #7769 Update job submission and query

Changeset: 07358b487929e6cba06b344e85127b432328fc14

comment:22 Changed 7 years ago by Mathieu Doucet

Re #7769 fix conflict

Changeset: 71e155633c0239c966d5bd44dde2ec1bc5ebab2e

comment:23 Changed 7 years ago by Mathieu Doucet

Re #7769 Add abort job/stop transaction button

Changeset: 51b43ca8c4b2d28d546d369594210c734c21f2ba

comment:24 Changed 7 years ago by Ross Miller

Forgot to prefix a call to make_pair with 'std::'

Strangely, it compiled on Linux anyway. Windows complained, though.

Refs #7769

Changeset: fc49886eb87de7b137aa91f5ab697dbf06fd061c

comment:25 Changed 7 years ago by Ross Miller

Fix a couple of windows-specific build issues

Refs #7769

Changeset: 280dd075fb934b2f7c406f3967b6998c6a7c72d0

comment:26 Changed 7 years ago by Ross Miller

Catch exceptions parsing the JSON data returned from the WEB API

Refs #7769

Changeset: 396da322a6cb3ac894c71f2f1a013204a2596a26

comment:27 Changed 7 years ago by Ross Miller

  • Status changed from inprogress to verify
  • Resolution set to fixed

comment:28 Changed 7 years ago by Ross Miller

  • Blocking 7907 added

comment:29 Changed 7 years ago by Vickie Lynch

  • Status changed from verify to verifying
  • Tester set to Vickie Lynch

comment:30 Changed 7 years ago by Vickie Lynch

  • Status changed from verifying to closed

My job ran on Fermi

comment:31 Changed 7 years ago by Ross Miller

Merge branch 'feature/7769_NewRemoteAPI' into develop

Conflicts:

Code/Mantid/Framework/Remote/src/RemoteJobManager.cpp

Refs #7769

Changeset: 9769e0aa34124358c6d8d6bf613378bea7ec1d5a

comment:32 Changed 7 years ago by Ross Miller

Fix call to ofstream constructor

Officially, the ofstream constructor takes a const char * for the file name, not a string reference. Interestingly, both Linux and apparently Windows have overloaded constructors that accept a string. Mac's don't, though.

Refs #7769

Changeset: dab389729305dfd8f55354ca89c15f51c9eaa070

comment:33 Changed 7 years ago by Ross Miller

Fix call to ifstream constructor

Similar to the previous commit: Mac's are picky about their constructors for ifstream. They want a const char* and not a string for the file name.

Refs #7769

Changeset: 6e643e628db4463bc371e8680af5afbd209af701

comment:34 Changed 7 years ago by Ross Miller

Merge branch 'feature/7769_NewRemoteAPI' of github.com:mantidproject/mantid into feature/7769_NewRemoteAPI

comment:35 Changed 7 years ago by Vickie Lynch

Refs #7769 rm Code/Mantid/Framework/Remote/src/RemoteJobManager.cpp

Changeset: d67b6a533377f4abc95bf965754637ecec93195f

comment:36 Changed 5 years ago by Stuart Campbell

This ticket has been transferred to github issue 8614

Note: See TracTickets for help on using tickets.