Ticket #4884 (closed: fixed)
Implement Job Submission to Remote Clusters
| Reported by: | Ross Miller | Owned by: | Ross Miller |
|---|---|---|---|
| Priority: | major | Milestone: | Release 2.6 |
| Component: | MantidPlot | Keywords: | Mantid-MPI ADARA |
| Cc: | Blocked By: | ||
| Blocking: | #6924 | Tester: | Mathieu Doucet |
Description
Implement functionality to submit and track jobs running on remote compute clusters.
Initial work will be via Moab Web Services, but there needs to be an abstraction layer so that other job submission schemes can be implemented.
See Chapter 6 of the ADARA design document.
Change History
comment:2 Changed 8 years ago by Ross Miller
Further work on the remote algorithms project.
Still not anywhere near complete, but some more functionality has been added: cluster info is save to / restored from the properties file and the code to fetch the specified configuration file from a URL works.
Refs #4884
Changeset: 79ec21f58dba0371f5d68b1cdbe8350b025363cc
comment:3 Changed 8 years ago by Ross Miller
Moving files related to remote job submisison to their own directory.
Refs #4884
Changeset: b3176dfcf9397eb041b93a4ede4190819574b552
comment:4 Changed 8 years ago by Ross Miller
Moving files related to remote job submisison to their own directory.
Refs #4884
Changeset: d4663554859ff1d4d45b968e3bc88e72c8403d77
comment:5 Changed 8 years ago by Ross Miller
Now have enough done to actually submit a remote job to Chadwick
refs #4884
Changeset: f9320e86d64abd86a2f70d22856fb97df0e2e466
comment:7 Changed 8 years ago by Ross Miller
Small tweaks to the remote job submission code.
Setting the job name (instead of leaving it as the default of STDIN) and picking up the user id from the RemoteJobManager class. Still need to figure out how to handle the password though!! refs #4884
Changeset: 0f50d82543c47ec1f5534fbcd28e16053249a0f7
comment:8 Changed 8 years ago by Ross Miller
Fixed a signed/unsigned comparison warning
refs #4884
Changeset: d43bc4467cafe7b3d9ce0a1729884b6e64e1d28c
comment:9 Changed 8 years ago by Ross Miller
Added job status display
Added code to display the status of previously submiitted remote jobs. Also added code to ask for a password the first time the user needs to send anything to Moab Web Services. (The password is held in memory after that.)
refs #4884
Changeset: 419ddcfd443ac976e10a12fb0611e4169d7d54fe
comment:10 Changed 8 years ago by Ross Miller
Minor cleanups in preparation for a code review
Declared default constructor for RemoteJob private (and left unimplemented), removed unnecessary #include's from MantidRemoteAlg.h and commented out a currently unused paramter in RemoteAlgorithmDockWidget::xmlParseServerAttributes()
Refs #4884
Changeset: 673ec2a23adfc942519bbcf2583482dd8a90c21c
comment:11 Changed 8 years ago by Ross Miller
Changed all instances of "Remote Algorithm" to "Remote Task"
Refs #4884
Changeset: e75089be537f10dba7615fc8f6d764828dd869d4
comment:12 Changed 8 years ago by Ross Miller
Replaced a few more instances of 'algorithm' with 'task'
Refs #4884
Changeset: 6afcfc862b0b048a62cd2fbff9d660d07839131a
comment:13 Changed 8 years ago by Ross Miller
Changes to the remote job submission code
Added code to automatically pick a name for the remote job's output file and pass that as a parameter to the job submission URL. This is needed to allow the user to download the results after the remote job has run.
Refs #4884
Changeset: 3cd2be453e12d5bd0b2ad6c08443697872f78f4c
comment:14 Changed 8 years ago by Ross Miller
Added ability to download output files from cluster
'Download' buttons in the Job Status dialog will now actually attempt to download the output file and open it in a new workspace. Also added code to verify that there actually is an output file available before adding the download button.
Refs #4884
Changeset: 3364c2bcb44897aebf8364b21aaec3786577e67c
comment:15 Changed 8 years ago by Ross Miller
Only show jobs submitted from MantidPlot in the job status dialog
Uses the 'variables' entry in the submitted JSON to set an entry called "SUBMITTING_APP" when the job is submitted. The jobStatusAll() function checks for this entry and only includes jobs that have it set in its result list.
Refs #4884
Changeset: 4af5da7d6f3b6da5b94c3d0a3f0831ee03c58c21
comment:16 Changed 8 years ago by Ross Miller
Disable previous change to job status dialog
Commit 4af5da7d6f3b6da5b94c3d0a3f0831ee03c58c21 changed the job status dialog box to only show jobs that were actually submitted from MantidPlot. Unfortunately, there's a bug in Moab Web Services (support case #15864) that renders this useless. Once the folks at Adaptive Computing fix their bug, I'll uncomment one line of code to re-enable this feature.
Refs #4884
Changeset: ec10df543ee43832c4954e96e07771a1f9ce5888
comment:17 Changed 8 years ago by Ross Miller
Several changes to job status dialog (and job submission in general)
1) HTTPS is now enabled and enforced. No unencrypted connections are allowed. (Note: We're accepting the certificate from the server as-is with no validation. That's not really correct, but it's the best we can do for now.) 2) Jobs in the job status dialog are now sorted by their ID's 3) Had to disable the use of QTemporaryFile objects for downloading the output files because the Load algorithm runs in a separate thread and the object was cleaning up its file before the algorithm had a chance to run.
Refs #4884
Changeset: ad09fd3970607afa7e36e4bbcbd8a1c3a3c20db9
comment:18 Changed 8 years ago by Ross Miller
Better error messages if there's a problem parsing the XML config file
Refs #4884
Changeset: 3e5d53c7c115c53d1b895382c2058da1b89de223
comment:19 Changed 8 years ago by Ross Miller
Jub submission GUI now supports checkboxes for boolean values
Refs #4884
Changeset: 17c7d011f41cebf8d2002ed3aa2d984e53b011da
comment:20 Changed 8 years ago by Ross Miller
Enable session cookies in MWSRemoteJobManager
Refs #4884
Changeset: 8fa9d837bf460a01574326a034e21d2484a08230
comment:21 Changed 8 years ago by Ross Miller
Starting to make GUI changes for the new job submission design
Refs #4884
Changeset: 93089021385e64a7a56c06e37fbc4d6b4860b4ab
comment:22 Changed 8 years ago by Ross Miller
More minor GUI changes plus changing a class name
Refs #4884
Changeset: 3574d978ec4778c23e29c98db91f33ac589d0f73
comment:23 Changed 8 years ago by Ross Miller
Changed the name of 2 files to match the new class name
Refs #4884
Changeset: e883109062cb549377be6d3e79803a214872809b
comment:24 Changed 8 years ago by Ross Miller
Make the remote cluster list a singleton object
This is in preparation for the new design for remote job submission. In the new design, individual algorithms will need a way to access the cluster list, so it can't be stored as part of the dock widget.
Refs #4884
Changeset: 20dce3e941acd477793f236702c867a25d536668
comment:25 Changed 8 years ago by Ross Miller
Starting to add functions necessary for the new design of the...
...remote job submission code. Specifically, adding functions for starting & stopping transactions.
Refs #4884
Changeset: 3d2eb6359cd92e950588c9b95381172b9305ad15
comment:26 Changed 8 years ago by Ross Miller
Comment & whitespace cleanups only - no code changes
Refs #4884
Changeset: 47f6f0ee8ac5cd34d15c050c4be62a6e34285103
comment:27 Changed 8 years ago by Ross Miller
Changing the RemoteJobManager infrastructure
Instances of RemoteJobManager are created from XML elements in Facilities.xml (rather than being entered by the user and stored in the user's properties file).
Refs #4884
Changeset: cb9c8e5c7d80868bd0d6011b068daa8ae8e84ea8
comment:28 Changed 8 years ago by Ross Miller
Add Fermi cluster to Facilities.xml
Adding a description of the Fermi cluster to Facilities.xml
Refs #4884
Changeset: 48bbc9e7347563746f42e4c744e47d361c0d5d58
comment:29 Changed 8 years ago by Ross Miller
comment:30 Changed 8 years ago by Ross Miller
First cut at GUI elements for the remote algorithsm work
refs #4884
Changeset: ae82af26c846f0ba01eb845d9ba6ce3f88942833
comment:31 Changed 8 years ago by Ross Miller
Moving files related to remote job submisison to their own directory.
Refs #4884
Changeset: 5d4672bb61fa97dbb97d571127a03533533a74a7
comment:32 Changed 8 years ago by Ross Miller
Now have enough done to actually submit a remote job to Chadwick
refs #4884
Changeset: 48b7d3018198c2709e1ea310e3f62e394e5b5ed4
comment:33 Changed 8 years ago by Ross Miller
Added job status display
Added code to display the status of previously submiitted remote jobs. Also added code to ask for a password the first time the user needs to send anything to Moab Web Services. (The password is held in memory after that.)
refs #4884
Changeset: 3e87fdd3d34cfd1c33d53794687544edf098dbae
comment:34 Changed 8 years ago by Ross Miller
Changed all instances of "Remote Algorithm" to "Remote Task"
Refs #4884
Changeset: cc19d7c17eca8c7e22b440b285e0b6fbfbb9c456
comment:35 Changed 8 years ago by Ross Miller
Update CMakeLists to compile the new JSON parser code
Refs #4884
Changeset: b8e273960b7473bb4ae9d3165b2b48defebf9813
comment:36 Changed 8 years ago by Ross Miller
Changed the name of 2 files to match the new class name
Refs #4884
Changeset: cf76adc6b9fdf417bcf45ec38e98cda54bc658dc
comment:37 Changed 8 years ago by Ross Miller
Changing the RemoteJobManager infrastructure
Instances of RemoteJobManager are created from XML elements in Facilities.xml (rather than being entered by the user and stored in the user's properties file).
Refs #4884
Changeset: e75d4b20bdef1aaa08ee365d866d117cc0f66aab
comment:38 Changed 8 years ago by Ross Miller
Cleaning up the mess caused by renaming the remote branch...
...and then re-basing off master. (Not sure why, but all kinds of manual conflict resolution was required.) These changes are probably because I did the conflict resolution incorrectly...
Refs #4884
Changeset: 3923b63695659e26b5b493037e09eedf21068871
comment:39 Changed 8 years ago by Ross Miller
First cut at GUI elements for the remote algorithsm work
Cherry-picked from original commit: 9d78735367a204d1058e8963f5c2ddc9ace76a0e
refs #4884
Conflicts:
Code/Mantid/MantidPlot/CMakeLists.txt
Changeset: 13f20a76ab5ef5e23b423e60b38a11137491cd82
comment:40 Changed 8 years ago by Ross Miller
Further work on the remote algorithms project.
Still not anywhere near complete, but some more functionality has been added: cluster info is save to / restored from the properties file and the code to fetch the specified configuration file from a URL works.
Refs #4884
Changeset: e968d027d25390abf7e40d8570f22d6f290db2b2
comment:41 Changed 8 years ago by Ross Miller
Moving files related to remote job submisison to their own directory.
Refs #4884
Changeset: 26bdc03dda83076df230859653c70e18d8dddd13
comment:42 Changed 8 years ago by Ross Miller
Moving files related to remote job submisison to their own directory.
Refs #4884
Cherry-picked from original commit: d4663554859ff1d4d45b968e3bc88e72c8403d77
Conflicts:
Code/Mantid/MantidPlot/CMakeLists.txt
Changeset: 9345cc4bcdc279c8e0cd9d61c29aaa544b7ee84f
comment:43 Changed 8 years ago by Ross Miller
Now have enough done to actually submit a remote job to Chadwick
refs #4884
Changeset: 11053ee634cf50b643d560a545eff926fe8badb4
comment:44 Changed 8 years ago by Ross Miller
comment:45 Changed 8 years ago by Ross Miller
Small tweaks to the remote job submission code.
Setting the job name (instead of leaving it as the default of STDIN) and picking up the user id from the RemoteJobManager class. Still need to figure out how to handle the password though!! refs #4884
Changeset: cad8e6dde07380ecc0164d89cc0d56af69ad9054
comment:46 Changed 8 years ago by Ross Miller
Fixed a signed/unsigned comparison warning
refs #4884
Changeset: 7208f5db8b53fadb3edd1e8a35ee3a7775d40d1a
comment:47 Changed 8 years ago by Ross Miller
Added job status display
Added code to display the status of previously submiitted remote jobs. Also added code to ask for a password the first time the user needs to send anything to Moab Web Services. (The password is held in memory after that.)
refs #4884
Cherry-picked from original commit: 419ddcfd443ac976e10a12fb0611e4169d7d54fe
Conflicts:
Code/Mantid/MantidPlot/CMakeLists.txt
Changeset: c3a6c2d5e81ada966eb645c7fb15df67b9c325c6
comment:48 Changed 8 years ago by Ross Miller
Minor cleanups in preparation for a code review
Declared default constructor for RemoteJob private (and left unimplemented), removed unnecessary #include's from MantidRemoteAlg.h and commented out a currently unused paramter in RemoteAlgorithmDockWidget::xmlParseServerAttributes()
Refs #4884
Changeset: 0ecc4c4f2c76e7681328d5807c233ec786b0aed2
comment:49 Changed 8 years ago by Ross Miller
Changed all instances of "Remote Algorithm" to "Remote Task"
Refs #4884
Changeset: c51f1d5617c1794697e82d2bc3ee356290f35e71
comment:50 Changed 8 years ago by Ross Miller
Replaced a few more instances of 'algorithm' with 'task'
Refs #4884
Changeset: 652231400e7361120f02409d0e52a6bff13b0210
comment:51 Changed 8 years ago by Ross Miller
Filter the job status dialog
Added a slider widget to the job status dialog so that users can filter out jobs older than a specified number of days. NOTE: This is just the GUI bits; the actual filtering code isn't ready yet.
Refs #4884
Changeset: c081756f62c00817f233c2f9b92c8f68ef102b68
comment:52 Changed 8 years ago by Ross Miller
Adding a JSON parser for Moab Web Services output
Refs #4884
Changeset: 0eb6462c0eada5ed56469fc4c3a809f63a45df47
comment:53 Changed 8 years ago by Ross Miller
Update CMakeLists to compile the new JSON parser code
Refs #4884
Changeset: 8316eaca0dd80b0d06032ad7ce59500bf6dfdf65
comment:54 Changed 8 years ago by Ross Miller
Change Job Status Dialog Box
Changed the job status dialog box to query the MWS server for all the jobs the user has submitted. Uses the new JSON parser code to parse the (rather complicated) output. Also includes a spinbox to let the user set the maximum age of jobs to be displayed.
Refs #4884
Changeset: 3084b6a879dfe70adde846ef68a6ee7dab783069
comment:55 Changed 8 years ago by Ross Miller
Changes to the remote job submission code
Added code to automatically pick a name for the remote job's output file and pass that as a parameter to the job submission URL. This is needed to allow the user to download the results after the remote job has run.
Refs #4884
Changeset: 48e45f219b1c89d6597d7440c3c3087545640396
comment:56 Changed 8 years ago by Ross Miller
Added ability to download output files from cluster
'Download' buttons in the Job Status dialog will now actually attempt to download the output file and open it in a new workspace. Also added code to verify that there actually is an output file available before adding the download button.
Refs #4884
Changeset: 9cfb785bc423a47fdb386c25391b51d2b40695ef
comment:57 Changed 8 years ago by Ross Miller
Only show jobs submitted from MantidPlot in the job status dialog
Uses the 'variables' entry in the submitted JSON to set an entry called "SUBMITTING_APP" when the job is submitted. The jobStatusAll() function checks for this entry and only includes jobs that have it set in its result list.
Refs #4884
Changeset: 99f408268ec0b825f11fce334eeadacd49038979
comment:58 Changed 8 years ago by Ross Miller
Disable previous change to job status dialog
Commit 4af5da7d6f3b6da5b94c3d0a3f0831ee03c58c21 changed the job status dialog box to only show jobs that were actually submitted from MantidPlot. Unfortunately, there's a bug in Moab Web Services (support case #15864) that renders this useless. Once the folks at Adaptive Computing fix their bug, I'll uncomment one line of code to re-enable this feature.
Refs #4884
Changeset: 9f72ea64f82a4053c8d9244993b7c7d09e157114
comment:59 Changed 8 years ago by Ross Miller
Several changes to job status dialog (and job submission in general)
1) HTTPS is now enabled and enforced. No unencrypted connections are allowed. (Note: We're accepting the certificate from the server as-is with no validation. That's not really correct, but it's the best we can do for now.) 2) Jobs in the job status dialog are now sorted by their ID's 3) Had to disable the use of QTemporaryFile objects for downloading the output files because the Load algorithm runs in a separate thread and the object was cleaning up its file before the algorithm had a chance to run.
Refs #4884
Changeset: 448df4f0fd7f053d33b707edab3a4431b948e549
comment:60 Changed 8 years ago by Ross Miller
Preliminary support for Combo Boxes in the job submission dialog
NOTE: NOT FULLY TESTED YET! Refs #4884
Changeset: b4a95c037ba4bbeed2c81e6cd59b8b3a3e3e3862
comment:61 Changed 8 years ago by Ross Miller
Better error messages if there's a problem parsing the XML config file
Refs #4884
Changeset: 2f7416bcbaba571f919756ed7c1ea8f92727bdbf
comment:62 Changed 8 years ago by Ross Miller
Jub submission GUI now supports checkboxes for boolean values
Refs #4884
Changeset: 97a3fa50851d69e67860136d873abbfba6c58630
comment:63 Changed 8 years ago by Ross Miller
Enable session cookies in MWSRemoteJobManager
Refs #4884
Changeset: 15a9098ecabad7350f12e73a3a02763d4984a770
comment:64 Changed 8 years ago by Ross Miller
Starting to make GUI changes for the new job submission design
Refs #4884
Changeset: 95df1703c0ec2365f000f8cbf861b1a22e7b6da9
comment:65 Changed 8 years ago by Ross Miller
More minor GUI changes plus changing a class name
Refs #4884
Changeset: 21d8ff219245bb7b164d92e28e2d10699026dcc2
comment:66 Changed 8 years ago by Ross Miller
Changed the name of 2 files to match the new class name
Refs #4884
Changeset: f4e10f00bbed1e86d64492f427cd7957ab0a09da
comment:67 Changed 8 years ago by Ross Miller
Make the remote cluster list a singleton object
This is in preparation for the new design for remote job submission. In the new design, individual algorithms will need a way to access the cluster list, so it can't be stored as part of the dock widget.
Refs #4884
Changeset: 557bbe552cdbad6742716e0d3fb622d1b9361409
comment:68 Changed 8 years ago by Ross Miller
Starting to add functions necessary for the new design of the...
...remote job submission code. Specifically, adding functions for starting & stopping transactions.
Refs #4884
Changeset: eee01d60d7630213cc3ae6bcc8df646b9e62f2bf
comment:69 Changed 8 years ago by Ross Miller
Comment & whitespace cleanups only - no code changes
Refs #4884
Changeset: c67b4a2904b11aae4d0e460ad3373e1c5f336162
comment:70 Changed 8 years ago by Ross Miller
Changing the RemoteJobManager infrastructure
Instances of RemoteJobManager are created from XML elements in Facilities.xml (rather than being entered by the user and stored in the user's properties file).
Refs #4884
Changeset: f6b283c06d007156fa40447cdd16251fdb543cfc
comment:71 Changed 8 years ago by Ross Miller
Add Fermi cluster to Facilities.xml
Adding a description of the Fermi cluster to Facilities.xml
Refs #4884
Changeset: 13357b1ad898a1eb056fe9049bd3246a16646cb4
comment:72 Changed 8 years ago by Ross Miller
comment:73 Changed 8 years ago by Ross Miller
First cut at GUI elements for the remote algorithsm work
Cherry-picked from original commit: 9d78735367a204d1058e8963f5c2ddc9ace76a0e
refs #4884
Conflicts:
Code/Mantid/MantidPlot/CMakeLists.txt
Changeset: 930d7005ba9f768e877c4978a7a5889fc076660b
comment:74 Changed 8 years ago by Ross Miller
Further work on the remote algorithms project.
Still not anywhere near complete, but some more functionality has been added: cluster info is save to / restored from the properties file and the code to fetch the specified configuration file from a URL works.
Refs #4884
Changeset: c75c2c193e286f2310ade7a4b2fb7da4b8daa9a6
comment:75 Changed 8 years ago by Ross Miller
Moving files related to remote job submisison to their own directory.
Refs #4884
Changeset: 43bc8606fc2787fe06bb9355347e38e36862ea24
comment:76 Changed 8 years ago by Ross Miller
Moving files related to remote job submisison to their own directory.
Refs #4884
Cherry-picked from original commit: d4663554859ff1d4d45b968e3bc88e72c8403d77
Conflicts:
Code/Mantid/MantidPlot/CMakeLists.txt
Changeset: 3ac6f19462799752fb1a96ac5e197eedca4f17d1
comment:77 Changed 8 years ago by Ross Miller
Now have enough done to actually submit a remote job to Chadwick
refs #4884
Changeset: 11e92f2261f477c63c2ad8bd82b81bff248086b1
comment:78 Changed 8 years ago by Ross Miller
comment:79 Changed 8 years ago by Ross Miller
Small tweaks to the remote job submission code.
Setting the job name (instead of leaving it as the default of STDIN) and picking up the user id from the RemoteJobManager class. Still need to figure out how to handle the password though!! refs #4884
Changeset: 9f28438b2f2826baa05eab6fd8d11d4c31fe0fbf
comment:80 Changed 8 years ago by Ross Miller
Fixed a signed/unsigned comparison warning
refs #4884
Changeset: e10c2ee5e14f7ab483e4dabe17869156cbf4de07
comment:81 Changed 8 years ago by Ross Miller
Added job status display
Added code to display the status of previously submiitted remote jobs. Also added code to ask for a password the first time the user needs to send anything to Moab Web Services. (The password is held in memory after that.)
refs #4884
Cherry-picked from original commit: 419ddcfd443ac976e10a12fb0611e4169d7d54fe
Conflicts:
Code/Mantid/MantidPlot/CMakeLists.txt
Changeset: 9bdad0d3d1fbefbc5de2d276a15ad31806d21f54
comment:82 Changed 8 years ago by Ross Miller
Minor cleanups in preparation for a code review
Declared default constructor for RemoteJob private (and left unimplemented), removed unnecessary #include's from MantidRemoteAlg.h and commented out a currently unused paramter in RemoteAlgorithmDockWidget::xmlParseServerAttributes()
Refs #4884
Changeset: 66dc9004125bab10a8e352d43934fcc51259d9d6
comment:83 Changed 8 years ago by Ross Miller
Changed all instances of "Remote Algorithm" to "Remote Task"
Refs #4884
Changeset: b0398533bc3bbf95a656833d5c83277eb3748330
comment:84 Changed 8 years ago by Ross Miller
Replaced a few more instances of 'algorithm' with 'task'
Refs #4884
Changeset: 0d275e5cf72badd8d1cb3e40dd995a65c91144de
comment:85 Changed 8 years ago by Ross Miller
Filter the job status dialog
Added a slider widget to the job status dialog so that users can filter out jobs older than a specified number of days. NOTE: This is just the GUI bits; the actual filtering code isn't ready yet.
Refs #4884
Changeset: 6bcef6c076f9ff48b1eaedb8f13b001c98a4a64a
comment:86 Changed 8 years ago by Ross Miller
Adding a JSON parser for Moab Web Services output
Refs #4884
Changeset: cf6d781198551cb1f8b14b6c5425058fbb4f02f5
comment:87 Changed 8 years ago by Ross Miller
Update CMakeLists to compile the new JSON parser code
Refs #4884
Changeset: 8a37c6fbc91ae7f5427a6f6aac2a443280c62502
comment:88 Changed 8 years ago by Ross Miller
Change Job Status Dialog Box
Changed the job status dialog box to query the MWS server for all the jobs the user has submitted. Uses the new JSON parser code to parse the (rather complicated) output. Also includes a spinbox to let the user set the maximum age of jobs to be displayed.
Refs #4884
Changeset: 840acc960f738cdc8b1fbb2b63e72d27dc62c30c
comment:89 Changed 8 years ago by Ross Miller
Changes to the remote job submission code
Added code to automatically pick a name for the remote job's output file and pass that as a parameter to the job submission URL. This is needed to allow the user to download the results after the remote job has run.
Refs #4884
Changeset: 085ff8c57b65f5d946daa42eacd87971e9935758
comment:90 Changed 8 years ago by Ross Miller
Added ability to download output files from cluster
'Download' buttons in the Job Status dialog will now actually attempt to download the output file and open it in a new workspace. Also added code to verify that there actually is an output file available before adding the download button.
Refs #4884
Changeset: b2678a7ea93efd265c5c8e1faa6f16b159fc7b99
comment:91 Changed 8 years ago by Ross Miller
Only show jobs submitted from MantidPlot in the job status dialog
Uses the 'variables' entry in the submitted JSON to set an entry called "SUBMITTING_APP" when the job is submitted. The jobStatusAll() function checks for this entry and only includes jobs that have it set in its result list.
Refs #4884
Changeset: d6927a27ed0517cb9bf3c7b1ec5802811ccbdd14
comment:92 Changed 8 years ago by Ross Miller
Disable previous change to job status dialog
Commit 4af5da7d6f3b6da5b94c3d0a3f0831ee03c58c21 changed the job status dialog box to only show jobs that were actually submitted from MantidPlot. Unfortunately, there's a bug in Moab Web Services (support case #15864) that renders this useless. Once the folks at Adaptive Computing fix their bug, I'll uncomment one line of code to re-enable this feature.
Refs #4884
Changeset: ee0ac80c469afed56339a7efc751706fb3d96c0d
comment:93 Changed 8 years ago by Ross Miller
Several changes to job status dialog (and job submission in general)
1) HTTPS is now enabled and enforced. No unencrypted connections are allowed. (Note: We're accepting the certificate from the server as-is with no validation. That's not really correct, but it's the best we can do for now.) 2) Jobs in the job status dialog are now sorted by their ID's 3) Had to disable the use of QTemporaryFile objects for downloading the output files because the Load algorithm runs in a separate thread and the object was cleaning up its file before the algorithm had a chance to run.
Refs #4884
Changeset: 92492c1a94fd593f3fa6ede2488053185d8e3294
comment:94 Changed 8 years ago by Ross Miller
Preliminary support for Combo Boxes in the job submission dialog
NOTE: NOT FULLY TESTED YET! Refs #4884
Changeset: 4e84edf560b9f7c4b0ca0ad1f3863103e5b48d68
comment:95 Changed 8 years ago by Ross Miller
Better error messages if there's a problem parsing the XML config file
Refs #4884
Changeset: c973a993b55e0cae2cabd2a634f069f1dbe8d218
comment:96 Changed 8 years ago by Ross Miller
Jub submission GUI now supports checkboxes for boolean values
Refs #4884
Changeset: c4909066f8608bd10a69c0c483531d1c02028854
comment:97 Changed 8 years ago by Ross Miller
Enable session cookies in MWSRemoteJobManager
Refs #4884
Changeset: dfbeaad22c67f492b5d150177e1d927408d3019d
comment:98 Changed 8 years ago by Ross Miller
Starting to make GUI changes for the new job submission design
Refs #4884
Changeset: 745792a3e06ed5de703d4894cfa2f4b2370d52ec
comment:99 Changed 8 years ago by Ross Miller
More minor GUI changes plus changing a class name
Refs #4884
Changeset: bfdb792ed4cb96a0f73b9de7ffcf347ca006f7b1
comment:100 Changed 8 years ago by Ross Miller
Changed the name of 2 files to match the new class name
Refs #4884
Changeset: 270ac746a886e607dcb51c98bffb293e4e9bfc82
comment:101 Changed 8 years ago by Ross Miller
Make the remote cluster list a singleton object
This is in preparation for the new design for remote job submission. In the new design, individual algorithms will need a way to access the cluster list, so it can't be stored as part of the dock widget.
Refs #4884
Changeset: 5ad2d4f50bf15f76c686a7d260437f609cc7f4a9
comment:102 Changed 8 years ago by Ross Miller
Starting to add functions necessary for the new design of the...
...remote job submission code. Specifically, adding functions for starting & stopping transactions.
Refs #4884
Changeset: 814763c4156f9c0b4f399a3d9f1f3f9134594376
comment:103 Changed 8 years ago by Ross Miller
Comment & whitespace cleanups only - no code changes
Refs #4884
Changeset: ccacbe7e5960dd6624176439b354b029e1b6b28e
comment:104 Changed 8 years ago by Ross Miller
Changing the RemoteJobManager infrastructure
Instances of RemoteJobManager are created from XML elements in Facilities.xml (rather than being entered by the user and stored in the user's properties file).
Refs #4884
Changeset: 84fd907f394ee9b2538e6514e734c54c17a8f3ea
comment:105 Changed 8 years ago by Ross Miller
Add Fermi cluster to Facilities.xml
Adding a description of the Fermi cluster to Facilities.xml
Refs #4884
Changeset: 386481b2cf85ebebc0c52b759dacbc9a3da6f080
comment:106 Changed 8 years ago by Ross Miller
comment:107 Changed 8 years ago by Ross Miller
First cut at GUI elements for the remote algorithsm work
Cherry-picked from original commit: 9d78735367a204d1058e8963f5c2ddc9ace76a0e
refs #4884
Conflicts:
Code/Mantid/MantidPlot/CMakeLists.txt
Changeset: 135b917f070497cc1c9dda77f155091ed9da95dd
comment:108 Changed 8 years ago by Ross Miller
Further work on the remote algorithms project.
Still not anywhere near complete, but some more functionality has been added: cluster info is save to / restored from the properties file and the code to fetch the specified configuration file from a URL works.
Refs #4884
Changeset: a1cea83ff4571479f71e6220aa330962aa23f43e
comment:109 Changed 8 years ago by Ross Miller
Moving files related to remote job submisison to their own directory.
Refs #4884
Changeset: 6852ff377c0496fa692421c78ebc2ffa39cb5d48
comment:110 Changed 8 years ago by Ross Miller
Moving files related to remote job submisison to their own directory.
Refs #4884
Cherry-picked from original commit: d4663554859ff1d4d45b968e3bc88e72c8403d77
Conflicts:
Code/Mantid/MantidPlot/CMakeLists.txt
Changeset: 9b2c8e25d20647277c702e67746d421237b30761
comment:111 Changed 8 years ago by Ross Miller
Now have enough done to actually submit a remote job to Chadwick
refs #4884
Changeset: eba0bdf30dee51f09b859d9f4bc8adfb6918ea11
comment:112 Changed 8 years ago by Ross Miller
comment:113 Changed 8 years ago by Ross Miller
Small tweaks to the remote job submission code.
Setting the job name (instead of leaving it as the default of STDIN) and picking up the user id from the RemoteJobManager class. Still need to figure out how to handle the password though!! refs #4884
Changeset: cfffff87fcf05c822cc903aeb1f8fddae8ee0aa9
comment:114 Changed 8 years ago by Ross Miller
Fixed a signed/unsigned comparison warning
refs #4884
Changeset: 3d3247a24fbfd4f1e4261309afcd7860f1201fb3
comment:115 Changed 8 years ago by Ross Miller
Added job status display
Added code to display the status of previously submiitted remote jobs. Also added code to ask for a password the first time the user needs to send anything to Moab Web Services. (The password is held in memory after that.)
refs #4884
Cherry-picked from original commit: 419ddcfd443ac976e10a12fb0611e4169d7d54fe
Conflicts:
Code/Mantid/MantidPlot/CMakeLists.txt
Changeset: 05c762c25e462ad9bbac0d69cbebba1b71a27e76
comment:116 Changed 8 years ago by Ross Miller
Minor cleanups in preparation for a code review
Declared default constructor for RemoteJob private (and left unimplemented), removed unnecessary #include's from MantidRemoteAlg.h and commented out a currently unused paramter in RemoteAlgorithmDockWidget::xmlParseServerAttributes()
Refs #4884
Changeset: 1d5935066c6c9687ff2f1425d0c1fcc509dcbcfc
comment:117 Changed 8 years ago by Ross Miller
Changed all instances of "Remote Algorithm" to "Remote Task"
Refs #4884
Changeset: 635de30b949e01f96542e4037740f23f2c1c644d
comment:118 Changed 8 years ago by Ross Miller
Replaced a few more instances of 'algorithm' with 'task'
Refs #4884
Changeset: 60f0aeb2292f28384141d293f646a002ab3aa3f7
comment:119 Changed 8 years ago by Ross Miller
Filter the job status dialog
Added a slider widget to the job status dialog so that users can filter out jobs older than a specified number of days. NOTE: This is just the GUI bits; the actual filtering code isn't ready yet.
Refs #4884
Changeset: e969b48f171bbda055a4003d7114801faffe297e
comment:120 Changed 8 years ago by Ross Miller
Adding a JSON parser for Moab Web Services output
Refs #4884
Changeset: 745ce2bd79e9eeccaa3685a7f3968e7dcb6d651e
comment:121 Changed 8 years ago by Ross Miller
Update CMakeLists to compile the new JSON parser code
Refs #4884
Changeset: a8e3146699c6b529147c5383514e65c399cc3bd6
comment:122 Changed 8 years ago by Ross Miller
Change Job Status Dialog Box
Changed the job status dialog box to query the MWS server for all the jobs the user has submitted. Uses the new JSON parser code to parse the (rather complicated) output. Also includes a spinbox to let the user set the maximum age of jobs to be displayed.
Refs #4884
Changeset: 94ebca1ca9db02e2dac3292aa6171ec517f17f8a
comment:123 Changed 8 years ago by Ross Miller
Changes to the remote job submission code
Added code to automatically pick a name for the remote job's output file and pass that as a parameter to the job submission URL. This is needed to allow the user to download the results after the remote job has run.
Refs #4884
Changeset: 8aa7c6c8b1959a007cf5092d13b3302afc2fc64f
comment:124 Changed 8 years ago by Ross Miller
Added ability to download output files from cluster
'Download' buttons in the Job Status dialog will now actually attempt to download the output file and open it in a new workspace. Also added code to verify that there actually is an output file available before adding the download button.
Refs #4884
Changeset: 8af7f7138026e07d60dcf57789440921cd863b58
comment:125 Changed 8 years ago by Ross Miller
Only show jobs submitted from MantidPlot in the job status dialog
Uses the 'variables' entry in the submitted JSON to set an entry called "SUBMITTING_APP" when the job is submitted. The jobStatusAll() function checks for this entry and only includes jobs that have it set in its result list.
Refs #4884
Changeset: c4cf4aab1072427fcf9ad637a482804952d95c07
comment:126 Changed 8 years ago by Ross Miller
Disable previous change to job status dialog
Commit 4af5da7d6f3b6da5b94c3d0a3f0831ee03c58c21 changed the job status dialog box to only show jobs that were actually submitted from MantidPlot. Unfortunately, there's a bug in Moab Web Services (support case #15864) that renders this useless. Once the folks at Adaptive Computing fix their bug, I'll uncomment one line of code to re-enable this feature.
Refs #4884
Changeset: 9b6cfc37c0a41539f77698de1389b5e6dff94fd4
comment:127 Changed 8 years ago by Ross Miller
Several changes to job status dialog (and job submission in general)
1) HTTPS is now enabled and enforced. No unencrypted connections are allowed. (Note: We're accepting the certificate from the server as-is with no validation. That's not really correct, but it's the best we can do for now.) 2) Jobs in the job status dialog are now sorted by their ID's 3) Had to disable the use of QTemporaryFile objects for downloading the output files because the Load algorithm runs in a separate thread and the object was cleaning up its file before the algorithm had a chance to run.
Refs #4884
Changeset: 466c4eef9014de8478615e516adace4efbff2c4b
comment:128 Changed 8 years ago by Ross Miller
Preliminary support for Combo Boxes in the job submission dialog
NOTE: NOT FULLY TESTED YET! Refs #4884
Changeset: 71e5c8a7ddaa8b907cd6f0c67a8a21c8da05e3df
comment:129 Changed 8 years ago by Ross Miller
Better error messages if there's a problem parsing the XML config file
Refs #4884
Changeset: fa1fad2863642f84ed17023236359f1a929f8565
comment:130 Changed 8 years ago by Ross Miller
Jub submission GUI now supports checkboxes for boolean values
Refs #4884
Changeset: bfcc456a03315baaae6abedaa2432212b9802a6b
comment:131 Changed 8 years ago by Ross Miller
Enable session cookies in MWSRemoteJobManager
Refs #4884
Changeset: 21a8d648167b8245ecdc73f27c3368d4dd3d0f3a
comment:132 Changed 8 years ago by Ross Miller
Starting to make GUI changes for the new job submission design
Refs #4884
Changeset: e74d96e93f755c53b47d0f9bf8cb31d487ed58f3
comment:133 Changed 8 years ago by Ross Miller
More minor GUI changes plus changing a class name
Refs #4884
Changeset: 3f6c36513cdc541f28b503276221032737941b17
comment:134 Changed 8 years ago by Ross Miller
Changed the name of 2 files to match the new class name
Refs #4884
Changeset: 52467d3b3287d1d419ba8aa7e5ec1d272a986c6f
comment:135 Changed 8 years ago by Ross Miller
Make the remote cluster list a singleton object
This is in preparation for the new design for remote job submission. In the new design, individual algorithms will need a way to access the cluster list, so it can't be stored as part of the dock widget.
Refs #4884
Changeset: d1e0afedfc492a73408980c4b9742abfb37903bf
comment:136 Changed 8 years ago by Ross Miller
Starting to add functions necessary for the new design of the...
...remote job submission code. Specifically, adding functions for starting & stopping transactions.
Refs #4884
Changeset: 2564cca89039018f3cfddeb02fa2ce00edcb630e
comment:137 Changed 8 years ago by Ross Miller
Comment & whitespace cleanups only - no code changes
Refs #4884
Changeset: bcdca96f966ab68c502f0e84b3f6876e3f268928
comment:138 Changed 8 years ago by Ross Miller
Changing the RemoteJobManager infrastructure
Instances of RemoteJobManager are created from XML elements in Facilities.xml (rather than being entered by the user and stored in the user's properties file).
Refs #4884
Changeset: 84b81913530ef85b21190870cd158ef2df16d597
comment:139 Changed 8 years ago by Ross Miller
Add Fermi cluster to Facilities.xml
Adding a description of the Fermi cluster to Facilities.xml
Refs #4884
Changeset: 519b60b8a75fe7754dc1f4fbfd9918ddfd3feefb
comment:140 Changed 8 years ago by Ross Miller
comment:141 Changed 8 years ago by Ross Miller
First cut at using Algorithms for job submission
Not anywhere near complete, but wanted to get it checked in.
Refs #4884
Changeset: b4d37c31970f1701930971f4be7371fb0b3bf475
comment:142 Changed 8 years ago by Ross Miller
First cut at GUI elements for the remote algorithsm work
Cherry-picked from original commit: 9d78735367a204d1058e8963f5c2ddc9ace76a0e
refs #4884
Conflicts:
Code/Mantid/MantidPlot/CMakeLists.txt
Changeset: ec00ec09792a5f7f85522d850045c707e40ab468
comment:143 Changed 8 years ago by Ross Miller
Further work on the remote algorithms project.
Still not anywhere near complete, but some more functionality has been added: cluster info is save to / restored from the properties file and the code to fetch the specified configuration file from a URL works.
Refs #4884
Changeset: 6dfe0b0b829a0437df381a904ed0727a9c3bd873
comment:144 Changed 8 years ago by Ross Miller
Moving files related to remote job submisison to their own directory.
Refs #4884
Changeset: 82db1d4d3b4717e6130f0969684992de1cc33417
comment:145 Changed 8 years ago by Ross Miller
Moving files related to remote job submisison to their own directory.
Refs #4884
Cherry-picked from original commit: d4663554859ff1d4d45b968e3bc88e72c8403d77
Conflicts:
Code/Mantid/MantidPlot/CMakeLists.txt
Changeset: d67ef5a631acf5ad1e465e301562ee1659811bcc
comment:146 Changed 8 years ago by Ross Miller
Now have enough done to actually submit a remote job to Chadwick
refs #4884
Changeset: eb67cdaf1219f972c8a15423b43d4e166132cfab
comment:147 Changed 8 years ago by Ross Miller
comment:148 Changed 8 years ago by Ross Miller
Small tweaks to the remote job submission code.
Setting the job name (instead of leaving it as the default of STDIN) and picking up the user id from the RemoteJobManager class. Still need to figure out how to handle the password though!! refs #4884
Changeset: bd85ec7c3628c37968480a7c958226bf8391d5b4
comment:149 Changed 8 years ago by Ross Miller
Fixed a signed/unsigned comparison warning
refs #4884
Changeset: 108c39bfac83f9d0e4bea098087c33f793c2b2e7
comment:150 Changed 8 years ago by Ross Miller
Added job status display
Added code to display the status of previously submiitted remote jobs. Also added code to ask for a password the first time the user needs to send anything to Moab Web Services. (The password is held in memory after that.)
refs #4884
Cherry-picked from original commit: 419ddcfd443ac976e10a12fb0611e4169d7d54fe
Conflicts:
Code/Mantid/MantidPlot/CMakeLists.txt
Changeset: cf4bd5b8ab2e6da6c5616faa06a16c683f8c6ce8
comment:151 Changed 8 years ago by Ross Miller
Minor cleanups in preparation for a code review
Declared default constructor for RemoteJob private (and left unimplemented), removed unnecessary #include's from MantidRemoteAlg.h and commented out a currently unused paramter in RemoteAlgorithmDockWidget::xmlParseServerAttributes()
Refs #4884
Changeset: a3870cd899ea237ec097180fc6ef75b640626e87
comment:152 Changed 8 years ago by Ross Miller
Changed all instances of "Remote Algorithm" to "Remote Task"
Refs #4884
Changeset: 88ce6ce3b325ada226a9385e437ba107c26e2791
comment:153 Changed 8 years ago by Ross Miller
Replaced a few more instances of 'algorithm' with 'task'
Refs #4884
Changeset: 72868707b829ca359799677b5ad550da8cc792cb
comment:154 Changed 8 years ago by Ross Miller
Filter the job status dialog
Added a slider widget to the job status dialog so that users can filter out jobs older than a specified number of days. NOTE: This is just the GUI bits; the actual filtering code isn't ready yet.
Refs #4884
Changeset: 6254c92f80440716bd232891d525b682b33c0c3a
comment:155 Changed 8 years ago by Ross Miller
Adding a JSON parser for Moab Web Services output
Refs #4884
Changeset: 218272c004b88677eda89a01b91489ba69656da6
comment:156 Changed 8 years ago by Ross Miller
Update CMakeLists to compile the new JSON parser code
Refs #4884
Changeset: fe13347d54d00fc3c382d9dd74cdc1f4ee6db339
comment:157 Changed 8 years ago by Ross Miller
Change Job Status Dialog Box
Changed the job status dialog box to query the MWS server for all the jobs the user has submitted. Uses the new JSON parser code to parse the (rather complicated) output. Also includes a spinbox to let the user set the maximum age of jobs to be displayed.
Refs #4884
Changeset: e7e320e051539f9bc9ced4cee35dd2ac6cb3cc76
comment:158 Changed 8 years ago by Ross Miller
Changes to the remote job submission code
Added code to automatically pick a name for the remote job's output file and pass that as a parameter to the job submission URL. This is needed to allow the user to download the results after the remote job has run.
Refs #4884
Changeset: 9ca71aced6df572af26a31fe89d10c45703e537f
comment:159 Changed 8 years ago by Ross Miller
Added ability to download output files from cluster
'Download' buttons in the Job Status dialog will now actually attempt to download the output file and open it in a new workspace. Also added code to verify that there actually is an output file available before adding the download button.
Refs #4884
Changeset: 7fffa8e241707a11225f18eab0e4c38fdbfc9564
comment:160 Changed 8 years ago by Ross Miller
Only show jobs submitted from MantidPlot in the job status dialog
Uses the 'variables' entry in the submitted JSON to set an entry called "SUBMITTING_APP" when the job is submitted. The jobStatusAll() function checks for this entry and only includes jobs that have it set in its result list.
Refs #4884
Changeset: 411d74401fd2d2f25b162e9810ad12b542cc6c14
comment:161 Changed 8 years ago by Ross Miller
Disable previous change to job status dialog
Commit 4af5da7d6f3b6da5b94c3d0a3f0831ee03c58c21 changed the job status dialog box to only show jobs that were actually submitted from MantidPlot. Unfortunately, there's a bug in Moab Web Services (support case #15864) that renders this useless. Once the folks at Adaptive Computing fix their bug, I'll uncomment one line of code to re-enable this feature.
Refs #4884
Changeset: 22ac676a9e8e287ee934170d8595bf2d96bb6374
comment:162 Changed 8 years ago by Ross Miller
Several changes to job status dialog (and job submission in general)
1) HTTPS is now enabled and enforced. No unencrypted connections are allowed. (Note: We're accepting the certificate from the server as-is with no validation. That's not really correct, but it's the best we can do for now.) 2) Jobs in the job status dialog are now sorted by their ID's 3) Had to disable the use of QTemporaryFile objects for downloading the output files because the Load algorithm runs in a separate thread and the object was cleaning up its file before the algorithm had a chance to run.
Refs #4884
Changeset: ac7dc5dcd4d21e83c8e2b11961799be7e4e002dd
comment:163 Changed 8 years ago by Ross Miller
Preliminary support for Combo Boxes in the job submission dialog
NOTE: NOT FULLY TESTED YET! Refs #4884
Changeset: 490d49be25cd09c828702c0ebcab89b2f48d37b2
comment:164 Changed 8 years ago by Ross Miller
Better error messages if there's a problem parsing the XML config file
Refs #4884
Changeset: 2ed5e81e5c8a7374a62b03efe2b1424a9875be59
comment:165 Changed 8 years ago by Ross Miller
Jub submission GUI now supports checkboxes for boolean values
Refs #4884
Changeset: 3c7a62280188f8e11c480b10f8c623bf9bb2eec4
comment:166 Changed 8 years ago by Ross Miller
Enable session cookies in MWSRemoteJobManager
Refs #4884
Changeset: ee44832ec5c4531754a5006508a0c877521acef1
comment:167 Changed 8 years ago by Ross Miller
Starting to make GUI changes for the new job submission design
Refs #4884
Changeset: 32d654c9c639a86ae7af4ec88b4506448fb05b57
comment:168 Changed 8 years ago by Ross Miller
More minor GUI changes plus changing a class name
Refs #4884
Changeset: 4144ba148206e904e2fff14171e591a16fd81cb7
comment:169 Changed 8 years ago by Ross Miller
Changed the name of 2 files to match the new class name
Refs #4884
Changeset: 889227e70e516f301963dad33d2b0e805676a868
comment:170 Changed 8 years ago by Ross Miller
Make the remote cluster list a singleton object
This is in preparation for the new design for remote job submission. In the new design, individual algorithms will need a way to access the cluster list, so it can't be stored as part of the dock widget.
Refs #4884
Changeset: fd221f5476f8bac6c966080d8897227b7ea8e9c4
comment:171 Changed 8 years ago by Ross Miller
Starting to add functions necessary for the new design of the...
...remote job submission code. Specifically, adding functions for starting & stopping transactions.
Refs #4884
Changeset: fee8ff5141295e878bff6cfb8a5e2f59b483fe06
comment:172 Changed 8 years ago by Ross Miller
Comment & whitespace cleanups only - no code changes
Refs #4884
Changeset: 03072663564a41c6d81b6c8c8edd7164f191fbe6
comment:173 Changed 8 years ago by Ross Miller
Changing the RemoteJobManager infrastructure
Instances of RemoteJobManager are created from XML elements in Facilities.xml (rather than being entered by the user and stored in the user's properties file).
Refs #4884
Changeset: 9779134e86d57048c4d97a78b5e5ca257238afdf
comment:174 Changed 8 years ago by Ross Miller
Add Fermi cluster to Facilities.xml
Adding a description of the Fermi cluster to Facilities.xml
Refs #4884
Changeset: bd4135b06d238fbae8fbcb274b6aa13a139cf64e
comment:175 Changed 8 years ago by Ross Miller
comment:176 Changed 8 years ago by Ross Miller
First cut at using Algorithms for job submission
Not anywhere near complete, but wanted to get it checked in.
Refs #4884
Changeset: 7ca75da08732369dc98ca9657e6e8451a93f8c23
comment:177 Changed 8 years ago by Ross Miller
Remove QtMwsRemoteJobManager
Got rid of the QtMwsRemoteJobManager class and made MWSRemoteJobManager the concrete class. Had to add functions to set the username and password to the RemoteJobManager API. (Need to do this so python scripts can use the remote job submission stuff.)
Refs #4884
Changeset: f6d474f12d152093508eccec79e85ff9c4a7b008
comment:178 Changed 8 years ago by Ross Miller
Remove a reference to QtMwsRemoteJobManager.hpp
MantidRemoteCluster.cpp (which itself is going to be deleted soon) still had a reference to QtMwsRemoteJobManager.hpp (which is now gone).
Refs #4884
Changeset: 1d3e6dbe59c049b037db9f6d3b835bf66d0bf944
comment:179 Changed 8 years ago by Ross Miller
First cut at GUI elements for the remote algorithsm work
Cherry-picked from original commit: 9d78735367a204d1058e8963f5c2ddc9ace76a0e
refs #4884
Conflicts:
Code/Mantid/MantidPlot/CMakeLists.txt
Changeset: acd23ed5b85af3b4211b71693a078c56e22af6e1
comment:180 Changed 8 years ago by Ross Miller
Further work on the remote algorithms project.
Still not anywhere near complete, but some more functionality has been added: cluster info is save to / restored from the properties file and the code to fetch the specified configuration file from a URL works.
Refs #4884
Changeset: 1002ef63845964b16f08e83f48c6d1d94f84c138
comment:181 Changed 8 years ago by Ross Miller
Moving files related to remote job submisison to their own directory.
Refs #4884
Changeset: 97c8d46c0733100a395c5785ffa7b70d43efae80
comment:182 Changed 8 years ago by Ross Miller
Moving files related to remote job submisison to their own directory.
Refs #4884
Cherry-picked from original commit: d4663554859ff1d4d45b968e3bc88e72c8403d77
Conflicts:
Code/Mantid/MantidPlot/CMakeLists.txt
Changeset: 2c4b9dcd38f65861d60d823ac60125d6d4becfe5
comment:183 Changed 8 years ago by Ross Miller
Now have enough done to actually submit a remote job to Chadwick
refs #4884
Changeset: 5149f32b930c3c12cd16e1c2a82b8d00cb80bd26
comment:184 Changed 8 years ago by Ross Miller
comment:185 Changed 8 years ago by Ross Miller
Small tweaks to the remote job submission code.
Setting the job name (instead of leaving it as the default of STDIN) and picking up the user id from the RemoteJobManager class. Still need to figure out how to handle the password though!! refs #4884
Changeset: be7836448d79f2ef7941bb8790819424fb755a73
comment:186 Changed 8 years ago by Ross Miller
Fixed a signed/unsigned comparison warning
refs #4884
Changeset: 61d983013b9ee565e4027f9c7efe05e2e4750efe
comment:187 Changed 8 years ago by Ross Miller
Added job status display
Added code to display the status of previously submiitted remote jobs. Also added code to ask for a password the first time the user needs to send anything to Moab Web Services. (The password is held in memory after that.)
refs #4884
Cherry-picked from original commit: 419ddcfd443ac976e10a12fb0611e4169d7d54fe
Conflicts:
Code/Mantid/MantidPlot/CMakeLists.txt
Changeset: 7260ebc6384b6b41fcf1a57f52a1b9dd22520546
comment:188 Changed 8 years ago by Ross Miller
Minor cleanups in preparation for a code review
Declared default constructor for RemoteJob private (and left unimplemented), removed unnecessary #include's from MantidRemoteAlg.h and commented out a currently unused paramter in RemoteAlgorithmDockWidget::xmlParseServerAttributes()
Refs #4884
Changeset: 6bc5e779210a0859112bf058219020b018eaa302
comment:189 Changed 8 years ago by Ross Miller
Changed all instances of "Remote Algorithm" to "Remote Task"
Refs #4884
Changeset: 3ee685c2e1cd170c278683d682c5551555f81c77
comment:190 Changed 8 years ago by Ross Miller
Replaced a few more instances of 'algorithm' with 'task'
Refs #4884
Changeset: 331e5fcbdd027baea57f5ca545be04e6ef1ad7e1
comment:191 Changed 8 years ago by Ross Miller
Filter the job status dialog
Added a slider widget to the job status dialog so that users can filter out jobs older than a specified number of days. NOTE: This is just the GUI bits; the actual filtering code isn't ready yet.
Refs #4884
Changeset: f4bd88afc4ba220db7f89924e116c01c78d84806
comment:192 Changed 8 years ago by Ross Miller
Adding a JSON parser for Moab Web Services output
Refs #4884
Changeset: 7fd8d66a3e55e9de4ef1005a9092974e04c6ecc2
comment:193 Changed 8 years ago by Ross Miller
Update CMakeLists to compile the new JSON parser code
Refs #4884
Changeset: 75026296d0780d4c46863ff90efa6c26d0fdb2b9
comment:194 Changed 8 years ago by Ross Miller
Change Job Status Dialog Box
Changed the job status dialog box to query the MWS server for all the jobs the user has submitted. Uses the new JSON parser code to parse the (rather complicated) output. Also includes a spinbox to let the user set the maximum age of jobs to be displayed.
Refs #4884
Changeset: 5e14b90530852332c8b65e6854149b7e7978e873
comment:195 Changed 8 years ago by Ross Miller
Changes to the remote job submission code
Added code to automatically pick a name for the remote job's output file and pass that as a parameter to the job submission URL. This is needed to allow the user to download the results after the remote job has run.
Refs #4884
Changeset: 0b279b4185f9e0e29427958fd8b4c5be675e9d9f
comment:196 Changed 8 years ago by Ross Miller
Added ability to download output files from cluster
'Download' buttons in the Job Status dialog will now actually attempt to download the output file and open it in a new workspace. Also added code to verify that there actually is an output file available before adding the download button.
Refs #4884
Changeset: 9d9cc258cbaf05e0780f827e5ad8250b358e1c8c
comment:197 Changed 8 years ago by Ross Miller
Only show jobs submitted from MantidPlot in the job status dialog
Uses the 'variables' entry in the submitted JSON to set an entry called "SUBMITTING_APP" when the job is submitted. The jobStatusAll() function checks for this entry and only includes jobs that have it set in its result list.
Refs #4884
Changeset: 411a8ace56ae04e1370c511dc864479c0e2ee6e5
comment:198 Changed 8 years ago by Ross Miller
Disable previous change to job status dialog
Commit 4af5da7d6f3b6da5b94c3d0a3f0831ee03c58c21 changed the job status dialog box to only show jobs that were actually submitted from MantidPlot. Unfortunately, there's a bug in Moab Web Services (support case #15864) that renders this useless. Once the folks at Adaptive Computing fix their bug, I'll uncomment one line of code to re-enable this feature.
Refs #4884
Changeset: dbe9e1178b1b242d1399a0e58a8c85d551302eff
comment:199 Changed 8 years ago by Ross Miller
Several changes to job status dialog (and job submission in general)
1) HTTPS is now enabled and enforced. No unencrypted connections are allowed. (Note: We're accepting the certificate from the server as-is with no validation. That's not really correct, but it's the best we can do for now.) 2) Jobs in the job status dialog are now sorted by their ID's 3) Had to disable the use of QTemporaryFile objects for downloading the output files because the Load algorithm runs in a separate thread and the object was cleaning up its file before the algorithm had a chance to run.
Refs #4884
Changeset: 7eca24da2fa29ed801976a47670282e4b2a2dc1c
comment:200 Changed 8 years ago by Ross Miller
Preliminary support for Combo Boxes in the job submission dialog
NOTE: NOT FULLY TESTED YET! Refs #4884
Changeset: e6fc785b94107486e39438c2fd9dac1ad04f9f6a
comment:201 Changed 8 years ago by Ross Miller
Better error messages if there's a problem parsing the XML config file
Refs #4884
Changeset: da20090faf9431f2f3d5cc06c20a986053afaf17
comment:202 Changed 8 years ago by Ross Miller
Jub submission GUI now supports checkboxes for boolean values
Refs #4884
Changeset: 04d6769d4bb012976a64224ea0ca99a7861a2670
comment:203 Changed 8 years ago by Ross Miller
Enable session cookies in MWSRemoteJobManager
Refs #4884
Changeset: 022abaf80a0588f71fd67d2481bd44be90dcf727
comment:204 Changed 8 years ago by Ross Miller
Starting to make GUI changes for the new job submission design
Refs #4884
Changeset: 84dcbafe3112d65ece99bda1427e229c879a444c
comment:205 Changed 8 years ago by Ross Miller
More minor GUI changes plus changing a class name
Refs #4884
Changeset: be731260287f6d10b004a4574f38c8ca1c76c7b0
comment:206 Changed 8 years ago by Ross Miller
Changed the name of 2 files to match the new class name
Refs #4884
Changeset: 0cbb6239c1bbb7c9cfeea5dc9b2c743da69404ba
comment:207 Changed 8 years ago by Ross Miller
Make the remote cluster list a singleton object
This is in preparation for the new design for remote job submission. In the new design, individual algorithms will need a way to access the cluster list, so it can't be stored as part of the dock widget.
Refs #4884
Changeset: e38537331aac9e57b5bbf7858e5faae17ba79f3f
comment:208 Changed 8 years ago by Ross Miller
Starting to add functions necessary for the new design of the...
...remote job submission code. Specifically, adding functions for starting & stopping transactions.
Refs #4884
Changeset: ba18afd13a67f9310ea5afb117a13297c582ab84
comment:209 Changed 8 years ago by Ross Miller
Comment & whitespace cleanups only - no code changes
Refs #4884
Changeset: 964e3cc8d5348db2f15ceff1c37eaaf256dde775
comment:210 Changed 8 years ago by Ross Miller
Changing the RemoteJobManager infrastructure
Instances of RemoteJobManager are created from XML elements in Facilities.xml (rather than being entered by the user and stored in the user's properties file).
Refs #4884
Changeset: c16e0d22bf81fb7c04d0921b25e9268813436e96
comment:211 Changed 8 years ago by Ross Miller
Add Fermi cluster to Facilities.xml
Adding a description of the Fermi cluster to Facilities.xml
Refs #4884
Changeset: d4690ce60d189d9e11742797b6757b2d04f37a85
comment:212 Changed 8 years ago by Ross Miller
comment:213 Changed 8 years ago by Ross Miller
First cut at using Algorithms for job submission
Not anywhere near complete, but wanted to get it checked in.
Refs #4884
Changeset: 76f1112d8df595e4ddbcb60cea4ff763724a16f8
comment:214 Changed 8 years ago by Ross Miller
Remove QtMwsRemoteJobManager
Got rid of the QtMwsRemoteJobManager class and made MWSRemoteJobManager the concrete class. Had to add functions to set the username and password to the RemoteJobManager API. (Need to do this so python scripts can use the remote job submission stuff.)
Refs #4884
Changeset: ffb69af2a2d38692b7c5611f340caf1a12486a5e
comment:215 Changed 8 years ago by Ross Miller
Remove a reference to QtMwsRemoteJobManager.hpp
MantidRemoteCluster.cpp (which itself is going to be deleted soon) still had a reference to QtMwsRemoteJobManager.hpp (which is now gone).
Refs #4884
Changeset: 3caa45a9926fe03b68e923dfc240e36aaaa95b6c
comment:216 Changed 8 years ago by Ross Miller
More work on the Remote Job Submission stuff
1) Added location of mpirun and python executables to Facilities.xml 2) Removed property for executable from submit algorithm (because of #1) 3) Changed the password property to a masked property 4) Added validators to the other properties
Refs #4884
Changeset: 8e571013b7bc8d6ce48e0df584cdff883fd95174
comment:217 Changed 8 years ago by Ross Miller
First cut at GUI elements for the remote algorithsm work
Cherry-picked from original commit: 9d78735367a204d1058e8963f5c2ddc9ace76a0e
refs #4884
Conflicts:
Code/Mantid/MantidPlot/CMakeLists.txt
Changeset: c32b68055c9e3d2741ffb094b8359bc4992abb88
comment:218 Changed 8 years ago by Ross Miller
Further work on the remote algorithms project.
Still not anywhere near complete, but some more functionality has been added: cluster info is save to / restored from the properties file and the code to fetch the specified configuration file from a URL works.
Refs #4884
Changeset: 92f9118a5b82805705a8be1a77c91f2bab9a93e2
comment:219 Changed 8 years ago by Ross Miller
Moving files related to remote job submisison to their own directory.
Refs #4884
Changeset: b4a1f0a91d19af0de9daba7ce9904a3d2bf418a0
comment:220 Changed 8 years ago by Ross Miller
Moving files related to remote job submisison to their own directory.
Refs #4884
Cherry-picked from original commit: d4663554859ff1d4d45b968e3bc88e72c8403d77
Conflicts:
Code/Mantid/MantidPlot/CMakeLists.txt
Changeset: c898d2cb236c7b1b3213f14ccc668c97f069f2b9
comment:221 Changed 8 years ago by Ross Miller
Now have enough done to actually submit a remote job to Chadwick
refs #4884
Changeset: a57827a817862c6dc7208e96cb828e029e8f9d92
comment:222 Changed 8 years ago by Ross Miller
comment:223 Changed 8 years ago by Ross Miller
Small tweaks to the remote job submission code.
Setting the job name (instead of leaving it as the default of STDIN) and picking up the user id from the RemoteJobManager class. Still need to figure out how to handle the password though!! refs #4884
Changeset: 5e9a192c0b8b91a6c5b786ced581250567876985
comment:224 Changed 8 years ago by Ross Miller
Fixed a signed/unsigned comparison warning
refs #4884
Changeset: 3e77ddba9b2cb75795b67db0033567449e78759b
comment:225 Changed 8 years ago by Ross Miller
Added job status display
Added code to display the status of previously submiitted remote jobs. Also added code to ask for a password the first time the user needs to send anything to Moab Web Services. (The password is held in memory after that.)
refs #4884
Cherry-picked from original commit: 419ddcfd443ac976e10a12fb0611e4169d7d54fe
Conflicts:
Code/Mantid/MantidPlot/CMakeLists.txt
Changeset: 56cf4830d456568d6bbd28f4fe96ce531cf4ef1d
comment:226 Changed 8 years ago by Ross Miller
Minor cleanups in preparation for a code review
Declared default constructor for RemoteJob private (and left unimplemented), removed unnecessary #include's from MantidRemoteAlg.h and commented out a currently unused paramter in RemoteAlgorithmDockWidget::xmlParseServerAttributes()
Refs #4884
Changeset: 29ac75aea9f4361a6bc7f01ac76af015927ca1d5
comment:227 Changed 8 years ago by Ross Miller
Changed all instances of "Remote Algorithm" to "Remote Task"
Refs #4884
Changeset: 8630681cdbda317f09ba057fe3512bee4e034d0b
comment:228 Changed 8 years ago by Ross Miller
Replaced a few more instances of 'algorithm' with 'task'
Refs #4884
Changeset: da2184e8b3e2a55a06c6baf52d435d3773ce9fbd
comment:229 Changed 8 years ago by Ross Miller
Filter the job status dialog
Added a slider widget to the job status dialog so that users can filter out jobs older than a specified number of days. NOTE: This is just the GUI bits; the actual filtering code isn't ready yet.
Refs #4884
Changeset: 636cd1dc41e5cf9826d4b1a4e35010386c7005c5
comment:230 Changed 8 years ago by Ross Miller
Adding a JSON parser for Moab Web Services output
Refs #4884
Changeset: ff5354b92ac4fcb3df08fc9a2ed726d6aba74522
comment:231 Changed 8 years ago by Ross Miller
Update CMakeLists to compile the new JSON parser code
Refs #4884
Changeset: 8550af43325ef4a065e4bd10f447882facb5e2be
comment:232 Changed 8 years ago by Ross Miller
Change Job Status Dialog Box
Changed the job status dialog box to query the MWS server for all the jobs the user has submitted. Uses the new JSON parser code to parse the (rather complicated) output. Also includes a spinbox to let the user set the maximum age of jobs to be displayed.
Refs #4884
Changeset: 400e162e54fd9e9fb7fc9332812a53b3619a7e18
comment:233 Changed 8 years ago by Ross Miller
Changes to the remote job submission code
Added code to automatically pick a name for the remote job's output file and pass that as a parameter to the job submission URL. This is needed to allow the user to download the results after the remote job has run.
Refs #4884
Changeset: b41f4e531d2a402801daa9d7d576c11f90003fb6
comment:234 Changed 8 years ago by Ross Miller
Added ability to download output files from cluster
'Download' buttons in the Job Status dialog will now actually attempt to download the output file and open it in a new workspace. Also added code to verify that there actually is an output file available before adding the download button.
Refs #4884
Changeset: dcf41df12a99c80fe797b4b06821d52cb6b45c06
comment:235 Changed 8 years ago by Ross Miller
Only show jobs submitted from MantidPlot in the job status dialog
Uses the 'variables' entry in the submitted JSON to set an entry called "SUBMITTING_APP" when the job is submitted. The jobStatusAll() function checks for this entry and only includes jobs that have it set in its result list.
Refs #4884
Changeset: 5f95f9f044563c8c0ae5977108639b4711d9b853
comment:236 Changed 8 years ago by Ross Miller
Disable previous change to job status dialog
Commit 4af5da7d6f3b6da5b94c3d0a3f0831ee03c58c21 changed the job status dialog box to only show jobs that were actually submitted from MantidPlot. Unfortunately, there's a bug in Moab Web Services (support case #15864) that renders this useless. Once the folks at Adaptive Computing fix their bug, I'll uncomment one line of code to re-enable this feature.
Refs #4884
Changeset: c7c9593afb45337a5b98e2680f31b007346bf2b2
comment:237 Changed 8 years ago by Ross Miller
Several changes to job status dialog (and job submission in general)
1) HTTPS is now enabled and enforced. No unencrypted connections are allowed. (Note: We're accepting the certificate from the server as-is with no validation. That's not really correct, but it's the best we can do for now.) 2) Jobs in the job status dialog are now sorted by their ID's 3) Had to disable the use of QTemporaryFile objects for downloading the output files because the Load algorithm runs in a separate thread and the object was cleaning up its file before the algorithm had a chance to run.
Refs #4884
Changeset: d5e9d3be96ce8e532a1dc1e3962c067cc092700e
comment:238 Changed 8 years ago by Ross Miller
Preliminary support for Combo Boxes in the job submission dialog
NOTE: NOT FULLY TESTED YET! Refs #4884
Changeset: b80419d13e20311398fac6edda6eb81282d45302
comment:239 Changed 8 years ago by Ross Miller
Better error messages if there's a problem parsing the XML config file
Refs #4884
Changeset: 31d1ad36ef05b23aa093216079e6ecfd3ad11c47
comment:240 Changed 8 years ago by Ross Miller
Jub submission GUI now supports checkboxes for boolean values
Refs #4884
Changeset: a170e05653fae853a81f6cbceff1f703a73d9db4
comment:241 Changed 8 years ago by Ross Miller
Enable session cookies in MWSRemoteJobManager
Refs #4884
Changeset: bc7689d2f63d0a7be900ef31989ddaddcdf4f921
comment:242 Changed 8 years ago by Ross Miller
Starting to make GUI changes for the new job submission design
Refs #4884
Changeset: 08ab1a58a4b969b289a31390971530e637f0dfb2
comment:243 Changed 8 years ago by Ross Miller
More minor GUI changes plus changing a class name
Refs #4884
Changeset: 429b0fd6fcb0270bf60167a7b186a4292f7b90c9
comment:244 Changed 8 years ago by Ross Miller
Changed the name of 2 files to match the new class name
Refs #4884
Changeset: b5547d1f9ed829f1e846e712b8bb284e8bca4a92
comment:245 Changed 8 years ago by Ross Miller
Make the remote cluster list a singleton object
This is in preparation for the new design for remote job submission. In the new design, individual algorithms will need a way to access the cluster list, so it can't be stored as part of the dock widget.
Refs #4884
Changeset: ac8388e8ae366d01a89b761d4b10e196fdb1f4cc
comment:246 Changed 8 years ago by Ross Miller
Starting to add functions necessary for the new design of the...
...remote job submission code. Specifically, adding functions for starting & stopping transactions.
Refs #4884
Changeset: 35e270b7a76240aa4f7e39f03074571f6c024b1b
comment:247 Changed 8 years ago by Ross Miller
Comment & whitespace cleanups only - no code changes
Refs #4884
Changeset: fff4d5223745072b7368ade218ffcc60bd02436d
comment:248 Changed 8 years ago by Ross Miller
Changing the RemoteJobManager infrastructure
Instances of RemoteJobManager are created from XML elements in Facilities.xml (rather than being entered by the user and stored in the user's properties file).
Refs #4884
Changeset: 39f1afb320bb4e4d317b47ee06e44fb18d81eaba
comment:249 Changed 8 years ago by Ross Miller
Add Fermi cluster to Facilities.xml
Adding a description of the Fermi cluster to Facilities.xml
Refs #4884
Changeset: f8e66f035f41bd53d281de43c3b67d9d21d6b3b4
comment:250 Changed 8 years ago by Ross Miller
comment:251 Changed 8 years ago by Ross Miller
First cut at using Algorithms for job submission
Not anywhere near complete, but wanted to get it checked in.
Refs #4884
Changeset: b5205b3f9c1d4269ca68238bd88a38eaad173901
comment:252 Changed 8 years ago by Ross Miller
Remove QtMwsRemoteJobManager
Got rid of the QtMwsRemoteJobManager class and made MWSRemoteJobManager the concrete class. Had to add functions to set the username and password to the RemoteJobManager API. (Need to do this so python scripts can use the remote job submission stuff.)
Refs #4884
Changeset: 34c61e072b3c3ab9360cbf25ca95f0d08a013f37
comment:253 Changed 8 years ago by Ross Miller
Remove a reference to QtMwsRemoteJobManager.hpp
MantidRemoteCluster.cpp (which itself is going to be deleted soon) still had a reference to QtMwsRemoteJobManager.hpp (which is now gone).
Refs #4884
Changeset: 746ca5ab7be635353183fbb9e89802f44624350c
comment:254 Changed 8 years ago by Ross Miller
More work on the Remote Job Submission stuff
1) Added location of mpirun and python executables to Facilities.xml 2) Removed property for executable from submit algorithm (because of #1) 3) Changed the password property to a masked property 4) Added validators to the other properties
Refs #4884
Changeset: 234a3d73a8de717fffc93ddd8f096704c852c129
comment:255 Changed 8 years ago by Ross Miller
Added algorithms for starting & stopping remote transactions
Added algorithms to start and stop remote transactions. Also fixed the exception throwing in the SubmitRemoteJob algorithm.
Refs #4884
Changeset: 9db4c771a2a7031ff919a14b4fa5d7a1071ae335
comment:256 Changed 7 years ago by Ross Miller
First cut at GUI elements for the remote algorithsm work
Cherry-picked from original commit: 9d78735367a204d1058e8963f5c2ddc9ace76a0e
refs #4884
Conflicts:
Code/Mantid/MantidPlot/CMakeLists.txt
Changeset: 06512bc7d1e4311de088e2c6b8129c9ba96e5004
comment:257 Changed 7 years ago by Ross Miller
Further work on the remote algorithms project.
Still not anywhere near complete, but some more functionality has been added: cluster info is save to / restored from the properties file and the code to fetch the specified configuration file from a URL works.
Refs #4884
Changeset: 8c2e125a979e453790d89479089d5d230fb68267
comment:258 Changed 7 years ago by Ross Miller
Moving files related to remote job submisison to their own directory.
Refs #4884
Changeset: 55ce5563168f569e9db7781e28714cc55b7ce6e0
comment:259 Changed 7 years ago by Ross Miller
Moving files related to remote job submisison to their own directory.
Refs #4884
Cherry-picked from original commit: d4663554859ff1d4d45b968e3bc88e72c8403d77
Conflicts:
Code/Mantid/MantidPlot/CMakeLists.txt
Changeset: 234221ef6676e8b0d42a3493682b9f57549546ad
comment:260 Changed 7 years ago by Ross Miller
Now have enough done to actually submit a remote job to Chadwick
refs #4884
Changeset: add18c1ca4a8ed2ca13e9e299c7a93269eac3a7e
comment:261 Changed 7 years ago by Ross Miller
comment:262 Changed 7 years ago by Ross Miller
Small tweaks to the remote job submission code.
Setting the job name (instead of leaving it as the default of STDIN) and picking up the user id from the RemoteJobManager class. Still need to figure out how to handle the password though!! refs #4884
Changeset: 2be11953f85ef3e040c7aec53e42bf6e6154a0f1
comment:263 Changed 7 years ago by Ross Miller
Fixed a signed/unsigned comparison warning
refs #4884
Changeset: 68ee3aea296397b501215ec668f3bf14c479df17
comment:264 Changed 7 years ago by Ross Miller
Added job status display
Added code to display the status of previously submiitted remote jobs. Also added code to ask for a password the first time the user needs to send anything to Moab Web Services. (The password is held in memory after that.)
refs #4884
Cherry-picked from original commit: 419ddcfd443ac976e10a12fb0611e4169d7d54fe
Conflicts:
Code/Mantid/MantidPlot/CMakeLists.txt
Changeset: dae804e9d4b13ba25acec63fa4f8ad2a3b0d8399
comment:265 Changed 7 years ago by Ross Miller
Minor cleanups in preparation for a code review
Declared default constructor for RemoteJob private (and left unimplemented), removed unnecessary #include's from MantidRemoteAlg.h and commented out a currently unused paramter in RemoteAlgorithmDockWidget::xmlParseServerAttributes()
Refs #4884
Changeset: da11bccbda7dd759d97bb4f3550fa92b97ce6a60
comment:266 Changed 7 years ago by Ross Miller
Changed all instances of "Remote Algorithm" to "Remote Task"
Refs #4884
Changeset: ffa99154278c036396feb6f04e5a3bfdb6fccc19
comment:267 Changed 7 years ago by Ross Miller
Replaced a few more instances of 'algorithm' with 'task'
Refs #4884
Changeset: 5ed43c6f75411ea3453645296874cb9f51911b0b
comment:268 Changed 7 years ago by Ross Miller
Filter the job status dialog
Added a slider widget to the job status dialog so that users can filter out jobs older than a specified number of days. NOTE: This is just the GUI bits; the actual filtering code isn't ready yet.
Refs #4884
Changeset: 00bbf214c0a9c1670c1bcacf35a8ee61a565044c
comment:269 Changed 7 years ago by Ross Miller
Adding a JSON parser for Moab Web Services output
Refs #4884
Changeset: 5a2d369b95730a20f9df4ff8705072152b758456
comment:270 Changed 7 years ago by Ross Miller
Update CMakeLists to compile the new JSON parser code
Refs #4884
Changeset: 83a401bcb4cadb792231cbd663a111b9d00ff301
comment:271 Changed 7 years ago by Ross Miller
Change Job Status Dialog Box
Changed the job status dialog box to query the MWS server for all the jobs the user has submitted. Uses the new JSON parser code to parse the (rather complicated) output. Also includes a spinbox to let the user set the maximum age of jobs to be displayed.
Refs #4884
Changeset: c8d690edb7394648f018a6df09d982e6c255e39a
comment:272 Changed 7 years ago by Ross Miller
Changes to the remote job submission code
Added code to automatically pick a name for the remote job's output file and pass that as a parameter to the job submission URL. This is needed to allow the user to download the results after the remote job has run.
Refs #4884
Changeset: cb7d4525570370b41cca1bdf4c2263f4a6771ec1
comment:273 Changed 7 years ago by Ross Miller
Added ability to download output files from cluster
'Download' buttons in the Job Status dialog will now actually attempt to download the output file and open it in a new workspace. Also added code to verify that there actually is an output file available before adding the download button.
Refs #4884
Changeset: b4fe43f6f94034fcd8bee366e627851b1b71f35d
comment:274 Changed 7 years ago by Ross Miller
Only show jobs submitted from MantidPlot in the job status dialog
Uses the 'variables' entry in the submitted JSON to set an entry called "SUBMITTING_APP" when the job is submitted. The jobStatusAll() function checks for this entry and only includes jobs that have it set in its result list.
Refs #4884
Changeset: 17bbb39f71268b1e3dda961258b4a8ac5f1a986f
comment:275 Changed 7 years ago by Ross Miller
Disable previous change to job status dialog
Commit 4af5da7d6f3b6da5b94c3d0a3f0831ee03c58c21 changed the job status dialog box to only show jobs that were actually submitted from MantidPlot. Unfortunately, there's a bug in Moab Web Services (support case #15864) that renders this useless. Once the folks at Adaptive Computing fix their bug, I'll uncomment one line of code to re-enable this feature.
Refs #4884
Changeset: fedc90fe9497e6d2ba25d889e0da322370a26a75
comment:276 Changed 7 years ago by Ross Miller
Several changes to job status dialog (and job submission in general)
1) HTTPS is now enabled and enforced. No unencrypted connections are allowed. (Note: We're accepting the certificate from the server as-is with no validation. That's not really correct, but it's the best we can do for now.) 2) Jobs in the job status dialog are now sorted by their ID's 3) Had to disable the use of QTemporaryFile objects for downloading the output files because the Load algorithm runs in a separate thread and the object was cleaning up its file before the algorithm had a chance to run.
Refs #4884
Changeset: 9d576b104981698d11c30cc5d477b7b5fb614afa
comment:277 Changed 7 years ago by Ross Miller
Preliminary support for Combo Boxes in the job submission dialog
NOTE: NOT FULLY TESTED YET! Refs #4884
Changeset: a497aea763620053259f8d605a8696428e47e640
comment:278 Changed 7 years ago by Ross Miller
Better error messages if there's a problem parsing the XML config file
Refs #4884
Changeset: 6950ad4cabef46a41af06e064b804e45a81074d7
comment:279 Changed 7 years ago by Ross Miller
Jub submission GUI now supports checkboxes for boolean values
Refs #4884
Changeset: d1018e969ab40c5a14e2366e840405668b78e7f4
comment:280 Changed 7 years ago by Ross Miller
Enable session cookies in MWSRemoteJobManager
Refs #4884
Changeset: ce0b1f34732d066f2db46fd6a97464d78be4b2ac
comment:281 Changed 7 years ago by Ross Miller
Starting to make GUI changes for the new job submission design
Refs #4884
Changeset: d3c66c48bdbf6353965c992f5264101c8b65bdff
comment:282 Changed 7 years ago by Ross Miller
More minor GUI changes plus changing a class name
Refs #4884
Changeset: 68085e56e97a6238e9b0d07f55f98e119634d16b
comment:283 Changed 7 years ago by Ross Miller
Changed the name of 2 files to match the new class name
Refs #4884
Changeset: 62dd60509854ba19830a718b28a25b5d65ca8066
comment:284 Changed 7 years ago by Ross Miller
Make the remote cluster list a singleton object
This is in preparation for the new design for remote job submission. In the new design, individual algorithms will need a way to access the cluster list, so it can't be stored as part of the dock widget.
Refs #4884
Changeset: 7baed1d0fe21f60661fdbbae80f68c56483c8ec0
comment:285 Changed 7 years ago by Ross Miller
Starting to add functions necessary for the new design of the...
...remote job submission code. Specifically, adding functions for starting & stopping transactions.
Refs #4884
Changeset: e8d7983852e498e03d3a442d8329bd06dc42d6b7
comment:286 Changed 7 years ago by Ross Miller
Comment & whitespace cleanups only - no code changes
Refs #4884
Changeset: dac682bf3c46fb0d2e137a2bd9b470cd8a39170f
comment:287 Changed 7 years ago by Ross Miller
Changing the RemoteJobManager infrastructure
Instances of RemoteJobManager are created from XML elements in Facilities.xml (rather than being entered by the user and stored in the user's properties file).
Refs #4884
Changeset: 77a040c1bffda739cf120a37f6284621ff86f519
comment:288 Changed 7 years ago by Ross Miller
Add Fermi cluster to Facilities.xml
Adding a description of the Fermi cluster to Facilities.xml
Refs #4884
Changeset: d3e6954396f2d4df146331857f5f3339b7bda04f
comment:289 Changed 7 years ago by Ross Miller
comment:290 Changed 7 years ago by Ross Miller
First cut at using Algorithms for job submission
Not anywhere near complete, but wanted to get it checked in.
Refs #4884
Changeset: f5227140e23df999e5836b69cc1c003cb88d7118
comment:291 Changed 7 years ago by Ross Miller
Remove QtMwsRemoteJobManager
Got rid of the QtMwsRemoteJobManager class and made MWSRemoteJobManager the concrete class. Had to add functions to set the username and password to the RemoteJobManager API. (Need to do this so python scripts can use the remote job submission stuff.)
Refs #4884
Changeset: 56ac8ad4ea649a071e8a4544e717c963cacee0cf
comment:292 Changed 7 years ago by Ross Miller
Remove a reference to QtMwsRemoteJobManager.hpp
MantidRemoteCluster.cpp (which itself is going to be deleted soon) still had a reference to QtMwsRemoteJobManager.hpp (which is now gone).
Refs #4884
Changeset: 3bdd2b579a0b0f86595cfbf8d7acef7676d7dc3d
comment:293 Changed 7 years ago by Ross Miller
More work on the Remote Job Submission stuff
1) Added location of mpirun and python executables to Facilities.xml 2) Removed property for executable from submit algorithm (because of #1) 3) Changed the password property to a masked property 4) Added validators to the other properties
Refs #4884
Changeset: 82ceec34249ff3ca0213b98c0d00cd41025fbbe6
comment:294 Changed 7 years ago by Ross Miller
Added algorithms for starting & stopping remote transactions
Added algorithms to start and stop remote transactions. Also fixed the exception throwing in the SubmitRemoteJob algorithm.
Refs #4884
Changeset: a8d469a10d231441bfe8f7d62731d686634e79f6
comment:295 Changed 7 years ago by Ross Miller
API changes to RemoteJobManager class
Replaced jobOutputRead() and getJobOutput() with more generic functions for directory listing and file download.
Refs #4884
Changeset: dd30d9ca77547b89c3489624f33b910398043ac4
comment:296 Changed 7 years ago by Ross Miller
New algorithms for querying and downloading remote files
Algorithms make use of the changes to RemoteJobManager that were just checked in.
Refs #4884
Changeset: 860f785eb42a8e3b2508fea873388a2780512681
comment:297 Changed 7 years ago by Ross Miller
Better validator on ComputeResource properties
The various algorithms that deal with remote job submission all have a property to select the compute cluster to be used. This property's validator now checks against the list of compute resource names found in Facilities.xml. (Previously, the property was just a free-form string.)
Refs #4884
Changeset: 68761b3a032d091cc176f9ee0e78ff40e2ef8740
comment:298 Changed 7 years ago by Ross Miller
Change error message in download algorithm
Had the wrong error message when throwing an exception.
Refs #4884
Changeset: ab6e19ee90aa86e5f3b153909c3fb4e3ea2833b0
comment:299 Changed 7 years ago by Ross Miller
Change RemoteJobManager API
Add upload function to the RemoteJobManager API.
Also changed the HTTP connection initialization code to allow us to make POST requests and remove the 'using namespace std' line.
Refs #4884
Changeset: 7227aaf7e575d2867cf5d990015c5eb3c6aba663
comment:300 Changed 7 years ago by Ross Miller
comment:301 Changed 7 years ago by Ross Miller
First cut at GUI elements for the remote algorithsm work
Cherry-picked from original commit: 9d78735367a204d1058e8963f5c2ddc9ace76a0e
refs #4884
Conflicts:
Code/Mantid/MantidPlot/CMakeLists.txt
Changeset: ada0de4e6dda82e629f49f80313f689ae77cfe81
comment:302 Changed 7 years ago by Ross Miller
Further work on the remote algorithms project.
Still not anywhere near complete, but some more functionality has been added: cluster info is save to / restored from the properties file and the code to fetch the specified configuration file from a URL works.
Refs #4884
Changeset: 7abd33dd02345dfc0f199db08ebd2f3a9fec044f
comment:303 Changed 7 years ago by Ross Miller
Moving files related to remote job submisison to their own directory.
Refs #4884
Changeset: acd8981031e3874f2b23e83df954873dbf4bc570
comment:304 Changed 7 years ago by Ross Miller
Moving files related to remote job submisison to their own directory.
Refs #4884
Cherry-picked from original commit: d4663554859ff1d4d45b968e3bc88e72c8403d77
Conflicts:
Code/Mantid/MantidPlot/CMakeLists.txt
Changeset: ffe4cd968340a080f57e19535a02d5f9d070977f
comment:305 Changed 7 years ago by Ross Miller
Now have enough done to actually submit a remote job to Chadwick
refs #4884
Changeset: 5afc572121faf22e870538522a148252d4dbff8f
comment:306 Changed 7 years ago by Ross Miller
comment:307 Changed 7 years ago by Ross Miller
Small tweaks to the remote job submission code.
Setting the job name (instead of leaving it as the default of STDIN) and picking up the user id from the RemoteJobManager class. Still need to figure out how to handle the password though!! refs #4884
Changeset: 0b57669fe743fab9b26fc94a01e3e1955e92eb19
comment:308 Changed 7 years ago by Ross Miller
Fixed a signed/unsigned comparison warning
refs #4884
Changeset: f6afd875d4bdf12a2d25d6498a07066e92b48a85
comment:309 Changed 7 years ago by Ross Miller
Added job status display
Added code to display the status of previously submiitted remote jobs. Also added code to ask for a password the first time the user needs to send anything to Moab Web Services. (The password is held in memory after that.)
refs #4884
Cherry-picked from original commit: 419ddcfd443ac976e10a12fb0611e4169d7d54fe
Conflicts:
Code/Mantid/MantidPlot/CMakeLists.txt
Changeset: 7a7f62981a6b6d636ead8b3b13780473bbf84270
comment:310 Changed 7 years ago by Ross Miller
Minor cleanups in preparation for a code review
Declared default constructor for RemoteJob private (and left unimplemented), removed unnecessary #include's from MantidRemoteAlg.h and commented out a currently unused paramter in RemoteAlgorithmDockWidget::xmlParseServerAttributes()
Refs #4884
Changeset: b90cd95b4e03b1bef3c89439a117ead61d347590
comment:311 Changed 7 years ago by Ross Miller
Changed all instances of "Remote Algorithm" to "Remote Task"
Refs #4884
Changeset: a236e94deed57146092889d6644bc4ff1fa8dfc1
comment:312 Changed 7 years ago by Ross Miller
Replaced a few more instances of 'algorithm' with 'task'
Refs #4884
Changeset: 27774438815e584f61847096c8ddb4827b60854e
comment:313 Changed 7 years ago by Ross Miller
Filter the job status dialog
Added a slider widget to the job status dialog so that users can filter out jobs older than a specified number of days. NOTE: This is just the GUI bits; the actual filtering code isn't ready yet.
Refs #4884
Changeset: f84fd7c9da8e06ca80b01284d6c84e50667b3a1d
comment:314 Changed 7 years ago by Ross Miller
Adding a JSON parser for Moab Web Services output
Refs #4884
Changeset: 88b6e25ce596e73de214592462ce83d6218596c3
comment:315 Changed 7 years ago by Ross Miller
Update CMakeLists to compile the new JSON parser code
Refs #4884
Changeset: e5bb89d1bd118c46a95985c8cd45e6496ac205f5
comment:316 Changed 7 years ago by Ross Miller
Change Job Status Dialog Box
Changed the job status dialog box to query the MWS server for all the jobs the user has submitted. Uses the new JSON parser code to parse the (rather complicated) output. Also includes a spinbox to let the user set the maximum age of jobs to be displayed.
Refs #4884
Changeset: 8d463724a58044679f9165ab22fda55c1468a853
comment:317 Changed 7 years ago by Ross Miller
Changes to the remote job submission code
Added code to automatically pick a name for the remote job's output file and pass that as a parameter to the job submission URL. This is needed to allow the user to download the results after the remote job has run.
Refs #4884
Changeset: 78b45eb309d59d3fd18c5256465a708ffff08280
comment:318 Changed 7 years ago by Ross Miller
Added ability to download output files from cluster
'Download' buttons in the Job Status dialog will now actually attempt to download the output file and open it in a new workspace. Also added code to verify that there actually is an output file available before adding the download button.
Refs #4884
Changeset: 2d2701d981831292f5880c222883f9fcc5ea6d15
comment:319 Changed 7 years ago by Ross Miller
Only show jobs submitted from MantidPlot in the job status dialog
Uses the 'variables' entry in the submitted JSON to set an entry called "SUBMITTING_APP" when the job is submitted. The jobStatusAll() function checks for this entry and only includes jobs that have it set in its result list.
Refs #4884
Changeset: afa38f736b721b6319437f79224f118bc121aa61
comment:320 Changed 7 years ago by Ross Miller
Disable previous change to job status dialog
Commit 4af5da7d6f3b6da5b94c3d0a3f0831ee03c58c21 changed the job status dialog box to only show jobs that were actually submitted from MantidPlot. Unfortunately, there's a bug in Moab Web Services (support case #15864) that renders this useless. Once the folks at Adaptive Computing fix their bug, I'll uncomment one line of code to re-enable this feature.
Refs #4884
Changeset: 290edda728eb8628ac31dc2509cee9b5ecc85402
comment:321 Changed 7 years ago by Ross Miller
Several changes to job status dialog (and job submission in general)
1) HTTPS is now enabled and enforced. No unencrypted connections are allowed. (Note: We're accepting the certificate from the server as-is with no validation. That's not really correct, but it's the best we can do for now.) 2) Jobs in the job status dialog are now sorted by their ID's 3) Had to disable the use of QTemporaryFile objects for downloading the output files because the Load algorithm runs in a separate thread and the object was cleaning up its file before the algorithm had a chance to run.
Refs #4884
Changeset: 61eebe0df4dfa902b7feb9ce28d2f62a9199950d
comment:322 Changed 7 years ago by Ross Miller
Preliminary support for Combo Boxes in the job submission dialog
NOTE: NOT FULLY TESTED YET! Refs #4884
Changeset: 3a5347a08d8a056848e0c4b4bc77f819cef04538
comment:323 Changed 7 years ago by Ross Miller
Better error messages if there's a problem parsing the XML config file
Refs #4884
Changeset: 7c62a6c23c2319f60358405d6ce5eea01fd0db41
comment:324 Changed 7 years ago by Ross Miller
Jub submission GUI now supports checkboxes for boolean values
Refs #4884
Changeset: 6b948d3f2dbb078b3ae4c8a6cf1ef0e021e2a66a
comment:325 Changed 7 years ago by Ross Miller
Enable session cookies in MWSRemoteJobManager
Refs #4884
Changeset: 60984d551fb8d0e155a40453a80bda0c7e19f103
comment:326 Changed 7 years ago by Ross Miller
Starting to make GUI changes for the new job submission design
Refs #4884
Changeset: 49bbf93517bb3b5f4756c7db178c3885b393c95e
comment:327 Changed 7 years ago by Ross Miller
More minor GUI changes plus changing a class name
Refs #4884
Changeset: b698eb7af2f09f8b97399065dc451ac69f774d94
comment:328 Changed 7 years ago by Ross Miller
Changed the name of 2 files to match the new class name
Refs #4884
Changeset: 372f4e0f770a9f1dda6521ad2ba6a3c02fa63220
comment:329 Changed 7 years ago by Ross Miller
Make the remote cluster list a singleton object
This is in preparation for the new design for remote job submission. In the new design, individual algorithms will need a way to access the cluster list, so it can't be stored as part of the dock widget.
Refs #4884
Changeset: f39dd36c9c028700564f1b2ff6710cc71ed9eab2
comment:330 Changed 7 years ago by Ross Miller
Starting to add functions necessary for the new design of the...
...remote job submission code. Specifically, adding functions for starting & stopping transactions.
Refs #4884
Changeset: f64c4c5926c0c4dfaa17cb8e0d864fd4b0300135
comment:331 Changed 7 years ago by Ross Miller
Comment & whitespace cleanups only - no code changes
Refs #4884
Changeset: 3cbd7aacbeef2882c4ee610993715624ad9eba6f
comment:332 Changed 7 years ago by Ross Miller
Changing the RemoteJobManager infrastructure
Instances of RemoteJobManager are created from XML elements in Facilities.xml (rather than being entered by the user and stored in the user's properties file).
Refs #4884
Changeset: dcf645ed809e0cccf8cac589b3691c86871b248b
comment:333 Changed 7 years ago by Ross Miller
Add Fermi cluster to Facilities.xml
Adding a description of the Fermi cluster to Facilities.xml
Refs #4884
Changeset: f973f6d3ffd233e61e9b6ca03e52506ed4187965
comment:334 Changed 7 years ago by Ross Miller
comment:335 Changed 7 years ago by Ross Miller
First cut at using Algorithms for job submission
Not anywhere near complete, but wanted to get it checked in.
Refs #4884
Changeset: 4f5b07b1ae2f98c374a1aa572f3cf2c72331e5d7
comment:336 Changed 7 years ago by Ross Miller
Remove QtMwsRemoteJobManager
Got rid of the QtMwsRemoteJobManager class and made MWSRemoteJobManager the concrete class. Had to add functions to set the username and password to the RemoteJobManager API. (Need to do this so python scripts can use the remote job submission stuff.)
Refs #4884
Changeset: 468d4be1302991d4061efdfc3f6491bdb9bef8d5
comment:337 Changed 7 years ago by Ross Miller
Remove a reference to QtMwsRemoteJobManager.hpp
MantidRemoteCluster.cpp (which itself is going to be deleted soon) still had a reference to QtMwsRemoteJobManager.hpp (which is now gone).
Refs #4884
Changeset: f473785e4c5a396171422b3e411754b9be373cf5
comment:338 Changed 7 years ago by Ross Miller
More work on the Remote Job Submission stuff
1) Added location of mpirun and python executables to Facilities.xml 2) Removed property for executable from submit algorithm (because of #1) 3) Changed the password property to a masked property 4) Added validators to the other properties
Refs #4884
Changeset: 02f52aa864c50883f43ce78211cd533d8c41758f
comment:339 Changed 7 years ago by Ross Miller
Added algorithms for starting & stopping remote transactions
Added algorithms to start and stop remote transactions. Also fixed the exception throwing in the SubmitRemoteJob algorithm.
Refs #4884
Changeset: 30842741d0c1b7fe31579fe8dd43bb484d9e0002
comment:340 Changed 7 years ago by Ross Miller
API changes to RemoteJobManager class
Replaced jobOutputRead() and getJobOutput() with more generic functions for directory listing and file download.
Refs #4884
Changeset: 4712fb3caadf90f1bd0554fc6ca5a6b9241369d5
comment:341 Changed 7 years ago by Ross Miller
New algorithms for querying and downloading remote files
Algorithms make use of the changes to RemoteJobManager that were just checked in.
Refs #4884
Changeset: b77cf6f763f8f654f0072224dc88e968f500bc08
comment:342 Changed 7 years ago by Ross Miller
Better validator on ComputeResource properties
The various algorithms that deal with remote job submission all have a property to select the compute cluster to be used. This property's validator now checks against the list of compute resource names found in Facilities.xml. (Previously, the property was just a free-form string.)
Refs #4884
Changeset: 698f1003926db9bfc20fdb024d2ec9f6d5cf6f31
comment:343 Changed 7 years ago by Ross Miller
Change error message in download algorithm
Had the wrong error message when throwing an exception.
Refs #4884
Changeset: 709443fa4398170bbf5da009a5f681ae2224c148
comment:344 Changed 7 years ago by Ross Miller
Change RemoteJobManager API
Add upload function to the RemoteJobManager API.
Also changed the HTTP connection initialization code to allow us to make POST requests and remove the 'using namespace std' line.
Refs #4884
Changeset: 81493ef6c7cc41c2bc0bee2848e2a1621ad75f89
comment:345 Changed 7 years ago by Ross Miller
comment:346 Changed 7 years ago by Ross Miller
Fixing bugs in the code that generates the command line...
...that's run on the remote cluster.
Refs #4884
Changeset: 299c73363ca6dffcd53d04a83bdade922e4983aa
comment:347 Changed 7 years ago by Ross Miller
Clean up SubmitRemoteJob algorithm
Changed the algorithm to simply call the requested script instead of trying to call mpirun directly. It's assumed the script will call mpirun.
Changed the generated JSON to pass some important values (number of nodes and cores per node) to the process via environment variables.
Refs #4884
Changeset: 453c84f228bc948da03d854d238f2deb578aa3c0
comment:348 Changed 7 years ago by Ross Miller
First cut at GUI elements for the remote algorithsm work
Cherry-picked from original commit: 9d78735367a204d1058e8963f5c2ddc9ace76a0e
refs #4884
Conflicts:
Code/Mantid/MantidPlot/CMakeLists.txt
Changeset: f3044d15c7d3c23e67f4f487a500568610ae1d61
comment:349 Changed 7 years ago by Ross Miller
Further work on the remote algorithms project.
Still not anywhere near complete, but some more functionality has been added: cluster info is save to / restored from the properties file and the code to fetch the specified configuration file from a URL works.
Refs #4884
Changeset: 60eaec74420aff2ac49e7595fa8d73583284c31b
comment:350 Changed 7 years ago by Ross Miller
Moving files related to remote job submisison to their own directory.
Refs #4884
Changeset: 889ea43875cefa5a9799b533bf9473eeae099628
comment:351 Changed 7 years ago by Ross Miller
Moving files related to remote job submisison to their own directory.
Refs #4884
Cherry-picked from original commit: d4663554859ff1d4d45b968e3bc88e72c8403d77
Conflicts:
Code/Mantid/MantidPlot/CMakeLists.txt
Changeset: 3c4a9dfa46469347b8d97169adee29d64a2f53d4
comment:352 Changed 7 years ago by Ross Miller
Now have enough done to actually submit a remote job to Chadwick
refs #4884
Changeset: ee5d40e01c84d4d8383746e85887803ad320ac2b
comment:353 Changed 7 years ago by Ross Miller
comment:354 Changed 7 years ago by Ross Miller
Small tweaks to the remote job submission code.
Setting the job name (instead of leaving it as the default of STDIN) and picking up the user id from the RemoteJobManager class. Still need to figure out how to handle the password though!! refs #4884
Changeset: 55a7ad31de988b8ea2ff6fed73f5ce914f029073
comment:355 Changed 7 years ago by Ross Miller
Fixed a signed/unsigned comparison warning
refs #4884
Changeset: 2ac92a48333ef30339a843f5657245e191263b78
comment:356 Changed 7 years ago by Ross Miller
Added job status display
Added code to display the status of previously submiitted remote jobs. Also added code to ask for a password the first time the user needs to send anything to Moab Web Services. (The password is held in memory after that.)
refs #4884
Cherry-picked from original commit: 419ddcfd443ac976e10a12fb0611e4169d7d54fe
Conflicts:
Code/Mantid/MantidPlot/CMakeLists.txt
Changeset: 126dc909b02b3575c627295740bb5407dc478a75
comment:357 Changed 7 years ago by Ross Miller
Minor cleanups in preparation for a code review
Declared default constructor for RemoteJob private (and left unimplemented), removed unnecessary #include's from MantidRemoteAlg.h and commented out a currently unused paramter in RemoteAlgorithmDockWidget::xmlParseServerAttributes()
Refs #4884
Changeset: c105db264888dc38316922f90aaef6b1f05940b6
comment:358 Changed 7 years ago by Ross Miller
Changed all instances of "Remote Algorithm" to "Remote Task"
Refs #4884
Changeset: 3ab681d4b10741baf3e597f85a1f2138484d8d64
comment:359 Changed 7 years ago by Ross Miller
Replaced a few more instances of 'algorithm' with 'task'
Refs #4884
Changeset: 5d02544eb136d8da5b06647953b9e8661815cab2
comment:360 Changed 7 years ago by Ross Miller
Filter the job status dialog
Added a slider widget to the job status dialog so that users can filter out jobs older than a specified number of days. NOTE: This is just the GUI bits; the actual filtering code isn't ready yet.
Refs #4884
Changeset: 048f7b5b973972b33afc7736e2297794f6163a78
comment:361 Changed 7 years ago by Ross Miller
Adding a JSON parser for Moab Web Services output
Refs #4884
Changeset: 55c724c621d679ea29bbc8b4a4f2fbc8017f00ce
comment:362 Changed 7 years ago by Ross Miller
Update CMakeLists to compile the new JSON parser code
Refs #4884
Changeset: b8274c6ac543e764ef36d4768cde7a4b3985dc02
comment:363 Changed 7 years ago by Ross Miller
Change Job Status Dialog Box
Changed the job status dialog box to query the MWS server for all the jobs the user has submitted. Uses the new JSON parser code to parse the (rather complicated) output. Also includes a spinbox to let the user set the maximum age of jobs to be displayed.
Refs #4884
Changeset: 5668ce9eea59d12dcbe887eca0921ad034d007e8
comment:364 Changed 7 years ago by Ross Miller
Changes to the remote job submission code
Added code to automatically pick a name for the remote job's output file and pass that as a parameter to the job submission URL. This is needed to allow the user to download the results after the remote job has run.
Refs #4884
Changeset: a3efb5491bac927f9bdc89d4fc335bde038ecadf
comment:365 Changed 7 years ago by Ross Miller
Added ability to download output files from cluster
'Download' buttons in the Job Status dialog will now actually attempt to download the output file and open it in a new workspace. Also added code to verify that there actually is an output file available before adding the download button.
Refs #4884
Changeset: cce428e28aaf13e1f11c3632d02b4c92e7636090
comment:366 Changed 7 years ago by Ross Miller
Only show jobs submitted from MantidPlot in the job status dialog
Uses the 'variables' entry in the submitted JSON to set an entry called "SUBMITTING_APP" when the job is submitted. The jobStatusAll() function checks for this entry and only includes jobs that have it set in its result list.
Refs #4884
Changeset: 0641a7112a0f5b3e8db720d8cc127f1ec10d5d6d
comment:367 Changed 7 years ago by Ross Miller
Disable previous change to job status dialog
Commit 4af5da7d6f3b6da5b94c3d0a3f0831ee03c58c21 changed the job status dialog box to only show jobs that were actually submitted from MantidPlot. Unfortunately, there's a bug in Moab Web Services (support case #15864) that renders this useless. Once the folks at Adaptive Computing fix their bug, I'll uncomment one line of code to re-enable this feature.
Refs #4884
Changeset: beb827e2d37898bb7cfde904c392e3e0c9783bb3
comment:368 Changed 7 years ago by Ross Miller
Several changes to job status dialog (and job submission in general)
1) HTTPS is now enabled and enforced. No unencrypted connections are allowed. (Note: We're accepting the certificate from the server as-is with no validation. That's not really correct, but it's the best we can do for now.) 2) Jobs in the job status dialog are now sorted by their ID's 3) Had to disable the use of QTemporaryFile objects for downloading the output files because the Load algorithm runs in a separate thread and the object was cleaning up its file before the algorithm had a chance to run.
Refs #4884
Changeset: 861efc74ccb7f14680519200b6bffed32f3b6f6d
comment:369 Changed 7 years ago by Ross Miller
Preliminary support for Combo Boxes in the job submission dialog
NOTE: NOT FULLY TESTED YET! Refs #4884
Changeset: 6c79c23d7a3b039b48e7aac4ab4d16e736b2aad2
comment:370 Changed 7 years ago by Ross Miller
Better error messages if there's a problem parsing the XML config file
Refs #4884
Changeset: 9dbc60a47a31a7bd2f3f5c87d24ba17697f0f59a
comment:371 Changed 7 years ago by Ross Miller
Jub submission GUI now supports checkboxes for boolean values
Refs #4884
Changeset: d0b62e3c8bdc92c99578d26e90a94ae3258ef0e9
comment:372 Changed 7 years ago by Ross Miller
Enable session cookies in MWSRemoteJobManager
Refs #4884
Changeset: f0da9dd933a6c255f37bf5153292a806eb21508b
comment:373 Changed 7 years ago by Ross Miller
Starting to make GUI changes for the new job submission design
Refs #4884
Changeset: 005c641e6f0f4da085ec9f4d3315273537ca1e96
comment:374 Changed 7 years ago by Ross Miller
More minor GUI changes plus changing a class name
Refs #4884
Changeset: aff3963ff71a529d0f4d30109d39bea7cabc838e
comment:375 Changed 7 years ago by Ross Miller
Changed the name of 2 files to match the new class name
Refs #4884
Changeset: 21782bbe0e7a268f3832ff72cd632360a9f31b5a
comment:376 Changed 7 years ago by Ross Miller
Make the remote cluster list a singleton object
This is in preparation for the new design for remote job submission. In the new design, individual algorithms will need a way to access the cluster list, so it can't be stored as part of the dock widget.
Refs #4884
Changeset: 7eebaca5b5843041ade23af7ea6217a3388242cb
comment:377 Changed 7 years ago by Ross Miller
Starting to add functions necessary for the new design of the...
...remote job submission code. Specifically, adding functions for starting & stopping transactions.
Refs #4884
Changeset: 1a0e7f61e1f94443cd690ab4d17313dd2f2e138c
comment:378 Changed 7 years ago by Ross Miller
Comment & whitespace cleanups only - no code changes
Refs #4884
Changeset: 69243ecb365628e617ae69d50b4e161e62885f2e
comment:379 Changed 7 years ago by Ross Miller
Changing the RemoteJobManager infrastructure
Instances of RemoteJobManager are created from XML elements in Facilities.xml (rather than being entered by the user and stored in the user's properties file).
Refs #4884
Changeset: 1025d01c47992a3059fab58169b3d7fff727a78f
comment:380 Changed 7 years ago by Ross Miller
Add Fermi cluster to Facilities.xml
Adding a description of the Fermi cluster to Facilities.xml
Refs #4884
Changeset: 3ebee527fe79ad5f6f66ed2adf8d10bd03d8ed29
comment:381 Changed 7 years ago by Ross Miller
comment:382 Changed 7 years ago by Ross Miller
First cut at using Algorithms for job submission
Not anywhere near complete, but wanted to get it checked in.
Refs #4884
Changeset: 9dbda10f1bb69251b1087678eb851d5ca3688d2a
comment:383 Changed 7 years ago by Ross Miller
Remove QtMwsRemoteJobManager
Got rid of the QtMwsRemoteJobManager class and made MWSRemoteJobManager the concrete class. Had to add functions to set the username and password to the RemoteJobManager API. (Need to do this so python scripts can use the remote job submission stuff.)
Refs #4884
Changeset: 8f72194740d39ca004c42461de3dababfd36e416
comment:384 Changed 7 years ago by Ross Miller
Remove a reference to QtMwsRemoteJobManager.hpp
MantidRemoteCluster.cpp (which itself is going to be deleted soon) still had a reference to QtMwsRemoteJobManager.hpp (which is now gone).
Refs #4884
Changeset: 7d9cbb9b43009dd4d4dd043f5f98372f9d3c105c
comment:385 Changed 7 years ago by Ross Miller
More work on the Remote Job Submission stuff
1) Added location of mpirun and python executables to Facilities.xml 2) Removed property for executable from submit algorithm (because of #1) 3) Changed the password property to a masked property 4) Added validators to the other properties
Refs #4884
Changeset: b475f32288533782c69af2a9440344cdda01b8eb
comment:386 Changed 7 years ago by Ross Miller
Added algorithms for starting & stopping remote transactions
Added algorithms to start and stop remote transactions. Also fixed the exception throwing in the SubmitRemoteJob algorithm.
Refs #4884
Changeset: f33468e919d757332d54297516001d0d7ab20a23
comment:387 Changed 7 years ago by Ross Miller
API changes to RemoteJobManager class
Replaced jobOutputRead() and getJobOutput() with more generic functions for directory listing and file download.
Refs #4884
Changeset: fc556fa1660cb338e5dd0cbbf4b8252815b299f1
comment:388 Changed 7 years ago by Ross Miller
New algorithms for querying and downloading remote files
Algorithms make use of the changes to RemoteJobManager that were just checked in.
Refs #4884
Changeset: 67cb95f540aa1552548ccea0c3b15fd61704952b
comment:389 Changed 7 years ago by Ross Miller
Better validator on ComputeResource properties
The various algorithms that deal with remote job submission all have a property to select the compute cluster to be used. This property's validator now checks against the list of compute resource names found in Facilities.xml. (Previously, the property was just a free-form string.)
Refs #4884
Changeset: 320291b60f13f539b0bd647a55c3544385566fc9
comment:390 Changed 7 years ago by Ross Miller
Change error message in download algorithm
Had the wrong error message when throwing an exception.
Refs #4884
Changeset: 649b3ce6d3c2d465c39abde864e4ff3ba2aa238f
comment:391 Changed 7 years ago by Ross Miller
Change RemoteJobManager API
Add upload function to the RemoteJobManager API.
Also changed the HTTP connection initialization code to allow us to make POST requests and remove the 'using namespace std' line.
Refs #4884
Changeset: 334d93d95a3be58c76360a148968e75427988377
comment:392 Changed 7 years ago by Ross Miller
comment:393 Changed 7 years ago by Ross Miller
Fixing bugs in the code that generates the command line...
...that's run on the remote cluster.
Refs #4884
Changeset: a2c48989f170306caf778683b405e13c79fcd59d
comment:394 Changed 7 years ago by Ross Miller
Clean up SubmitRemoteJob algorithm
Changed the algorithm to simply call the requested script instead of trying to call mpirun directly. It's assumed the script will call mpirun.
Changed the generated JSON to pass some important values (number of nodes and cores per node) to the process via environment variables.
Refs #4884
Changeset: 162a8a701dff16f50b44b654c7e923fcb031ef33
comment:395 Changed 7 years ago by Ross Miller
Removed last of the remote job submission GUI code...
...from the MantidPlot tree. Most of it has been supercedeed by the *Remote* algorithms and the parts that hadn't been were out of date anyway. About the only thing we're likely to need in the future is a dialog box for querying jobs on the remote system and it will probably be easier to create a new one than to try and salvage the old code.
Refs #4884
Changeset: 4eccc8daa19924adc8ba659c2aac9acec6b95dba
comment:396 Changed 7 years ago by Ross Miller
Add CMake flag to enable/disable compiling
Add a flag to CMake to enable/disable building the entire remote job submission package. Flag defaults to 'disable'.
Note: A few things are still compiled even if the flag is disabled: the RemoteJobManagerFactory and a few minor changes to FacilityInfo.
Refs #4884
Changeset: 31ff385c4f8be16594c086272552e1c7593a5ded
comment:397 Changed 7 years ago by Ross Miller
Update algorithm categories
The various algorithms for handling remote job submission had categories left as 'User Algorithm' (from createAlg.py). Change this to 'Remote'.
Refs #4884
Changeset: 7b3154f60a1967b62a0137e87ab79d5c849d6dba
comment:398 Changed 7 years ago by Ross Miller
First cut at GUI elements for the remote algorithsm work
Cherry-picked from original commit: 9d78735367a204d1058e8963f5c2ddc9ace76a0e
refs #4884
Conflicts:
Code/Mantid/MantidPlot/CMakeLists.txt
Changeset: 4ddbdff99390bf766c234e4921fc85baf5a9269a
comment:399 Changed 7 years ago by Ross Miller
Further work on the remote algorithms project.
Still not anywhere near complete, but some more functionality has been added: cluster info is save to / restored from the properties file and the code to fetch the specified configuration file from a URL works.
Refs #4884
Changeset: 3ffcbf2409c122b440639147668073a1705cc2b6
comment:400 Changed 7 years ago by Ross Miller
Moving files related to remote job submisison to their own directory.
Refs #4884
Changeset: 50c37411ae7b968c11454512cad94ca489a5f85c
comment:401 Changed 7 years ago by Ross Miller
Moving files related to remote job submisison to their own directory.
Refs #4884
Cherry-picked from original commit: d4663554859ff1d4d45b968e3bc88e72c8403d77
Conflicts:
Code/Mantid/MantidPlot/CMakeLists.txt
Changeset: c005d09a433a4d010e613198106461ce81da9a6d
comment:402 Changed 7 years ago by Ross Miller
Now have enough done to actually submit a remote job to Chadwick
refs #4884
Changeset: d7dc42f00eec6cd29e90788451d8df689f18c0b8
comment:403 Changed 7 years ago by Ross Miller
comment:404 Changed 7 years ago by Ross Miller
Small tweaks to the remote job submission code.
Setting the job name (instead of leaving it as the default of STDIN) and picking up the user id from the RemoteJobManager class. Still need to figure out how to handle the password though!! refs #4884
Changeset: 5e772b83c6dc1f0b8f8d2473aedf36bba20f570b
comment:405 Changed 7 years ago by Ross Miller
Fixed a signed/unsigned comparison warning
refs #4884
Changeset: 274f0e2b1ba032858907bc2b7f094a6f5098f08c
comment:406 Changed 7 years ago by Ross Miller
Added job status display
Added code to display the status of previously submiitted remote jobs. Also added code to ask for a password the first time the user needs to send anything to Moab Web Services. (The password is held in memory after that.)
refs #4884
Cherry-picked from original commit: 419ddcfd443ac976e10a12fb0611e4169d7d54fe
Conflicts:
Code/Mantid/MantidPlot/CMakeLists.txt
Changeset: 70d1908d1b00c52a74daa5d52ca8e6e13760cae6
comment:407 Changed 7 years ago by Ross Miller
Minor cleanups in preparation for a code review
Declared default constructor for RemoteJob private (and left unimplemented), removed unnecessary #include's from MantidRemoteAlg.h and commented out a currently unused paramter in RemoteAlgorithmDockWidget::xmlParseServerAttributes()
Refs #4884
Changeset: fddd5433261e90d10811a6c2f16ae34e23c6fcd2
comment:408 Changed 7 years ago by Ross Miller
Changed all instances of "Remote Algorithm" to "Remote Task"
Refs #4884
Changeset: b03f9268b6701f38c263856001d35cf7d997344b
comment:409 Changed 7 years ago by Ross Miller
Replaced a few more instances of 'algorithm' with 'task'
Refs #4884
Changeset: 9a20f5de6fb932a645cc547aa1c8c49c6c21bb9a
comment:410 Changed 7 years ago by Ross Miller
Filter the job status dialog
Added a slider widget to the job status dialog so that users can filter out jobs older than a specified number of days. NOTE: This is just the GUI bits; the actual filtering code isn't ready yet.
Refs #4884
Changeset: 7b3b568e67356843936b4c4f0168d6acf61d1c12
comment:411 Changed 7 years ago by Ross Miller
Adding a JSON parser for Moab Web Services output
Refs #4884
Changeset: 3568e56425e0015167fe3f57251595d5cff213b1
comment:412 Changed 7 years ago by Ross Miller
Update CMakeLists to compile the new JSON parser code
Refs #4884
Changeset: 15eaf2c9bdf7b36b806cc2e7bec00b77dc6cd737
comment:413 Changed 7 years ago by Ross Miller
Change Job Status Dialog Box
Changed the job status dialog box to query the MWS server for all the jobs the user has submitted. Uses the new JSON parser code to parse the (rather complicated) output. Also includes a spinbox to let the user set the maximum age of jobs to be displayed.
Refs #4884
Changeset: c5f85e4446308195efb0ad90bca8c5111d604b16
comment:414 Changed 7 years ago by Ross Miller
Changes to the remote job submission code
Added code to automatically pick a name for the remote job's output file and pass that as a parameter to the job submission URL. This is needed to allow the user to download the results after the remote job has run.
Refs #4884
Changeset: 9b03653a135c61eea3fee571dab044baf982573a
comment:415 Changed 7 years ago by Ross Miller
Added ability to download output files from cluster
'Download' buttons in the Job Status dialog will now actually attempt to download the output file and open it in a new workspace. Also added code to verify that there actually is an output file available before adding the download button.
Refs #4884
Changeset: 8d34372a5ae4fa6fdf78742e07d44be0729c9fc8
comment:416 Changed 7 years ago by Ross Miller
Only show jobs submitted from MantidPlot in the job status dialog
Uses the 'variables' entry in the submitted JSON to set an entry called "SUBMITTING_APP" when the job is submitted. The jobStatusAll() function checks for this entry and only includes jobs that have it set in its result list.
Refs #4884
Changeset: 869d42dfe05933141b1a797ede755878b27424d3
comment:417 Changed 7 years ago by Ross Miller
Disable previous change to job status dialog
Commit 4af5da7d6f3b6da5b94c3d0a3f0831ee03c58c21 changed the job status dialog box to only show jobs that were actually submitted from MantidPlot. Unfortunately, there's a bug in Moab Web Services (support case #15864) that renders this useless. Once the folks at Adaptive Computing fix their bug, I'll uncomment one line of code to re-enable this feature.
Refs #4884
Changeset: 5f6e396ebb1545f4b0bf2423aec6d08d7763cbef
comment:418 Changed 7 years ago by Ross Miller
Several changes to job status dialog (and job submission in general)
1) HTTPS is now enabled and enforced. No unencrypted connections are allowed. (Note: We're accepting the certificate from the server as-is with no validation. That's not really correct, but it's the best we can do for now.) 2) Jobs in the job status dialog are now sorted by their ID's 3) Had to disable the use of QTemporaryFile objects for downloading the output files because the Load algorithm runs in a separate thread and the object was cleaning up its file before the algorithm had a chance to run.
Refs #4884
Changeset: 641734c23bfe7d4f8bb3149efce7f1ea7d3458de
comment:419 Changed 7 years ago by Ross Miller
Preliminary support for Combo Boxes in the job submission dialog
NOTE: NOT FULLY TESTED YET! Refs #4884
Changeset: 0711ae53df232b2a9a29d69365059b64a775b5cb
comment:420 Changed 7 years ago by Ross Miller
Better error messages if there's a problem parsing the XML config file
Refs #4884
Changeset: 4d4b8f6819b48565fd7bdd5a363a74670a0204d5
comment:421 Changed 7 years ago by Ross Miller
Jub submission GUI now supports checkboxes for boolean values
Refs #4884
Changeset: 48cbaa6fed6ad890882671931ca0dc8f1f4b70f0
comment:422 Changed 7 years ago by Ross Miller
Enable session cookies in MWSRemoteJobManager
Refs #4884
Changeset: 2495f74a74ec847a2baf96b424e0f0bc14ca71a5
comment:423 Changed 7 years ago by Ross Miller
Starting to make GUI changes for the new job submission design
Refs #4884
Changeset: 9d45e9a346c0136c055997e138a380c47795dce2
comment:424 Changed 7 years ago by Ross Miller
More minor GUI changes plus changing a class name
Refs #4884
Changeset: 14785e131ad1199e7926d09915e5460681cd587d
comment:425 Changed 7 years ago by Ross Miller
Changed the name of 2 files to match the new class name
Refs #4884
Changeset: 40d2b7b41b67b397608e13d7caeb756a337347f7
comment:426 Changed 7 years ago by Ross Miller
Make the remote cluster list a singleton object
This is in preparation for the new design for remote job submission. In the new design, individual algorithms will need a way to access the cluster list, so it can't be stored as part of the dock widget.
Refs #4884
Changeset: 713be803af3127d0e1149e6a29bf4c61035bdda8
comment:427 Changed 7 years ago by Ross Miller
Starting to add functions necessary for the new design of the...
...remote job submission code. Specifically, adding functions for starting & stopping transactions.
Refs #4884
Changeset: ffc4b8b8b64a23455d91fc2a66f010efef516e51
comment:428 Changed 7 years ago by Ross Miller
Comment & whitespace cleanups only - no code changes
Refs #4884
Changeset: 58b426b87c1c08315c11fb359db4ba8a32fcb80b
comment:429 Changed 7 years ago by Ross Miller
Changing the RemoteJobManager infrastructure
Instances of RemoteJobManager are created from XML elements in Facilities.xml (rather than being entered by the user and stored in the user's properties file).
Refs #4884
Changeset: bbc44f2fc8375986c00231d1285f8de834277cfb
comment:430 Changed 7 years ago by Ross Miller
Add Fermi cluster to Facilities.xml
Adding a description of the Fermi cluster to Facilities.xml
Refs #4884
Changeset: c6cd12914726b93ea8a39f4d75620c3a32a1af97
comment:431 Changed 7 years ago by Ross Miller
comment:432 Changed 7 years ago by Ross Miller
First cut at using Algorithms for job submission
Not anywhere near complete, but wanted to get it checked in.
Refs #4884
Changeset: 2c62ba4bf39e6f79331a4a34b945ba41ca0a4836
comment:433 Changed 7 years ago by Ross Miller
Remove QtMwsRemoteJobManager
Got rid of the QtMwsRemoteJobManager class and made MWSRemoteJobManager the concrete class. Had to add functions to set the username and password to the RemoteJobManager API. (Need to do this so python scripts can use the remote job submission stuff.)
Refs #4884
Changeset: f984fc9353a42de31395c00be5a8d74afab61339
comment:434 Changed 7 years ago by Ross Miller
Remove a reference to QtMwsRemoteJobManager.hpp
MantidRemoteCluster.cpp (which itself is going to be deleted soon) still had a reference to QtMwsRemoteJobManager.hpp (which is now gone).
Refs #4884
Changeset: 861598f7e0c1163942df63a01d0205bcda8e7266
comment:435 Changed 7 years ago by Ross Miller
More work on the Remote Job Submission stuff
1) Added location of mpirun and python executables to Facilities.xml 2) Removed property for executable from submit algorithm (because of #1) 3) Changed the password property to a masked property 4) Added validators to the other properties
Refs #4884
Changeset: 9e4e1814ac2bc005e8300e66b07360f0bac6780a
comment:436 Changed 7 years ago by Ross Miller
Added algorithms for starting & stopping remote transactions
Added algorithms to start and stop remote transactions. Also fixed the exception throwing in the SubmitRemoteJob algorithm.
Refs #4884
Changeset: 4a7eeb6fed8a53a247994f17824e03b2213c7449
comment:437 Changed 7 years ago by Ross Miller
API changes to RemoteJobManager class
Replaced jobOutputRead() and getJobOutput() with more generic functions for directory listing and file download.
Refs #4884
Changeset: 744c3d4fa66324ed9c5d872abc8a7a4d0a96166b
comment:438 Changed 7 years ago by Ross Miller
New algorithms for querying and downloading remote files
Algorithms make use of the changes to RemoteJobManager that were just checked in.
Refs #4884
Changeset: fe7907d329c63cdad1d23655b6aebec94dec4e3c
comment:439 Changed 7 years ago by Ross Miller
Better validator on ComputeResource properties
The various algorithms that deal with remote job submission all have a property to select the compute cluster to be used. This property's validator now checks against the list of compute resource names found in Facilities.xml. (Previously, the property was just a free-form string.)
Refs #4884
Changeset: 4fb4a96b276400c8b5b2a56fc88b35f707abeb80
comment:440 Changed 7 years ago by Ross Miller
Change error message in download algorithm
Had the wrong error message when throwing an exception.
Refs #4884
Changeset: 13e0747f101f608c68c3217fe06356916ba2de0f
comment:441 Changed 7 years ago by Ross Miller
Change RemoteJobManager API
Add upload function to the RemoteJobManager API.
Also changed the HTTP connection initialization code to allow us to make POST requests and remove the 'using namespace std' line.
Refs #4884
Changeset: 8e2c570851ca17f854c883a737ca7767ef6bf36c
comment:442 Changed 7 years ago by Ross Miller
comment:443 Changed 7 years ago by Ross Miller
Fixing bugs in the code that generates the command line...
...that's run on the remote cluster.
Refs #4884
Changeset: 30b5413ea470c6bb3d52a35019f4d3c13034c9d6
comment:444 Changed 7 years ago by Ross Miller
Clean up SubmitRemoteJob algorithm
Changed the algorithm to simply call the requested script instead of trying to call mpirun directly. It's assumed the script will call mpirun.
Changed the generated JSON to pass some important values (number of nodes and cores per node) to the process via environment variables.
Refs #4884
Changeset: 531a230b5bb701e740bf82460f5d9a276eb0975b
comment:445 Changed 7 years ago by Ross Miller
Removed last of the remote job submission GUI code...
...from the MantidPlot tree. Most of it has been supercedeed by the *Remote* algorithms and the parts that hadn't been were out of date anyway. About the only thing we're likely to need in the future is a dialog box for querying jobs on the remote system and it will probably be easier to create a new one than to try and salvage the old code.
Refs #4884
Changeset: 72d310a812051b554297df278fb0475c50279dd1
comment:446 Changed 7 years ago by Ross Miller
Add CMake flag to enable/disable compiling
Add a flag to CMake to enable/disable building the entire remote job submission package. Flag defaults to 'disable'.
Note: A few things are still compiled even if the flag is disabled: the RemoteJobManagerFactory and a few minor changes to FacilityInfo.
Refs #4884
Changeset: 2efa9aa506b60b513aab8fff4c877dff33558265
comment:447 Changed 7 years ago by Ross Miller
Update algorithm categories
The various algorithms for handling remote job submission had categories left as 'User Algorithm' (from createAlg.py). Change this to 'Remote'.
Refs #4884
Changeset: 57c98adcf614f0ac944747112af13d03c8706e8d
comment:448 Changed 7 years ago by Ross Miller
First cut at GUI elements for the remote algorithsm work
Cherry-picked from original commit: 9d78735367a204d1058e8963f5c2ddc9ace76a0e
refs #4884
Conflicts:
Code/Mantid/MantidPlot/CMakeLists.txt
Changeset: 4ddbdff99390bf766c234e4921fc85baf5a9269a
comment:449 Changed 7 years ago by Ross Miller
Further work on the remote algorithms project.
Still not anywhere near complete, but some more functionality has been added: cluster info is save to / restored from the properties file and the code to fetch the specified configuration file from a URL works.
Refs #4884
Changeset: 3ffcbf2409c122b440639147668073a1705cc2b6
comment:450 Changed 7 years ago by Ross Miller
Moving files related to remote job submisison to their own directory.
Refs #4884
Changeset: 50c37411ae7b968c11454512cad94ca489a5f85c
comment:451 Changed 7 years ago by Ross Miller
Moving files related to remote job submisison to their own directory.
Refs #4884
Cherry-picked from original commit: d4663554859ff1d4d45b968e3bc88e72c8403d77
Conflicts:
Code/Mantid/MantidPlot/CMakeLists.txt
Changeset: c005d09a433a4d010e613198106461ce81da9a6d
comment:452 Changed 7 years ago by Ross Miller
Now have enough done to actually submit a remote job to Chadwick
refs #4884
Changeset: d7dc42f00eec6cd29e90788451d8df689f18c0b8
comment:453 Changed 7 years ago by Ross Miller
comment:454 Changed 7 years ago by Ross Miller
Small tweaks to the remote job submission code.
Setting the job name (instead of leaving it as the default of STDIN) and picking up the user id from the RemoteJobManager class. Still need to figure out how to handle the password though!! refs #4884
Changeset: 5e772b83c6dc1f0b8f8d2473aedf36bba20f570b
comment:455 Changed 7 years ago by Ross Miller
Fixed a signed/unsigned comparison warning
refs #4884
Changeset: 274f0e2b1ba032858907bc2b7f094a6f5098f08c
comment:456 Changed 7 years ago by Ross Miller
Added job status display
Added code to display the status of previously submiitted remote jobs. Also added code to ask for a password the first time the user needs to send anything to Moab Web Services. (The password is held in memory after that.)
refs #4884
Cherry-picked from original commit: 419ddcfd443ac976e10a12fb0611e4169d7d54fe
Conflicts:
Code/Mantid/MantidPlot/CMakeLists.txt
Changeset: 70d1908d1b00c52a74daa5d52ca8e6e13760cae6
comment:457 Changed 7 years ago by Ross Miller
Minor cleanups in preparation for a code review
Declared default constructor for RemoteJob private (and left unimplemented), removed unnecessary #include's from MantidRemoteAlg.h and commented out a currently unused paramter in RemoteAlgorithmDockWidget::xmlParseServerAttributes()
Refs #4884
Changeset: fddd5433261e90d10811a6c2f16ae34e23c6fcd2
comment:458 Changed 7 years ago by Ross Miller
Changed all instances of "Remote Algorithm" to "Remote Task"
Refs #4884
Changeset: b03f9268b6701f38c263856001d35cf7d997344b
comment:459 Changed 7 years ago by Ross Miller
Replaced a few more instances of 'algorithm' with 'task'
Refs #4884
Changeset: 9a20f5de6fb932a645cc547aa1c8c49c6c21bb9a
comment:460 Changed 7 years ago by Ross Miller
Filter the job status dialog
Added a slider widget to the job status dialog so that users can filter out jobs older than a specified number of days. NOTE: This is just the GUI bits; the actual filtering code isn't ready yet.
Refs #4884
Changeset: 7b3b568e67356843936b4c4f0168d6acf61d1c12
comment:461 Changed 7 years ago by Ross Miller
Adding a JSON parser for Moab Web Services output
Refs #4884
Changeset: 3568e56425e0015167fe3f57251595d5cff213b1
comment:462 Changed 7 years ago by Ross Miller
Update CMakeLists to compile the new JSON parser code
Refs #4884
Changeset: 15eaf2c9bdf7b36b806cc2e7bec00b77dc6cd737
comment:463 Changed 7 years ago by Ross Miller
Change Job Status Dialog Box
Changed the job status dialog box to query the MWS server for all the jobs the user has submitted. Uses the new JSON parser code to parse the (rather complicated) output. Also includes a spinbox to let the user set the maximum age of jobs to be displayed.
Refs #4884
Changeset: c5f85e4446308195efb0ad90bca8c5111d604b16
comment:464 Changed 7 years ago by Ross Miller
Changes to the remote job submission code
Added code to automatically pick a name for the remote job's output file and pass that as a parameter to the job submission URL. This is needed to allow the user to download the results after the remote job has run.
Refs #4884
Changeset: 9b03653a135c61eea3fee571dab044baf982573a
comment:465 Changed 7 years ago by Ross Miller
Added ability to download output files from cluster
'Download' buttons in the Job Status dialog will now actually attempt to download the output file and open it in a new workspace. Also added code to verify that there actually is an output file available before adding the download button.
Refs #4884
Changeset: 8d34372a5ae4fa6fdf78742e07d44be0729c9fc8
comment:466 Changed 7 years ago by Ross Miller
Only show jobs submitted from MantidPlot in the job status dialog
Uses the 'variables' entry in the submitted JSON to set an entry called "SUBMITTING_APP" when the job is submitted. The jobStatusAll() function checks for this entry and only includes jobs that have it set in its result list.
Refs #4884
Changeset: 869d42dfe05933141b1a797ede755878b27424d3
comment:467 Changed 7 years ago by Ross Miller
Disable previous change to job status dialog
Commit 4af5da7d6f3b6da5b94c3d0a3f0831ee03c58c21 changed the job status dialog box to only show jobs that were actually submitted from MantidPlot. Unfortunately, there's a bug in Moab Web Services (support case #15864) that renders this useless. Once the folks at Adaptive Computing fix their bug, I'll uncomment one line of code to re-enable this feature.
Refs #4884
Changeset: 5f6e396ebb1545f4b0bf2423aec6d08d7763cbef
comment:468 Changed 7 years ago by Ross Miller
Several changes to job status dialog (and job submission in general)
1) HTTPS is now enabled and enforced. No unencrypted connections are allowed. (Note: We're accepting the certificate from the server as-is with no validation. That's not really correct, but it's the best we can do for now.) 2) Jobs in the job status dialog are now sorted by their ID's 3) Had to disable the use of QTemporaryFile objects for downloading the output files because the Load algorithm runs in a separate thread and the object was cleaning up its file before the algorithm had a chance to run.
Refs #4884
Changeset: 641734c23bfe7d4f8bb3149efce7f1ea7d3458de
comment:469 Changed 7 years ago by Ross Miller
Preliminary support for Combo Boxes in the job submission dialog
NOTE: NOT FULLY TESTED YET! Refs #4884
Changeset: 0711ae53df232b2a9a29d69365059b64a775b5cb
comment:470 Changed 7 years ago by Ross Miller
Better error messages if there's a problem parsing the XML config file
Refs #4884
Changeset: 4d4b8f6819b48565fd7bdd5a363a74670a0204d5
comment:471 Changed 7 years ago by Ross Miller
Jub submission GUI now supports checkboxes for boolean values
Refs #4884
Changeset: 48cbaa6fed6ad890882671931ca0dc8f1f4b70f0
comment:472 Changed 7 years ago by Ross Miller
Enable session cookies in MWSRemoteJobManager
Refs #4884
Changeset: 2495f74a74ec847a2baf96b424e0f0bc14ca71a5
comment:473 Changed 7 years ago by Ross Miller
Starting to make GUI changes for the new job submission design
Refs #4884
Changeset: 9d45e9a346c0136c055997e138a380c47795dce2
comment:474 Changed 7 years ago by Ross Miller
More minor GUI changes plus changing a class name
Refs #4884
Changeset: 14785e131ad1199e7926d09915e5460681cd587d
comment:475 Changed 7 years ago by Ross Miller
Changed the name of 2 files to match the new class name
Refs #4884
Changeset: 40d2b7b41b67b397608e13d7caeb756a337347f7
comment:476 Changed 7 years ago by Ross Miller
Make the remote cluster list a singleton object
This is in preparation for the new design for remote job submission. In the new design, individual algorithms will need a way to access the cluster list, so it can't be stored as part of the dock widget.
Refs #4884
Changeset: 713be803af3127d0e1149e6a29bf4c61035bdda8
comment:477 Changed 7 years ago by Ross Miller
Starting to add functions necessary for the new design of the...
...remote job submission code. Specifically, adding functions for starting & stopping transactions.
Refs #4884
Changeset: ffc4b8b8b64a23455d91fc2a66f010efef516e51
comment:478 Changed 7 years ago by Ross Miller
Comment & whitespace cleanups only - no code changes
Refs #4884
Changeset: 58b426b87c1c08315c11fb359db4ba8a32fcb80b
comment:479 Changed 7 years ago by Ross Miller
Changing the RemoteJobManager infrastructure
Instances of RemoteJobManager are created from XML elements in Facilities.xml (rather than being entered by the user and stored in the user's properties file).
Refs #4884
Changeset: bbc44f2fc8375986c00231d1285f8de834277cfb
comment:480 Changed 7 years ago by Ross Miller
Add Fermi cluster to Facilities.xml
Adding a description of the Fermi cluster to Facilities.xml
Refs #4884
Changeset: c6cd12914726b93ea8a39f4d75620c3a32a1af97
comment:481 Changed 7 years ago by Ross Miller
comment:482 Changed 7 years ago by Ross Miller
First cut at using Algorithms for job submission
Not anywhere near complete, but wanted to get it checked in.
Refs #4884
Changeset: 2c62ba4bf39e6f79331a4a34b945ba41ca0a4836
comment:483 Changed 7 years ago by Ross Miller
Remove QtMwsRemoteJobManager
Got rid of the QtMwsRemoteJobManager class and made MWSRemoteJobManager the concrete class. Had to add functions to set the username and password to the RemoteJobManager API. (Need to do this so python scripts can use the remote job submission stuff.)
Refs #4884
Changeset: f984fc9353a42de31395c00be5a8d74afab61339
comment:484 Changed 7 years ago by Ross Miller
Remove a reference to QtMwsRemoteJobManager.hpp
MantidRemoteCluster.cpp (which itself is going to be deleted soon) still had a reference to QtMwsRemoteJobManager.hpp (which is now gone).
Refs #4884
Changeset: 861598f7e0c1163942df63a01d0205bcda8e7266
comment:485 Changed 7 years ago by Ross Miller
More work on the Remote Job Submission stuff
1) Added location of mpirun and python executables to Facilities.xml 2) Removed property for executable from submit algorithm (because of #1) 3) Changed the password property to a masked property 4) Added validators to the other properties
Refs #4884
Changeset: 9e4e1814ac2bc005e8300e66b07360f0bac6780a
comment:486 Changed 7 years ago by Ross Miller
Added algorithms for starting & stopping remote transactions
Added algorithms to start and stop remote transactions. Also fixed the exception throwing in the SubmitRemoteJob algorithm.
Refs #4884
Changeset: 4a7eeb6fed8a53a247994f17824e03b2213c7449
comment:487 Changed 7 years ago by Ross Miller
API changes to RemoteJobManager class
Replaced jobOutputRead() and getJobOutput() with more generic functions for directory listing and file download.
Refs #4884
Changeset: 744c3d4fa66324ed9c5d872abc8a7a4d0a96166b
comment:488 Changed 7 years ago by Ross Miller
New algorithms for querying and downloading remote files
Algorithms make use of the changes to RemoteJobManager that were just checked in.
Refs #4884
Changeset: fe7907d329c63cdad1d23655b6aebec94dec4e3c
comment:489 Changed 7 years ago by Ross Miller
Better validator on ComputeResource properties
The various algorithms that deal with remote job submission all have a property to select the compute cluster to be used. This property's validator now checks against the list of compute resource names found in Facilities.xml. (Previously, the property was just a free-form string.)
Refs #4884
Changeset: 4fb4a96b276400c8b5b2a56fc88b35f707abeb80
comment:490 Changed 7 years ago by Ross Miller
Change error message in download algorithm
Had the wrong error message when throwing an exception.
Refs #4884
Changeset: 13e0747f101f608c68c3217fe06356916ba2de0f
comment:491 Changed 7 years ago by Ross Miller
Change RemoteJobManager API
Add upload function to the RemoteJobManager API.
Also changed the HTTP connection initialization code to allow us to make POST requests and remove the 'using namespace std' line.
Refs #4884
Changeset: 8e2c570851ca17f854c883a737ca7767ef6bf36c
comment:492 Changed 7 years ago by Ross Miller
comment:493 Changed 7 years ago by Ross Miller
Fixing bugs in the code that generates the command line...
...that's run on the remote cluster.
Refs #4884
Changeset: 30b5413ea470c6bb3d52a35019f4d3c13034c9d6
comment:494 Changed 7 years ago by Ross Miller
Clean up SubmitRemoteJob algorithm
Changed the algorithm to simply call the requested script instead of trying to call mpirun directly. It's assumed the script will call mpirun.
Changed the generated JSON to pass some important values (number of nodes and cores per node) to the process via environment variables.
Refs #4884
Changeset: 531a230b5bb701e740bf82460f5d9a276eb0975b
comment:495 Changed 7 years ago by Ross Miller
Removed last of the remote job submission GUI code...
...from the MantidPlot tree. Most of it has been supercedeed by the *Remote* algorithms and the parts that hadn't been were out of date anyway. About the only thing we're likely to need in the future is a dialog box for querying jobs on the remote system and it will probably be easier to create a new one than to try and salvage the old code.
Refs #4884
Changeset: 72d310a812051b554297df278fb0475c50279dd1
comment:496 Changed 7 years ago by Ross Miller
Add CMake flag to enable/disable compiling
Add a flag to CMake to enable/disable building the entire remote job submission package. Flag defaults to 'disable'.
Note: A few things are still compiled even if the flag is disabled: the RemoteJobManagerFactory and a few minor changes to FacilityInfo.
Refs #4884
Changeset: 2efa9aa506b60b513aab8fff4c877dff33558265
comment:497 Changed 7 years ago by Ross Miller
Update algorithm categories
The various algorithms for handling remote job submission had categories left as 'User Algorithm' (from createAlg.py). Change this to 'Remote'.
Refs #4884
Changeset: 57c98adcf614f0ac944747112af13d03c8706e8d
comment:499 Changed 7 years ago by Ross Miller
Merge branch 'RACleanup' into develop into 6856_ConvertToDiffractionMDWS_v2
comment:500 Changed 7 years ago by Ross Miller
Merge branch 'RACleanup' into develop into 6856_ConvertToDiffractionMDWS_v2
comment:501 Changed 7 years ago by Ross Miller
Moved the RemoteAlgorithms classes into their own namespace
Refs #4884
Changeset: 797dc5cd845f6bd31c55423e631b8f77f05918df
comment:503 Changed 7 years ago by Ross Miller
- Status changed from accepted to verify
- Resolution set to fixed
Closing this ticket. The code is by no means complete, but it's got enough functionality that others want to start using it. The CMake flag to enable this code still defaults to off, so most people won't ever see it. Testing it on the build servers requires using the special builds under the "Remote Jobs Builds" tab.
comment:504 Changed 7 years ago by Ross Miller
- Status changed from verify to reopened
- Resolution fixed deleted
Re-opening the ticket because of build failures on MacOS
comment:505 Changed 7 years ago by Ross Miller
Build cleanups for MacOS
std::ifstream and std::ofstream constructors require const char * rather than strings
Refs #4884
Changeset: 2c0aab6c0fa70ddb0f1df5c7ca49cb21c3de4847
comment:506 Changed 7 years ago by Ross Miller
Changed the order of a couple of libraries in CMakeLists
I'm hoping this will fix the 'undefined symbol' erro on the Mac build
Refs #4884
Changeset: b034b8a30dc539190a71f9358cf3e8ce0563e9f6
comment:508 Changed 7 years ago by Ross Miller
- Status changed from accepted to verify
- Resolution set to fixed
Builds are working on Linux, Mac & Windows now, so closing the ticket. Again, code is not yet complete, but others want to start using it. The CMake flag that enables this chunk of code still defaults to off.
comment:509 Changed 7 years ago by Mathieu Doucet
- Status changed from verify to verifying
- Milestone changed from Unassigned to Release 2.6
- Tester set to Mathieu Doucet
comment:510 Changed 7 years ago by Mathieu Doucet
- Status changed from verifying to closed
The submission works.
Note: The job querying part will be done in another ticket
comment:511 Changed 5 years ago by Stuart Campbell
This ticket has been transferred to github issue 5731
First cut at GUI elements for the remote algorithsm work
refs #4884