Ticket #6923 (closed: fixed)

Opened 7 years ago

Last modified 5 years ago

Implement AutoUpdate on ScriptRepository

Reported by: Gesner Passos Owned by: Gesner Passos
Priority: critical Milestone: Release 2.6
Component: GUI Keywords: ScriptRepository,Maintenance
Cc: Blocked By:
Blocking: #6893 Tester: Martyn Gigg

Description

Currently the auto-update is not imlement. This means, allowing users to set/disable auto update as well as downloading automatically, when new files are available.

Change History

comment:1 Changed 7 years ago by Gesner Passos

  • Status changed from new to accepted

comment:2 Changed 7 years ago by Gesner Passos

Implement the auto_update feature

re #6923

Changeset: 827916367b4296764d8197e01b98f1561c832d64

comment:3 Changed 7 years ago by Gesner Passos

Improve the interaction with autoupdate column on Gui

re #6923

Changeset: 4d677826d790213a1938e9b309d205df14b83bf6

comment:4 Changed 7 years ago by Gesner Passos

Merge branch 'feature/6923_auto_update' into develop

Conflicts:

Code/Mantid/Framework/ScriptRepository/src/ScriptRepositoryImpl.cpp

Solve conflict with the correct syntax to update the property tree, local json files.

re #6923

Changeset: 7f4575705e634242d5e4cd6b5455330ced81e52f

comment:5 Changed 7 years ago by Gesner Passos

Implement the auto_update feature

re #6923

Changeset: 827916367b4296764d8197e01b98f1561c832d64

comment:6 Changed 7 years ago by Gesner Passos

Improve the interaction with autoupdate column on Gui

re #6923

Changeset: 4d677826d790213a1938e9b309d205df14b83bf6

comment:7 Changed 7 years ago by Gesner Passos

Merge branch 'feature/6923_auto_update' into develop

Conflicts:

Code/Mantid/Framework/ScriptRepository/src/ScriptRepositoryImpl.cpp

Solve conflict with the correct syntax to update the property tree, local json files.

re #6923

Changeset: 7f4575705e634242d5e4cd6b5455330ced81e52f

comment:8 Changed 7 years ago by Gesner Passos

Implement the auto_update feature

re #6923

Changeset: 827916367b4296764d8197e01b98f1561c832d64

comment:9 Changed 7 years ago by Gesner Passos

Improve the interaction with autoupdate column on Gui

re #6923

Changeset: 4d677826d790213a1938e9b309d205df14b83bf6

comment:10 Changed 7 years ago by Gesner Passos

Merge branch 'feature/6923_auto_update' into develop

Conflicts:

Code/Mantid/Framework/ScriptRepository/src/ScriptRepositoryImpl.cpp

Solve conflict with the correct syntax to update the property tree, local json files.

re #6923

Changeset: 7f4575705e634242d5e4cd6b5455330ced81e52f

comment:11 Changed 7 years ago by Gesner Passos

Implement the auto_update feature

re #6923

Changeset: 827916367b4296764d8197e01b98f1561c832d64

comment:12 Changed 7 years ago by Gesner Passos

Improve the interaction with autoupdate column on Gui

re #6923

Changeset: 4d677826d790213a1938e9b309d205df14b83bf6

comment:13 Changed 7 years ago by Gesner Passos

Merge branch 'feature/6923_auto_update' into develop

Conflicts:

Code/Mantid/Framework/ScriptRepository/src/ScriptRepositoryImpl.cpp

Solve conflict with the correct syntax to update the property tree, local json files.

re #6923

Changeset: 7f4575705e634242d5e4cd6b5455330ced81e52f

comment:14 Changed 7 years ago by Gesner Passos

Implement the auto_update feature

re #6923

Changeset: 827916367b4296764d8197e01b98f1561c832d64

comment:15 Changed 7 years ago by Gesner Passos

Improve the interaction with autoupdate column on Gui

re #6923

Changeset: 4d677826d790213a1938e9b309d205df14b83bf6

comment:16 Changed 7 years ago by Gesner Passos

Merge branch 'feature/6923_auto_update' into develop

Conflicts:

Code/Mantid/Framework/ScriptRepository/src/ScriptRepositoryImpl.cpp

Solve conflict with the correct syntax to update the property tree, local json files.

re #6923

Changeset: 7f4575705e634242d5e4cd6b5455330ced81e52f

comment:17 Changed 7 years ago by Gesner Passos

Implement the auto_update feature

re #6923

Changeset: 827916367b4296764d8197e01b98f1561c832d64

comment:18 Changed 7 years ago by Gesner Passos

Improve the interaction with autoupdate column on Gui

re #6923

Changeset: 4d677826d790213a1938e9b309d205df14b83bf6

comment:19 Changed 7 years ago by Gesner Passos

Merge branch 'feature/6923_auto_update' into develop

Conflicts:

Code/Mantid/Framework/ScriptRepository/src/ScriptRepositoryImpl.cpp

Solve conflict with the correct syntax to update the property tree, local json files.

re #6923

Changeset: 7f4575705e634242d5e4cd6b5455330ced81e52f

comment:20 Changed 7 years ago by Gesner Passos

  • Milestone changed from Release 2.5 to Release 2.6

comment:21 Changed 7 years ago by Nick Draper

  • Priority changed from blocker to critical
  • Keywords ScriptRepository,Maintenance added; ScriptRepository removed

comment:22 Changed 7 years ago by Gesner Passos

Implement the auto_update feature

process the auto_update string from local.json file and set it when necessary.

re #6923

Changeset: c8a70c745586c2b0377762c89e57f2334a94953e

comment:23 Changed 7 years ago by Gesner Passos

Extend signature check4update returning list of downloaded entries

If the check4update has to download the new version of files, these files are listed to provide feedback on what has been done.

re #6923

Changeset: 31e834c7fedfc4b7e5f098164aba513f9654c2f2

comment:24 Changed 7 years ago by Gesner Passos

Update unit test for check4update

re #6923

Changeset: ee48387584bc270727b39a25354df1bf4e5fa972

comment:25 Changed 7 years ago by Gesner Passos

Enable AutoUpdate from Script Repository Interface

re #6923

Changeset: ea851f608b0176422fe8a0ba6fe68548503d40dd

comment:26 Changed 7 years ago by Gesner Passos

Creation of the UpdateScriptRepository Algorithm

This algorithm will update the ScriptRepository and return a TableWorkspace with all the entries downloaded.

re #6923

Changeset: 07db0d632b264d781329701830734eafd1955392

comment:27 Changed 7 years ago by Gesner Passos

Remove the check4update from the ScriptRepositoryView

The call for the update for the Script Repository will be triggered from the UpdateScriptRepository Algorithm.

re #6923

Changeset: cbd96b8b87468efdde7e5efd0b57090d87d2f89d

comment:28 Changed 7 years ago by Gesner Passos

Remove the creation of TableWorkspace

I've discussed with Martyn this feature and hi suggested to remove it.

re #6923

Changeset: a6f751f15cd790de4bcf6f4fc182c236085313c7

comment:29 Changed 7 years ago by Gesner Passos

Improve the interaction with autoupdate column on Gui

re #6923

Changeset: 173f8229a4cdec4bc407522ae74433da6dbdc9d5

comment:30 Changed 7 years ago by Gesner Passos

Forgot to finish one line

add ; at the end of one line. shame on me.

re #6923

Changeset: 4333d6427c033d6d8e4f97f78d57351cb8e14592

comment:31 Changed 7 years ago by Gesner Passos

Triggering Update Script Repository on Mantid Startup

The requirement were:

  • in a different thread (so not to block on network issues)
  • At Mantid initialization (so to check frequently)

In order to use a different thread, it was necessary to execute trigger the algorithm after the initialization of event loop. So, a new slot was added to ApplicationWindow that is the first thing to be executed after starting the event loop. about2Start.

re #6923

Changeset: c9271afe91c65b5eb1313e5cfed5d434c3faff8a

comment:32 Changed 7 years ago by Gesner Passos

Merge branch 'feature/6923_implement_auto_update' into develop

Conflicts:

Code/Mantid/Framework/ScriptRepository/src/ScriptRepositoryImpl.cpp

re #6923

Changeset: 923130f0e8870b7a5856f73b5bbc71cba97a038c

comment:33 Changed 7 years ago by Gesner Passos

  • Blocking 6893 added

comment:34 Changed 7 years ago by Gesner Passos

  • Status changed from accepted to verify
  • Resolution set to fixed

Tester:

  • Open ScriptRepository Interface
  • You must see a column AutoUpdate
  • Download README.md file
  • Select this file Update automatically.
  • Shutdown Mantid
  • Now, let's simulate a new version of this file is available (if you want, you can really push a new version of README.md file).
    • Open a file editor and look for the files .repository.json and .local.json inside the folder where your scriptrepository was installed
    • At .local.json
      • Change the "downloaded_pubdate": "2013-May-08 09:40:02" from README.md to "downloaded_pubdate": "2013-May-08 09:30:02". (Which means, say that the current version was created at the Central Repository at 2013-May-08 09:30:02 (10 minutes before).
      • Keep the information of downloaded_date of README.md entry
    • At .repository.json
      • Change the "pub_date": "2013-May-08 09:40:02" to "pub_date": "2013-May-08 09:30:02"
  • The scriptRepository will now assume that there is a new version of the file README.md at the repository.
  • Restart MantidPlot. You will see a warning message informing you that ScriptRepository has downloaded the new version of README.md file available at the central repository.
  • Open again the .local.json file and check that the downloaded_date is ahead the previous one.

It seems complicate... but it is not ;)

comment:35 Changed 7 years ago by Gesner Passos

Last edited 7 years ago by Gesner Passos (previous) (diff)

comment:36 Changed 7 years ago by Gesner Passos

Last edited 7 years ago by Gesner Passos (previous) (diff)

comment:37 Changed 7 years ago by Gesner Passos

Last edited 7 years ago by Gesner Passos (previous) (diff)

comment:38 Changed 7 years ago by Gesner Passos

I've removed the 3 comments above because, I wrongly put one commit in the wrong branch. I've fixed it, but the history remained. So, I remove them manually.

comment:39 Changed 7 years ago by Nick Draper

  • Status changed from verify to verifying
  • Tester set to Nick Draper

comment:40 Changed 7 years ago by Nick Draper

  • Status changed from verifying to closed

Passed this but raised another ticket that must get into this release alongside this. #7122 it needs to handle folders better.

comment:41 Changed 7 years ago by Gesner Passos

Implement the auto_update feature

process the auto_update string from local.json file and set it when necessary.

re #6923

Changeset: c8a70c745586c2b0377762c89e57f2334a94953e

comment:42 Changed 7 years ago by Gesner Passos

Extend signature check4update returning list of downloaded entries

If the check4update has to download the new version of files, these files are listed to provide feedback on what has been done.

re #6923

Changeset: 31e834c7fedfc4b7e5f098164aba513f9654c2f2

comment:43 Changed 7 years ago by Gesner Passos

Update unit test for check4update

re #6923

Changeset: ee48387584bc270727b39a25354df1bf4e5fa972

comment:44 Changed 7 years ago by Gesner Passos

Enable AutoUpdate from Script Repository Interface

re #6923

Changeset: ea851f608b0176422fe8a0ba6fe68548503d40dd

comment:45 Changed 7 years ago by Gesner Passos

Creation of the UpdateScriptRepository Algorithm

This algorithm will update the ScriptRepository and return a TableWorkspace with all the entries downloaded.

re #6923

Changeset: 07db0d632b264d781329701830734eafd1955392

comment:46 Changed 7 years ago by Gesner Passos

Remove the check4update from the ScriptRepositoryView

The call for the update for the Script Repository will be triggered from the UpdateScriptRepository Algorithm.

re #6923

Changeset: cbd96b8b87468efdde7e5efd0b57090d87d2f89d

comment:47 Changed 7 years ago by Gesner Passos

Remove the creation of TableWorkspace

I've discussed with Martyn this feature and hi suggested to remove it.

re #6923

Changeset: a6f751f15cd790de4bcf6f4fc182c236085313c7

comment:48 Changed 7 years ago by Gesner Passos

Improve the interaction with autoupdate column on Gui

re #6923

Changeset: 173f8229a4cdec4bc407522ae74433da6dbdc9d5

comment:49 Changed 7 years ago by Gesner Passos

Forgot to finish one line

add ; at the end of one line. shame on me.

re #6923

Changeset: 4333d6427c033d6d8e4f97f78d57351cb8e14592

comment:50 Changed 7 years ago by Gesner Passos

Triggering Update Script Repository on Mantid Startup

The requirement were:

  • in a different thread (so not to block on network issues)
  • At Mantid initialization (so to check frequently)

In order to use a different thread, it was necessary to execute trigger the algorithm after the initialization of event loop. So, a new slot was added to ApplicationWindow that is the first thing to be executed after starting the event loop. about2Start.

re #6923

Changeset: c9271afe91c65b5eb1313e5cfed5d434c3faff8a

comment:51 Changed 7 years ago by Gesner Passos

Remove the import SANSWideAngleCorrection from old python

re #6923

Changeset: 7db0bf86f89654350416099b31ce127eb2eb5f0c

comment:52 Changed 7 years ago by Gesner Passos

  • Status changed from closed to reopened
  • Resolution fixed deleted

There is an issue when the user does not install the scriptrepository in his machine.

comment:53 Changed 7 years ago by Gesner Passos

Protect the ScriptRepository from empty string

the Algorithm UpdateScriptRepository first check if the repository is valid, meaning, it is installed and correct.

Protect ScriptRepositoryImpl from empty ScriptRepositoryLocal string.

re #6923

Changeset: 7d79ef2dae694a2e717dd17963e6165ab693d1ed

comment:54 Changed 7 years ago by Gesner Passos

Do not trigger UpdateScriptRepository if it is not installed

if the user never installed the ScriptRepository, it should not try to update it. As so now.

re #6923

Changeset: 37e7736292e9d23d1c8548541e1a5398568e5c0d

comment:55 Changed 7 years ago by Gesner Passos

  • Status changed from reopened to accepted

comment:56 Changed 7 years ago by Gesner Passos

Error reported. If you do not have a ScriptRepository installed at your machine, on opening MantidPlot you will see:

UpdateScriptRepository-[Notice] UpdateScriptRepository started
UpdateScriptRepository-[Error] Error in execution of algorithm UpdateScriptRepository:
UpdateScriptRepository-[Error] File not found

For Windows users, if you try to run MantidPlot in debug it will crash as well.

Tester:

remove the entry ScriptLocalRepository from your Mantid.user.properties file and start MantidPlot.

comment:57 Changed 7 years ago by Gesner Passos

  • Status changed from accepted to verify
  • Resolution set to fixed

comment:58 Changed 7 years ago by Gesner Passos

Merge branch 'feature/6923_implement_auto_update' into develop into 6856_ConvertToDiffractionMDWS_v2

comment:59 Changed 7 years ago by Gesner Passos

Merge branch 'bugfix/6923_auto_update' into develop into 6856_ConvertToDiffractionMDWS_v2

comment:60 Changed 7 years ago by Gesner Passos

Merge branch 'feature/6923_implement_auto_update' into develop into 6856_ConvertToDiffractionMDWS_v2

comment:61 Changed 7 years ago by Gesner Passos

Merge branch 'bugfix/6923_auto_update' into develop into 6856_ConvertToDiffractionMDWS_v2

comment:62 Changed 7 years ago by Martyn Gigg

  • Status changed from verify to verifying
  • Tester changed from Nick Draper to Martyn Gigg

comment:63 Changed 7 years ago by Martyn Gigg

  • Status changed from verifying to reopened
  • Resolution fixed deleted

So the update doesn't happen now if the repository is not installed but when I then go to click on "Script Repository" I get an error in the results log:

ScriptRepository: Corrupted database : .repository.json
Details: json_parser_error: .repository.json: cannot open file
ScriptRepository failed to list all entries inside the repository. Details: Corrupted database : .repository.json

and the dialog pops up but is blank.

My user properties file doesn't have anything in concerning the script repository so it should look like a it wants to install it for the first time.

comment:64 Changed 7 years ago by Gesner Passos

Correct the bug found for the installation of ScriptRepository

Ubuntu used to define non initialized bool attribute as false, but, windows does seems to initialize them as true. So, the error presented by the test:

ScriptRepository: Corrupted database : .repository.json

was not seem at my development environment (ubuntu). This commit does initialize the valid attribute to fix this issue.

re #6923

Changeset: 7d58bac56ebd0c5cdd2118ac767021edd2111603

comment:65 Changed 7 years ago by Gesner Passos

Merge branch 'bugfix/6923_auto_update' into develop

comment:66 Changed 7 years ago by Gesner Passos

  • Status changed from reopened to accepted

comment:67 Changed 7 years ago by Martyn Gigg

  • Status changed from accepted to verify
  • Resolution set to fixed

comment:68 Changed 7 years ago by Martyn Gigg

  • Status changed from verify to verifying

comment:69 Changed 7 years ago by Martyn Gigg

  • Status changed from verifying to reopened
  • Resolution fixed deleted

It attempts to install it but now I get

ScriptRepositoryView-[Error] Unknown error occurred to install ScriptRepository. It will not be shown.

comment:70 Changed 7 years ago by Gesner Passos

Addition of a unit test that reproduce the error found

This is to protect for future erros like this.

The error is that the remote url is empty.

re #6923

Changeset: 357082a1a2b30270f68c36fe257146f64d638117

comment:71 Changed 7 years ago by Gesner Passos

Protection from empty remote url

Correct the steps of construction, first get the remote url information, and them the local paths. Besides, protect install method to attempt to install on an empty remote_url.

re #6923

Changeset: 238636f07ee6671557ff37de69247822630048c5

comment:72 Changed 7 years ago by Gesner Passos

Merge branch 'bugfix/6923_auto_update' into develop

comment:73 Changed 7 years ago by Gesner Passos

  • Status changed from reopened to accepted

I think(hope) it is solved right now.

comment:74 Changed 7 years ago by Gesner Passos

  • Status changed from accepted to verify
  • Resolution set to fixed

comment:75 Changed 7 years ago by Martyn Gigg

  • Status changed from verify to verifying

comment:76 Changed 7 years ago by Martyn Gigg

Merge remote-tracking branch 'origin/bugfix/6923_auto_update' into master

comment:77 Changed 7 years ago by Martyn Gigg

  • Status changed from verifying to closed

Looks fine now. Tried with no repository then when I had one installed and finally when I removed the key from my properties file.

comment:78 Changed 7 years ago by Russell Taylor

Merge remote-tracking branch 'origin/feature/7056_data_stitching_api2' into 7090_runpythonscript

comment:79 Changed 7 years ago by Martyn Gigg

Merge remote-tracking branch 'origin/bugfix/6923_auto_update' into 7090_runpythonscript

comment:80 Changed 7 years ago by Nick Draper

  • Component changed from Mantid to Framework

comment:81 Changed 7 years ago by Nick Draper

  • Component changed from Framework to User Interface

comment:82 Changed 5 years ago by Stuart Campbell

This ticket has been transferred to github issue 7769

Note: See TracTickets for help on using tickets.