Ticket #9069 (closed: fixed)

Opened 7 years ago

Last modified 5 years ago

IDS catalog interface

Reported by: Jay Rainey Owned by: Jay Rainey
Priority: major Milestone: Release 3.2
Component: Framework Keywords: ICAT,IDS
Cc: martyn.gigg@… Blocked By:
Blocking: #7640, #9026 Tester: Martyn Gigg

Description (last modified by Jay Rainey) (diff)

The methods getDownloadURL and getUploadURL in ICatalog need to be re-factored into their own interface as it's not possible to call them within a compositeCatalog (#9026). They are part of the Information Data Service (IDS), which is a separate API from ICAT. (Though, it uses investigationIds etc that are obtained via ICAT)

To resolve this issue I need to:

  1. Create a new interface (CatalogInfoService) that contains the two method signatures.
  2. Inherit this interface in ICat4Catalog. (I should not have to change much within this class)
  3. Remove the related methods from ICat3Catalog as it does not have an IDS.
  4. Remove keepAliveinminutes method from ICatalog as it's not used, and will not function as expected in the compositeCatalog. The keepAlive method is more than sufficient.

Note: the catalog algorithms that use these methods should not be affected as they call the catalog in use. Better exception handling should be put in place to ensure if a catalog is used that does not support an IDS then the user will know. The algorithms are:

  • CatalogDownloadDataFiles(uses getDownloadURL)
  • CatalogDownloadDataSets (uses getDownloadURL)
  • CatalogPublish (uses getUploadURL)

Attachments

ICatBefore.jpg (24.2 KB) - added by Jay Rainey 7 years ago.
The catalog structure before changes
ICatAfter.jpg (62.6 KB) - added by Jay Rainey 7 years ago.

Change History

comment:1 Changed 7 years ago by Jay Rainey

  • Blocking 7640 added

comment:1 Changed 7 years ago by Jay Rainey

  • Milestone changed from Release 3.3 to Release 3.2
  • Description modified (diff)
  • Blocking 7640 removed

comment:2 Changed 7 years ago by Owen Arnold

  • Status changed from new to infoneeded
  • Owner changed from Jay Rainey to Owen Arnold

This sounds fine, but a quick 5-min sketch (class diagram) done in any application showing what the class hierarchy will look like before and after would be very useful. Export and attach to this ticket.

comment:3 Changed 7 years ago by Jay Rainey

  • Blocking 7640 added

(In #7640) This ticket will act as an umbrella ticket for all functionality that will enable cross facility querying of ICAT within Mantid.

Changed 7 years ago by Jay Rainey

The catalog structure before changes

Changed 7 years ago by Jay Rainey

comment:4 Changed 7 years ago by Jay Rainey

ICatBefore

The current structure of the catalog functionality within Mantid does not allow for multiple catalogs to be created.This was to be addressed in #9026, which I am unable to do as several methods in ICatalog do not belong there. E.g. the IDS upload & download functionality.

ICatAfter

This diagram illustrates the changes that I plan to make both in this ticket, and #9026, which involves:

  1. Creating a new interface (ICatalogInfoService) that holds the IDS functionality.
  2. Creating a CompositeCatalog (#9026) to allow the catalog functionality to be called on each instance of a catalog.
  3. The session class will be removed, and related functionality added to each catalog.

comment:5 Changed 7 years ago by Martyn Gigg

  • Owner changed from Owen Arnold to Jay Rainey

comment:7 Changed 7 years ago by Jay Rainey

Added new CatalogInfoService interface. Refs #9069.

Changeset: 9a1aaf15893b22d8e38895cd1bc3f24051dcdfa7

comment:8 Changed 7 years ago by Jay Rainey

Updated layout of ICatalog. Refs #9069.

Changeset: 78fe1ea7eeaaf97aa9509961e511d6e92ba74793

comment:9 Changed 7 years ago by Jay Rainey

Using ICatalogInfoService interface in ICAT4. Refs #9069.

  • Removed IDS methods from ICatalog interface.
  • Added typedef to easily use CatalogInfoService in algorithms.

Changeset: 2cf4b5e38a84775e28813df15d732ec44acf7e31

comment:10 Changed 7 years ago by Jay Rainey

Algorithms to use ICatalogInfoService interface methods. Refs #9069.

Changeset: e6b95ccd29abbe243e8455f715e9b88b64040ca9

comment:11 Changed 7 years ago by Jay Rainey

ICat3Catalog now uses CatalogInfoService. Refs #9069.

Changeset: dd73e653675df1f5963c82ecda2b21c2e8e4b34f

comment:12 Changed 7 years ago by Jay Rainey

Merge branch 'feature/9069_refactor_catalog_interface' into develop.Refs #9069.

Conflicts:

  • Code/Mantid/Framework/API/inc/MantidAPI/ICatalog.h
  • Code/Mantid/Framework/ICat/inc/MantidICat/ICat3/ICat3Catalog.h

Changeset: 23ac661b83ce2c043722604f49377a8866091b6f

comment:13 Changed 7 years ago by Jay Rainey

  • Status changed from infoneeded to new

comment:14 Changed 7 years ago by Jay Rainey

  • Status changed from new to assigned

comment:15 Changed 7 years ago by Jay Rainey

  • Status changed from assigned to verify
  • Resolution set to fixed

Overview

This ticket addresses the issues found when creating a composite catalog in #9026. That is, several methods did not belong in the ICatalog interface, and should instead be separated out into their own interface (they both use the IDS).

To test

External downloading

In order to test the external downloading functionality you must not have access to the archives. I have written instructions to disable access to the archives on Ubuntu here.

  1. Log into the catalog and perform a search.
  2. Double click on an investigation in the Search Results table. The Datafile information table will be populated.
  3. Select a datafile, and click Download to..., and save the datafile on your desktop.
  4. Verify the file was downloaded correctly.

Publishing

You will need to verify the publishing functionality works. To do this:

  1. Log into the catalog.
  2. Select Publish from the catalog menu.
  3. Select a datafile to upload.
  4. Enter a name to save the file as in the Name in catalog: field. Copy this name to your clipboard.
  5. Select an investigation (The id is: 1193002) and click Run.

To verify the datafile was uploaded correctly:

  1. Open the catalog interface and click Advanced search.
  2. Paste the name of the file you uploaded into the Datafile name field.
  3. Click Search. The investigation containing the datafile was now be displayed in the Search results table. Double click on it, and verify the datafile is there.
  4. Download the datafile and compare it with the one you uploaded.

comment:16 Changed 7 years ago by Martyn Gigg

  • Status changed from verify to verifying
  • Tester set to Martyn Gigg

comment:17 Changed 7 years ago by Martyn Gigg

All of the steps above still work after the refactoring and the code changes look good.

comment:18 Changed 7 years ago by Martyn Gigg

  • Status changed from verifying to closed

Merge remote-tracking branch 'origin/feature/9069_refactor_catalog_interface'

Full changeset: 9da084670a7b56b44f252ddd857f8f848566cc57

comment:19 Changed 7 years ago by Jay Rainey

  • Blocking 9084 added

comment:20 Changed 7 years ago by Jay Rainey

  • Blocking 9084 removed

comment:21 Changed 7 years ago by Martyn Gigg

Put back initializer character for MPI build

Refs #9069

Changeset: 019c2930128cfeb5529ca1a5b6021b72172996e8

comment:22 Changed 5 years ago by Stuart Campbell

This ticket has been transferred to github issue 9912

Note: See TracTickets for help on using tickets.