Ticket #10260 (closed: fixed)

Opened 6 years ago

Last modified 5 years ago

Add progress reporting to LoadParameterFile

Reported by: Harry Jeffery Owned by: Harry Jeffery
Priority: major Milestone: Release 3.3
Component: Framework Keywords:
Cc: Blocked By: #10185
Blocking: Tester: Dan Nixon

Description

Now that SaveParameterFile has begun to output very large XML files (~50MB), it takes a considerable amount of time to load these parameter files back into Mantid.

Currently, LoadParameterFile does not report its progress to the user (and consequently cannot be cancelled).

It should provide progress updates on a regular basis.

Change History

comment:1 Changed 6 years ago by Harry Jeffery

  • Status changed from new to assigned

comment:2 Changed 6 years ago by Harry Jeffery

  • Status changed from assigned to inprogress

comment:3 Changed 6 years ago by Harry Jeffery

Refs #10260 Remove usage of LoadParameterFile::execManually

Replace all external calls to this static method with use of a child algorithm instead.

Changeset: abb99045c84a700c1df1b4435cd57d58c7812f1e

comment:4 Changed 6 years ago by Harry Jeffery

Refs #10260 Remove LoadParameterFile::execManually

Changeset: 2116279f31c618325a38413797ef8e8c921b4aef

comment:5 Changed 6 years ago by Harry Jeffery

Refs #10260 Improve LoadParameterFile progress reporting

Changeset: d7e9c176bc0144e0b4b615b933bc507a17f4a91a

comment:6 Changed 6 years ago by Harry Jeffery

Refs #10260 Optimise setComponentLinks

Internally Poco::XML::NodeList::item performs an O(n) linked-list traversal to look up an item by index. We were calling item within an O(n) loop, pushing our performance up to O(n2). Let's not do that. Let's traverse the DOM ourselves at O(n).

On my machine this brought the execution time of LoadParameterFile down from ~1,400 seconds to ~9 seconds. That's about 140x faster.

Changeset: 21597ea53fdaa34bb74ef6056dd51584cc42259a

comment:7 Changed 6 years ago by Harry Jeffery

Refs #10260 Fix doxygen warning

Changeset: a0e9628fb2ab4a2e46d23569f09492eecf946c01

comment:8 Changed 6 years ago by Harry Jeffery

  • Status changed from inprogress to verify
  • Resolution set to fixed

Testing

  • Run LoadParameterFile on a large parameter file. Progress is reported and it will run much more quickly whilst behaving the same.
  • Verify no tests were broken
  • Inspect code

Branch: https://github.com/mantidproject/mantid/compare/feature/10260_loadparamfile_report_progress

comment:9 Changed 6 years ago by Dan Nixon

  • Status changed from verify to verifying
  • Tester set to Dan Nixon

comment:10 Changed 6 years ago by Dan Nixon

  • Status changed from verifying to verify
  • Tester Dan Nixon deleted

comment:11 Changed 6 years ago by Dan Nixon

  • Status changed from verify to verifying
  • Tester set to Dan Nixon

comment:12 Changed 6 years ago by Dan Nixon

  • Status changed from verifying to closed

Merge remote-tracking branch 'origin/feature/10260_loadparamfile_report_progress'

Full changeset: b9ddaa464db796ef5bde0bc294ef33a726e4844a

comment:13 Changed 6 years ago by Dan Nixon

Merge remote-tracking branch 'origin/feature/10260_loadparamfile_report_progress'

Full changeset: b9ddaa464db796ef5bde0bc294ef33a726e4844a

comment:14 Changed 5 years ago by Stuart Campbell

This ticket has been transferred to github issue 11102

Note: See TracTickets for help on using tickets.