Ticket #4787 (closed: fixed)

Opened 9 years ago

Last modified 5 years ago

Create a LoadLiveData algorithm

Reported by: Janik Zikovsky Owned by: Janik Zikovsky
Priority: critical Milestone: Release 2.1
Component: Mantid Keywords: LiveData, ADARA
Cc: Blocked By: #4801, #4830, #4873, #4902
Blocking: #4803, #4808 Tester: Stuart Campbell

Description (last modified by Janik Zikovsky) (diff)

As described in ​https://github.com/mantidproject/documents/blob/master/Design/Live_Event_Data_Detailed_Design_Document.doc

Change History

comment:1 Changed 9 years ago by Janik Zikovsky

  • Description modified (diff)
  • Blocked By 4801 added
  • Summary changed from Live Event Processing to Create a LoadLiveData algorithm

comment:2 Changed 9 years ago by Janik Zikovsky

  • Blocking 4803 added

comment:3 Changed 9 years ago by Janik Zikovsky

  • Keywords LiveData, ADARA added added

comment:4 Changed 9 years ago by Janik Zikovsky

  • Keywords added removed

comment:5 Changed 9 years ago by Janik Zikovsky

  • Blocking 4808 added

comment:6 Changed 9 years ago by Janik Zikovsky

  • Status changed from new to accepted

comment:7 Changed 9 years ago by Janik Zikovsky

Refs #4787 start of LoadLiveData algo

Changeset: d2371897d4f10afa2528132f3181911b779d4af8

comment:8 Changed 9 years ago by Janik Zikovsky

Refs #4787 start of LoadLiveData algo

Changeset: d2371897d4f10afa2528132f3181911b779d4af8

comment:9 Changed 9 years ago by Janik Zikovsky

Refs #4787 filled in LoadLiveData

Changeset: a579d6f37cd0da72caa8af1c4bc4771759b9e524

comment:10 Changed 9 years ago by Janik Zikovsky

Refs #4787 attempt to fix ConjoinWorkspaces

Changeset: 332b82815fe3008a9bf00f593542eb5dd4f9c4b5

comment:11 Changed 9 years ago by Janik Zikovsky

  • Blocked By 4830 added

comment:12 Changed 9 years ago by Janik Zikovsky

Refs #4787 test for LiveDataAlgorithm::makeAlgorithm()

Changeset: 6d68af8fb9c475d729dfeeecf182fb1f9f3c72c7

comment:13 Changed 9 years ago by Janik Zikovsky

Refs #4787: fix LiveData test

Changeset: 970f373f2455862934b235f5aa908018bcb2e26a

comment:14 Changed 9 years ago by Janik Zikovsky

Refs #4787 test for Conjoin on LoadLiveData

Also refs #4830 fix spectrum numbers for event workspaces in ConjoinWorkspaces2

Changeset: 561a7c0b9d135ceca206f1073de0d82c6867f872

comment:15 Changed 9 years ago by Janik Zikovsky

Refs #4787 extra tests and planning to handle postProcessing

Changeset: 77e38e7bcd41da709fc538c37d5692fd556da3e5

comment:16 Changed 9 years ago by Janik Zikovsky

Refs #4787: simple chunk processing works

Changeset: a9a0f94fa44492eb545c44e5f236250417d1198b

comment:17 Changed 9 years ago by Russell Taylor

Copy existing FakeEventDataListener code to test/TestDataListener.

This can then serve as a 'predictable' listener for testing while the FakeEventDataListener is scaled up to generate a certain number of events/sec. Use it in LoadLiveDataTest & StartLiveDataTest. Re #4812, Re #4787, Re #4803.

Changeset: 06893b4a11e9b16d678fcc096651bc71db54e53b

comment:18 Changed 9 years ago by Janik Zikovsky

  • Blocked By 4873 added

comment:19 Changed 9 years ago by Janik Zikovsky

Refs #4787: LoadLiveData does PostProcessing too.

Changeset: 537bcd00ed9b2117038b6561fe1066d485dbe315

comment:20 Changed 9 years ago by Janik Zikovsky

  • Blocked By 4902 added

comment:21 Changed 9 years ago by Janik Zikovsky

Refs #4787 using custom locking in LoadLiveData

I get segfaults that seem related to simultaneous calls to ADS::addOrReplace

Changeset: b317d3844fa69e8c07bccc163bf8010b2b06e738

comment:22 Changed 9 years ago by Janik Zikovsky

Refs #4787 LoadLiveData can run a bit of python script code

Changeset: 83b132a85005934b4d658ec6566443e9c4a69e28

comment:23 Changed 9 years ago by Janik Zikovsky

Refs #4787: Python LoadLiveDataTest to test python processing step

Changeset: 8f741547afe4ce3de01c84f785f6c7239ba3e349

comment:24 Changed 9 years ago by Janik Zikovsky

Refs #4808 StartLiveData clears existing output workspace

Also, refs #4787 fix processing workspace calls. Use unique names for temp. workspaces. More tests.

Changeset: 68cc37cf772afe942a04a58f32d5901bf10bab17

comment:25 Changed 9 years ago by Janik Zikovsky

Refs #4787 handle any type of OutputWorkspace

Changeset: 1e690854659868e54c98d9a6aa916f77cb3f9b66

comment:26 Changed 9 years ago by Janik Zikovsky

Refs #4787 handle MDWorkspace outputs when processing

Changeset: 713a7b32bf2d6a67e1d0a2d321e1cb6f511538c4

comment:27 Changed 9 years ago by Janik Zikovsky

Refs #4787 LoadLiveData graphic for wiki

Changeset: c043d19b27e98eaa595ba124d083842ea02951b0

comment:28 Changed 9 years ago by Janik Zikovsky

Refs #4787 LoadLiveData can run a bit of python script code

Changeset: 83b132a85005934b4d658ec6566443e9c4a69e28

comment:29 Changed 9 years ago by Janik Zikovsky

Refs #4787: Python LoadLiveDataTest to test python processing step

Changeset: 8f741547afe4ce3de01c84f785f6c7239ba3e349

comment:30 Changed 9 years ago by Janik Zikovsky

Refs #4808 StartLiveData clears existing output workspace

Also, refs #4787 fix processing workspace calls. Use unique names for temp. workspaces. More tests.

Changeset: 68cc37cf772afe942a04a58f32d5901bf10bab17

comment:31 Changed 9 years ago by Janik Zikovsky

Refs #4787 handle any type of OutputWorkspace

Changeset: 1e690854659868e54c98d9a6aa916f77cb3f9b66

comment:32 Changed 9 years ago by Janik Zikovsky

Refs #4787 handle MDWorkspace outputs when processing

Changeset: 713a7b32bf2d6a67e1d0a2d321e1cb6f511538c4

comment:33 Changed 9 years ago by Janik Zikovsky

Refs #4787 LoadLiveData graphic for wiki

Changeset: c043d19b27e98eaa595ba124d083842ea02951b0

comment:34 Changed 9 years ago by Janik Zikovsky

Refs #4787: Dynamic validator, does not work with python

Changeset: 0210a55d42e8fa483a879e0fd25f7fd88669d78f

comment:35 Changed 9 years ago by Janik Zikovsky

Revert "Refs #4787: Dynamic validator, does not work with python"

This reverts commit a08d3e772436dca9ace13cc3853df0d080407444.

Changeset: b625074e6647967e1389b35e44b23f81da020af4

comment:36 Changed 9 years ago by Janik Zikovsky

Refs #4970 mechanism to validate all inputs before execution,

API side and also refs #4787 added to LoadLiveData

Changeset: 36f56a6dd9560862c908d9e75d54c54165c3380c

comment:37 Changed 9 years ago by Janik Zikovsky

Refs #4787 fix validateInputs() and tests

Changeset: 0916126c52a487094c15fdf993308d37e360842c

comment:38 Changed 9 years ago by Janik Zikovsky

Refs #4787: added option to NOT PreserveEvents in live data summing

Changeset: 3a70f46c7ef77f1f8076b61b1858f9010e19b879

comment:39 Changed 9 years ago by Janik Zikovsky

Refs #4787: Dynamic validator, does not work with python

Changeset: 0210a55d42e8fa483a879e0fd25f7fd88669d78f

comment:40 Changed 9 years ago by Janik Zikovsky

Revert "Refs #4787: Dynamic validator, does not work with python"

This reverts commit a08d3e772436dca9ace13cc3853df0d080407444.

Changeset: b625074e6647967e1389b35e44b23f81da020af4

comment:41 Changed 9 years ago by Janik Zikovsky

Refs #4970 mechanism to validate all inputs before execution,

API side and also refs #4787 added to LoadLiveData

Changeset: 36f56a6dd9560862c908d9e75d54c54165c3380c

comment:42 Changed 9 years ago by Janik Zikovsky

Refs #4787 fix validateInputs() and tests

Changeset: 0916126c52a487094c15fdf993308d37e360842c

comment:43 Changed 9 years ago by Janik Zikovsky

Refs #4787: added option to NOT PreserveEvents in live data summing

Changeset: 3a70f46c7ef77f1f8076b61b1858f9010e19b879

comment:44 Changed 9 years ago by Janik Zikovsky

Refs #4787 make test more stable

Changeset: b503f28a6a32e20fe0c8547c0302e7a7783ed252

comment:45 Changed 9 years ago by Janik Zikovsky

Refs #4787 still trying to make test more stable

Changeset: 3236779cac6195043930e3a91bb0422eb40e7f86

comment:46 Changed 9 years ago by Janik Zikovsky

Refs #4787 wiki docs for LiveData algos

Changeset: 3c16f69030541fac083dde948630f8811470da80

comment:47 Changed 9 years ago by Janik Zikovsky

  • Status changed from accepted to verify
  • Resolution set to fixed

comment:48 Changed 9 years ago by Janik Zikovsky

Refs #4787 accidental paste error

Changeset: a6efa022fef01e2b3fc873f7f4e6a123d01746da

comment:49 Changed 8 years ago by Stuart Campbell

  • Status changed from verify to verifying
  • Tester set to Stuart Campbell

comment:50 Changed 8 years ago by Stuart Campbell

  • Status changed from verifying to closed

Been playing with the Live data stuff for about a day trying to break it. Seems to be very stable, been running 7 (or 8 - depends on OS) instances, all with preserve events=true, and all seems happy.

tested on Mac, Windows and Linux

Looks excellent!

comment:51 Changed 5 years ago by Stuart Campbell

This ticket has been transferred to github issue ​5634

Note: See TracTickets for help on using tickets.