Ticket #10095 (closed: fixed)

Opened 6 years ago

Last modified 5 years ago

Speed up isis live data listener throughput

Reported by: Nick Draper Owned by: Nick Draper
Priority: blocker Milestone: Release 3.2.1
Component: Framework Keywords:
Cc: Blocked By:
Blocking: #9671 Tester: Roman Tolchenov

Description

The isis live event listener was having trouble keeping up with the white beam event speed from MERLIN (500K events per second).

A paired code review with Martyn lead to several improvments, with the fastest event rate measured increasing from 100K to 3.3M events per sec.

Also some code documentation improvements (microseconds to milliseconds)

Change History

comment:1 Changed 6 years ago by Nick Draper

  • Keywords PatchCandidate added

comment:2 Changed 6 years ago by Nick Draper

to test

  1. Code review
  2. set up a live data localhost loop (or use 2 pc's)
    1. number of spectra 80000
    2. number of events per packet 12500
    3. rate 20
  3. Live listener to Facility TEST_LIVE, ISIS_EVENT
    1. processing Rebin, 100,10,200, preserve events
    2. Preserve events and replace for accumulation
    3. Update every 1 second
    4. Then the number of eventws in the workspace gives you the troughput in events per second
    5. You are looking for an event rate close to 600,000 events per second. (to get this you might have to build in Release).
Last edited 6 years ago by Nick Draper (previous) (diff)

comment:3 Changed 6 years ago by Martyn Gigg

  • Status changed from new to assigned

comment:4 Changed 6 years ago by Nick Draper

  • Status changed from assigned to inprogress

re #10095 speed up, made fakepacket sending more like real

Changeset: 7924f9dc601fbde11471d3ff8e8bb59818fa8203

comment:5 Changed 6 years ago by Nick Draper

re #10095 speed up, made fakepacket sending more like real

Changeset: bfb8a90c46d93e973df3da0594d031065cb64772

comment:6 Changed 6 years ago by Nick Draper

re #10095 fix name clash with unit test

Changeset: a846630235658e2f39ceb2d849db10a1a59554ce

comment:7 Changed 6 years ago by Nick Draper

re #10095 revert from develop

Changeset: a68e64a500aab3f9820628339cb0f0a605e0653d

comment:8 Changed 6 years ago by Nick Draper

re #10095 revert the revert and avoid the facility name TEST

Changeset: 2187b54b11655a24ec590eccd82151f4dac7dc76

comment:9 Changed 6 years ago by Nick Draper

re #10095 Minor change to the TEST_LIVE facility

Changeset: d11d495902b96183fd7dd0dd9bf9233f665f5040

comment:10 Changed 6 years ago by Nick Draper

  • Status changed from inprogress to verify
  • Resolution set to fixed

see earlier to test section

comment:11 Changed 6 years ago by Nick Draper

re #10095 clear a warning

Changeset: ca1be6fa14bbb651fef42c4fe2e35eca3aa88a17

comment:12 Changed 6 years ago by Nick Draper

  • Blocking 9671 added

comment:13 Changed 6 years ago by Owen Arnold

  • Status changed from verify to closed

Merge remote-tracking branch 'origin/feature/10095_speed_up_ISIS_LiveListener'

Full changeset: ​04e2e72c61920e9d4e37f76ca77aa909e2f2bf7f

comment:14 Changed 6 years ago by Peter Peterson

  • Keywords PatchCandidate removed
  • Milestone changed from Release 3.3 to Release 3.2.1

comment:15 Changed 6 years ago by Nick Draper

Cherry-pick fixes from feature/10095_speed_up_ISIS_LiveListener

into next

Changes:

re #10095 speed up, made fakepacket sending more like real (cherry picked from commit 7924f9dc601fbde11471d3ff8e8bb59818fa8203)

re #10095 speed up, made fakepacket sending more like real (cherry picked from commit bfb8a90c46d93e973df3da0594d031065cb64772)

re #10095 fix name clash with unit test (cherry picked from commit a846630235658e2f39ceb2d849db10a1a59554ce)

re #10095 revert from develop (cherry picked from commit a68e64a500aab3f9820628339cb0f0a605e0653d)

re #10095 revert the revert and avoid the facility name TEST (cherry picked from commit 2187b54b11655a24ec590eccd82151f4dac7dc76)

re #10095 Minor change to the TEST_LIVE facility (cherry picked from commit d11d495902b96183fd7dd0dd9bf9233f665f5040)

re #10095 clear a warning (cherry picked from commit ca1be6fa14bbb651fef42c4fe2e35eca3aa88a17)

Changeset: b84427b2b3454b438b4c737f7457c98e7230d738

comment:16 Changed 5 years ago by Stuart Campbell

This ticket has been transferred to github issue ​10937

Note: See TracTickets for help on using tickets.