SNS live listener needs to carry over the state of all the logs to each new chunk

[Russell Taylor]

At the moment, it clears all the time-series logs after they are copied over into a fresh 'chunk'. This is presenting a problem with logs that don't change - in particular EnergyRequest - as you then end up with an empty TSP that presents a problem to DGSReduction.

At the moment we're employing nasty hacks in the python we run within LoadLiveData, but we need a better solution.

My expectation is that the repeated logs in the combined workspace will not present a problem, but obviously we need to keep an eye on that.

[Russell Taylor]

Re #7822. Add a method to clear all but the last value.

This will modify a TimeSeriesProperty to keep only the last entry. Note that 'last' means the last added, unless the property has been sorted first.

Changeset: 769ff9f775fbaee23fdb9e34efac4747314b0ca1

[Russell Taylor]

Re #7822. A method to clear all log values except the current one.

Takes all TimeSeriesProperty logs and clears out all entries except the last one. Single value logs are left unchanged.

Changeset: 685d1474cc24b80d61a10db4f56785a2ff866758

[Russell Taylor]

Re #7822. Keep the last log entries instead of clearing.

The previous way presented problems with unchanging or slowly varying logs as a chunk could be left with an empty log that the analysis relied on having. The particular problem one was 'EnergyRequest', which was only appearing in the first chunk. I think that we can count on the logs coming in in time order, so the fact that the underlying methods don't check or sort should not be a problem.

Changeset: 2dec7164a718513848368b0cba87f3e41ae2f14e

[Russell Taylor]

Re #7822. Remove hoop jumping.

We no longer need this now the live listener has been updated to carry over the current value of all logs.

Changeset: 26d78b5fcc87ae6beab7ca79861faea5ab673bcb

[Russell Taylor]

Tester: Branch is feature/7822_carry_log_state_to_new_live_chunk

This should be tested at the SNS

A quick way to check that this has had an effect is to set the following as a processing script in a live data run (connect to SEQUOIA):

print input.getRun().getProperty('EnergyRequest').value

Before the changes this would only have an entry the first time around. Now it will always be populated.

The code I removed from dgs_reduction_script was previously required to stop it falling over on the second time around. Now it should keep going without problem.

My thinking is that the duplicated values that will get into the added logs will not pose a significant problem. I notice that logs with only a single entry (such as EnergyRequest) do not get duplicated anyway. The exception might be proton_charge which would integrate (very) slightly too high. I'm keen, though, not to have to take the hit from calling eliminateDuplicates after every chunk addition.

[Andrei Savici]

Merge remote-tracking branch 'origin/feature/7822_carry_log_state_to_new_live_chunk'

Full changeset: ​fc3f1b480967d4febbfb435e3e260de94790df91

[Andrei Savici]

Works on RHEL6, not on my machine (Ubuntu 12.10). On my machine Load Live data causes segfaults.

[Stuart Campbell]

This ticket has been transferred to github issue ​8667