Filtering Logs between times

[Owen Arnold]

This is currently not possible for the end user. The following should provide the basis of an algorithm that would do it for us. In addition, It should be possible to:

• Choose to filter keep only the selected interval
• Choose to apply mean, median, max ... functions to the output.

[Owen Arnold]

Updated Filtering example. Much better than previously attached example.

from mantid.kernel import DateAndTime
mtd.importAll()
ws = WISH00026110

def meanWithinRange(logname, starttime=None, endtime=None, operation="mean", inside = True):
        run = ws.getRun()
        tstart = run.startTime()
        tend = run.endTime()
        if starttime:
                tstart = tstart + (starttime * int(1e9))
        if  endtime:
                tend = run.startTime() + (endtime * int(1e9))
        temperature = run.getLogData(logname)
        times = numpy.array(temperature.times)
        values = numpy.array(temperature.value)
        if inside:
                mask = (tstart < times) & (times < tend) # Get times between filter start and end.
        else:
                mask = (tstart > times) | (times > tend)
        method = getattr(numpy, operation)
        return method(values[mask])
        

''' Gets mean from logs from start of run till end of run. '''
logname = 'Sample_Temp'
print meanWithinRange(logname)

''' Gets mean from logs from specified start till end of run. '''
starttime =  -10000 # seconds since the start of the run. Also accepts negative values
endtime = None # Defaults to the end of the run.
logname = 'Sample_Temp'
print meanWithinRange(logname, starttime=starttime, endtime=endtime, operation="max", inside=False)

[Owen Arnold]

refs #8300. Filtering algorithm made.

Changeset: f2608d03a389004bc80997569d73ec6e559ef7e7

[Owen Arnold]

refs #8300. Add Start and End time filtering.

Unit tested too.

Changeset: b95d71755442b815010fa526d3df4c806d11b891

[Owen Arnold]

refs #8300. Add statistical methods.

Changeset: f0a699bf01ea3a9ca3a5f6a79ce39a3636f4844c

[Owen Arnold]

refs #8300. Add wiki documentation.

Changeset: b51084e88e51889fe5ae47100ce44c4f505b87a2

[Owen Arnold]

Tester. The unit tests show this working. You should however do the following.

• Load ARGUS0026287 out of the AutoTest dir
• Open MantidPlot and use the algorithm FilterLogByTime
• Use the temperature-2-log in association with the algorithm documentation to apply filtering according to this algorithm.

[Gesner Passos]

This script pass if you uncomment the second line. But, fail as it is.

f, log = '/apps/mantid/systemtests/Data/SANS2D/SANS2D00005547.raw', 'temp2'
#f, log = '/apps/mantid/mantid/Test/AutoTestData/argus0026287.nxs', 'temperature_2_log'

ws = Load(Filename=f)

if isinstance(ws, tuple):
   ws = ws[0]
   
tempfilter,  stat = FilterLogByTime(ws, log )

temps = ws.getRun().getLogData(log).value
print type(temps)
print len(tempfilter)
print len(temps)
assert(len(tempfilter) == len(temps))

Besides, it is worth pointing out in the documentation that it works only for float numeric series.

[Owen Arnold]

refs #8300. Make more robust about log type.

Changeset: 0eac2fe5c8146ccd07b412cebf5fa861cb71a177

[Owen Arnold]

There actually isn't a problem here. The reason that the script above fails is because the run endTime is prior to the last collected log value. Note that if you do not specify StartTime or EndTime the algorithm takes these to be the startTime of the run and the endTime of the run respectively. That is why a single log value is being chopped off the end.

[Wenduo Zhou]

The algorithm works. But there should be more introduction on 'Method', especially the math on 'mean', 'median', 'mode' and etc.

[Owen Arnold]

refs #8300 Documentation for Method property

Changeset: f08b4c237f6b62411518e1ebaafb907eb781e054

[Wenduo Zhou]

Merge remote-tracking branch 'origin/feature/8300_filter_time'

Full changeset: ​6ddc8b9d349b3dce41a8c1630e3a46c31675e169

[Stuart Campbell]

This ticket has been transferred to github issue ​9145