Ticket #8301 (closed: fixed)

Opened 7 years ago

Last modified 5 years ago

Filter Log Values

Reported by: Owen Arnold Owned by: Wenduo Zhou
Priority: blocker Milestone: Release 3.1
Component: Framework Keywords:
Cc: Blocked By:
Blocking: Tester: Keith Brown

Description

Filtering by log values just isn't working properly.

  • There's no way to generate a filter other than via a log that already exists.
  • There's no way to AddSampleLog for a time series boolean log (so can filter anyway)
  • There's no way to create a TimeSeries log in python.

This needs to be sorted out ASAP!

Change History

comment:1 in reply to: ↑ description Changed 7 years ago by Owen Arnold

Replying to Owen Arnold:

Filtering by log values just isn't working properly.

  • There's no way to generate a filter other than via a log that already exists.
  • There's no way to AddSampleLog for a time series boolean log (so can filter anyway)
  • There's no way to create a TimeSeries log in python.

This needs to be sorted out ASAP! Don't even think about moving this to another iteration or closing this without a fix. This functionality is required.

comment:2 follow-up: ↓ 5 Changed 7 years ago by Andrei Savici

The last point is wrong:

w=Load("CNCS_7860")
from time import gmtime, strftime,mktime
from numpy import *
tsp=kernel.FloatTimeSeriesProperty("tsp")
for i in arange(25):
   tsp.addValue(strftime("%Y-%m-%d %H:%M:%S", gmtime(mktime(gmtime())+i)),1.0*i*i)
w.mutableRun()['tsp']=tsp

comment:3 Changed 7 years ago by Wenduo Zhou

  • Owner changed from Anyone to Wenduo Zhou

comment:4 Changed 7 years ago by Russell Taylor

I'm wondering if there's some confusion here because we have more than one method of filtering logs. Filtering by duplicate entries/booleans is used up in the MantidPlot log viewing functionality (and unfortunately intrudes into TimeSeriesProperty when it would be better if it were isolated) but AFAIK filtering elsewhere (by time or log value) doesn't work like this (it creates new logs with the entries that pass the filter).

comment:5 in reply to: ↑ 2 Changed 7 years ago by Owen Arnold

Replying to Andrei Savici:

The last point is wrong:

w=Load("CNCS_7860")
from time import gmtime, strftime,mktime
from numpy import *
tsp=kernel.FloatTimeSeriesProperty("tsp")
for i in arange(25):
   tsp.addValue(strftime("%Y-%m-%d %H:%M:%S", gmtime(mktime(gmtime())+i)),1.0*i*i)
w.mutableRun()['tsp']=tsp

I stand corrected.Thanks for this.

comment:6 Changed 7 years ago by Owen Arnold

I think that we need to figure out how log filtering is actually working across the whole project first. There doesn't seem to be a good understanding of how this is working. A wikipage or design document answering some of these questions for the existing behaviour might be a good place to start.

comment:7 Changed 7 years ago by Wenduo Zhou

  • Status changed from new to inprogress

Added web link to event filtering alg. Refs #8301.

Changeset: 3839b39470a0ec010a846b6d39b04acb23221f0d

comment:8 Changed 7 years ago by Wenduo Zhou

  • Status changed from inprogress to verify
  • Resolution set to fixed

An additional wiki page (http://www.mantidproject.org/EventFiltering) is created to introduce the event filtering algorithms in Mantid.

This link is also inserted to some related algorithms' wiki.

For tester

Pleas read the wiki to see whether the information is clear and sufficient.

comment:9 Changed 7 years ago by Karl Palmen

  • Status changed from verify to verifying
  • Tester set to Karl Palmen

comment:10 Changed 7 years ago by Karl Palmen

  • Status changed from verifying to verify
  • Tester Karl Palmen deleted

I am unable to determine whether the new wikipage addresses the issue. I'll return to testing pool.

comment:11 Changed 7 years ago by Keith Brown

  • Status changed from verify to verifying
  • Tester set to Keith Brown

comment:12 Changed 7 years ago by Keith Brown

  • Status changed from verifying to reopened
  • Resolution fixed deleted

I'm reopening it.

The grammar is awful overall, and that's something coming form me, and that's impacting on the understanding of the reader.

Bullets are used too much in instances where they needn't be; number agreement and pluralisation is non-existent in places; and there are some sentences that don't flow, so badly in fact I don't know which subject a verb is acting on.

Examples of specific instances:

  • The "Generating filter explicitly" heading should be "Generating filters explicitly"

Quote from page:

GenerateEventsFilter is able to generate event filters, are stored in SplittersWorkspace and taken as input property of FilterEvents, by either

  • log value or
  • time.
  • Sentences shouldn't flow through a bulletted list in most cases, and that second level shouldn't be bulletted at all and just flow from "taken as input property of FilterEvents, by either..."
  • "which" is required after the first comma

Quote from page:

User can create SplittersWorkspace from Python script because SplittersWorkspace is indeed inherited from TableWorkspace.

  • 'User' should be 'Users' or 'The user', see 'Python script' for the same number-agreement problem
  • 'indeed' is not required

There are more but I haven't got time to list the rest.

There are also a lot of links to non-existent pages, fix them or create a ticket that will fix them as the amount of red links is rather bad.

comment:13 Changed 7 years ago by Wenduo Zhou

  • Status changed from reopened to verify
  • Resolution set to fixed

Wiki page has been modified according to comments.

comment:14 Changed 7 years ago by Keith Brown

  • Status changed from verify to verifying

comment:15 Changed 7 years ago by Keith Brown

I understand what is being said now. I made a couple more minor tweaks but it's good now

comment:16 Changed 7 years ago by Keith Brown

  • Status changed from verifying to closed

comment:17 Changed 7 years ago by Russell Taylor

There's a branch on github relating to this ticket - https://github.com/mantidproject/mantid/tree/feature/8301_filter_wiki It has not been merged to master - should it be?

comment:18 Changed 7 years ago by Keith Brown

Forgot to do that, yes it should have. I was more focused on getting the grammar up to scratch on the new wiki page and forgot about the branch. I'll do it now.

Luckily it's only Wiki changes in the code.

comment:19 Changed 7 years ago by Keith Brown

or i will when it's checkbuilt

comment:20 Changed 7 years ago by Keith Brown

Merge remote-tracking branch 'origin/feature/8301_filter_wiki'

Full changeset: 5d3eded0db520b1ad3300f731478a8c4b11c5720

comment:21 Changed 7 years ago by Keith Brown

merged now

comment:22 Changed 5 years ago by Stuart Campbell

This ticket has been transferred to github issue 9146

Note: See TracTickets for help on using tickets.