Ticket #9455 (closed: fixed)

Opened 6 years ago

Last modified 5 years ago

FilterClusterFaces enhancements

Reported by: Owen Arnold Owned by: Owen Arnold
Priority: major Milestone: Release 3.2
Component: Diffraction Keywords:
Cc: Blocked By: #9428
Blocking: Tester: Samuel Jackson

Description (last modified by Owen Arnold) (diff)

  • If FilteringWorkspace is provided, then output an additional column with the radius of the face from the peak centre. This will allow client code to determine an effective radius of each cluster
  • Check that the MDHistoWorkspace provided actually contains labels ids rather than being a generic MDHistoWorkspace. I think the algorithm would probably badly misbehave if the latter case was currently provided. To check. For each signal value perform signal_value%1 == 0.
  • Parallelise the algorithm according to the criteria already defined in the algorithm comments. As an intermediate data structure, TableWorkspaces themselves may work best.
  • Add the ability to limit the number of output rows, and report that truncation has occurred in that eventuality.

Change History

comment:1 Changed 6 years ago by Owen Arnold

For the performance side of things - current timings on my Release Windows 7 build are:

test_execution_unfiltered = 0.015 seconds

test_execution_filtered = 0.018 seconds

Last edited 6 years ago by Owen Arnold (previous) (diff)

comment:2 Changed 6 years ago by Nick Draper

  • Status changed from new to assigned

comment:3 Changed 6 years ago by Owen Arnold

  • Status changed from assigned to inprogress

refs #9455. Create and use intermediate structure

This is easy and safe to do, because unit tests will pick up any changes in the final output TableWorkspace.

Changeset: 989d5e068e0b67a2f97afd8e95fe41bdec151e95

comment:4 Changed 6 years ago by Owen Arnold

refs #9455. Break jobs into distributed parts.

This is not yet executing in the context of an openMP loop, but the working is now fully separated, and should be thread-safe.

Changeset: eb592e4edfd2dc0c512a59791aa5c81d02796b06

comment:5 Changed 6 years ago by Owen Arnold

refs #9455. Run jobs in parallel.

Changeset: 5ba3891cdb71d7cce2dbff1d90bf5cede6a2734b

comment:6 Changed 6 years ago by Owen Arnold

The refactoring to distributable execution step did not slow the processing down by any noticeable amount. I expect that it would slow it down given that the rows could not be written until the end. The last stage where the open mp loop was switched on resulted in a good speed improvement.

test_execution_unfiltered = 0.008 seconds

test_execution_filtered = 0.009 seconds

comment:7 Changed 6 years ago by Owen Arnold

refs #9455. Add progress reporting.

Changeset: 62486340c7210f7f137ca831a3f8a7f2a0c63fc9

comment:8 Changed 6 years ago by Owen Arnold

refs #9455. Add signal value checking.

Add tests for this too.

Changeset: 49a64918a64edc9bbb3241425e1287b62e6a9ebd

comment:9 Changed 6 years ago by Owen Arnold

refs #9455. Add radius calculation.

The code is functionally correct, as the tests show, but the implementation is not good. Needs some refactoring work before this is complete.

Changeset: 98a5b8a27b861b6c82db43568bb263f6a2f0111c

comment:10 Changed 6 years ago by Owen Arnold

refs #9455. Refactor to separate processing.

Refactor to separate out unfiltered from filtered processing. Code duplication still needs to be addressed.

Changeset: 927ae6c47c5904aebf86af83ed853a5e283c021e

comment:11 Changed 6 years ago by Owen Arnold

refs #9455. Refactoring to achieve better code reuse.

Changeset: b0ef8e344b472ab1b2eadc402ae82f30f4ae3f29

comment:12 Changed 6 years ago by Owen Arnold

refs #9455. Add row limit features.

Fully tested functionality provided.

Changeset: 7593279b23fe2b6adfa02ba7a41e7e17200a321b

comment:13 Changed 6 years ago by Owen Arnold

refs #9455. Fix warning.

Changeset: bee839acc5e0d774f3d1ebf2c21e4a84261d7348

comment:14 Changed 6 years ago by Owen Arnold

  • Description modified (diff)

comment:15 Changed 6 years ago by Owen Arnold

refs #9455. Remove qualifier.

Changeset: fe2552e74e9385337e0b1db01b2dae52970f812e

comment:16 Changed 6 years ago by Owen Arnold

refs #9455. Fix type conversion warnings.

Changeset: 6eb03a2453201f32ae176a62e356c89646cf9516

comment:17 Changed 6 years ago by Owen Arnold

refs #9455. All column names uppercase.

Changeset: 9901b25fc3709cba3c927a91e209236efcfb70b6

comment:18 Changed 6 years ago by Owen Arnold

  • Status changed from inprogress to verify
  • Resolution set to fixed

Tester:

There are quite a few changes here. All changes are either covered by functional unit tests or performance tests.

To test the new radius functionality. Run this script. You should see that the main cluster has the same radius extents (in the SliceViewer) as reported at the end of the script.

ws_name = "TOPAZ_3132"
filename = ws_name +"_event.nxs"
ws = LoadEventNexus(Filename=filename,FilterByTofMin=3000, FilterByTofMax=16000)

# Convert to Q space
LabQ = ConvertToDiffractionMDWorkspace(InputWorkspace=ws, LorentzCorrection='0',
        OutputDimensions='Q (lab frame)', SplitInto=2, SplitThreshold=150)
        
# Find peaks
PeaksLattice = FindPeaksMD(InputWorkspace=LabQ,MaxPeaks=100)

        
# Find the UB matrix using the peaks and known lattice parameters
FindUBUsingLatticeParameters(PeaksWorkspace=PeaksLattice, a=10.3522, b=6.0768, c=4.7276,
                alpha=90, beta=90, gamma=90, NumInitial=20, Tolerance=0.12)
                
# And index to HKL            
IndexPeaks(PeaksWorkspace=PeaksLattice, Tolerance=0.12)

HistoMDQLab = BinMD(InputWorkspace=LabQ,AlignedDim0='Q_lab_x, 2.7, 3, 50',AlignedDim1='Q_lab_y, -0.9, -0.6, 50',AlignedDim2='Q_lab_z, 9.2, 9.5,  50')   
PeaksLattice_Integrated_Clusters, ClusterImage = IntegratePeaksUsingClusters(InputWorkspace=HistoMDQLab, PeaksWorkspace=PeaksLattice, Threshold=100000)

# Show Cluster with Peak Overlayed. Only One point where a PeaksWorkspace intersects with a Cluster!
svw = plotSlice(ClusterImage, xydim=['Q_lab_x', 'Q_lab_z'])
sv = svw.getSlicer()
pp_all = sv.setPeaksWorkspaces([PeaksLattice_Integrated_Clusters])
pp = pp_all.getPeaksPresenter(PeaksLattice_Integrated_Clusters)
pp.zoomToPeak(30)
svw.zoomBy(0.03)

# So finding with the PeaksWorkspace as a filter should yield results that have the SAME value for 'ClusterId 'in all rows!
with_filter, truncated = FindClusterFaces(InputWOrkspace=ClusterImage, FilterWorkspace=PeaksLattice)

def max_radius(t):
        max = -0.0
        for i in range(t.rowCount()):
                _row =  t.row(i)
                current_radius = _row['Radius'] 
                if  current_radius > max:
                        max = current_radius
        return max
        

radius_estimate =  max_radius(with_filter)
print "Radius estimate for the cluster is : ", radius_estimate

comment:19 Changed 6 years ago by Samuel Jackson

  • Status changed from verify to verifying
  • Tester set to Samuel Jackson

comment:20 Changed 6 years ago by Owen Arnold

  • Status changed from verifying to closed

Merge remote-tracking branch 'origin/feature/9455_cluster_faces_enhance'

Full changeset: 4f5df39b581350d6b6f931b15fc60608f621fbd1

comment:21 Changed 5 years ago by Stuart Campbell

This ticket has been transferred to github issue 10298

Note: See TracTickets for help on using tickets.