Ticket #9115 (closed: fixed)

Opened 7 years ago

Last modified 5 years ago

Implement Connected Component Analysis in Mantid

Reported by: anonymous Owned by: Owen Arnold
Priority: major Milestone: Release 3.2
Component: Diffraction Keywords:
Cc: Blocked By:
Blocking: #9194 Tester: Martyn Gigg

Description (last modified by Owen Arnold) (diff)

Connected component analysis: This is an exploratory method for peak finding and integration. Theoretically both can be achieved in the same set of steps very rapidly. Arbitrary shaped peaks, in both detector space, and in reciprocal space can be identified.

Initial implementation will use MDHistoWorkspaces, but the intention is to open this up to Workspace2Ds later.

Attachments

neighbour_grid.py (3.0 KB) - added by Owen Arnold 7 years ago.

Change History

comment:1 Changed 7 years ago by Nick Draper

  • Status changed from new to infoneeded

Please put the proposed Algorithm name in the title

comment:2 Changed 7 years ago by Owen Arnold

  • Status changed from infoneeded to new
  • Summary changed from Connected Component Peak Finding and Integration to FindPeaksUsingClusters

comment:3 Changed 7 years ago by Nick Draper

  • Status changed from new to assigned

Changed 7 years ago by Owen Arnold

comment:4 Changed 7 years ago by Owen Arnold

Just attached the results of some preliminary work in python to establish how we can determine nearest touching neighbours in an n-dimensional fashion. This was a useful exercise prior to coding in c++.

comment:5 Changed 7 years ago by Owen Arnold

refs #9155. 1D and 2D neighbour cases verified.

Extensive tests for these. Refactoring still neeeded.

Changeset: 7305ba6a6512f77848e4a5100d654a7c13fbdd55

comment:6 Changed 7 years ago by Owen Arnold

refs #9155. Extend test cases to 3D.

Still need to refactor the working code for this.

Changeset: 8a1fb0f9b83d49c4fcb77f8542dfdae29de63dce

comment:7 Changed 7 years ago by Owen Arnold

refs #9155. Refactor current working.

Avoid all vector resizing possible. Pre-cache as much as possible.

Changeset: 8da661ae9976b0b25b6c5dfda0bc1420f13017cb

comment:8 Changed 7 years ago by Owen Arnold

Ticket references above should be against 9115 not 9155

comment:9 Changed 7 years ago by Owen Arnold

  • Status changed from assigned to inprogress

refs #9115. Implement ConnectedComponentsLabelling with 1d tests.

Implement simple testing for connected component labelling.

Changeset: 7228a39fa6430e7b3545e4c1837d30f52520d179

comment:10 Changed 7 years ago by Owen Arnold

refs #9115. Extensive testing to check that MD clustering is working.

Changeset: 411b8edde83b0726a76df8562fce962bda0312cf

comment:11 Changed 7 years ago by Owen Arnold

refs #9115. Performance tests added.

As of yet, I have paid no attention at all to performance. This will be the subject of future work, but I want to make sure that any performance improvements are actually incremental improvements, and not unmeasured hand-waving type changes. Hence this performance test.

Changeset: cc0453469aa519b4137dbc93b7c69932dad2d92f

comment:12 Changed 7 years ago by Owen Arnold

refs #9115. Rename class and associated files.

Changeset: 7e08cf1c1d8d36fc38b33a7b24ee88cbc47f365e

comment:13 Changed 7 years ago by Owen Arnold

  • Summary changed from FindPeaksUsingClusters to Implement Connected Component Analysis in Mantid
  • Description modified (diff)
  • Blocked By 9194 added
  • Reporter Owen Arnold deleted

comment:14 Changed 7 years ago by Owen Arnold

refs #9115. Warns and errors fixed. Perf tests.

I also added some peformance tests directly over the implementation of findNeighbourIndexes. I noticed that one of the performance tests around masking in the iterators was taking far too long, of the orer of 10s of seconds. So I refactored that peformance test to make the test itself do less work.

Changeset: ef4c1e36dc228b6827f3ad87853890169806fc2f

comment:15 Changed 7 years ago by Owen Arnold

For future improvements. Benchmarks on my win64 debug build are:

  • 1.7 seconds for findNearestNeighbours of IMDHistoWorkspaceTestPerformance
  • 1.86 seconds for ConnectedComponentLabelingTestPeformance

comment:16 Changed 7 years ago by Owen Arnold

refs #9115. Fix issue around auto usage.

Changeset: 1232ae9bf71f6d153004ef5080421052330ddf6d

comment:17 Changed 7 years ago by Owen Arnold

refs #9115. Fix GCC 4.4 errors.

Changeset: 3b71da06c295286c4ff4e28c9cc9c9f8e2f35ca4

comment:18 Changed 7 years ago by Owen Arnold

refs #9115. Fix pointless check warning.

Changeset: fb93911acbe004f1c50586e14c0ecf8a6e705522

comment:19 Changed 7 years ago by Owen Arnold

refs #9115. Fix boost list_of usage.

Changeset: 79145a4c1585acedeb2607f84764e89f8d2fe3e7

comment:20 Changed 7 years ago by Owen Arnold

refs #9115. Add missing tests.

Changeset: 763729618204d6d1581783ebf89c32746938eeed

comment:21 Changed 7 years ago by Owen Arnold

refs #9115. Fix type warnings.

Changeset: 78fae90e875a10073f716546216da1363e7e3a1d

comment:22 Changed 7 years ago by Owen Arnold

Tester: The major ouput of this work is Code/Mantid/Framework/Crystal/test/ConnectedComponentLabelingTest.h which should be used to demonstrate that the functionality is working correctly. The code has not yet been wired-up into algorithm form. That will be the subject of a future ticket to prevent the tickets from getting too big. Please review the changes by eye and check that the relevant jenkins jobs are passing correctly.

comment:23 Changed 7 years ago by Owen Arnold

refs #9115. Fix more type conversion warnings.

Changeset: 16046401f7c9a64c5629642079c610019d07aa4a

comment:24 Changed 7 years ago by Owen Arnold

  • Blocking 9194 added
  • Blocked By 9194 removed

comment:25 Changed 7 years ago by Owen Arnold

  • Status changed from inprogress to verify
  • Resolution set to fixed

comment:26 Changed 7 years ago by Martyn Gigg

  • Status changed from verify to verifying
  • Tester set to Martyn Gigg

comment:27 Changed 7 years ago by Martyn Gigg

The code looks good and a nice use of strategies for the background determination.

All tests are passing.

comment:28 Changed 7 years ago by Martyn Gigg

  • Status changed from verifying to closed

Merge remote-tracking branch 'origin/feature/9115_connected_components'

Full changeset: 2611b7dc48eb65904bc36eb4248131fe6723391f

comment:29 Changed 5 years ago by Stuart Campbell

This ticket has been transferred to github issue 9958

Note: See TracTickets for help on using tickets.