Refactor FindPeaks with FitPeak

[Wenduo Zhou]

In FindPeaks, replace the part of code to fit peaks by algorithm FindPeak, which is implemented in ticket #7788.

Speed issue should be considered. Before the ticket is closed, a comparison on the computation time used by CalbirateRectangularDetector between original and new version should be made.

[Wenduo Zhou]

Some comments. Refs #7789.

Changeset: cfe95a2b89be441ca1bfa35ae18a4f742313726c

[Wenduo Zhou]

Checkpointing the work. Refs #7789.

Changeset: d0412d932ce48ea22574f3f78486b66dcd26cdbf

[Wenduo Zhou]

Added some more debug output. Refs #7789.

Changeset: af6084df06cfba60d173a19093b6a5d2e61402a4

[Wenduo Zhou]

Added one output property. Refs #7789.

Changeset: e96e5ecce2321b398bc07c1015f1ff1b6c1e5b90

[Wenduo Zhou]

Completed most of the refactoring work. Refs #7789.

Changeset: 9ad0c60435ea42326610b557b288b7310d6611d4

[Wenduo Zhou]

Added 1 more unit test. Refs #7789.

Changeset: bbe9741e342f5a5327afe77d5e933e9c7e4758e8

[Wenduo Zhou]

Fixed some bugs in cpp and unit test. Refs #7789.

Changeset: 4273cd3c615fb0606b719520cb5f66115e12d8dd

[Wenduo Zhou]

Improved error output message. Refs #7789.

Changeset: b7bb8bf538fcf8c2b4a0576c3a00f9c1d88f9993

[Wenduo Zhou]

Fixed bugs in code and unit test. Refs #7789.

Changeset: 8cf30f65ee58af5bbfa20abb91a4999be313346a

[Wenduo Zhou]

Added one more column in output to indicate fit quality. Refs #7789.

Changeset: 6448b86e98b1d110f43e4fb5d0f694a4d2e59b9c

[Wenduo Zhou]

Corrected some error during unit test. Refs #7789.

Changeset: e40d855378a5d1b617e5a38ba1cb9f7484fa3211

[Wenduo Zhou]

Corrected some errors. Refs #7789.

Changeset: ce70feb7c5ba6545175276f0771b40841f95de2f

[Wenduo Zhou]

Some comments. Refs #7789.

Changeset: bf2765a56f461d4da63adb1ee4caaf927783f1ed

[Wenduo Zhou]

Checkpointing the work. Refs #7789.

Changeset: 27e717369bbe5210b727dd833c99ed35cf5b2561

[Wenduo Zhou]

Added some more debug output. Refs #7789.

Changeset: f1578a2f32e4a700af3da24e27930a6def31c06c

[Wenduo Zhou]

Completed most of the refactoring work. Refs #7789.

Changeset: 89d5176943aeedaabbcb9ba629904f43489ed7e5

[Wenduo Zhou]

Added 1 more unit test. Refs #7789.

Changeset: b09a78107912c2368ec766205fa58564bbd058cc

[Wenduo Zhou]

Fixed some bugs in cpp and unit test. Refs #7789.

Changeset: b3f8084ce851798fc11b58484a4ee328c628404c

[Wenduo Zhou]

Improved error output message. Refs #7789.

Changeset: f579b6820df637200285e6f54b905321e4947203

[Wenduo Zhou]

Fixed bugs in code and unit test. Refs #7789.

Changeset: c3ab979cd25a5703264942cc83bc24c3af26dfb0

[Wenduo Zhou]

Added one more column in output to indicate fit quality. Refs #7789.

Changeset: a3df418e34c361076ae4c8f56e0595993f1451f0

[Wenduo Zhou]

Corrected some error during unit test. Refs #7789.

Changeset: f3cc8fd27c72140311292771e2feaf382c65f1a8

[Wenduo Zhou]

Corrected some errors. Refs #7789.

Changeset: dd1ec31e3eacb4884a7a2776b5b0facce6abc475

[Wenduo Zhou]

Cleaned code. Refs #7789.

Changeset: 993b8c3a27c56cf379ab1f74f7ed2229f87f8780

[Wenduo Zhou]

Added some more error message. Refs #7789.

Changeset: eabd1e8e6ae1da1038dd2641a1cdc9fe01183aaf

[Wenduo Zhou]

Checkpointing work. Refs #7789.

Changeset: a79cab0f32e0de90e614e3b88bba1e6b6ec0126b

[Wenduo Zhou]

Enabled FitPeak to have a light-weight output. Refs #7789.

Changeset: fa7053d88082113927c36d874a2f25796924533c

[Nick Draper]

The title of this ticket is very similar to #8646 is it a duplicate?

[Wenduo Zhou]

Replying to Nick Draper:

The title of this ticket is very similar to #8646 is it a duplicate?

These two tickets are different though somehow related. In ticket #7789, the section of codes in FindPeaks to fit a single peak will be replaced by calling algorithm FitPeak.

Some tests, which I did in order to close this ticket, show that calling FitPeak as a sub algorithm is very costly. And it slowed down some algorithm, such as GetDetOffsetsMultiPeaks that calls FindPeaks thousands of times, significantly. So I created ticket #8646 to implement a class to fit peak. And this class is used by FitPeak and will be used by FindPeaks in order to avoid the overhead to start sub algorithms.

[Nick Draper]

Moved to backlog at the end of Release 3.1

[Wenduo Zhou]

Some comments. Refs #7789.

Changeset: 407d006c74d7485f234ea24116deb3fba805d5a1

[Wenduo Zhou]

Checkpointing the work. Refs #7789.

Changeset: 4dfc206e6fd224e37908a4b746c95328d89c46d4

[Wenduo Zhou]

Added some more debug output. Refs #7789.

Changeset: fb653c3155c256a62e209a28f3d93e0b879ad1b2

[Wenduo Zhou]

Completed most of the refactoring work. Refs #7789.

Changeset: e01a53fffbd979a5e082408be6e4472ee6a28b88

[Wenduo Zhou]

Added 1 more unit test. Refs #7789.

Changeset: e6c092aa459b0655c28703acffcc5cba283be3c8

[Wenduo Zhou]

Fixed some bugs in cpp and unit test. Refs #7789.

Changeset: 9bb9b5680f42d3422a059dcfe85af7ff222de779

[Wenduo Zhou]

Improved error output message. Refs #7789.

Changeset: 95e88f551b45de90168828b36ace4f151b4ae44b

[Wenduo Zhou]

Fixed bugs in code and unit test. Refs #7789.

Changeset: 199169618abde996d784fed2fdaca5bbc29caa01

[Wenduo Zhou]

Added one more column in output to indicate fit quality. Refs #7789.

Changeset: 5644b762f0adf187dca833df631acf2889eb0279

[Wenduo Zhou]

Corrected some error during unit test. Refs #7789.

Changeset: cd1f2c999245450ff33398b3163f48a809329bfb

[Wenduo Zhou]

Cleaned code. Refs #7789.

Changeset: 53bedb337d9769a8b18fbf15f9a664c65a0fff88

[Wenduo Zhou]

Added some more error message. Refs #7789.

Changeset: 94192c6ad455df76bb70116e613ecd31aabe60d4

[Wenduo Zhou]

Checkpointing work. Refs #7789.

Changeset: 7ed7c55992106e5a536ff48a0323208b7cefdc9f

[Wenduo Zhou]

Enabled FitPeak to have a light-weight output. Refs #7789.

Changeset: 2ffe30ffe21b0885ee487a43cc9748dde7fd86c6

[Wenduo Zhou]

Resolved some conflict during rebase. Refs #7789.

Changeset: 4ba164ee604f4c2ab746475c8269303d13117040

[Wenduo Zhou]

Made FindPeaks to call a FitOneSinglePeak. Refs #7789.

Changeset: b4174c59a984244a4793388f6f478d0c7080ffd7

[Wenduo Zhou]

Removed a deleted property. Refs #7789.

Changeset: 4fa14a7885c88b3ea54551bee2060d6e36264647

[Wenduo Zhou]

Enabled cost function parsing. Refs #7789.

Changeset: f2b919fc9256c6b1bed932e0905d97083e8ea91e

[Wenduo Zhou]

Resolved conflict. Refs #7789.

Changeset: 4b8468c235956036f02c2d3212ce85b607a2c761

[Wenduo Zhou]

Fixed an error from rebase. Refs #7789.

Changeset: e6d4c98c4bc420bc37437378464437292a7ea163

[Wenduo Zhou]

Added a status monitor parameter. Refs #7789.

Changeset: f049a5b208112797ca53a8ffda85133c61c71806

[Wenduo Zhou]

Removed (commented out) log output for speeding. Refs #7789.

Changeset: a37a1ac4ca856e5857fd13a1e669a8d6744eec98

[Wenduo Zhou]

Fixed log and unit test. Refs #7789.

Changeset: 3e57561b4412871dd72c19e2efca90c457c7db13

[Wenduo Zhou]

Fixed unit test. Refs #7789.

Changeset: a597859004e8f65dd3d2a6df5a28e276d65955f1

[Wenduo Zhou]

Added detailed error message. Refs #7789.

Changeset: 323643afdbe8a1c657568b5d4d5b0a1928e85a5a

[Wenduo Zhou]

Modified 2 log output to information. Refs #7789.

The reason to change them from warning to information is that GetDetOffsetsMultiPeak() will call Fit() millions of time in a typical calibration. There is no way to hide warning log in MantidPlot.

Changeset: badbbec283c9bba082e29b1fb7eb32852d254b5b

[Wenduo Zhou]

Made a series minor changes including ... . Refs #7789.

1. Corrected discrpencies in cost function name;
2. Refactored logging in class FitOneSinglePeak;
3. Added debug output

Changeset: 6fda58896cd55e2ee524dc20f758fae306405b5a

[Wenduo Zhou]

Added one more column in output table workspace. Refs #7789.

Changeset: 0c56253689f934315d5c3cfb746a1df4ab9be7be

[Wenduo Zhou]

Replaced code to fit peak by FitOneSinglePeak. Refs #7789.

Changeset: fef02f49fc0f248a3955af10ff7b491cd5bb4bd0

[Wenduo Zhou]

Fixed an error caused by change in FindPeaksBackground. Refs #7789.

Changeset: 906573c6895af396ae92fd796399032e12f40181

[Wenduo Zhou]

Modified to fix unit test failure of StripPeaks. Refs #7789.

Changeset: 3dcd619a874a96ed09c4c7810cc4de4264942d8f

[Wenduo Zhou]

Cleaned the code in unit test. Refs #7789.

Changeset: 725cef8c66f4c134540413638c8f792744cdc330

[Wenduo Zhou]

Modifed due to change in output of FindPeaks. Refs #7789.

Changeset: 952d3fa034b76082308f22b3014d308bddda65f1

[Wenduo Zhou]

Changed unit test for better performance. Refs #7789.

Changeset: 09c41b90ceb19b2fc7c66e145584afb7160397ce

[Wenduo Zhou]

Added more detailed error message to track bug. Refs #7789.

Changeset: 355f5e0581cf9a0f6ae80dfd7d4c422933480789

[Wenduo Zhou]

New golden file due to change in FindPeaks. Refs #7789.

Changeset: cb4b73726dae62b2036d66c6401fa63b39667410

[Wenduo Zhou]

Added a check to find out occasional OSX error. Refs #7789.

Changeset: bc56ba0a1b248d4e0f083c78a59fb90db7913a6e

[Martyn Gigg]

There is a new cppcheck warning in ​StripPeaks

[Martyn Gigg]

The windows debug build is currently failing due to a problem in FindPeaks. I've run it through the debugger and it looks like the m_numTableParams variable is not set in the void FindPeaks::generateOutputPeakParameterTable() method for the case where you have a valid m_rawPeaksTable already. It looks to me as if you just need an extra line

m_numTableParams = numpeakpars + numbkgdpars

on line 310 of FindPeaks.

I'm not sure why none of the other platforms have shown this but it's usually windows debug that is the most picky. This is also a case where it would be more consistent if member variables were initialized in the constructor.

[Wenduo Zhou]

Fixed a bug. Refs #7789.

Changeset: cb2a91b824eb9995e8c6936e6df94429584c5f17

[Wenduo Zhou]

Removed the doxygen warnings. Refs #7789.

Changeset: 2c15ab313a6b22c05442ce8e25439d21a2f1d677

[Wenduo Zhou]

Fixed some doxygen warnings. Refs #7789.

Changeset: 84d06e727b3270cc2f2f98110c83a650202377c0

[Wenduo Zhou]

Fixed a cppcheck warning. Refs #7789.

Changeset: 5208fc02a17f85ce12df715fc3d1dd4d854f169f

[Wenduo Zhou]

Fixed a doxygen error. Refs #7789.

Changeset: 1f4b3d378d7f61c4e3bb863b81184a87b3563af3

[Wenduo Zhou]

Removed unused codes. Refs #7789.

Changeset: 671a4cba452d9c2a41b3b9a7df0e2237b84fcc5f

[Wenduo Zhou]

Cleaned some logging. Refs #7789.

Changeset: 0892c381ee4c5661a4a60dcced32ec036a6115fe

[Wenduo Zhou]

Removed some unused declarations of methods. Refs #7789.

Changeset: 8510507ec58d5a2aece3ef503caf00392c9de392

[Wenduo Zhou]

Corrected an error. Refs #7789.

Changeset: b08d3306326634b3698ba9c86979ca3ba71d1712

[Wenduo Zhou]

Fixed some error in setup of FWHM. Refs #7789.

Changeset: 0ccffe219fb148779d45dfc648055129ac7fa4d9

[Wenduo Zhou]

Modifed cost function. Refs #7789.

Changeset: dcd1722bc6fed4cb819b7c23a221897a5466131d

[Wenduo Zhou]

Modified cost function. Refs #7789.

Changeset: 05b1f5a24b7e8d4b229d4702801ee3dd59d1ad19

[Wenduo Zhou]

Fixed a wiki error. Refs #7789.

Changeset: 30d1a433358d803a62d1d0ab17093dbbcfe27d76

[Wenduo Zhou]

Updated reference file with minor changes. Refs #7789.

Changeset: d83c431ae65dc6f0c1873689c826c2e0a4e7e4f8

[Wenduo Zhou]

Updated reference file with minor change. Refs #7789.

Changeset: 5a1b97c20609d057142a58f6be5160e336627025

[Russell Taylor]

There's one doxygen warning left:

GetDetOffsetsMultiPeaks.cpp:724, Doxygen, Priority: Normal

The following parameters of Mantid::Algorithms::GetDetOffsetsMultiPeaks::fitSpectra(const int64_t wi, API::MatrixWorkspace_sptr inputW, const std::vector< double > &peakPositions, const std::vector< double > &fitWindows, size_t &nparams, double &minD, double &maxD, std::vector< double > &peakPosToFit, std::vector< double > &peakPosFitted, std::vector< double > &chisq, std::vector< double > &peakHeights, int &i_highestpeak) are not documented: parameter 'peakHeights'

[Wenduo Zhou]

Fixed a doxygen error. Refs #7789.

Changeset: aed01c6a9b248115957281f9c54e651dd261b2f6

[Russell Taylor]

FYI: The FindPeaks ​performance test has doubled in execution time following changes in this ticket.

[Wenduo Zhou]

Deleted a debug output. Refs #7789.

Changeset: 324a4f7eb02909e7516137a7af8a840775117c9a

[Wenduo Zhou]

Modified some logging. Refs #7789.

Changeset: aa08dfb17440b0dac79d617cb540184a13729b5a

[Wenduo Zhou]

Refs #7789. Fixed an error about best result.

And also modified for better logging.

Changeset: da43d18e8959f72bcfe07e09d64e734efc8f50d1

[Wenduo Zhou]

Refs #7789. Enhanced for low signal noise ratio peak.

1. Add option to use given peak position other than use observed peak

centre.

2. If FindPeaksBackground fails, then use some heuristic algorithm to

give a peak range for fitting background.

Changeset: 8d3c683dcfe106f41d732bc8133762fc4444ac83

[Wenduo Zhou]

Diabled some failed unit tests temporarily. Refs #7789.

Changeset: 4831bd3ee22c14c8b01ddd26f228e6b12f0c6c68

[Wenduo Zhou]

Refs #7789. Refactored methods for estimation.

Refactored the methods to estimate peak background, peak range and peak parameters.

Changeset: 453963abfe74f578979b688b248b76ed029c6ace

[Wenduo Zhou]

Fixed a typo but critical. Refs #7789.

Changeset: a254ab409a18f23702e1b415c6df2fc9bb203417

[Wenduo Zhou]

Adapted to new UI. Refs #7789.

Changeset: 2ec6cad3dc8db405208868a093c2efc0dd47d3c2

[Wenduo Zhou]

Adapted to new UI. Refs #7789.

Changeset: 59d48781d26bf9e7de5f440a38b19f4ac564f28f

[Wenduo Zhou]

Turned back on previously failed unit tests. Refs #7789.

Changeset: 5c7cf2da735ab27f23800f8395e13805b16ac784

[Wenduo Zhou]

Resolved conflict. Refs #7789.

Changeset: 4909f11dd80a4fd0854135faac34d723e212a344

[Wenduo Zhou]

Cleaned up the code. Refs #7789.

1. Optimized the logging;
2. Cleaned out some unused code;
3. Modified a method's argument.

Changeset: 2851224acfee4f5031c31ca6b08befbe72a25633

[Wenduo Zhou]

Modified reference due to change of FindPeaks. Refs #7789.

Changeset: c55053ae498e31449629908e2381e5daf9149208

[Wenduo Zhou]

For tester

There are huge amount of changes in this ticket including FitPeak, FindPeak, StripPeaks and GetDetOffsetsMultiPeaks. Though the core purpose of this ticket is just to replace the peak fitting section in FindPeaks by FitPeak. The algorithm in FitPeak is not exactly the same as what was in FindPeaks. Therefore, the fitting results are somehow different. Some more changes are made in order to make the new FindPeaks works better or at least no worse than the old version.

Here is what I suggest that a tester can do

1. Check all the unit tests and system tests related. The algorithms call FindPeaks include StripPeaks, StripVanadiumPeaks, GetDetOffsetsMultiPeaks and FindPeaksMD.

2. Compare the reference GSAS file PG3_4866_reference.gsa in system test before and after the change in this ticket. This file is for testing StripVanadiumPeaks, which calls FindPeaks underneath. The new FindPeaks avoids fit the peak to background when the data is noisy;

3. Compare the calibrated diamond data. You can find the data in system test (CalibrateRectangleDetectorsTest). The new one is better.

[Wenduo Zhou]

New reference files. Refs #7789.

Changeset: ab96eb456c53fb9c350252b1bc84376bf8f4b354

[Wenduo Zhou]

Cleaned the codes and skip a method temporarily. Refs #7789.

Changeset: c9b70b2c04e55160c83a90b904646f9b64be9c99

[Alex Buts]

refs #7789 renamed file

Changeset: 9859c64aaea3d9b96a0101f81052864af2bda65b

[Alex Buts]

refs #7789 Conflicts with develop

Merge branch 'feature/7789_fitpeak_to_findpeaks_new' into develop

Conflicts:

.gitattributes Code/Mantid/Framework/Algorithms/src/FindPeaks.cpp

Changeset: a16adebec1cab426d968adcba90a43b2aefcaeec

[Alex Buts]

refs #7789 Correct SANS2D_definition_Tubes case

fixing errod from merging with master.

Changeset: 6ff24cfbc41148d9b1cbe2d1f364fa97e8494dbb

[Alex Buts]

Its complex ticket refactoring the code, so assuringly not changing its functionality, but I see number of unit tests which check the previous functionality disabled.

Looking at range and amount of changes, I do not have confidence to confirm the ticket fixed.

[Wenduo Zhou]

Turned on unit tests. Refs #7789.

Changeset: 36ce6a6b329fb1120c09cce90b3a7abd67d143cc

[Wenduo Zhou]

Refs #7789. Added single peak offset as a criteria.

If an individual peak's offset on position exceeds the maximum offset, then it will be deleted from the peaks for calculating offset.

Changeset: 95ba621424b3ea40465132b6ff01bc324f64b889

[Wenduo Zhou]

Refs #7789. Refine auto peak fit range.

Rename a parameter.

Changeset: d6d74166691696a23d1b1bde6a009809eb13f11c

[Wenduo Zhou]

Added Zscore on each peak's offset. Refs #7789.

Changeset: 842ab840e615348c7a220bc9e56d96ab28d7e742

[Wenduo Zhou]

Fixed a doxygen error. Refs #7789.

Changeset: 088fe90038d904c941b69be4f8abf5e67dd358c4

[Wenduo Zhou]

New golden file for PG3Calibration. Refs #7789.

Changeset: 513c6379f4cf5c97d19572c902f48870018d9535

[Peter Peterson]

The various tests I did (f20) worked. I'm in favor of this one.

[Vickie Lynch]

Merge remote branch 'origin/feature/7789_fitpeak_to_findpeaks_new'

Full changeset: ​14526efc76835a99bc74a470389f5eef836205e3

[Vickie Lynch]

Was going to ask you to remove MultiDomainCreator warnings, but Pete says that is a ticket for Roman. Calibrations and profile fitting for cylindrical integration look great.

[Vickie Lynch]

Merge remote branch 'origin/feature/7789_fitpeak_to_findpeaks_new'

Full changeset: ​5f6be2dac96abd7aef6f9085b049aa4f8a8a4a83

[Stuart Campbell]

This ticket has been transferred to github issue ​8634