Ticket #9244 (closed: fixed)

Opened 7 years ago

Last modified 5 years ago

Improve sort in TimeSeriesProperty

Reported by: Wenduo Zhou Owned by: Wenduo Zhou
Priority: major Milestone: Release 3.2
Component: Framework Keywords:
Cc: Blocked By: #8685
Blocking: Tester: Russell Taylor

Description

TimeSeriesProperty uses stable_sort in method sort. The flag to show whether a series is sorted is set to 'unsorted' if the sorted status is unsorted or unsure. For example, after add() operation, the flag is always set to 'unsorted' because it is unknown whether it is sorted with a new entry.

Sort() uses stable_sort, which is very costly if the vector is sorted while the sorting status is 'unsorted' for being unsure. This problem should be solved.

Attachments

verify9244.py (707 bytes) - added by Wenduo Zhou 7 years ago.

Change History

comment:1 Changed 7 years ago by Wenduo Zhou

Check sorted status before stable_sort. Refs #9244.

Changeset: a11f6a7556c1c376a37fe3562624fd206abec55a

comment:2 Changed 7 years ago by Russell Taylor

  • Status changed from new to assigned

The check should be accomplished using boost::is_sorted rather than hand-crafting.

It's possibly better to handle things on the 'other' side - i.e. make sure the flag is always right.

comment:3 Changed 7 years ago by Wenduo Zhou

  • Status changed from assigned to inprogress

Use boost::is_sorted. Refs #9244.

Changeset: 3c7de6aded860420f398705fecc0dcf015e2cde2

comment:4 Changed 7 years ago by Wenduo Zhou

Enhanced series property sorted status. Refs #9244.

Changeset: f3ca507242ce8bfd6d9b477206e69e027adb1869

comment:5 Changed 7 years ago by Wenduo Zhou

Minor modification on comments. Refs #9244.

Changeset: 3b086d8a7dfdbcb5c49bf137170daa50512a4b13

comment:6 Changed 7 years ago by Wenduo Zhou

Removed included header file. Refs #9244.

Changeset: 64b7da1f886b6e8dfedf0d5875cf42475542d06f

comment:7 Changed 7 years ago by Wenduo Zhou

Used is_sorted by identifying compiler. Refs #9244.

Changeset: 4512b4a0f46e89b2bcc994540bc05f8e8de19d1c

comment:8 Changed 7 years ago by Wenduo Zhou

Fixed osx build. Refs #9244.

Changeset: 5473d4e9334f6b5c5d288e45ae66cf1837156e10

comment:9 Changed 7 years ago by Wenduo Zhou

  • Blocking 8685 removed

comment:10 Changed 7 years ago by Wenduo Zhou

  • Blocked By 8685 added

comment:11 Changed 7 years ago by Wenduo Zhou

For tester

Check the unit test and system test that are not broken due to change in this ticket.

There is a way to see the improvement made by this ticket. Download attached script verify9244.py. Run it with Mantid before merging the change. Compare to the performance of GenerateEventsFilter after merging the change, you can see the difference in execution speed. It is because the new version won't sort a sorted time series in an unknown-sorted-or-not state.

Changed 7 years ago by Wenduo Zhou

comment:12 Changed 6 years ago by Wenduo Zhou

  • Status changed from inprogress to verify
  • Resolution set to fixed

comment:13 Changed 6 years ago by Russell Taylor

  • Status changed from verify to verifying
  • Tester set to Russell Taylor

comment:14 Changed 6 years ago by Wenduo Zhou

Removed some commented out codes. Refs #9244.

Changeset: 87c9d0cb929b863d7329830596c6a7c2987ff1d0

comment:15 Changed 6 years ago by Russell Taylor

  • Status changed from verifying to reopened
  • Resolution fixed deleted

Some issues spotted during code review:

  • The "else if" at line 833 does nothing!!!
  • In TimeSeriesProperty<TYPE>::addValue, you could also switch from UNKNOWN to UNSORTED if the new value is out of sequence
  • Use newvalue or m_values.back() instead of first *m_values.rbegin() at line 840
  • The addValues overload at line 881 would be better calling the first addValue method instead of (partially) doing the work itself.
  • It also needs its indentation fixing.
  • Times are guaranteed to be sorted after calling first TimeSeriesProperty<TYPE>::create method.
  • Could check whether they are in second TimeSeriesProperty<TYPE>::create method. I'm not sure what the performance implications of this would be though.
  • Please remove the outdated comment at line 1182

comment:16 Changed 6 years ago by Wenduo Zhou

  • Status changed from reopened to inprogress

Refs #9244. Made changes according to code review

used m_values.back() instead of first *m_values.rbegin() at line 840; removed the outdated comment at line 1182;

Changeset: 932018364aad820fe619fbd7c8a66737b8cd52a1

comment:17 Changed 6 years ago by Wenduo Zhou

Made change according to code review. Refs #9244.

  1. Add a case in addValue to switch from unknown to unsorted;
  2. Fixed an indentation issue.

Changeset: 1661db24ea904005934efc5728be6ab14eed288e

comment:18 Changed 6 years ago by Wenduo Zhou

Refs #9244. Modification according to code review.

  1. Remove a do-nothing "else if" line 833;
  2. Enable 'create' to check whether the series is sorted;

Changeset: 94bd99adbfa31dbe7bda45656f929f712ab6a140

comment:19 Changed 6 years ago by Wenduo Zhou

  • Status changed from inprogress to verify
  • Resolution set to fixed

comment:20 Changed 6 years ago by Russell Taylor

  • Status changed from verify to verifying

comment:21 Changed 6 years ago by Russell Taylor

  • Status changed from verifying to closed

Merge remote branch 'origin/feature/9244_timeseries_sort'

Full changeset: 82a3ffc3abe0c04395589c8ffbcf040fb3a8682a

comment:22 Changed 5 years ago by Stuart Campbell

This ticket has been transferred to github issue 10087

Note: See TracTickets for help on using tickets.