Windows: Deleting large Workspace2Ds is very slow. Seen on MERLIN, WISH

[Martyn Gigg]

whitebeamfile="MER06398"
LoadRaw(Filename=whitebeamfile,OutputWorkspace="wb_wksp",LoadLogFiles="0")
white_ws = ConvertUnits(InputWorkspace="wb_wksp",OutputWorkspace="wb_wksp", Target= "Energy", AlignBins=0)

Running the above script on Windows produces a workspace with ~70000 spectra and 2637 bins. The ConvertUnits call causes the X vectors to "unshare" and so that spectrum has a unique X vector. Hitting delete on the workspace locks Mantid while it deletes.

[Martyn Gigg]

Put Workspace2D spectra delete in OpenMP loop.

On MSVC this improves the time taken to delete large workspaces that have had their memory allocated on multiple threads, i.e. running ConvertUnits and breaking the X-data sharing. For example, a workspace with ~70000 spectra & ~2600 bins (~4.3Gb) takes ~45 seconds to delete after a ConvertUnits call. This drops to 1-2s if the deallocation of the memory also happens within a multi-threaded loop. Refs #8216

Changeset: e81d0d89be3d0b5697b5e2937cd30350b3e2f471

[Martyn Gigg]

Use DeleteWorkspace algorithm behind GUI delete button.

This then won't block on the main thread if the deletion takes a bit of time. Refs #8216

Changeset: 63296e3cc9ff4ff04caf4c3d351dac1dc264db3f

[Martyn Gigg]

Define a set of GlobalFixtures for the unittests.

Currently these clear the AnalysisDataService & AlgorithmManager when their tearDownWorld() methods are called, which ensures that all workspaces placed in these singletons are cleared up before the end of main.

If this it not done then the process hangs on MSVC while trying to delete a Workspace2D, which contains an OpenMP loop in its destructor and omp is not allowed to be called on Windows outside of main.

Refs #8216

Changeset: f564331e9738a97cb113d522c0e665e929abaf7e

[Martyn Gigg]

Add comments for Workspace2D destructor & test tearDowns

Refs #8216

Changeset: 015ca434cd7ce2380be5b73c543685c7103dc9d2

[Martyn Gigg]

Add the PropertyManagerDataService to the TearDownWorld test helper.

Refs #8216

Changeset: f0a3080f0c4261b665a28d180837d0f96def0c6c

[Martyn Gigg]

Formatting only changes to remove tabs.

Refs #8216

Changeset: 69e528aba108598fca2edc11a7fbf8e4c693d677

[Martyn Gigg]

Empty out the DataService objects before python exits.

The instance methods are tracked using a reference count. When the count reaches zero then it is assumed that the final object is being deleted and the service is cleared. Refs #8216

Changeset: 9ed8f06b0487d886b2832e2431b4c4429ddd4a27

[Martyn Gigg]

Add required includes for non-precompiled header platforms.

Refs #8216

Changeset: 184995f2a31438030c41245708516c9ee9ae9979

[Martyn Gigg]

Add required namespace using statement.

Refs #8216

Changeset: 041595bfb6172c0a83ebd7545c88541436bfe3f6

[Martyn Gigg]

Remove unecessary typename statement from non-template scope.

Refs #8216

Changeset: 83c16b067ef932455804efec326df8f2ef6fd1da

[Martyn Gigg]

Only run the Workspace2D delete in parallel on MSVC.

It looks as if some performance tests on Linux have suffered slightly from the running the delete in parallel there too. Refs #8216

Changeset: 5b5be7ccd1e345274ed72e3a542f37cc181e371a

[Martyn Gigg]

Branch: feature/8216_slow_ws_del_win

Tester: This bug only affected windows so it needs to be tested there. From the ​system test timings you can now see that the MERLIN reduction takes about 2 minutes, whereas it used to take 11 minutes on the test machine.

Run the script defined in the description and delete the workspace. It should now only take moments rather than minutes.

The only change actually required to improve the performance was the OpenMP loop in the Workspace2D destructor. The other changes were required so that no workspaces would be deleted after the main() function has exited, i.e in the singleton destruction.

[Alex Buts]

Merge remote-tracking branch 'origin/feature/8216_slow_ws_del_win'

Full changeset: ​079e1adf1d2b26cbd3d1206868bddaf0b2451a14

[Alex Buts]

Large reduction under win have accelerated ~twofold so its a big gain.

It is still substantially slower then on equivalent Unix machine so there are possibilities for improvement.

Do not properly understand changes to python interface so have to trust the developer on that.

[Stuart Campbell]

This ticket has been transferred to github issue ​9061