Move tcmalloc to a runtime-only library

[Martyn Gigg]

#7798 prompted an investigation into memory allocation issues while using tcmalloc under Linux. The findings are listed ​here.

It was decided by the ​TSC that tcmalloc would become a library that is only used at runtime and is loaded via the LD_PRELOAD variable within a startup script.

This will ensure that all libraries that are loaded by the program use the same allocation routines. Also, adding the TCMALLOC_RELEASE_RATE will remove the requirement for "Release Free Memory" calls as it will be handled by the library.

[Martyn Gigg]

Remove tcmalloc from list of globally linked libraries.

It will become a library loaded through LD_PRELOAD so we keep the search for it but move tcmalloc_minimal to be found first if it exists. Refs #10271

Changeset: 076399a69163fdcaab182cb2f705192755404049

[Martyn Gigg]

Simplify to process of loading the Python plugins on Linux.

Remove any need to force some libraries to load first. Newer linkers seem to have removed the need for this. Refs #10271

Changeset: afa941f09d461ca46cb46c8713f8762936e0a711

[Martyn Gigg]

Fix library link order for older linkers.

stdc++ is required to be linked first to avoid a segfault when exceptions are thrown and pass into the Python layer.

Refs #10271

Changeset: a58da79b7447f15c69a5573f5c5c5a1d2adcd4f1

[Martyn Gigg]

Separate packaging from more general Linux setup.

Refs #10271

Changeset: 6c8911a1459733c3f858caf292298b79e1e3552a

[Martyn Gigg]

Move maintainer scripts to be in their own files.

Refs #10271

Changeset: 00c7a09f055f29994f0da8893fdd9a9ec5305975

[Martyn Gigg]

Use eval to call MantidPlot when not using scl

Refs #10271

Changeset: 8c4df195d292e186f463192b51f5f7c0f8fbf8fc

[Martyn Gigg]

Merge branch 'master' into feature/10271_tcmalloc_ldpreload

Conflicts:

Code/Mantid/Build/CMake/FindTcmalloc.cmake Code/Mantid/Framework/API/src/MemoryManager.cpp Code/Mantid/Framework/Kernel/CMakeLists.txt Code/Mantid/Framework/Kernel/src/Memory.cpp Code/Mantid/Framework/PythonInterface/mantid/api/CMakeLists.txt Code/Mantid/Framework/PythonInterface/mantid/geometry/CMakeLists.txt Code/Mantid/Framework/PythonInterface/mantid/kernel/CMakeLists.txt

Refs #10271

Changeset: 3b15699e20468a71e9925ce1a55c2de0ab6c0ae5

[Martyn Gigg]

Implement a post install script for Debian

Refs #10271

Changeset: 0b53a3afde093c9c0da11ee7e1851ef7d7363482

[Martyn Gigg]

Add profile links if requested in post install

Refs #10271

Changeset: fa84dd348ed17041d829521fd8408be201e1bbbf

[Martyn Gigg]

Add Debian pre & post install scripts

Refs #10271

Changeset: 91db9e83e78d9b20eb080ce2d1db65e0e29d4ec0

[Martyn Gigg]

Only remove profile scripts if they were installed by us in RPM

Refs #10271

Changeset: ef214aeb32ff6a7bd6e72049895af7d60cfe65da

[Martyn Gigg]

Use RPM_INSTALL_PREFIX0 directly in rpm maintainer scripts

Refs #10271

Changeset: 77055060698bb109db4dac3817541517c7f8b04b

[Martyn Gigg]

Fix install prefix for debian post install script

Refs #10271

Changeset: b31f758c3ec4254e8f5c831281e1917db12c0e31

[Martyn Gigg]

Don't wipe LD_PRELOAD in startup scripts.

Refs #10271

Changeset: 35206479cda0aecbd6940f6e194aa63b70426c0a

[Martyn Gigg]

Add LD_PRELOAD/TCMALLOC_RELEASE_RATE to MantidPlot launch script

Refs #10271

Changeset: a89897786407fd50a77704e18cc05982b1bb6f1c

[Martyn Gigg]

Add LD_PRELOAD/TCMALLOC_RELEASE to profile scripts

Refs #10271

Changeset: b67ee4ee835574216559916a3eda1583e339d547

[Martyn Gigg]

Merge branch 'master' into feature/10271_tcmalloc_ldpreload

Conflicts:

Code/Mantid/Framework/Kernel/CMakeLists.txt

Refs #10271

Changeset: 19552b9bf21a44d4786052c374053bdbf74749af

[Martyn Gigg]

This is being verified as ​pull request #81.

[Martyn Gigg]

Fix install path for MantidPlot launcher script

Refs #10271

Changeset: 50ddf1f6f03db576dee8c825e01649b80f060ffa

[Martyn Gigg]

Remove LD_PRELOAD from profile scripts.

Tcmalloc causes emacs to crash so don't pollute the standard environment with it. Refs #10271

Changeset: b6cf1acc88c4686374b44a64b1899196c4bd92ab

[Stuart Campbell]

Which kits are the best to use for testing ? The latest unstable/develop with this change in, or a kit built from the pull request ?

[Martyn Gigg]

I've added package archiving to the pull request builds so it would be good to try those first. I've just merged to develop so those would be a good fall back.

[Martyn Gigg]

retest this please

[Martyn Gigg]

I think the current failures of the doctests are due to the launcher script. Don't merge this yet

[Martyn Gigg]

Fix silly error in launcher script.

It was calling the script recursively and running out of file descriptors Refs #10271

Changeset: ecd9f8f842948743d25cd67e0773d30294fc965b

[Martyn Gigg]

Fix launch_mantidplot script when not run under bash

dirname is a bash function and is not available in all shells. Refs #10271

Changeset: dd80a6e02e4a7f3b8f5917c33f62d14907dfbc6d

[Martyn Gigg]

Fix install of launcher script.

Refs #10271

Changeset: 73c8303462ea533891584db1410e4f963cdb3e9b

[Martyn Gigg]

Fix quotes in launcher script

Refs #10271

Changeset: 0672fd9c13afac4768530a0a81f4cd745dd75358

[Martyn Gigg]

More quote fixes after moving from CMake string

Refs #10271

Changeset: c0c7063c1242f9229d675e28356634535227ca49

[Martyn Gigg]

Fix path to MantidPlot exe for squish

Refs #10271

Changeset: b4bf7f03f27d609ad59b4c8fec250fc6963d8c31

[Martyn Gigg]

retest this please

[Martyn Gigg]

The build servers are happy at long last!

[Stuart Campbell]

@martyngigg Can you merge master back in to get rid of the merge problems ?

[Martyn Gigg]

Merge branch 'master' into feature/10271_tcmalloc_ldpreload

Conflicts:

Code/Mantid/Framework/CMakeLists.txt

Refs #10271

Changeset: 8af22ffdd98ef4e1d3f253d1dcac87ee26312665

[Stuart Campbell]

I had some issues with the packages, as in path conflicts and similar - but these should be really tested on later kits (although it looks like the archiving of packages is not working for the pull request jobs)

Otherwise, the preloading of tcmalloc seems to work and MantidPlot is not linked against it.

[Martyn Gigg]

Merge branch 'master' into feature/10271_tcmalloc_ldpreload

Conflicts:

Code/Mantid/Build/CMake/FindTcmalloc.cmake Code/Mantid/Framework/API/src/MemoryManager.cpp Code/Mantid/Framework/Kernel/CMakeLists.txt Code/Mantid/Framework/Kernel/src/Memory.cpp Code/Mantid/Framework/PythonInterface/mantid/api/CMakeLists.txt Code/Mantid/Framework/PythonInterface/mantid/geometry/CMakeLists.txt Code/Mantid/Framework/PythonInterface/mantid/kernel/CMakeLists.txt

Refs #10271

Full changeset: ​3b15699e20468a71e9925ce1a55c2de0ab6c0ae5

[Martyn Gigg]

Merge branch 'master' into feature/10271_tcmalloc_ldpreload

Conflicts:

Code/Mantid/Framework/Kernel/CMakeLists.txt

Refs #10271

Full changeset: ​19552b9bf21a44d4786052c374053bdbf74749af

[Martyn Gigg]

Merge branch 'master' into feature/10271_tcmalloc_ldpreload

Conflicts:

Code/Mantid/Framework/CMakeLists.txt

Refs #10271

Full changeset: ​8af22ffdd98ef4e1d3f253d1dcac87ee26312665

[Stuart Campbell]

Merge pull request #81 from mantidproject/feature/10271_tcmalloc_ldpreload

Use tcmalloc via ld_preload

Full changeset: ​b04d8438bd8a74c88c5595938de317af7da74107

[Stuart Campbell]

This ticket has been transferred to github issue ​11113