Ticket #9265 (closed: fixed)

Opened 7 years ago

Last modified 5 years ago

Memory leak with Python fit functions

Reported by: Martyn Gigg Owned by: Martyn Gigg
Priority: major Milestone: Release 3.2
Component: Python Keywords:
Cc: Blocked By:
Blocking: Tester: Arturs Bekasovs

Description

From James Lord, ISIS:

I've found that a Python fit function can leak LOTS of memory, especially if used for repetitive fits in a script. Try the attached example which uses either ExpDecayOsc (built-in) or my Python equivalent version. Give names for the 2 output workspaces and choose the Python or built in function. It does 20 fits to a 5000 point data set and it'll leak 300-500 Mbytes! The actual fit results are correct and there's no detectable leak with the built in fit function.

Attachments

LeakyFit.py (3.3 KB) - added by Martyn Gigg 7 years ago.

Change History

Changed 7 years ago by Martyn Gigg

comment:1 Changed 7 years ago by Peter Peterson

  • Status changed from new to assigned

comment:2 Changed 6 years ago by Martyn Gigg

  • Status changed from assigned to inprogress

comment:3 Changed 6 years ago by Martyn Gigg

Fix memory leak in python fit functions.

The result returned from the function1D/functionLocal call needs to have its reference count decremented as the returned array is a new numpy array. Refs #9265

Changeset: fa24a168004122b77305706395cef61404617a7d

comment:4 Changed 6 years ago by Martyn Gigg

  • Status changed from inprogress to verify
  • Resolution set to fixed

Branch: bugfix/9265_python_function_memleak

Tester: The attached python algorithm demonstrates the bug. Before starting testing run the check that you see a memory leak:

  • open a system monitor that allows you to monitor memory usage of the MantidPlot process
  • start MantidPlot & note the amount of memory used
  • run the attached script to register the new algorithm LeakyFit and start this from the algorithms box
  • enter two workspace names & tick the UsePythonFunc box & click run
  • after the algorithm completes go to File->Clear All Memory

Look at the memory usage, it should still be over 500Mb.

Merge the branch, build the code & then run the steps above again and you should see the memory return to the same level as when MantidPlot started.

comment:5 Changed 6 years ago by Arturs Bekasovs

  • Status changed from verify to verifying
  • Tester set to Arturs Bekasovs

comment:6 Changed 6 years ago by Arturs Bekasovs

Reproduced the bug on current stable version. Is leaking only with "UsePythonFunc".

Code changes make sense: PyObject-s reference counts stay correct now, which makes them to be released when the time comes.

The problem appear to be solved with the changes merged. Memory usage is stable at <100mb throughout the fit, with both "UsePythonFunc" enabled and disabled.

comment:7 Changed 6 years ago by Arturs Bekasovs

  • Status changed from verifying to closed

Merge remote-tracking branch 'origin/bugfix/9265_python_function_memleak'

Full changeset: f573dd385a236728cebdf7f7989594685c95bed5

comment:8 Changed 5 years ago by Stuart Campbell

This ticket has been transferred to github issue 10108

Note: See TracTickets for help on using tickets.