Ticket #5760 (closed: fixed)
Windows: Upgrade shipped numpy version
Reported by: | Martyn Gigg | Owned by: | Martyn Gigg |
---|---|---|---|
Priority: | blocker | Milestone: | Release 2.3 |
Component: | Mantid | Keywords: | |
Cc: | Blocked By: | ||
Blocking: | Tester: | Andrei Savici |
Description
Make sure we don't lag too far behind the current version.
Change History
comment:3 Changed 8 years ago by Martyn Gigg
Remove numpy 1.5.1. Refs #5760
Ensure all old files have been removed before updating.
Changeset: ce754ad12abfa5a3baef3abe718f762d88d596c5
comment:4 Changed 8 years ago by Martyn Gigg
Upgrade numpy to 1.6.2 Refs #5760
Built with Visual Studio 10 and includes debug libraries.
Changeset: 74f570c4b9336fc183585d3683932b6f95a71e59
comment:5 Changed 8 years ago by Martyn Gigg
Add numpy f2py scripts. Refs #5760
Changeset: 2aed79497d406e80bca283fbe56199fe451a0f78
comment:6 Changed 8 years ago by Martyn Gigg
Add patched msvccompiler (+patch) to build with MSVC 2010. Refs #5760
Changeset: 7e3f3ef142e23545f259b9134508c45ca532843d
comment:7 Changed 8 years ago by Martyn Gigg
Add batch file to build numpy with MSVC. Refs #5760
Changeset: 083ae98b325325cd7edb9bf0fd1a78c8c2fc3d46
comment:8 Changed 8 years ago by Martyn Gigg
Remove numpy 1.5.1 Refs #5760
Changeset: 69e5b5d2d380fdee8cb317087205d4ea7b464510
comment:9 Changed 8 years ago by Martyn Gigg
Add ctypes debug module for Python. Refs #5760
It can be useful to have it in debug.
Changeset: 6a4f96cd83edbb54ef84de8c84ebf51374f3725e
comment:10 Changed 8 years ago by Martyn Gigg
Add f2py scripts. Refs #5760
Changeset: 01cc66c1a74157e6406baedef62e6fc79caf5149
comment:11 Changed 8 years ago by Martyn Gigg
Update precompiled Inelastic fortran to use numpy 1.6.2. Refs #5760
Changeset: 94db7996dd6f1d6b09b2f9484f69995bc55347cf
comment:12 Changed 8 years ago by Martyn Gigg
Force recompile of numpy pyc modules. Refs #5760
First import of old ones seem to mark them as changed.
Changeset: 6e87605f4e98686ee64f420e8c420dca287bc404
comment:13 Changed 8 years ago by Martyn Gigg
Add scipy-0.11. Refs #5760
Changeset: 99d23ee44553565a16f5220f912eb3e5294d06db
comment:14 Changed 8 years ago by Martyn Gigg
- Status changed from accepted to verify
- Resolution set to fixed
comment:15 Changed 8 years ago by Nick Draper
- Status changed from verify to verifying
- Tester set to Nick Draper
comment:16 Changed 8 years ago by Nick Draper
NumPy Tests
NumPy Version - Passed
import numpy as nm print nm.version.version 1.6.2
NumPy array - Passed
from numpy import * a= arange(12) a=a.reshape(3,2,2) print a [[[ 0 1] [ 2 3]] [[ 4 5] [ 6 7]] [[ 8 9] [10 11]]]
Transpose - Passed
a=array([[1,2,3],[4,5,6]]) print a.shape b=a.transpose() print b.shape (2L, 3L) (3L, 2L)
comment:17 Changed 8 years ago by Nick Draper
NumPy with workspace data - Passed
ws=Load("GEM38370") a=ws.readY(0) print a.shape b= a.reshape(505,10) print b.shape (5050L,) (505L, 10L)
comment:18 Changed 8 years ago by Nick Draper
- Status changed from verifying to reopened
- Resolution fixed deleted
SciPy
Version - passed
import scipy print scipy.version.version 0.11.0
convolve - Passed
from scipy import * x=r_[1:4] h1=zeros(15) h1[0] =1 h2=zeros(15) h2[6] =1 h3=zeros(15) h3[12] =1 print convolve(h1+h2+h3,x) [ 1. 2. 3. 0. 0. 0. 1. 2. 3. 0. 0. 0. 1. 2. 3. 0. 0.]
Optimize - Failed
from pylab import * from numpy import * from scipy.optimize import leastsq ## Parametric function: 'v' is the parameter vector, 'x' the independent varible fp = lambda v, x: v[0]/(x**v[1])*sin(v[2]*x) ## Noisy function (used to generate data to fit) v_real = [1.5, 0.1, 2.] fn = lambda x: fp(v_real, x) ## Error function e = lambda v, x, y: (fp(v,x)-y) ## Generating noisy data to fit n = 30 xmin = 0.1 xmax = 5 x = linspace(xmin,xmax,n) y = fn(x) + rand(len(x))*0.2*(fn(x).max()-fn(x).min()) ## Initial parameter value v0 = [3., 1, 4.] ## Fitting v, success = leastsq(e, v0, args=(x,y), maxfev=10000) ## Plot def plot_fit(): print 'Estimater parameters: ', v print 'Real parameters: ', v_real X = linspace(xmin,xmax,n*5) plot(x,y,'ro', X, fp(v,X)) plot_fit() show() Error on line 3: ImportError: DLL load failed: The specified module could not be found. at line 3 in '' caused by line 144 in 'C:\MantidInstall\bin\lib\site-packages\scipy\optimize\__init__.py' caused by line 30 in 'C:\MantidInstall\bin\lib\site-packages\scipy\optimize\optimize.py' caused by line 14 in 'C:\MantidInstall\bin\lib\site-packages\scipy\optimize\linesearch.py'
comment:19 Changed 8 years ago by Martyn Gigg
Rebuilt scipy .pyd to link with gfortran statically. Refs #5760
Changeset: e9804d3749bb49b17c8de69ae4f69be5832dae43
comment:20 Changed 8 years ago by Martyn Gigg
Only keep numpy .pyc files in repo Refs #5760
If the py files are present then they pyc files keep getting recompiled when numpy is imported and look modified to git. The correct procedure would be to include only py files & compile on packaging. Left for future work.
Changeset: 18bcdf9c605bccfe58dc76790a8e18c56c88f8c0
comment:21 Changed 8 years ago by Martyn Gigg
Only keep numpy .pyc files in repo Refs #5760
If the py files are present then they pyc files keep getting recompiled when numpy is imported and look modified to git. The correct procedure would be to include only py files & compile on packaging. Left for future work.
Changeset: 2880d4ea887ff9ff308123ac9a4581c5997554b5
comment:22 Changed 8 years ago by Martyn Gigg
Replace numpy & scipy with MKL variants. Refs #5760
It turns out we can redistribute as we have an appropriate license.
Changeset: 44a98a192e249ccf95ae790413c04b71112e6393
comment:23 Changed 8 years ago by Martyn Gigg
Replace numpy & scipy with MKL variants. Refs #5760
Consistent with 64-bit platform then.
Changeset: ce44725367e30ffdd50fd0ab7f04c1384113c1be
comment:24 Changed 8 years ago by Martyn Gigg
Add gfortran libraries for mingw for Inelastic pyd files. Refs #5760
Changeset: 8228335ca81316daad927bbf10071f2ae895fdc2
comment:25 Changed 8 years ago by Martyn Gigg
Install fortran libraries for inelastic modules... Refs #5760
on win32. Not quite sure how these have worked in the past without them, they must have been statically linked but the latest f2py seems to dynamically link them.
Changeset: b532aa2ddadbea990e2bb42be475f5e73b6803fa
comment:27 Changed 8 years ago by Martyn Gigg
- Status changed from accepted to verify
- Resolution set to fixed
Tester: I've had to slightly modifiy the test script to do the plotting. Anything that will perform a GUI command needs to be wrapped with our function that ensures it happens on the correct thread.
There is also a mode under the execute menu for serialised running that will work but that's less ideal I think.
Should I think about having an alias for the threadsafe_call function with a more understandable name: gui_call or something?
from pylab import * from numpy import * from scipy.optimize import leastsq ## Parametric function: 'v' is the parameter vector, 'x' the independent varible fp = lambda v, x: v[0]/(x**v[1])*sin(v[2]*x) ## Noisy function (used to generate data to fit) v_real = [1.5, 0.1, 2.] fn = lambda x: fp(v_real, x) ## Error function e = lambda v, x, y: (fp(v,x)-y) ## Generating noisy data to fit n = 30 xmin = 0.1 xmax = 5 x = linspace(xmin,xmax,n) y = fn(x) + rand(len(x))*0.2*(fn(x).max()-fn(x).min()) ## Initial parameter value v0 = [3., 1, 4.] ## Fitting v, success = leastsq(e, v0, args=(x,y), maxfev=10000) ## Plot def plot_fit(): print 'Estimater parameters: ', v print 'Real parameters: ', v_real X = linspace(xmin,xmax,n*5) plot(x,y,'ro', X, fp(v,X)) threadsafe_call(plot_fit) threadsafe_call(show)
comment:29 Changed 8 years ago by Nick Draper
- Status changed from verifying to reopened
- Resolution fixed deleted
Still fails with same error on version 2.2.1133
comment:30 Changed 8 years ago by Martyn Gigg
Put numpy & scipy dependencies in bin on packaging. Refs #5760
That way they both can find them.
Changeset: 65e0e07cf6f2aea9a4e722653bc0fe5a25b41e5a
comment:32 Changed 8 years ago by Martyn Gigg
- Status changed from accepted to verify
- Resolution set to fixed
comment:33 Changed 8 years ago by Andrei Savici
- Status changed from verify to verifying
- Tester changed from Nick Draper to Andrei Savici
comment:35 Changed 8 years ago by Martyn Gigg
Update precompiled Inelastic fortran to use numpy 1.6.2. Refs #5760
Changeset: 94db7996dd6f1d6b09b2f9484f69995bc55347cf
comment:36 Changed 8 years ago by Martyn Gigg
Install fortran libraries for inelastic modules... Refs #5760
on win32. Not quite sure how these have worked in the past without them, they must have been statically linked but the latest f2py seems to dynamically link them.
Changeset: b532aa2ddadbea990e2bb42be475f5e73b6803fa
comment:37 Changed 8 years ago by Martyn Gigg
Put numpy & scipy dependencies in bin on packaging. Refs #5760
That way they both can find them.
Changeset: 65e0e07cf6f2aea9a4e722653bc0fe5a25b41e5a
comment:38 Changed 8 years ago by Martyn Gigg
Install fortran libraries for inelastic modules... Refs #5760
on win32. Not quite sure how these have worked in the past without them, they must have been statically linked but the latest f2py seems to dynamically link them.
Changeset: b532aa2ddadbea990e2bb42be475f5e73b6803fa
comment:39 Changed 8 years ago by Martyn Gigg
Put numpy & scipy dependencies in bin on packaging. Refs #5760
That way they both can find them.
Changeset: 65e0e07cf6f2aea9a4e722653bc0fe5a25b41e5a
comment:40 Changed 5 years ago by Stuart Campbell
This ticket has been transferred to github issue 6606
And ship a compatible version of scipy at the same time.