Ticket #8006 (closed: fixed)

Opened 7 years ago

Last modified 5 years ago

Update IPython and dependencies for Windows

Reported by: Russell Taylor Owned by: Russell Taylor
Priority: critical Milestone: Release 3.0
Component: Framework Keywords: IPython
Cc: Blocked By:
Blocking: #5939 Tester: Martyn Gigg

Description

IPython needs to be updated to 1.1.0 and pyzmq needs to be newer than the one we have as well. I think the other dependencies are OK.

This needs to be done in the 3rdpartylibs repositories for both 32 & 64 bit. The packages live in Python27/Lib/site-packages.

To make pyzmq importable within MantidPlot, the following command needs to be run to put the correct Visual Studio runtime into the manifest:

mt.exe -inputresource:python27.dll;#2 -outputresource:Lib\site-packages\zmq\libzmq.pyd;#2

Attachments

Windows8x64 Error.jpg (31.3 KB) - added by Russell Taylor 7 years ago.

Change History

comment:1 Changed 7 years ago by Russell Taylor

  • Status changed from new to inprogress

Re #8006. Upgrade pyzmq to version 13.1.0.

One change was necessary from the vanilla download - the manifest from the python dll was embedded in libzmq.pyd so that the package can be imported into MantidPlot.

Changeset: 31d5f5157dab306dfa3e459f8b3cbe6f3983aed8

comment:2 Changed 7 years ago by Russell Taylor

Re #8006. Upgrade pyzmq to version 13.1.0.

One change was necessary from the vanilla download - the manifest from the python dll was copied into libzmq.pyd so that the package can be imported into MantidPlot.

Changeset: 5324091893178399797854930e43ed89c38508c8

Changed 7 years ago by Russell Taylor

comment:3 Changed 7 years ago by Russell Taylor

I encountered the above error with the nightly builds (64 & 32 bit) on my Windows 8 machine when trying to start the external IPython console. I don't see it on Windows 7. This should be checked again when #5939 is finished, though I'm hopeful it will go away when we move to a single-process solution (the existing one starts a new process). If I just import zmq, I get no error.

comment:4 Changed 7 years ago by Russell Taylor

The only remaining change is to update the version of IPython to 1.1. I'm leaving that until the last minute as doing so will immediately break the existing IPython console mechanism (on the nightly build as we don't have branches in the thirdparty repos).

comment:5 Changed 7 years ago by Russell Taylor

I still see the error from comment:3. Seems like there's something wrong with the manifest, or else it's related to the fact that we don't ship the VC runtime that python is build against. We'll need to keep an eye out for it, but I still haven't seen it on Windows 7.

comment:6 Changed 7 years ago by Russell Taylor

  • Keywords IPython added

comment:7 Changed 7 years ago by Russell Taylor

Re #8006. Remove IPython version 0.12.

Changeset: 2a308494db3575de9216bdb0f53c745fd2262924

comment:8 Changed 7 years ago by Russell Taylor

Re #8006. Add IPython version 1.1.0.

Changeset: affa130553374a4abb4f1cfed71957bb88678e2f

comment:9 Changed 7 years ago by Russell Taylor

Re #8006. Remove IPython version 0.12.

Changeset: a2eeff9ac4a067534d4c22aa784dbaf80f9eecb4

comment:10 Changed 7 years ago by Russell Taylor

Re #8006. Add IPython version 1.1.0.

Changeset: 6c5354060af17a89abb4a420c55b0b9c9dd63bdb

comment:11 Changed 7 years ago by Russell Taylor

  • Status changed from inprogress to verify
  • Resolution set to fixed

Tester: This needs testing with both 32 & 64 bit builds, and both for a local build and a downloaded (or built) binary package (it's fine to test the local build only on your own bit-ness, I think, but both binary packages should definitely be checked). There are no branches as all the changes are in the 3rdpartylibs repos - so the binaries can be ones built off master. For your local build, make sure run fetch_Third_Party.bat before building.

In all cases, the following commands need to work without error and produce the output shown:

>> import zmq
>> print zmq.__version__
13.1.0
>> import IPython
>> print IPython.__version__
1.1.0
Last edited 7 years ago by Russell Taylor (previous) (diff)

comment:12 Changed 7 years ago by Martyn Gigg

  • Status changed from verify to verifying
  • Tester set to Martyn Gigg

comment:13 Changed 7 years ago by Martyn Gigg

  • Status changed from verifying to closed

Both packages look fine as does the release build on my 64-bit machine.

It doesn't work in debug, unsurprisingly. The error is:

ImportError: DLL load failed: A dynamic link library (DLL) initialization routine failed.
Are you trying to `import zmq` from the pyzmq source dir?
  at line 1 in 'New script'
  caused by line 44 in 'c:\GitHub\mantid\build\bin\Debug\lib\site-packages\zmq\__init__.py'

which I assume is that there is no debug build of zmq.

I certainly don't think this should hold us up as there are parts of Python itself that we don't have debug builds for. It will just mean it is limited to release mode on Windows.

comment:14 Changed 7 years ago by Russell Taylor

Anticipating this kind of issue, I diverted from my usual way and deliberately used a release build of Mantid to use when doing this - and then forgot to even go back and check the debug build.

I certainly wouldn't want things held up for this, but this does mean that the IPython script interpreter will not work at all for a debug build, which is not ideal for developers. I've created #8135 to look at this, but I'm not sure it's a trivial task (I just took pre-build binaries for the release build).

Why, oh why, does debug python on Windows have to behave like this?! Why couldn't it just fall back to the release libraries if it can't find debug ones?

Last edited 7 years ago by Russell Taylor (previous) (diff)

comment:15 Changed 5 years ago by Stuart Campbell

This ticket has been transferred to github issue 8851

Note: See TracTickets for help on using tickets.