Friday, April 27, 2018

How to ignore the annoying Cython warnings in PyPy 6.0


If you install any Cython-based module in PyPy 6.0.0, it is very likely that you get a warning like this:
>>>> import numpy
/data/extra/pypy/6.0.0/site-packages/numpy/random/__init__.py:99: UserWarning: __builtin__.type size changed, may indicate binary incompatibility. Expected 888, got 408
  from .mtrand import *
The TL;DR version is: the warning is a false alarm, and you can hide it by doing:
$ pypy -m pip install pypy-fix-cython-warning
The package does not contain any module, only a .pth file which installs a warning filter at startup.

Technical details

This happens because whenever Cython compiles a pyx file, it generates C code which does a sanity check on the C size of PyType_Type. PyPy versions up to 5.10 are buggy and report the incorrect size, so Cython includes a workaround to compare it with the incorrect value, when on PyPy.
PyPy 6 fixed the bug and now PyType_Type reports the correct size; however, Cython still tries to compare it with the old, buggy value, so it (wrongly) emits the warning.
Cython 0.28.2 includes a fix for it, so that C files generated by it no longer emit the warning. However, most packages are distributed with pre-cythonized C files. For example, numpy-1.14.2.zip include C files which were generated by Cython 0.26.1: if you compile it you still get the warning, even if you locally installed a newer version of Cython.

There is not much that we can do on the PyPy side, apart for waiting for all the Cython-based packages to do a new release which include C files generated by a newer Cython.  In the mean time, installing this module will silence the warning.

3 comments:

  1. I've opened an issue to allow easier workarounds via a Cython compile flag: https://github.com/cython/cython/issues/2221

    ReplyDelete
  2. Thanks, although note that this flag would not help much in this case. Even if it were there, the package author would still have to recompile/republish each package in order to get rid of them. And once you do that, the warning vanishes anyway in the case of PyPy :)

    ReplyDelete
  3. PyPy looks awesome. I can't wait for Python 3.6.5 support and/or a Windows x86-64 version! Though it would be unlikely, Anaconda and/or IdleX support would be awesome too.

    ReplyDelete

See also PyPy's IRC channel: #pypy at freenode.net, or the pypy-dev mailing list.
If the blog post is old, it is pointless to ask questions here about it---you're unlikely to get an answer.