PyPy2 v5.3 released - major C-extension support improvements

We have released PyPy2.7 v5.3, about six weeks after PyPy 5.1 and a week after PyPy3.3 v5.2 alpha 1, the first PyPy release targeting 3.3 compatibility. This new PyPy2.7 release includes major improvements for the C-API compatibility layer. In addition to complete support for lxml, we now pass most (more than 95%) of the upstream numpy test suite. We can build and run scipy and matplotlib as well. Most of the failures have to do with (ab) use of the C-API, for instance writing to a read-only pointer obtained from PyString_AsString().

Note that the C-API compatibility layer is significantly slower than CPython, as explained in the blog post about the new strategy for reflection of C objects into the PyPy interpreter.

We updated cffi to version 1.7 (incremental changes which provide a nicer developer experience, documented here). We would encourage developers to move their C-extension modules to cffi, but are willing to help you work through issues with existing code; come to #pypy on IRC and let us know how we can help you help us do better.

You can download the PyPy2 v5.3 release here:

http://pypy.org/download.html

We would like to thank our donors for their continued support of the PyPy project. We would also like to thank our contributors and encourage new people to join the project. PyPy has many layers and we need help with all of them: PyPy and RPython documentation improvements, tweaking popular modules to run on PyPy, or general help with making RPython’s JIT even better.

What is PyPy?

PyPy is a very compliant Python interpreter, almost a drop-in replacement for CPython 2.7. It’s fast (PyPy and CPython 2.7 performance comparison) due to its integrated tracing JIT compiler.

We also welcome developers of other dynamic languages to see what RPython can do for them.

This release supports:

• x86 machines on most common operating systems (Linux 32/64, Mac OS X 64, Windows 32, OpenBSD, FreeBSD)
• newer ARM hardware (ARMv6 or ARMv7, with VFPv3) running Linux
• big- and little-endian variants of PPC64 running Linux
• s390x running Linux

Other Highlights

(since the release of PyPy 5.1 in April, 2016)

• New features:
  • Merge a major expansion of the C-API support in cpyext, also expand cpyext tests to allow running them after translation as well as untranslated
  • Instead of “GIL not held when a CPython C extension module calls PyXxx”, we now silently acquire/release the GIL. Helps with C extension modules that call some PyXxx() functions without holding the GIL (arguably, they are theoretically buggy).
  • Support command line -v to trace import statements
  • Revive traceviewer, a tool to use pygame to view traces
    
    
• Numpy via our internal _numpypy module:
  • Implement ufunc.outer
  • Move PyPy-specific numpypy headers to a subdirectory (also changed the repo accordingly)
   
  
• Performance improvements:
  • Use bitstrings to compress lists of descriptors that are attached to an EffectInfo
  • Remove most of the _ovf, _zer and _val operations from RPython. Kills quite some code internally, and allows the JIT to do better optimizations: for example, app-level code like x / 2 or x % 2 can now be turned into x >> 1 or x & 1, even if x is possibly negative.
  • Rework the way registers are moved/spilled in before_call()
  
  
• Internal refactorings:
  • Refactor code to better support Python3-compatible syntax
  • Reduce the size of generated C sources during translation by eliminating many many unused struct declarations (Issue #2281)
  • Reduce the size of generated code by using the same function objects in all generated subclasses
  • Share cpyext Py* function wrappers according to the signature, shrinking the translated libpypy.so by about 10% (without the JIT)

Please update, and continue to help us make PyPy better. Cheers
The PyPy Team