Thursday, October 5, 2017

PyPy v5.9 Released, Now Supports Pandas, NumPy

The PyPy team is proud to release both PyPy3.5 v5.9 (a beta-quality interpreter for Python 3.5 syntax) and PyPy2.7 v5.9 (an interpreter supporting Python 2.7 syntax).

  • NumPy and Pandas now work on PyPy2.7 (together with Cython 0.27.1). Many other modules based on C-API extensions work on PyPy as well.

  • Cython 0.27.1 (released very recently) supports more projects with PyPy, both on PyPy2.7 and PyPy3.5 beta. Note version 0.27.1 is now the minimum version that supports this version of PyPy, due to some interactions with updated C-API interface code.

  • We optimized the JSON parser for recurring string keys, which should decrease memory use by up to 50% and increase parsing speed by up to 15% for large JSON files with many repeating dictionary keys (which is quite common).

  • CFFI, which is part of the PyPy release, has been updated to 1.11.1, improving an already great package for interfacing with C. CFFI now supports complex arguments in API mode, as well as char16_t and char32_t and has improved support for callbacks.

  • Issues in the C-API compatibility layer that appeared as excessive memory use were cleared up and other incompatibilities were resolved. The C-API compatibility layer does slow down code which crosses the python-c interface often. Some fixes are in the pipelines for some of the performance issues, and we still recommend using pure python on PyPy or interfacing via CFFI

Please let us know if your use case is slow, we have ideas how to make things faster but need real-world examples (not micro-benchmarks) of problematic code.

Work sponsored by a Mozilla grant continues on PyPy3.5; we continue on the path to the goal of a complete python 3.5 implementation. Of course the bug fixes and performance enhancements mentioned above are part of both PyPy2.7 and PyPy3.5 beta.

As always, this release fixed many other issues and bugs raised by the growing community of PyPy users. We strongly recommend updating.

You can download the v5.9 releases here (note that we provide PyPy3.5 binaries for only Linux 64bit for now):

We would like to thank our donors and contributors, and encourage new people to join the project. PyPy has many layers and we need help with all of them: PyPy and RPython documentation improvements, tweaking popular modules to run on PyPy, or general help with making RPython’s JIT even better.

What is PyPy?

PyPy is a very compliant Python interpreter, almost a drop-in replacement for CPython 2.7 (stdlib version 2.7.13), and CPython 3.5 (stdlib version 3.5.3). It’s fast (PyPy and CPython 2.7.x performance comparison) due to its integrated tracing JIT compiler.

We also welcome developers of other dynamic languages to see what RPython can do for them.

The PyPy 2.7 release supports:
  • x86 machines on most common operating systems (Linux 32/64 bits, Mac OS X 64 bits, Windows 32 bits, OpenBSD, FreeBSD)
  • newer ARM hardware (ARMv6 or ARMv7, with VFPv3) running Linux,
  • big- and little-endian variants of PPC64 running Linux,
  • s390x running Linux

What else is new?

PyPy 5.8 was released in June, 2017.
There are many incremental improvements to RPython and PyPy, the complete listing is here.
 
Please update, and continue to help us make PyPy better.

Cheers, The PyPy team

7 comments:

  1. Pypy3 works very well with flask. Good Job and thanx.



    cheers Rob

    ReplyDelete
  2. This comment has been removed by the author.

    ReplyDelete
  3. Good job ! 😀🎉
    It would be great if you could update http://packages.pypy.org/
    I'm going to donate again ! your work is awesome.

    ReplyDelete
  4. Pypy test run pands two or three times slower than pyhon

    df = sparkSession.sql("select * from test_users_dt").toPandas()
    for index, row in df.iterrows():
    result = 0

    for key in range(0, 10000000):
    event_type = row.event_type
    if key > 234:
    result = result + 1
    len(event_type + "123")

    print(result)

    ReplyDelete
  5. @melin

    We know that. We're in the process of improving that by merging various cpyext improvement branches. Stay tuned.

    ReplyDelete
  6. So this means the numpy port for pypy is redundant now right? We can use the original python numpy package?

    ReplyDelete
  7. @Eitan

    yes, but look at this FAQ for a longer explanation: http://doc.pypy.org/en/latest/faq.html#what-about-numpy-numpypy-micronumpy

    ReplyDelete

See also PyPy's IRC channel: #pypy at freenode.net, or the pypy-dev mailing list.
If the blog post is old, it is pointless to ask questions here about it---you're unlikely to get an answer.