Friday, February 10, 2012

PyPy 1.8 - business as usual

We're pleased to announce the 1.8 release of PyPy. As habitual this release brings a lot of bugfixes, together with performance and memory improvements over the 1.7 release. The main highlight of the release is the introduction of list strategies which makes homogenous lists more efficient both in terms of performance and memory. This release also upgrades us from Python 2.7.1 compatibility to 2.7.2. Otherwise it's "business as usual" in the sense that performance improved roughly 10% on average since the previous release.

you can download the PyPy 1.8 release here:

http://pypy.org/download.html

What is PyPy?

PyPy is a very compliant Python interpreter, almost a drop-in replacement for CPython 2.7. It's fast (pypy 1.8 and cpython 2.7.1 performance comparison) due to its integrated tracing JIT compiler.

This release supports x86 machines running Linux 32/64, Mac OS X 32/64 or Windows 32. Windows 64 work has been stalled, we would welcome a volunteer to handle that.

Highlights

  • List strategies. Now lists that contain only ints or only floats should be as efficient as storing them in a binary-packed array. It also improves the JIT performance in places that use such lists. There are also special strategies for unicode and string lists.

  • As usual, numerous performance improvements. There are many examples of python constructs that now should be faster; too many to list them.

  • Bugfixes and compatibility fixes with CPython.

  • Windows fixes.

  • NumPy effort progress; for the exact list of things that have been done, consult the numpy status page. A tentative list of things that has been done:

    • multi dimensional arrays
    • various sizes of dtypes
    • a lot of ufuncs
    • a lot of other minor changes

    Right now the numpy module is available under both numpy and numpypy names. However, because it's incomplete, you have to import numpypy first before doing any imports from numpy.

  • New JIT hooks that allow you to hook into the JIT process from your python program. There is a brief overview of what they offer.

  • Standard library upgrade from 2.7.1 to 2.7.2.

Ongoing work

As usual, there is quite a bit of ongoing work that either didn't make it to the release or is not ready yet. Highlights include:

  • Non-x86 backends for the JIT: ARMv7 (almost ready) and PPC64 (in progress)
  • Specialized type instances - allocate instances as efficient as C structs, including type specialization
  • More numpy work
  • Since the last release there was a significant breakthrough in PyPy's fundraising. We now have enough funds to work on first stages of numpypy and py3k. We would like to thank again to everyone who donated.
  • It's also probably worth noting, we're considering donations for the Software Transactional Memory project. You can read more about our plans

Cheers,
The PyPy Team

8 comments:

Anonymous said...

As usual, excellent work!
The faster Pypy becomes, the less the need to use limited languages just for speed considerations.
List specialization is really cool and seems to boost performance and reduce memory usage considerably. I'd love seeing specializations for tuples of ints/floats/strings as structs.
On a side note, what stops people from using RPython as a compiled language (in terms of speed) with a nicer syntax?

Daivd said...

Well done!

I find nothing on the comparison page about memory (maybe because it's called speed.pypy.org...). How are you stacking up against CPython there, on benchmarks and real word examples? I realize a JIT will always need some memory overhead, but perhaps you have done enough clever things now, like list strategies, to be competitive anyway?

halfaleague said...

I would donate to this.
Would this give us 'true' multithreading? a la clojure?

Francis said...

Seems like you guys are ahead of the curve with STM: http://arstechnica.com/business/news/2012/02/transactional-memory-going-mainstream-with-intel-haswell.ars

kurdakov said...

Did anybody test if it works with
pypy?

https://github.com/mvantellingen/psycopg2-ctypes

would be great to have out of the box postgresql support for Django

Joko Susilo said...

i will try it first

Anonymous said...

Just donated for py3k. I think it would make sense to allow donations for STM as well.

Trulium said...

I will try it.