Tuesday, March 4, 2008

As fast as CPython (for carefully taken benchmarks)

Good news everyone. A tuned PyPy compiled to C is nowadays as fast as CPython on the richards benchmark and slightly faster on the gcbench benchmark.
IMPORTANT: These are very carefully taken benchmarks where we expect pypy to be fast! PyPy is still quite slower than CPython on other benchmarks and on real-world applications (but we're working on it). The point of this post is just that for the first time (not counting JIT experiments) we are faster than CPython on *one* example :-)
The exact times as measured on my notebook (which is a Core Duo machine) are here:
Compiled pypy with options:
./translate.py --gcrootfinder=asmgcc --gc=generation targetpypystandalone.py --allworkingmodules --withmod-_rawffi --faassen (allworkingmodules and withmod-_rawffi are very likely irrelevant to those benchmarks)
CPython version 2.5.1, release.
  • richards 800ms pypy-c vs 809ms cpython (1% difference)
  • gcbench 53700ms pypy-c vs 60215ms cpython (11% difference)
PyPy shines on gcbench, which is mostly just about allocating and freeing many objects. Our gc is simply better than refcounting, even though we've got shortcomings in other places.
About richards, there is a catch. We use a method cache optimization, and have an optimization which helps to avoid creating bound methods each time a method is called. This speeds up the benchmark for about 20%. Although method cache was even implemented for CPython, it didn't make its way to the core because some C modules directly modify the dictionary of new-style classes. In PyPy, the greater level of abstraction means that this operation is just illegal.

8 comments:

Nicholas said...

This is GREAT news!

Keep up the good work guys, i will be closely following you all!

Cheers!

Anonymous said...

I have been watching PyPy for some time now and this news along with the ctypes news has me excited.

Geoff said...

Great work.

It is wonderful to see PyPy making progress towards the overall goal!

Brandon Singer said...

Awsome. :)

Anonymous said...

Will PyPy be released before Duke Nukem gets released? Pray please tell and enlighten!
Cursing all you skilled hackers for not doing an amd64 port of Psycho and pursuing something that will be irrelevant when it materializes.
Have fun any way.

Maciej Fijalkowski said...

Yeah, it will be released usable. For real. We're getting into having nice and usable python interpreter, ctypes is a good example of feature that is ready to use. How fast it'll be? I don't know, hopefully faster than psyco.

Paul Boddie said...

"Will PyPy be released before Duke Nukem gets released?"

I doubt it: Duke Nukem was released in 1991...

http://en.wikipedia.org/wiki/Duke_Nukem_%28computer_game%29

If you want to make a wisecrack, at least try to deliver it correctly.

Anonymous said...

And pypy 1.0 was released one year ago ... http://aspn.activestate.com/ASPN/Mail/Message/python-announce/3461501