Monday, May 3, 2010

Running wxPython on top of pypy

Hello,

These last three weeks we have been busy working on the cpyext subsystem, which allows pypy to execute extension modules written with the Python C API.

Today we hacked enough to have wxPython compile, and run its wonderful demo. This: cannot be distinguished from the same run with a standard python interpreter, but this: shows an exception that CPython never produces.

wxPython is a big extension module: it has more than 500 classes and 7500 functions, most of the code is automatically generated by swig. It uses advanced techniques, like "Original Object Return" and cross-platform polymorphism, that effectively allows the developer to seamlessly subclass C++ objects in Python and write GUI applications efficiently.

The demo application runs reasonably fast, it feels slower than with CPython, but I did not activate the JIT option of pypy. It still crashes in some places (the demo is very comprehensive and covers all the aspects of wxPython), and threads are expected to not work at the moment.

We had to modify a little the code of wxPython, mainly because it often stores borrowed references into C++ objects. This does not work well in pypy, where all other counted references can disappear, and allows the address of the object to change. The solution is to use weak references instead. The patch is here, it will eventually be merged into the upstream wxPython version.

This first real test proves that CPython extensions can be migrated to pypy without much pain. It also points some places which can be improved, like better diagnostics in crashes, better support of distutils...

Amaury Forgeot d'Arc

11 comments:

illume said...

sweet as!

Dan Villiom Podlaski Christiansen said...

Nice! Do you have any plans for making Mac nightlies with this available? I'd love to try out PyPy, but the one time I tried bootstrapping, it used all available memory. After I had let it run overnight but it didn't finish, I killed it…

Bourne said...

This is very good news.

Finishing wxPython and the JIT is probably all that's needed to make PyPy a **great** alternative to CPython. (but I guess you figured that already)

Thanks!

Stu said...

Sweet ! I wonder if pycairo and pygtk... at the moment I don't know if it's cairo or python slowing down my app (I have an idea it's both, but running it in pypy does seem attractive).

illume said...

Are there docs for how to compile extensions somewhere? I had a quick look, but couldn't find them.

The Cannon Family said...

this is a major accomplishment in terms of usability, many people use Python extension modules, way to go. (and next steps, PIL).

Amaury Forgeot d'Arc said...

PIL also works with PyPy. I've only tried basic tests though (like gif->png conversion)

Amaury Forgeot d'Arc said...

@illume: you have to compile pypy with the option "--withmod-cpyext", then it should be enough to run "/path/to/pypy-c setup.py build"

Simosito said...

Well done! The Italian Python Community has an article on this (here, in Italian)

daves said...
This comment has been removed by a blog administrator.
Anonymous said...

Wow. PyPy is coming along quite nicely :-)