Monday, May 31, 2010

PyPy in Google's Summer of Code 2010

Good news everyone.

This year, thanks to google generosity and PSF support, we got two and a half of students for PyPy's summer of code. We didn't cut any students, but one of the projects is a joint project of PyPy and numpy. Hereby I present descriptions, in my own words with my own opinions and in arbitrary order. For more details please follow links to particular blogs.

Jason Creighton: 64bit JIT backend for PyPy

Intel 64bit (and I mean x86_64) compatibility for JIT has been one of the top requested features (along with GIL removal). While GIL removal is not really an easy task, having our JIT emit 64bit assembler is sort of easy, thanks to our JIT backend abstraction. It will likely be faster, thanks to abundance of registers.

Bartosz Skowron: Fast ctypes for PyPy

Historically weak point of PyPy was compatibility with extension modules. We have progressed quite a bit in recent years, first introducing ctypes for pypy then progressing towards CPython extension modules. However, ctypes is well known to be slow (and it's even slower on PyPy) and writing CPython extension modules is ugly, and it's going to be only with compatibility layer that'll keep this slow. What happens if we try to employ JIT technology to ctypes? Maybe we can compile calls to C code from Python as a direct calls in compiled assembler? Why not?

This project will look how the JIT technology can be employed to do some sort of FFI. There is no guarantee we'll get super-fast ctypes as a result, but it's good to see progress in that area.

Dan Roberts: Numpy in PyPy

This is a joint project of numpy and PyPy. The main objective is to bring numpy to PyPy, possibly fast. The official mentor for this project is Stefan van der Walt from numpy community. During initial meeting it was agreed that probably the best way to go would be to support original numpy with CPython extension compatibility and then provide a minimal native numpy framework for pypy. The former would retain full compatibility, while the latter would have JIT integration, with line of our previous numeric experiments. There would be an explicit interface from converting one array to another for convinience.

Overall, I'm very happy to see so much support for PyPy from SoC. I hope all three proposals will be successful!

fijal & pypy team.


Michael Twomey said...

Some really nice stuff in there, very interested in the potential for JIT + numpy, keep up the good work!

Anonymous said...

Cool projects. Two of them live as PyPy branches:

Where can we follow the NumPy work? :)

Unknown said...

when will pypy catch up with python 3.1? will it happen during the python language moratorium (pep 3003)?

Maciej Fijalkowski said...


Depends when you can help :)