Friday, April 6, 2012

Py3k status update #3

This is the third status update about my work on the py3k branch, which I can work on thanks to all of the people who donated to the py3k proposal.

A lot of work has been done during the last month: as usual, the list of changes is too big to be reported in a detalied way, so this is just a summary of what happened.

One of the most active areas was killing old and deprecated features. In particular, we killed support for the __cmp__ special method and its counsins, the cmp builtin function and keyword argument for list.sort() and sorted(). Killing is easy, but then you have to fix all the places which breaks because of this, including all the types which relied on __cmp__ to be comparable,, fixing all the tests which tried to order objects which are no longer ordeable now, or implementing new behavior like forbidding calling hash() on objects which implement __eq__ but not __hash__.

Among the other features, we killed lots of now-gone functions in the operator module, the builtins apply(), reduce() and buffer, and the os.* functions to deal with temporary files, which has been deprecated in favour of the new tempfile module.

The other topic which can't miss in a py3k status update is, as usual, string-vs-unicode. At this round, we fixed bugs in string formatting (in particular to teach format() to always use unicode strings) and various corner cases about when calling the (possibly overridden) __str__ method on subclasses of str. Believe me, you don't want to know the precise rules :-).

Other features which we worked on and fixed tests include, but are not limited to, marshal, hashlib, zipimport, _socket and itertools, plus the habitual endless lists of tests which fail for shallow reasons such as the syntactic differences, int vs long, range() vs list(range()) etc. As a result, the number of failing tests dropped from 650 to 235: we are beginning to see the light at the end of the tunnel :-)

Benjamin finished implementing Python 3 syntax. Most of it was small cleanups and tweaks to be compatible with CPython such as making True and False keywords and preventing . . . (note spaces between dots) from being parsed as Ellipsis. Larger syntax additions included keyword only arguments and function annotations.

Finally, we did some RPython fixes, so that it is possible again to translate PyPy in the py3k branch. However, the resuling binary is a strange beast which mixes python 2 and python 3 semantics, so it is unusable for anything but showing friends how cool it is.

I would like to underline that I was not alone in doing all this work. In particular, a lot of people joined the PyPy sprint at Pycon and worked on the branch, as you can clearly see in this activity graph. I would like to thank all who helped!

Antonio and Benjamin


ArneBab said...

Very cool work!

Thanks for the update! I‘ll need to see if I can already let it hit my own python3 project (I had to convert that to python2.x to make it run with pypy, being able to get rid of that step would be really cool!)

Do you already have prebuilt binaries of pypy3?

Antonio Cuni said...

I don't think that there is any chance that a python3 project will run as of now, there are still tons of features missing. So far my job as mostly been to fix all the failing tests in the PyPy testsuite. When I'll have finished, I'll be able to start with new features.

And, for the same reason: no prebuilt binaries yet, sorry.

ArneBab said...

OK, thanks for the info!

I’m anxious to test it, once you give it a chance to run simple unicode-using code!

Anonymous said...

Pocoo's pastebin has unfortunately permanently shut down. Any chance you could repaste how cool it is somewhere else?