PyPy San Francisco Sprint Dec 1st - Dec 2nd 2012

The next PyPy sprint will be in San Francisco, California. It is a
public sprint, suitable for newcomers. It will run on Saturday December 1st and
Sunday December 2nd. The goals for the sprint are continued work towards the
2.0 release as well as code cleanup, we of course welcome any topic which
contributors are interested in working on.

Some other possible topics are:

• running your software on PyPy
• work on PyPy's numpy (status)
• work on STM (status)
• JIT improvements
• any exciting stuff you can think of

If there are newcomers, we'll run the usual introduction to hacking on
PyPy.

Location

The sprint will be held at the Rackspace Office:

620 Folsom St, Ste 100
San Francisco

The doors will open at 10AM both days, and run until 6PM both days.

Thanks to David Reid for helping get everything set up!

PyPy 2.0 beta 1

We're pleased to announce the 2.0 beta 1 release of PyPy. This release is not a typical beta, in a sense the stability is the same or better than 1.9 and can be used in production. It does however include a few performance regressions documented below that don't allow us to label is as 2.0 final. (It also contains many performance improvements.)

The main features of this release are support for ARM processor and compatibility with CFFI. It also includes numerous improvements to the numpy in pypy effort, cpyext and performance.

You can download the PyPy 2.0 beta 1 release here:

http://pypy.org/download.html

What is PyPy?

PyPy is a very compliant Python interpreter, almost a drop-in replacement for CPython 2.7.3. It's fast (pypy 2.0 beta 1 and cpython 2.7.3 performance comparison) due to its integrated tracing JIT compiler.

This release supports x86 machines running Linux 32/64, Mac OS X 64 or Windows 32. It also supports ARM machines running Linux. Windows 64 work is still stalling, we would welcome a volunteer to handle that.

How to use PyPy?

We suggest using PyPy from a virtualenv. Once you have a virtualenv installed, you can follow instructions from pypy documentation on how to proceed. This document also covers other installation schemes.

Regressions

Reasons why this is not PyPy 2.0:

• the ctypes fast path is now slower than it used to be. In PyPy 1.9 ctypes was either incredibly faster or slower than CPython depending whether you hit the fast path or not. Right now it's usually simply slower. We're probably going to rewrite ctypes using cffi, which will make it universally faster.
• cffi (an alternative to interfacing with C code) is very fast, but it is missing one optimization that will make it as fast as a native call from C.
• numpypy lazy computation was disabled for the sake of simplicity. We should reenable this for the final 2.0 release.

Highlights

• cffi is officially supported by PyPy. You can install it normally by using pip install cffi once you have installed PyPy and pip. The corresponding 0.4 version of cffi has been released.
• ARM is now an officially supported processor architecture. PyPy now work on soft-float ARM/Linux builds. Currently ARM processors supporting the ARMv7 and later ISA that include a floating-point unit are supported.
• This release contains the latest Python standard library 2.7.3 and is fully compatible with Python 2.7.3.
• It does not however contain hash randomization, since the solution present in CPython is not solving the problem anyway. The reason can be found on the CPython issue tracker.
• gc.get_referrers() is now faster.
• Various numpy improvements. The list includes:
  • axis argument support in many places
  • full support for fancy indexing
  • complex128 and complex64 dtypes
• JIT hooks are now a powerful tool to introspect the JITting process that PyPy performs.
• **kwds usage is much faster in the typical scenario
• operations on long objects are now as fast as in CPython (from roughly 2x slower)
• We now have special strategies for dict/set/list which contain unicode strings, which means that now such collections will be both faster and more compact.

Things we're working on

There are a few things that did not make it to the 2.0 beta 1, which are being actively worked on. Greenlets support in the JIT is one that we would like to have before 2.0 final. Two important items that will not make it to 2.0, but are being actively worked on, are:

• Faster JIT warmup time.
• Software Transactional Memory.

Cheers,
Maciej Fijalkowski, Armin Rigo and the PyPy team

Py3k status update #7

This is the seventh status update about our work on the py3k branch, which
we can work on thanks to all of the people who donated to the py3k
proposal.

The biggest news is that this month Philip started to work on py3k in parallel
to Antonio. As such, there was an increased amount of activity.

The py3k buildbots now fully translate the branch every night and run the
Python standard library tests.

We currently pass 160 out of approximately 355 modules of CPython's standard
test suite, fail 144 and skip approximately 51.

Some highlights:

• dictviews (the objects returned by dict.keys/values/items) has been greatly
  improved, and now they full support set operators
• a lot of tests has been fixed wrt complex numbers (and in particular the
  __complex__ method)
• _csv has been fixed and now it correctly handles unicode instead of bytes
• more parser fixes, py3k list comprehension semantics; now you can no longer
  access the list comprehension variable after it finishes
• 2to3'd most of the lib_pypy modules (pypy's custom standard lib
  replacements/additions)
• py3-enabled pyrepl: this means that finally readline works at the command
  prompt, as well as builtins.input(). pdb seems to work, as well as
  fancycompleter to get colorful TAB completions :-)
• py3 round
• further tightening/cleanup of the unicode handling (more usage of
  surrogateescape, surrogatepass among other things)
• as well as keeping up with some big changes happening on the default branch
  and of course various other fixes.

Finally, we would like to thank Amaury Forgeot d'Arc for his significant
contributions.

cheers,
Philip&Antonio

NumPy status update #5

Hello.

I'm quite excited to inform that work on NumPy in PyPy has been restarted and there has been quite a bit of progress on the NumPy front in PyPy in the past two months. Things that happened:

• complex dtype support - thanks to matti picus, NumPy on PyPy now supports complex dtype (only complex128 so far, there is work on the other part)
• big refactoring - probably the biggest issue we did was finishing a big refactoring that disabled some speedups (notably lazy computation of arrays), but lowered the barrier of implementing cool new features.
• fancy indexing support - all fancy indexing tricks should now work, including a[b] where b is an array of integers.
• newaxis support - now you can use newaxis features
• improvements to ``intp``, ``uintp``, ``void``, ``string`` and record dtypes

Features that have active branches, but hasn't been merged:

• float16 dtype support
• missing ndarray attributes - this is a branch to finish all attributes on ndarray, hence ending one chapter.
• pickling support for numarray - hasn't started yet, but next on the list

More importantly, we're getting very close to able to import the python part of the original numpy with only import modifications and running it's tests. Most tests will fail at this point, however it'll be a good start for another chapter :-)

Cheers,
fijal