NumPyPy 2013 Developer Position

Introduction

Proposed herein is a part-time fellowship for developing NumPy in PyPy. The work will initially consist of 100 hours with the possibility of extension, until the funds run out. Development and improvement of PyPy's NumPyPy (as with most Open Source and Free Software) is done as a collaborative process between volunteer, paid, and academic contributors. Due to a successful funding drive but a lack of contributors willing to work directly for PyPy, we find ourselves in the enviable situation of being able to offer this position.

Background

PyPy's developers make all PyPy software available to the public without charge, under PyPy's Open Source copyright license, the permissive MIT License. PyPy's license assures that PyPy is equally available to everyone freely on terms that allow both non-commercial and commercial activity. This license allows for academics, for-profit software developers, volunteers and enthusiasts alike to collaborate together to make a better Python implementation for everyone.

NumPy support for PyPy is licensed similarly, and therefore NumPy in PyPy support can directly help researchers and developers who seek to do numeric computing but want an easier programming language to use than Fortan or C, which is typically used for these applications. Being licensed freely to the general public means that opportunities to use, improve and learn about how NumPy in PyPy works itself will be generally available to everyone.

The Need for a Part-Time Developer

NumPy project in PyPy has seen some slow, but steady progress since we started working about a year ago. On one hand, it's actually impressive what we could deliver with the effort undertaken, on the other hand, we would like to see the development accelerated.

PyPy has strict coding, testing, documentation, and review standards, which ensures excellent code quality, continually improving documentation and code test coverage, and minimal regressions. A part-time developer will be able to bring us closer to the goal of full numpy-api implementation and speed improvements.

Work Plan

The current proposal is split into two parts:

• Compatibility:

  This part covers the core NumPy Python API. We'll implement most NumPy APIs that are officially documented and we'll pass most of NumPy's tests that cover documented APIs and are not implementation details. Specifically, we don't plan to:

  • implement NumPy's C API
  • implement other scientific libraries, like SciPy, matplotlib or biopython
  • implement details that are otherwise agreed by consensus to not have a place in PyPy's implementation of NumPy or agreed with NumPy community to be implementation details

• Speed:

  This part will cover significant speed improvements in the JIT that would make numeric computations faster. This includes, but is not necesarilly limited to:

  • write a set of benchmarks covering various use cases
  • teaching the JIT backend (or multiple backends) how to deal with vector operations, like SSE
  • experiments with automatic parallelization using multiple threads, akin to numexpr
  • improving the JIT register allocator that will make a difference, especially for tight loops

  As with all speed improvements, it's relatively hard to predict exactly how it'll cope, however we expect the results to be withing an order of magnitude of handwritten C equivalent.

Position Candidate

We would like people who are proficient in NumPy and PyPy (but don't have to be core developers of either) to step up. The developer selection will be done by consensus of PyPy core developers and consulted with the Software Freedom Conservancy for lack of conflict of interest. The main criterium will be past contributions to the PyPy project, but they don't have to be significant in size.

A candidate for the Developer position will demonstrate the following:

• The ability to write clear, stable, suitable and tested code
• The ability to understand and extend the JIT capabilities used in NumPyPy.
• A positive presence in PyPy's online community on IRC and the mailing list.

Ideally the Developer will also:

• Have familiarity with the infrastructure of the PyPy project (including bug tracker and buildbot).
• Have Worked to provide education or outreach on PyPy in other forums such as workshops, conferences, and user groups.

Conservancy and PyPy are excited to announce the Developer Position. Renumeration for the position will be at the rate of 60 USD per hour, through the Software Freedom Conservancy.

PyPy community is promising to provide necessary guidance and help into the current codebase, however we expect a successful candidate to be able to review code and incorporate external patches within two months of the starting date of the contract.

Candidates should submit their proposal (including their CV) to:

pypy-z@python.org

The deadline for this initial round of proposals is February 1, 2013.

Py3k status update #9

This is the ninth status update about our work on the py3k branch, which
we can work on thanks to all of the people who donated to the py3k
proposal.

Just a very short update on December's work: we're now passing about 223 of
approximately 355 modules of CPython's regression test suite, up from passing
194 last month.

Some brief highlights:

• More encoding related issues were addressed. e.g. now most if not all the
  multibytecodec test modules pass.
• Fixed some path handling issues (test_os, test_ntpath and
  test_posixpath now pass)
• We now pass test_class, test_descr and almost test_builtin (among
  other things): these are notable as they are fairly extensive test suites of
  core aspects of the langauge.
• Amaury Forgeot d'Arc continued making progress on CPyExt (thanks again!)

cheers,
Phil