Monday, March 17, 2008

ctypes configuration tool

As a part of implementing ctypes, we decided to make coding using ctypes better on its own (irrelevant what python interpreter you use). The concrete problem we're trying to solve is to make ctypes code more platform-independent than it is. Say you want to create a ctypes type for size_t: ctypes itself provides no mechanism for doing that, so you need to use a concrete integer type (c_int, c_long, c_short etc.). Your code either becomes platform dependent if you pick one of them or is littered with conditionals for all sorts of platforms. We created a small library, called ctypes_configure (which is actually a variation of something we use somewhere in the PyPy source tree), which tries to solve some platform dependencies by compiling and running small chunks of C code through a C compiler. It's sort of like configure in the Linux world, except for Python using ctypes.

To install the library, you can just type easy_install ctypes_configure. The code is in an svn repository on codespeak and there is even some documentation and sample code. Also, even though the code lives in the pypy repository, it depends only on pylib, not on the whole of pypy.

The library is in its early infancy (but we think it is already rather useful). In the future we could add extra features, it might be possible to check whether the argtypes that are attached to the external functions are consistent with what is in the C headers), so that the following code wouldn't segfault but give a nice error
libc = ctypes.CDLL("libc.so")
time = libc.time
time.argtypes = [ctypes.c_double, ctypes.c_double]
time(0.0, 0.0)
Also, we plan to add a way to install a package that uses ctypes_configure in such a way that the installed library doesn't need to call the C compiler any more later.

4 comments:

Thomas Heller said...

Cool - it even works on Windows!.

BTW: The content-type of the documentation seems wrong, firefox displays the html instead of rendering it.

PJE said...

Since easy_install can compile C code, why not just compile an extension module with the configuration? Then, other modules can just import the pre-built configuration.

Maciej Fijalkowski said...

Sure. It's an obvious extension. I just got this from pypy source code and released separately. If it'll happen to be useful, I'll add more features.

Stephan said...

Re: PJE

I'm no expert (and I'm half asleep), but your approach sounds like it might run afoul of changes introduced by upgrading something without regenerating the pre-built configuration.