Monday, May 16, 2011

PyPy Usage Survey

We've been working on PyPy for a long time. But readers of this blog will know that in the past year something has changed: we think PyPy is production ready. And it's not just us, this week LWN.net wrote an article about how PyPy sped up one of their scripts by a factor of three, noting that, "plans are to run gitdm under PyPy from here on out". All in all we think PyPy is pretty great, but not everyone is using it yet, and we want to know why. We want your feedback on why PyPy isn't ready to be your only Python yet, and how we can improve it to make that happen.

Therefore, we've put together a quick survey, whether you're using PyPy or not if you could take a few minutes to fill it out and let us know how we're doing we'd really appreciate it. You can find the form here.

Thanks, The PyPy team

at

24 comments:

  1. AnonymousMay 16, 2011 at 7:23 PM

    We are very interested in using PyPy in production but our project is based on lxml library and both are incompatible. Do you suggest any fix for this? I'm not sure if PyPy would compensate the reduction if performance of a pure Python XML library.

    ReplyDelete
  2. AnonymousMay 16, 2011 at 7:55 PM

    Biggest blocker right now is gevent, which I believe would require pypy stackless and JIT to get along plus some work to make gevent use ctypes in place of cpython api.

    ReplyDelete
  3. AnonymousMay 16, 2011 at 8:12 PM

    I suggest that you reproduce this survey on StackOverflow (if it's acceptable there, maybe Programmers?) and Quora, maybe Convore too. Posting to comp.lang.python would also help.

    ReplyDelete
  4. AnonymousMay 16, 2011 at 8:22 PM

    Pypy needs to either be a dropin replacement for python or provide a significant (order of magnitude) difference in performance that moving to pypy won't be as big of a deal when you lose the support of so many 3rd party libraries.

    ReplyDelete
  5. AnonymousMay 16, 2011 at 8:35 PM

    1. Installation is long and non-intuitive. I'd like to see PyPy packaged up for all the major distros + Mac OSX via Fink, Homebrew, and MacPorts.

    2. A comprehensive listing of modules that can and cannot be used with PyPy. I'm still not quite clear as to how PyPy interacts with the major web frameworks and WSGI (haven't researched it much either).

    3. Drop-in replacement for Python 2.7. I want my scripts that I wrote in Python to run in PyPy with no complications.

    ReplyDelete
  6. PavelMay 16, 2011 at 8:46 PM

    Could you provide the downloads with PGP signatures, please? We would like to use PyPy in production to run our payment processing system backend, but verified integrity and authenticity of its source code is strictly required.

    ReplyDelete
  7. VictorMay 16, 2011 at 9:05 PM

    2. A comprehensive listing of modules that can and cannot be used with PyPy. I'm still not quite clear as to how PyPy interacts with the major web frameworks and WSGI (haven't researched it much either).

    This is available at the PyPy Compatibility Wiki (I should update it this week, lots of new information around).

    ReplyDelete
  8. AnonymousMay 16, 2011 at 9:20 PM

    We would use it across all our deployments (hundreds of thousands of LOCs) and gladly contribute and invest in pypy as soon as you guys implement python3 spec. Literally can't wait.

    ReplyDelete
  9. Daniel KluevMay 17, 2011 at 7:52 AM

    I'd love to use PyPy in some of my projects, but they rely on lots of 3rd-party C/C++-based libraries.

    1) lxml, thats an absolute must for most of my applications. Original ETree now lacks many features lxml has, so there is no ready pure-python replacement avail.
    2) Some my own boost::python libraries. I didn't actually try to compile them on PyPy, but as I was told on IRC, support for b::p is still marginal.
    3) PycURL, PyV8, PyQt, wxPython and so on.

    ReplyDelete
  10. Martin GfellerMay 17, 2011 at 10:14 AM

    We would like to profit from the speedup, but it would be a major piece of work for us, as we're currently running Zope 2.13 (which we could replace, because we make only limited use of it and have our own hybrid database). However, before making an investment, we need to be sure that:


    - PyPy won't go away like Psyco did. A kind of "mainstream endorsement" by PSF would be helpful


    - numpy and scipy are available

    - a decent ODBC package is available (we're using mxODBC) at the moment

    - full support on Windows 32 and 64 bit



    Best regards, Martin

    Swisscom IT Services Finance

    ReplyDelete
  11. Maciej FijalkowskiMay 17, 2011 at 10:18 AM

    @martin

    * numpy, scipy support is on the way

    * 32bit windows is done, 64bit windows will happen, it's on the todo list

    * PSF has just endorsed PyPy in front of 1000 people crowd on pycon giving us a 10000$ check (http://3.bp.blogspot.com/-yLUKuyRgjdg/TYfklB5Jg4I/AAAAAAAABKM/_5Rv2thqzA0/s1600/pycon_cheque.jpg).

    That answers roughly half to 3/4 of your issues, no bad, we're getting there :)

    ReplyDelete
  12. AnonymousMay 17, 2011 at 4:48 PM

    I would like to repeat the numpy and scipy thing. I have to add matplotlib, which a lot of people use for plotting. Personally I also cannot live without h5py, which is awesome for storing and handling numerical data. I have no idea if it will work with pypy, because it does require numpy first.

    I'm looking forward to pypy becoming faster, better supported, and more popular! I am convinced that it will.

    ReplyDelete
  13. wilkMay 17, 2011 at 5:38 PM

    I've a project wich use psyco with a factor 15 (computation of train path) ! yes really, this project is in production (unfortunately not open source) ! I just tried it with pypy 1.5, and it works with the same factor (congratulation to you). So i'm sure that we'll use pypy.

    But like my other project, i don't change something wich already works. Most of them don't need speed improvement.

    On one scrabble game i'd like to replace a scrabble solver in C (if someone wants to help, it's opensource ?)

    I also hope to see a debian package in the next debian release...

    Thanks for your work, i follow it !

    ReplyDelete
  14. AnonymousMay 18, 2011 at 2:26 PM

    On my server I'm running couple of Django based ecommerce systems. I hope to be running more of them soon (hopefully). There is also PostgreSQL. Still not using PyPy but I just can't wait to check if it will be faster and if so then how much. I don't know yet how to run Django app on production on PyPy but as soon I check and run couple of performance tests I will surely give some feedback.

    ReplyDelete
  15. raptorMay 23, 2011 at 1:53 AM

    Its all about compatibility with 3rd party libs, C libs or boost::python. Otherwise those who want to JIT their Python are just going to wait a bit longer for PEP 3146 so they can have a good LLVM based JIT in standard Python.

    http://www.python.org/dev/peps/pep-3146/

    ReplyDelete
  16. AnonymousMay 23, 2011 at 4:48 AM

    The pypy group should make a full featured ide with a gui designer with built in packaging to .exe and linux .deb and .rpm that only runs the pypy vm. That would bring the desktop application programmers in by the droves.

    ReplyDelete
  17. Carl Friedrich Bolz-TereickMay 23, 2011 at 8:35 AM

    @Hart: unladen swallow is dead:

    http://qinsb.blogspot.com/2011/03/unladen-swallow-retrospective.html

    ReplyDelete
  18. AnonymousMay 23, 2011 at 4:21 PM

    Well, basically, it's NumPy, SciPy, Matplotlib and MayaVi. I'm also using Cython to optimize computation intensive code paths, but of course it would be nicer to stick to pure Python and let JIT do it's magic.

    ReplyDelete
  19. lazywebMay 23, 2011 at 7:44 PM

    Arrgh, gevent does not work with pypy? There's my blocker.

    ReplyDelete
  20. Gaëtan de MentenMay 30, 2011 at 1:59 PM

    How long are you planning to keep this poll open? I hope you will blog about its results when it's closed...

    ReplyDelete
  21. Almir KaricJune 2, 2011 at 3:49 AM

    would love to see the results

    ReplyDelete
  22. AnonymousJune 2, 2011 at 4:21 PM

    I'm interesting in psycopg2and PIL libraries.

    ReplyDelete
  23. CaetanoJune 2, 2011 at 4:29 PM

    The only thing that makes me not using pypy is the lack of supporting python bynaries .so, .pyd, etc.
    I know that is a hard feature to implement because is needed to stub the CPython api.
    but I think when its done will there is no reasons to not using pypy for anybody.

    ReplyDelete
  24. AnonymousAugust 4, 2011 at 10:21 PM

    Numpy, scipy, matplotlib, and image are the stick ups for me.

    ReplyDelete

See also PyPy's IRC channel: #pypy at freenode.net, or the pypy-dev mailing list.
If the blog post is old, it is pointless to ask questions here about it---you're unlikely to get an answer.