Over the years PyPy has (ab?)used various external software for different purposes, and we've discovered bugs in nearly all of them, mostly by pushing them to their limits. For example, many compilers are not happy with 200MB of source in one file. The Microsoft C compiler has a limit of 65536 lines of code per file and the CLI was raising "System.InvalidProgramException: Method pypy.runtime.Constants:.cctor () is too complex.", where too complex probably means "too long". Just for fun, today we collected all projects we could think of in which we found bugs:
So one could say that PyPy is really just the most expensive debugging tool ever :-).
You know, the one piece of external software we most depend on is one we haven't found bugs in: gcc (at least, I can't remember any problems). That's pretty impressive.
ReplyDeleteI think you can probably add gdb to the list though.
Can you, maybe, give a few examples of what you did. Linking to items about them if you wrote about it.
ReplyDeleteI'd be interested in knowing which projects were the most receptive to the bug reports.
ReplyDeleteHi Brett,
ReplyDeleteI think the project most receptive to bug reports is LLVM, where bugs that we find are usually fixed within a small number of days. I think in general Open Source projects react quite well, as you would expect. A negative example is graphviz, which still segfaults despite us producing a patch which fixes the problem.
Microsoft proves to be completely unapproachable, it seems you have to pay them if you want to report a bug (should be the other way round, of course :-)).
@Carl:
ReplyDeleteThanks for the info, Carl. I have been contemplating trying to rely on them for compiling Python for testing purposes, especially with clang coming along (although I am waiting for them to address a bug I found =). Good to know they are responsive.
And yes, it is really unfortunate that Microsoft doesn't make reporting bugs easy, but I guess no one wants to deal with the number of reports they would most likely get. =)
Regarding Microsoft bug reports, there's Connect. And I've got some degree of success with it.
ReplyDelete