PyPy learned to directly download CPython extensions

PyPy can now load and run CPython extensions (that is, .pyd and .so files) directly using a new subsystem called CPyExt. Unlike the solution presented in another blog post (in which numpy extension modules were launched on CPython and proxied via TCP), the new solution no longer requires running CPython. We have not yet achieved full binary compatibility (like Ironclad), but, in most cases, it is enough just to recompile the extension.

The only requirement is that the necessary C API functions from CPython must be implemented in PyPy. If you are a user or author of a module, and you lack some functions in PyPy, we invite you to implement them. Already quite a few people (including many new contributors) have already joined the development and implemented some functions in order for their favorite modules to work. At the end of the post - a list of names.

Speaking of speed. Despite the fact that there are overhead costs for running these modules, when we run the regular expression engine from CPython ( _sre.so ) and go through the spambayes benchmark from the Unladen Swallow benchmark set (see speed.pypy.org ), we get a significant acceleration: test became twice as fast as the regulars engine built into PyPy. From the _sre.so law it follows that _sre.so should work several times faster than the built-in engine.
')
Currently working on PIL and other modules. For example, distutils support is almost complete. And if you want to participate in this or get more information on how to use this new feature, go to our #pypy IRC channel on freenode .

CPyExt contributors:

• Alex Gaynor
  
• Benjamin peterson
  
• Jean-Paul Calderone
  
• Maciej fijalkowski
  
• Jan de Mooij
  
• Lucian Branescu Mihaila
  
• Andreas Stührk
  
• Zooko Wilcox-O Hearn