Python Module Usage Stats - Feb 2011
Below is a list of the 30 most used “base modules” based on the number of downloads from the PyPI project. These results are based on 11,204 downloaded packages. Details below.
Some observations:
Back in 2007, I wrote a simple script that counts importing modules in code. I used it to generate statistics. One or two weeks ago I decided to repeat this experiment - and see the difference between 2007 and 2011.
I started working on my old code and decided to update it. Import analysis in the code is not a simple task, as it seems at first glance. I thought to use tokenize and parser modules, but refused this idea. Using parser made my code dependent on versions, and after a while I decided to use tokenize, since I already had a bulky regular expression for this purpose.
')
Input : PyPI and source codes of the standard library version 2.7. I wrote a simple script (cheese_getter.py) for processing modules. He read the PyPI page, and then used easy_install to select each module. Since there are more than 13,000 modules in the PyPI project, it took some time.
Parsing : I wrote a simple search code "import x" and "from x import y" expressions in the code. Not without difficulties: the expression "from x import a, b", "from. import bla "and
must also be supported. I converted each import statement into a series of modules with a dot. For example, “import ab” was replaced by “a” and “ab”, and “from b import c, d” was replaced by “b”, “bc”, and “bd”.
Processing : I created three types of result.
I think the third option is the most informative.
Code : Full code is available .
results
Discussion
Some observations:
- It seems that django scored "some popularity", and Zope rose strongly in the list.
- The number of projects using unittest is depressing, especially in terms of setuptools.
- optparse 1875 vs. getopt 515.
- cPickle 690 vs. pickle 598.
- simplejson 760 vs. json 593.
In the past
Back in 2007, I wrote a simple script that counts importing modules in code. I used it to generate statistics. One or two weeks ago I decided to repeat this experiment - and see the difference between 2007 and 2011.
I started working on my old code and decided to update it. Import analysis in the code is not a simple task, as it seems at first glance. I thought to use tokenize and parser modules, but refused this idea. Using parser made my code dependent on versions, and after a while I decided to use tokenize, since I already had a bulky regular expression for this purpose.
')
Methodology
Input : PyPI and source codes of the standard library version 2.7. I wrote a simple script (cheese_getter.py) for processing modules. He read the PyPI page, and then used easy_install to select each module. Since there are more than 13,000 modules in the PyPI project, it took some time.
Parsing : I wrote a simple search code "import x" and "from x import y" expressions in the code. Not without difficulties: the expression "from x import a, b", "from. import bla "and
from bla import \ some_module\ some_module2 must also be supported. I converted each import statement into a series of modules with a dot. For example, “import ab” was replaced by “a” and “ab”, and “from b import c, d” was replaced by “b”, “bc”, and “bd”.
Processing : I created three types of result.
- The total number of imports.
- The number of packages importing the module.
- The number of packages importing the module, only for the first module in the record with a dot.
I think the third option is the most informative.
Code : Full code is available .
Source: https://habr.com/ru/post/114673/
All Articles