Useful Python Tools
Good evening friends! We have great news; we have opened a new set for the Python Developer course. The group starts in early July, and right now, according to the established tradition, we are sharing a useful translation prepared for the students of this course.
When you first start learning Python, someone explains to you that you can add your source folder to the PYTHONPATH environment variable and then your code can be imported from other directories. Very often, an explanatory forgets to say that in most cases this is a bad idea. Some people find it on the Internet, others just understand it. But too many people (especially inexperienced programmers) think that there can be no other alternatives.
')
This article is mainly for them, because even if you know that there is an alternative, it is not always easy to accept it and start using it. Python tools are confusing because they are a large amount of software built on one another, with many intersections and problems arising from it. It is not easy to understand how to use these tools correctly in your project.
For this reason, I decided to write this article and consider the most popular tools in it, find out when and where they are used and what tasks they solve. I will try to explain on the fingers how to use each of these tools. If the tool is in this list, it means that you, as a Pythonist, need at least to know about its existence. I will only talk about the tools that can be applied to any project or workflow, and you should keep them in mind when you start a new project. However, this does not mean that you should use all the tools presented in each of your projects. It is unnecessary to overload the project with tools, in some cases this may complicate its support.
Setuptools is the standard way to create packages in Python. He works anywhere and does his job well.
What for : creating egg, zip or wheel files from sources, defining metadata for your project, collaborative structured and standardized work on the code.
When used : Always when you write code that should run on someone else’s machine.
Alternatives : Poetry, Flit
Virtualenv is a virtual environment manager. These isolated environments are a standalone python with a specific set of pre-installed packages. Using virtualenv means that you do not need to install packages into the python system by default.
For what : separation of dependencies, support for different versions of python by one system, easy movement of dependencies.
When used : You need to write code, and this requires a version of python different from your default system version of python.
Alternatives : Docker or something similar.
Pip is the most common package manager in python. It allows you to install local or remote packages in your virtual environment or Python system.
For what : installing and removing packages, tracking versions of packages that you use.
When used : Always.
Alternatives : Poetry, Conda
distutils is the predecessor of setuptools. The latter actively uses the functional distutils, so it is often necessary to interact with this tool. Distutils is not exactly the tool you should have in your arsenal, but you need to know how it fits into the big picture.
Pypi or Python Package Index is a large repository that contains all of your favorite Python modules. For example, the same pip takes builds from there.
For what : To publish your code.
When used : When there is a package that you want to show the community.
Pypiserver is one of the implementations of the Package Index API used by Pypi. You can create your own repository, for example, for your entire company and publish packages without making public releases.
What for : Create your own repositories within the organization.
When used : When your code does not need publicity, but you need full control over it.
Alternatives : Warehouse (used by Pypi), djangopypi
Poetry is an alternative package management system that replaces setuptools, pip and some other tools built on their basis. This is an attempt to completely revise how the package system works in Python. Currently, poetry has a lot of positive reviews, but is not the most common tool.
For what : processing and distributing packages, managing dependencies, preventing problems with resolving dependencies.
When used : When you are planning a new project and you are not afraid to use highly specialized tools.
Alternatives : Pipenv
Pipenv , like Poetry, is a tool for structuring dependencies and configuring Python projects in a more sane way. With Pipfile, it manages the dependencies of your project and ensures consistency and ease of use.
For what : processing and distributing packages, managing dependencies.
When used : you need a tool like Poetry that will cause less questions.
Alternatives : Poetry.
Sphinx is a tool for creating documentation. It was originally created to handle Python documentation, but has become a public tool. It is the most common option for projects in Python.
For what : creating PDF or HTML documents using a markup language from reStructuredText sources.
When used : When your project, API or code requires external documentation.
Alternatives : Docutils, Doxygen
autodoc is a fundamental extension for Sphinx that allows you to create reStructuredText files from Python source code with signatures for each class, function, module, and so on.
For what : documenting your code or API.
When used : In fact, every time you use Sphinx.
Alternatives : autosummary
py.test - in my opinion, is the best package for testing in Python. It has many functions, although not all of them are properly disclosed, so it will take some time to search for all the features that py.test provides.
For what : testing your code.
When used : Always when you are too lazy to test manually.
Alternatives : unittest, nose
Hypothesis is a tool for testing individual properties. In short, it generates random test scripts according to your specifications until it finds a script in which the test fails. Spend some time learning the principles before you start using this tool.
For what : testing the code, especially data processing.
When used : When you need to test non-trivial logic with a wide range of input values (numbers, strings, structured data).
tox is a virtual environment manager for testing. This means that you can configure it to run tests in clean, customizable virtual environments to ensure that your code can work in a variety of environments.
For what : for code that should run in different conditions and environments. Also useful for CI.
When used : When it is necessary for your code to be supported by different versions of Python, run in different environments and on different operating systems.
Alternatives : bash scrips, CI pipelines
pyenv is the python version manager. It aims to simplify the local workflow of developers when working with multiple versions.
For what : start different projects with different versions of Python.
When used : You need to work with global versions of Python and you have a lot of them.
Alternatives : manual management, virtualenv, Poetry, Pipenv
PyScaffold is a tool for initializing a project structure in a standardized way and providing some of the tools listed above without having to manually configure them. Very flexible.
For what : to load projects, work with several projects with the same toolkit and structure.
When used : always (if you are familiar with this tool, but do not try to use it for the first time when you are in a hurry)
Alternatives : python-project-template, Cookiecutter
flake8 is one of the most popular Python linters. It runs various scripts to verify that your code meets the requirements of the Python style guide ( PEP-8 ).
For what : checking your project for a good style of writing code.
When used : every time your project needs to be read by someone or by you.
Alternatives : pylint
Black automatically formats the code. This means that instead of simply checking your code for compliance with the standards, Black will change it himself so that it meets them.
For what : automatic code formatting.
When used : When you have no problems in order to refuse manual control of your code.
Alternatives : autopep8, yapf
That's all. We are waiting for your comments ;-).
When you first start learning Python, someone explains to you that you can add your source folder to the PYTHONPATH environment variable and then your code can be imported from other directories. Very often, an explanatory forgets to say that in most cases this is a bad idea. Some people find it on the Internet, others just understand it. But too many people (especially inexperienced programmers) think that there can be no other alternatives.
')
This article is mainly for them, because even if you know that there is an alternative, it is not always easy to accept it and start using it. Python tools are confusing because they are a large amount of software built on one another, with many intersections and problems arising from it. It is not easy to understand how to use these tools correctly in your project.
For this reason, I decided to write this article and consider the most popular tools in it, find out when and where they are used and what tasks they solve. I will try to explain on the fingers how to use each of these tools. If the tool is in this list, it means that you, as a Pythonist, need at least to know about its existence. I will only talk about the tools that can be applied to any project or workflow, and you should keep them in mind when you start a new project. However, this does not mean that you should use all the tools presented in each of your projects. It is unnecessary to overload the project with tools, in some cases this may complicate its support.
Basic tools
Setuptools
Setuptools is the standard way to create packages in Python. He works anywhere and does his job well.
What for : creating egg, zip or wheel files from sources, defining metadata for your project, collaborative structured and standardized work on the code.
When used : Always when you write code that should run on someone else’s machine.
Alternatives : Poetry, Flit
virtualenv
Virtualenv is a virtual environment manager. These isolated environments are a standalone python with a specific set of pre-installed packages. Using virtualenv means that you do not need to install packages into the python system by default.
For what : separation of dependencies, support for different versions of python by one system, easy movement of dependencies.
When used : You need to write code, and this requires a version of python different from your default system version of python.
Alternatives : Docker or something similar.
Pip
Pip is the most common package manager in python. It allows you to install local or remote packages in your virtual environment or Python system.
For what : installing and removing packages, tracking versions of packages that you use.
When used : Always.
Alternatives : Poetry, Conda
Package creation and distribution
For more thorough review, python.org has a separate page: packaging.python.org
distutils
distutils is the predecessor of setuptools. The latter actively uses the functional distutils, so it is often necessary to interact with this tool. Distutils is not exactly the tool you should have in your arsenal, but you need to know how it fits into the big picture.
Pypi
Pypi or Python Package Index is a large repository that contains all of your favorite Python modules. For example, the same pip takes builds from there.
For what : To publish your code.
When used : When there is a package that you want to show the community.
Pypiserver
Pypiserver is one of the implementations of the Package Index API used by Pypi. You can create your own repository, for example, for your entire company and publish packages without making public releases.
What for : Create your own repositories within the organization.
When used : When your code does not need publicity, but you need full control over it.
Alternatives : Warehouse (used by Pypi), djangopypi
Poetry
Poetry is an alternative package management system that replaces setuptools, pip and some other tools built on their basis. This is an attempt to completely revise how the package system works in Python. Currently, poetry has a lot of positive reviews, but is not the most common tool.
For what : processing and distributing packages, managing dependencies, preventing problems with resolving dependencies.
When used : When you are planning a new project and you are not afraid to use highly specialized tools.
Alternatives : Pipenv
Pipenv
Pipenv , like Poetry, is a tool for structuring dependencies and configuring Python projects in a more sane way. With Pipfile, it manages the dependencies of your project and ensures consistency and ease of use.
For what : processing and distributing packages, managing dependencies.
When used : you need a tool like Poetry that will cause less questions.
Alternatives : Poetry.
Documentation
Sphinx
Sphinx is a tool for creating documentation. It was originally created to handle Python documentation, but has become a public tool. It is the most common option for projects in Python.
For what : creating PDF or HTML documents using a markup language from reStructuredText sources.
When used : When your project, API or code requires external documentation.
Alternatives : Docutils, Doxygen
autodoc
autodoc is a fundamental extension for Sphinx that allows you to create reStructuredText files from Python source code with signatures for each class, function, module, and so on.
For what : documenting your code or API.
When used : In fact, every time you use Sphinx.
Alternatives : autosummary
Testing
py.test
py.test - in my opinion, is the best package for testing in Python. It has many functions, although not all of them are properly disclosed, so it will take some time to search for all the features that py.test provides.
For what : testing your code.
When used : Always when you are too lazy to test manually.
Alternatives : unittest, nose
Hypothesis
Hypothesis is a tool for testing individual properties. In short, it generates random test scripts according to your specifications until it finds a script in which the test fails. Spend some time learning the principles before you start using this tool.
For what : testing the code, especially data processing.
When used : When you need to test non-trivial logic with a wide range of input values (numbers, strings, structured data).
tox
tox is a virtual environment manager for testing. This means that you can configure it to run tests in clean, customizable virtual environments to ensure that your code can work in a variety of environments.
For what : for code that should run in different conditions and environments. Also useful for CI.
When used : When it is necessary for your code to be supported by different versions of Python, run in different environments and on different operating systems.
Alternatives : bash scrips, CI pipelines
Other tools
pyenv
pyenv is the python version manager. It aims to simplify the local workflow of developers when working with multiple versions.
For what : start different projects with different versions of Python.
When used : You need to work with global versions of Python and you have a lot of them.
Alternatives : manual management, virtualenv, Poetry, Pipenv
PyScaffold
PyScaffold is a tool for initializing a project structure in a standardized way and providing some of the tools listed above without having to manually configure them. Very flexible.
For what : to load projects, work with several projects with the same toolkit and structure.
When used : always (if you are familiar with this tool, but do not try to use it for the first time when you are in a hurry)
Alternatives : python-project-template, Cookiecutter
flake8
flake8 is one of the most popular Python linters. It runs various scripts to verify that your code meets the requirements of the Python style guide ( PEP-8 ).
For what : checking your project for a good style of writing code.
When used : every time your project needs to be read by someone or by you.
Alternatives : pylint
Black
Black automatically formats the code. This means that instead of simply checking your code for compliance with the standards, Black will change it himself so that it meets them.
For what : automatic code formatting.
When used : When you have no problems in order to refuse manual control of your code.
Alternatives : autopep8, yapf
That's all. We are waiting for your comments ;-).
Source: https://habr.com/ru/post/456624/
All Articles