Introduction to Python Type Annotations

Introduction

Illustration author - Magdalena Tomczyk

Python is a language with dynamic typing and allows us to quite freely operate with variables of different types. However, when writing code, we somehow assume which types of variables will be used (this may be caused by a restriction of the algorithm or a business logic). And for the correct operation of the program, it is important for us to find errors related to the transfer of data of the wrong type as early as possible.

Keeping the idea of ​​dynamic duck typing in modern versions of Python (3.6+) supports annotations for the types of variables, class fields, arguments, and return values ​​of functions:

• PEP 3107 - Function Annotations
• PEP 484 - Type Hints
• PEP 526 - Syntax for Variable Annotations
• typing

Type annotations are simply read by the Python interpreter and are no longer processed, but are available for use from third-party code and are primarily designed for use by static analyzers.

My name is Andrey Tikhonov and I am engaged in backend-development in Lamoda.

In this article, I want to explain the basics of using type annotations and look at typical examples implemented by typing annotations.

Tools supporting annotations

Type annotations are supported by many Python IDEs that highlight incorrect code or provide hints during the typing process.

For example, this is how Pycharm looks like:

Error highlighting

Tips:

Also, type annotations are processed by console linters.

Here is the output of pylint:

 $ pylint example.py ************* Module example example.py:7:6: E1101: Instance of 'int' has no 'startswith' member (no-member) 

But for the same file that mypy found:

 $ mypy example.py example.py:7: error: "int" has no attribute "startswith" example.py:10: error: Unsupported operand types for // ("str" and "int") 

The behavior of different analyzers may differ. For example, mypy and pycharm handle the change of variable type in different ways. Further in the examples I will focus on the output of mypy.

In some examples, the code at startup can work without exceptions, but may contain logical errors due to the use of variables of the wrong type. And in some examples it may not even be executed.

The basics

Unlike older versions of Python, type annotations are not written in comments or docstring, but directly in the code. On the one hand, it breaks backward compatibility, on the other, it clearly means that it is part of the code and can be processed accordingly.

In the simplest case, the annotation contains the directly expected type. More complex cases will be discussed below. If the base class is specified as an annotation, it is acceptable to pass instances of its heirs as values. However, you can use only those features that are implemented in the base class.

Annotations for variables are written with a colon after the identifier. After that, there may be an initialization of the value. For example,

 price: int = 5 title: str 

The function parameters are annotated in the same way as variables, and the return value is indicated after the arrow -> and before the terminating colon. For example,

 def indent_right(s: str, width: int) -> str: return " " * (max(0, width - len(s))) + s 

For class fields, annotations must be specified explicitly when defining a class. However, analyzers can output them automatically based on the __init__ method, but in this case they will not be available during program execution. Learn more about working with annotations in runtime in the second part of the article.

 class Book: title: str author: str def __init__(self, title: str, author: str) -> None: self.title = title self.author = author b: Book = Book(title='Fahrenheit 451', author='Bradbury') 

By the way, when using dataclass, field types must be specified in the class. Read more about dataclass

Built-in types

Although you can use standard types as annotations, many useful things are hidden in the typing module.

Optional

If you mark a variable with type int and try to assign it None , there will be an error:

Incompatible types in assignment (expression has type "None", variable has type "int")

For such cases, an Optional type annotation is provided in the typing module indicating the specific type. Note that the type of the optional variable is indicated in square brackets.

 from typing import Optional amount: int amount = None # Incompatible types in assignment (expression has type "None", variable has type "int") price: Optional[int] price = None 

Any

Sometimes you don't want to limit the possible types of a variable. For example, if it is really not important, or if you plan to do the processing of different types on your own. In this case, you can use the Any annotation. The following code mypy will not swear:

 unknown_item: Any = 1 print(unknown_item) print(unknown_item.startswith("hello")) print(unknown_item // 0) 

The question may arise, why not use an object ? However, in this case it is assumed that at least any object can be transmitted, it can only be treated as an instance of object .

 unknown_object: object print(unknown_object) print(unknown_object.startswith("hello")) # error: "object" has no attribute "startswith" print(unknown_object // 0) # error: Unsupported operand types for // ("object" and "int") 

Union

For cases where it is necessary to allow the use of not any type, but only some, you can use typing.Union annotation with the list of types in square brackets.

 def hundreds(x: Union[int, float]) -> int: return (int(x) // 100) % 10 hundreds(100.0) hundreds(100) hundreds("100") # Argument 1 to "hundreds" has incompatible type "str"; expected "Union[int, float]" 

By the way, the Optional[T] annotation is equivalent to Union[T, None] , although such an entry is not recommended.

Collections

The type annotation mechanism supports the generics mechanism ( Generics , in more detail in the second part of the article), which allow specifying types of elements stored in containers for containers.

Lists

In order to indicate that a variable contains a list, you can use the list type as an annotation. However, if you want to specify which elements the list contains, it will no longer be suitable for such an annotation. For this, there is typing.List . In the same way that we specified the type of an optional variable, we specify the type of list elements in square brackets.

 titles: List[str] = ["hello", "world"] titles.append(100500) # Argument 1 to "append" of "list" has incompatible type "int"; expected "str" titles = ["hello", 1] # List item 1 has incompatible type "int"; expected "str" items: List = ["hello", 1] 

It is assumed that the list contains an indefinite number of similar elements. But there are no restrictions on the annotation of the element: you can use Any , Optional , List and others. If the element type is not specified, it is assumed to be Any .

In addition to the list, there are similar annotations for sets: typing.Set and typing.FrozenSet .

Tuples

Tuples, unlike lists, are often used for elements of different types. The syntax is similar with one difference: the type of each element of the tuple is indicated in square brackets separately.

If you plan to use a tuple similar to the list: store an unknown number of the same type of elements, you can use the ellipsis ( ... ).

Tuple annotation without specifying element types works like Tuple[Any, ...]

 price_container: Tuple[int] = (1,) price_container = ("hello") # Incompatible types in assignment (expression has type "str", variable has type "Tuple[int]") price_container = (1, 2) # Incompatible types in assignment (expression has type "Tuple[int, int]", variable has type "Tuple[int]") price_with_title: Tuple[int, str] = (1, "hello") prices: Tuple[int, ...] = (1, 2) prices = (1, ) prices = (1, "str") # Incompatible types in assignment (expression has type "Tuple[int, str]", variable has type "Tuple[int, ...]") something: Tuple = (1, 2, "hello") 

Dictionaries

Dictionaries use typing.Dict . The key type and value type are annotated separately:

 book_authors: Dict[str, str] = {"Fahrenheit 451": "Bradbury"} book_authors["1984"] = 0 # Incompatible types in assignment (expression has type "int", target has type "str") book_authors[1984] = "Orwell" # Invalid index type "int" for "Dict[str, str]"; expected type "str" 

Similarly, typing.DefaultDict and typing.OrderedDict

The result of the function

To specify the type of the result of the function, you can use any annotation. But there are some special cases.

If the function returns nothing (for example, like print ), its result is always None . For annotations we also use None .

Valid options for completing such a function are: explicitly returning None , returning without specifying a value, and ending without calling return .

 def nothing(a: int) -> None: if a == 1: return elif a == 2: return None elif a == 3: return "" # No return value expected else: pass 

If the function never returns control (for example, as sys.exit ), you should use the NoReturn annotation:

 def forever() -> NoReturn: while True: pass 

If this is a generator function, that is, its body contains a yield operator, you can use the Iterable[T] annotation or the Generator[YT, ST, RT] for the return function:

 def generate_two() -> Iterable[int]: yield 1 yield "2" # Incompatible types in "yield" (actual type "str", expected type "int") 

Instead of conclusion

For many situations in the typing module there are suitable types, however I will not consider everything, since the behavior is similar to that considered.
For example, there is an Iterator as a generic version for collections.abc.Iterator , typing.SupportsInt to indicate that an object supports the __int__ method, or Callable for functions and objects that support the __call__ method

The standard also defines the format of annotations in the form of comments and stub files that contain information only for static analyzers.

In the next article I would like to dwell on the mechanism of generics, processing annotations in runtime and overloading methods.