Notes on the Python object system part 1

A few notes about the python object system. Designed for those who already know how to program in python. We are talking only about new classes (new-style classes) in python 2.3 and higher. This article describes what objects are and how attributes are searched.

Objects

All data in python are objects. Each object has 2 special attributes __class__ and __dict__.

• __class__ - defines a class or type, of which the object is an instance. The type (or class of the object) determines its behavior; all objects, including embedded ones, have it. Type and class are different names for the same. x .__ class__ <==> type (x).
• __dict__ dictionary, giving access to the internal namespace, almost all objects have it, many built-in types do not have it.

Examples

>>> def foo (): pass
...
>>> foo . __class__
<type 'function'>
>>> foo . __dict__
{}
>>> ( 42 ) . __dict__
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'int' object has no attribute '__dict__'
>>> ( 42 ) . __class__
<type 'int'>
>>> class A ( object ):
... qux = 'A'
... def __init__ ( self , name):
... self . name = name
... def foo ( self ):
... print 'foo'
...
>>> a = A( 'a' )

A also has __dict__ and __class__:
')
>>> a . __dict__ {'name': 'a'}
>>> a . __class__
<class '__main__.A'>
>>> type (a)
<class '__main__.A'>
>>> a . __class__ is type (a)
True

Class and type are the same.

>>> a . __class__ is type (a) is A
True

a .__ dict__ is a dictionary that contains internal (or object-specific) attributes, in this case 'name'. And in a .__ class__ class (type).

And, for example, in the class methods, the assignment self.foo = bar is almost identical to self .__ dict __ ['foo'] = bar or comes down to a similar call.

In __dict__ of an object there are no class methods, descriptors, class variables, properties, static class methods, they are all determined dynamically using a class from the __class__ attribute, and are specific to the class (type) of the object, and not to the object itself.

Example. Redefine the object class a:

>>> class B ( object ):
... qux = 'B'
... def __init__ ( self ):
... self . name = 'B object'
... def bar ( self ):
... print 'bar'
...
>>> a . __dict__
{'name': 'a'}
>>> a . foo()
foo
>>> a . __class__
<class '__main__.A'>
>>> a . __class__ = B
>>> a . __class__
<class '__main__.B'>

Look what has changed.

The value of a.name remains the same, i.e. __init__ was not called when changing class.

>>> a . __dict__
{'name': 'a'}
Access to class variables and “past” methods of class A is missing:
>>> a . foo()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'B' object has no attribute 'foo'
But class variables and class B methods are accessed:
>>> a . bar()
bar
>>> a . qux
'B'

Working with the attributes of an object: setting, deleting and searching, is equivalent to calling the built-in functions settattr, delattr, getattr:

ax = 1 <==> setattr (a, 'x', 1)
del ax <==> delattr (a, 'x')
ax <==> getattr (a, 'x')

It should be understood that setattr and delattr affect and change only the object itself (more precisely, a .__ dict__), and do not change the class of the object.

qux - is a class variable, i.e. it "belongs" to class B, and not to object a:

>>> a . qux
'B'
>>> a . __dict__
{'name': 'a'}

If we try to remove this attribute, we get an error, because delattr will attempt to remove an attribute from a .__ dict__

>>> delattr (a, 'qux' )
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: qux
>>> del a . qux
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: qux
>>> a . qux
'B'
>>>

Further, if we try to change (set) an attribute, setattr will put it in __dict__, specific to this particular object.

>>> b = B()
>>> b . qux
'B'
>>> a . qux = 'myB'
>>> a . qux
'myB'
>>> a . __dict__
{'qux': 'myB', 'name': 'a'}
>>> b . qux
'B'
>>>

Well, since there is a 'qux' in the __dict__ object, you can delete it with delattr:

>>> del a . qux

After deletion, a.qux will return the value of the class variable:

>>> a . qux
'B'
>>> a . __dict__
{'name': 'a'}

So:

• The class for an object is the value of the special attribute __class__ and can be changed. (Although the official documentation states that there are no guarantees, but in fact you can)
• almost every object has its own namespace (attribute), access (not always full), which is performed using the special attribute __dict__
• the actual class only affects the search for attributes that are not found in __dict__, such as: class methods, descriptors, magic methods, class variables, and so on.

Objects and Classes

Classes are objects, and they also have special attributes __class__ and __dict__.

>>> class A ( object ):
... pass
...

The class type is type.

>>> A . __class__
<type 'type'>

True __dict__ for classes is not quite a dictionary

>>> A . __dict__
<dictproxy object at 0x1111e88>

But __dict__ is responsible for accessing the internal namespace, which stores methods, descriptors, variables, properties, and so on:

>>> dict (A . __dict__)
{'__module__': '__main__', 'qux': 'A', '__dict__': <attribute '__dict__' of 'A' objects>, 'foo': <function foo at 0x7f7797a25c08>, '__weakref__': <attribute '__weakref__' of 'A' objects>, '__doc__': None}
>>> A . __dict__ . keys()
['__module__', 'qux', '__dict__', 'foo', '__weakref__', '__doc__']<

In classes besides __class__ and __dict__, there are several other special attributes: __bases__ is the list of direct parents, __name__ is the name of the class. [one]

Classes can be considered such extensions of ordinary objects that implement the type interface. The set of all classes (or types) belong to the set of all objects, or rather, is its subset. In other words, any class is an object, but not every object is a class. Let's agree to call regular objects regular objects that are not classes.

A small demonstration that will be better understood later.
A class is an object.
>>> class A ( object ):
... pass
...

>>> isinstance (A, object )
True

The number is also an object.

>>> isinstance ( 42 , object )
True

A class is a class (that is, a type).

>>> isinstance (A, type )
True

But the number is not a class (type). (What type will be explained later)

>>> isinstance ( 42 , type )
False
>>>

Well, a is also an ordinary object.

>>> a = A()
>>> isinstance (a, A)
True
>>> isinstance (a, object )
True
>>> isinstance (a, type )
False

And A has just one direct parent class, object.

>>> A . __bases__
(<type 'object'>,)

Some of the special parameters can even be changed:

>>> A . __name__
'A'
>>> A . __name__ = 'B'
>>> A
<class '__main__.B'>

Using getattr, we get access to class attributes:

>>> A . qux
'A'
>>> A . foo
<unbound method A.foo>
>>>

Attribute search in regular object

In the first approximation, the search algorithm looks like this: first it is searched in __dict__ of an object, then it is searched in __dict__ dictionaries of the class of an object (which is determined using __class__) and __dict__ of its base classes in recursive order.

Example.

>>> class A ( object ):
... qux = 'A'
... def __init__ ( self , name):
... self . name = name
... def foo ( self ):
... print 'foo'
...
>>> a = A()
>>> b = A()

Because in ordinary objects a and b are not in __dict__ the attribute 'qux', then the search continues in the internal dictionary __dict__ of their type (class), and then according to __dict__ to the dictionaries of the parents in a certain order:

>>> b . qux
'A'
>>> A . qux
'A'

We change the qux attribute of class A. And accordingly, the values ​​that return instances of class A - a and b should change:

>>> A . qux = 'B'
>>> a . qux
'B'
>>> b . qux
'B'
>>>

Similarly, in runtime, you can add a method to the class:

>>> A . quux = lambda self : 'i have quux method'
>>> A . __dict__[ 'quux' ]
<function <lambda> at 0x7f7797a25b90>
>>> A . quux
<unbound method A.<lambda>>

And access to it will appear in copies:

>>> a . quux()
'i have quux method'

Just like with any other objects, you can remove a class attribute, for example, the class variable qux:

>>> del A . qux

She retire from __dict__

>>> A . __dict__[ 'qux' ]
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
KeyError: 'qux'

And access at copies will be gone.

>>> a . qux
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'A' object has no attribute 'qux'
>>>

Classes have almost the same attribute search as regular objects, but there are differences: the search begins with its own __dict__ dictionary, and then comes the search for __dict__ dictionaries of superclasses (which are stored in __bases__) by a certain algorithm, and then by class in __class__ and its superclasses. (More on this later.)

Links

• Unifying types and classes in Python is the main document explaining what, how and why in new classes.
• Making Types Look More Like Classes - PEP 252, describing the difference between the old classes and the new ones.
• Built-in functions - a detailed description of the work of all built-in functions.
• Data model - a detailed description of the python data model.
• Python types and objects - an explanation of the python object model on simple examples with pictures.

Notes

[1] About __module__ and __doc__ for simplicity, for now let's forget. A complete list of class attributes can be found in the documentation.

Read more

Notes on the Python object system part 2
Notes on the Python object system part 3