Notes on the Python object system part 2

The second part of the notes about the python object system (the first part is here ). This article tells you what classes are, metaclasses, type, object, and how attributes are searched for in a class.

Classes

Classes (types) are object factories. Their main task is to create objects with specific behavior.

Classes define the behavior of objects using their attributes (which are stored in the __dict__ class): methods, properties, class variables, descriptors, as well as using attributes inherited from the parent classes.
')
The instantiation of a regular object occurs in 2 stages: first, its creation, then initialization. Accordingly, the class __new__ class method is first launched, which returns an object of this class, then the class __init__ method is executed, which initializes the already created object.

def __new __ (cls, ...) is a static method (but you can not declare it as such), which creates an object of class cls.

def __init __ (self, ...) is a class method that initializes the created object.

For example, declare a class:

>>> class A ( object ):
... pass
...
>>>

For class A, neither __new__ nor __init__ is defined. In accordance with the attribute search algorithm for a class (type), which is not to be confused with the attribute search algorithm for ordinary objects, when the class does not find them in its __ dict__, it will look for these methods in __ ict __ its base (parent) classes.

Class A has a built-in object class as a parent. So he will look for them in object .__ dict__

And find:

>>> object . __dict__[ '__init__' ]
<slot wrapper '__init__' of 'object' objects>
>>> object . __dict__[ '__new__' ]
<built-in method __new__ of type object at 0x82e780>
>>>

Since there are such methods, it means that a = A () is similar to a sequence of calls:

a = object .__ new __ (A)
object .__ init __ (a)

In general, using super, which implements the attribute search algorithm for parent classes [1]:

a = super (A, A) .__ new __ (A)
super (A, A) .__ init __ (a)

Example.

>>> class A ( object ):
... def __new__ (cls):
... obj = super (A, cls) . __new__(cls)
... print 'created object' , obj
... return obj
... def __init__ ( self ):
... print 'initing object' , self
...
>>> A()
created object <__main__.A object at 0x1620ed0>
initing object <__main__.A object at 0x1620ed0>
<__main__.A object at 0x1620ed0>
>>>

Singleton v.1

Understanding how the object is created, you can write the implementation of the loner pattern.

We need to ensure that the class has only one instance. Those. when calling a class constructor, always return the same instance of the class.

And this means that when calling the __new__ method, the same object must be returned each time. You can store the object itself, for example, in the instance class variable.

The result is:

>>> class C ( object ):
... instance = None
... def __new__ (cls):
... if cls . instance is None :
... cls . instance = super (C, cls) . __new__(cls)
... return cls . instance
...
>>> C() is C()
True
>>> C() . x = 1
>>> c = C()
>>> d = C()
>>> c . x
1
>>> d . x
1
>>> c . x =2
>>> d . x
2
>>> c . x
2

Classes and metaclasses.

For a class (type), as well as for a regular object, there is a class (type) that creates classes and defines the behavior of the class . This class is called a metaclass.

Creating a class, just like an ordinary object, happens by calling the constructor, but since There are several additional special attributes in the class that must be initialized, and the corresponding required parameters are passed to the constructor.

XClass = XMetaClass (name, bases, attrs)

Then, right after creation
XClass .__ name__ equals name,
XClass .__ bases__ equals bases,
XClass .__ dict__ is equal to attrs, and
XClass .__ class__ is equal to XMetaClass

By default, for all classes defined, the metaclass is type.

In this way.

>>> class A ( object ):
... pass
...

Equivalently, by analogy with ordinary objects:

>>> type ( 'A' , ( object ,), {})
<class '__main__.A'>

And this:

>>> class B (A):
... def foo ( self ):
... 42
...

is equivalent to

>>> type ( 'B' , (A,), { 'foo' : lambda self : 42 })

When defining a class, you can define your metaclass with
class variable __metaclass__:

>>> class A ( object ):
... __metaclass__ = Meta
...
>>>

Which is equivalent: A = Meta ('A', (object,), {})

About type and object

First of all, type and object are objects. And, like all decent objects, they have special attributes __class__ and __dict__:

>>> object . __class__
<type 'type'>
>>> type . __class__
<type 'type'>

>>> object . __dict__
<dictproxy object at 0x7f7797a1cf30>
>>> type . __dict__
<dictproxy object at 0x7f7797a1cfa0>

Moreover, object and type are objects of type (classes), and they also have special attributes __name__, __bases___:

>>> object . __name__
'object'
>>> type . __name__
'type'
>>> object . __bases__
()
>>> type . __bases__
(<type 'object'>,)
>>>

Instances of the type or class object are objects (any). Those. any object is an instance of the object class:

>>> isinstance ( 1 , object )
True
>>> isinstance ( setattr , object )
True
>>> isinstance ( "foo" , object )
True
>>> isinstance (A, object )
True

Even a function is an object:
>>> def bar ():
... pass
...
>>> isinstance (bar, object )
True

In addition, the object class itself is its own instance:

>>> isinstance ( object , object )
True

type is also an instance of it:

>>> isinstance ( type , object )
True

Instantiation - object () returns the simplest and most common object:

>>> o = object ()

Which even __dict__ is not, there is only __class__.

>>> o . __dict__
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'object' object has no attribute '__dict__'
>>> o . __class__
<type 'object'>
>>>

Instances of a class or type type are just other classes or other types:

The number is not a class

>>> isinstance ( 1 , type )
False

String too

>>> isinstance ( "foo" , type )
False

The built-in setattr function is also not a class.

>>> isinstance ( setattr , type )
False

A class is a class.

>>> isinstance (A, type )
True

The row type is a class.

>>> isinstance ( "foo" . __class__, type )
True

Because object and type are also classes, they are instances of the type class:

>>> isinstance ( object , type )
True
>>> isinstance ( type , type )
True
>>>

Because the set of classes (types) are a subset of the set of objects, it is logical to assume that type is a subclass of object, i.e.

>>> issubclass ( type , object )
True
>>> issubclass ( object , type )
False

type is just a class whose instances are other classes. (i.e. metaclass). And the classes themselves can be considered an extension of simple, ordinary objects.

Thus, when we inherit a class from object, this class automatically inherits the behavior of the object class, i.e. when instantiating, it will return a regular object. And when we inherit from the type class, we also automatically inherit the behavior of the type class, i.e. when installed, a class will be created. And the class that creates the class is called the metaclass.

So, to define just a class, you need to inherit it from object, to define a metaclass - we inherit it from type.

And yet: do not confuse type (a) and type (name, bases, attrs).
type (a) is a call with one argument, returns the type of the object,
a type (name, bases, attrs) - a call with three arguments is a call to the class constructor.

About finding attributes in a class

As already noted, the algorithm for finding attributes in a regular object, but there are some subtleties, because types (classes) have __bases__ - parent classes (types).

If the attribute is in __dict__ it is returned, then the search for base classes from __bases__ takes place, and then there is an appeal to __dict__ __class __ 'a (that is, in fact, the metaclass) and its (metaclass) parent classes (metaclasses).

A small example:

>>> class Ameta ( type ):
... def foo (cls):
... print 'Ameta.foo'
...
>>> class A ( object ):
... __metaclass__ = Ameta
...
>>> A . foo()
Ameta.foo

All that is defined in the metaclass is available for the class, but not available for the class instance — ordinary objects, since Attribute search in a regular object is conducted only in class __dict__ dictionaries.

>>> a = A()
>>> a . foo
Traceback (most recent call last):
File "<stdin>" , line 1 , in <module>
AttributeError : 'A' object has no attribute 'foo'

In A .__ dict__ 'foo' is not:

>>> A . __dict__[ 'foo' ]
Traceback (most recent call last):
File "<stdin>" , line 1 , in <module>
KeyError : 'foo'

But he is in the metaclass, therefore:

>>> A . foo()
Ameta.foo

>>> class B (A):
... @classmethod
... def foo (cls):
... print 'B.foo'
...
>>> B . foo # .. foo B.__dict__ B.__dict__['foo']
<bound method Ameta.foo of <class '__main__.B'>>
>>> B . foo()
B.foo

>>> class C (B):
... pass
...
>>> C . foo() # B.
B.foo

C foo B.

>>> c = C()
>>> c . foo()
B.foo

>>> class D (A):
... pass
...
>>> D . foo()
Ameta.foo

And instance D will not find:

>>> d = D()
>>> d . foo()
Traceback (most recent call last):
File "<stdin>" , line 1 , in <module>
AttributeError : 'D' object has no attribute 'foo'

Metaclasses

Metaclasses are factories of classes (or types). The instantiation of a class also takes place in 2 stages - the creation of an object of type (class) and its initialization. This is also done using two metaclass methods. First, the __new__ method of the metaclass is called with the parameters necessary to create a class - name, bases, attrs, and then __init__ with the same parameters and the class already created.

Example.

>>> class Meta ( type ):
... pass
...
>>> Meta( 'A' , ( object ,), {})
<class '__main__.A'>

At the beginning, the metaclass Meta searches for the __new__ method in its __dict__ dictionary, does not find it there, and begins to search in its parental classes __dict__ (i.e., metaclasses, in this case type), i.e. there is a normal attribute search in the class. As a result of the execution of __new__ with the appropriate parameters, it receives a new class, which is then initialized by calling the __init__ method of the metaclass.

In a completely expanded form it turns out:

cls = type .__ dict __ ['__ new __'] (Meta, 'A', (object,), {})
type .__ dict __ ['__ init __'] (cls, 'A', (object,), {})

Or using super

cls = super (Meta, Meta) .__ new __ (Meta, 'A', (object,), {})
super (Meta, Meta) .__ init __ (cls, 'A', (object,), {})

It should be noted that, in contrast to instantiating ordinary objects, it is not the object .__ new__ and object .__ init__ that is used, but the type .__ new__ and type .__ init__. Object .__ new__ and type .__ new__ have different signatures, and object .__ new__ returns a regular object (regular object), and type .__ new__ - an object of type (typeobject), i.e. classroom

Let's see how this all works on an example.

>>> class Meta ( type ):
... def __new__ (mcls, name, bases, attrs):
... print 'creating new class' , name
... return super (Meta, mcls) . __new__(mcls, name, bases, attrs)
... def __init__ (cls, name, bases, attrs):
... print 'initing new class' , name
...
...
>>> class A ( object ):
... __metaclass__ = Meta
...
creating new class A
initing new class A

During the instantiation of a simple object, no label is displayed.

>>> a = A()
>>>

In addition, respectively, in the methods __new__ and __init__ of the metaclass you can change everything: name, list of superclasses, attributes.

Links

• Unifying types and classes in Python is the main document explaining what, how and why in new classes.
• Making Types Look More Like Classes - PEP 252, describing the difference between the old classes and the new ones.
• Built-in functions - a detailed description of the work of all built-in functions.
• Data model - a detailed description of the python data model.
• Python types and objects - an explanation of the python object model on simple examples with pictures.

Notes

[1] For more information about super - here .

Read more

Notes on the Python object system part 3