As one of the StackOverflow users said, “using SO is like doing a list with a hashtable instead of a linked list”. We again turn to this remarkable resource, which comes across extremely detailed and understandable answers to a variety of questions.

This time we will discuss what metaclasses are, how, where and why to use them, and also why it is usually not worth doing.

Classes as objects

Before you study metaclasses, you need to understand the classes well, and the classes in Python are a very specific thing (based on ideas from the Smalltalk language).

In most languages, a class is just a piece of code that describes how to create an object. In general, this is also true for Python:

>>> class ObjectCreator(object): ... pass ... >>> my_object = ObjectCreator() >>> print my_object <__main__.ObjectCreator object at 0x8974f2c> 

But in Python, a class is something more - classes are also objects.
')
Once the class keyword is used, Python executes the command and creates an object . Instruction

  >>> class ObjectCreator(object): ... pass ... 

will create an object in memory called ObjectCreator .

This object (class) itself can create objects (instances), therefore it is a class.

However, this is an object, and therefore:

• it can be assigned to a variable,
• you can copy it
• you can add an attribute to it,
• it can be passed to the function as an argument,

Dynamic class creation

Since classes are objects, you can create them on the go, just like any object.

For example, you can create a class in a function using the class keyword:

  >>> def choose_class(name): ... if name == 'foo': ... class Foo(object): ... pass ... return Foo #  ,    ... else: ... class Bar(object): ... pass ... return Bar ... >>> MyClass = choose_class('foo') >>> print MyClass #   ,    <class '__main__.Foo'> >>> print MyClass() #      <__main__.Foo object at 0x89c6d4c> 

However, this is not very dynamic, because you still need to write the whole class yourself.

Since classes are objects, they must be generated by something.

When the class keyword is used, Python creates this object automatically. But like most things in Python, there is a way to do it manually.

Remember the type function? Good old function that allows you to determine the type of object:

 >>> print type(1) <type 'int'> >>> print type("1") <type 'str'> >>> print type(ObjectCreator) <type 'type'> >>> print type(ObjectCreator()) <class '__main__.ObjectCreator'> 

In fact, the type function has a completely different use: it can also create classes on the fly. type takes a class description as input and convokes the class.

(I know, it’s foolish that the same function can be used for two completely different things depending on the arguments passed. This is done for backward compatibility)

type works as follows:

  type(< >, <  >, #  ,    <,     >) 

For example,

 >>> class MyShinyClass(object): ... pass 

can be created manually as follows:

  >>> MyShinyClass = type('MyShinyClass', (), {}) #  - >>> print MyShinyClass <class '__main__.MyShinyClass'> >>> print MyShinyClass() #    <__main__.MyShinyClass object at 0x8997cec> 

You may have noticed that we use “MyShinyClass” both as the name of the class and as the name for a variable containing a reference to the class. They may be different, but why complicate things?

type accepts a dictionary that defines class attributes:

 >>> class Foo(object): ... bar = True 

can be rewritten as

  >>> Foo = type('Foo', (), {'bar':True}) 

and use as a regular class

  >>> print Foo <class '__main__.Foo'> >>> print Foo.bar True >>> f = Foo() >>> print f <__main__.Foo object at 0x8a9b84c> >>> print f.bar True 

Of course, you can inherit from it:

  >>> class FooChild(Foo): ... pass 

will turn into

  >>> FooChild = type('FooChild', (Foo,), {}) >>> print FooChild <class '__main__.FooChild'> >>> print FooChild.bar # bar is inherited from Foo True 

At some point, you will want to add methods to your class. To do this, simply define a function with the desired signature and assign it as an attribute:

 >>> def echo_bar(self): ... print self.bar ... >>> FooChild = type('FooChild', (Foo,), {'echo_bar': echo_bar}) >>> hasattr(Foo, 'echo_bar') >>> hasattr(FooChild, 'echo_bar') True >>> my_foo = FooChild() >>> my_foo.echo_bar() True 

It is already clear what I'm getting at: in Python, classes are objects and you can create classes on the go.

This is exactly what Python does when the class keyword is used, and it does this using metaclasses.

What is a metaclass (finally)

Metaclass is the “thing” that creates classes.

We create a class to create objects, right? And classes are objects. Metaclass is what creates these very objects. They are classes of classes, you can imagine it as follows:

  MyClass = MetaClass() MyObject = MyClass() 

We have already seen that type allows you to do something like this:

  MyClass = type('MyClass', (), {}) 

This is because the type function is actually a metaclass. type is a metaclass that Python internally uses to create all classes.

The natural question is: why does he spell his name in lower case, not Type ?

I suppose it's just to match str , the class for creating string objects, and int , the class for creating integer objects. type is just a class for creating class objects.

This is easily verified using the __class__ attribute:

In python, everything (in general, everything!) Is objects. Including numbers, strings, functions, and classes — they are all objects and all were created from a class:

  >>> age = 35 >>> age.__class__ <type 'int'> >>> name = 'bob' >>> name.__class__ <type 'str'> >>> def foo(): pass >>> foo.__class__ <type 'function'> >>> class Bar(object): pass >>> b = Bar() >>> b.__class__ <class '__main__.Bar'> 

And what is __class__ for each __class__ ?

  >>> a.__class__.__class__ <type 'type'> >>> age.__class__.__class__ <type 'type'> >>> foo.__class__.__class__ <type 'type'> >>> b.__class__.__class__ <type 'type'> 

So, a metaclass is just a thing that creates class objects.

If you want, you can call it a “class factory”

type is a built-in metaclass that Python uses, but of course you can create your own.

__metaclass__ attribute

When writing a class, you can add the attribute __metaclass__ :

 class Foo(object): __metaclass__ = something... [...] 

In this case, Python will use the specified metaclass when creating the Foo class.

Carefully, there is a subtlety!

Although you write class Foo(object) , an object class is not yet created in memory.

Python will look for __metaclass__ in the class definition. If he finds it, he uses it to create the class Foo . If not, it will use type .

That is when you write

 class Foo(Bar): pass 

Python does the following:

Does the Foo class have the attribute __metaclass__ ?

If so, it creates in memory an object class named Foo , using what is specified in __metaclass__ .

If Python does not find __metaclass__ , he searches for __metaclass__ in the parent class Bar and tries to do the same.

If __metaclass__ is not in any of the parents, Python will search for __metaclass__ at the module level.

And if he can't find any __metaclass__ at all, he uses type to create a class object.

Now an important question: what can be put in __metaclass__ ?

Answer: anything that can create classes.

And what creates classes? type or any of its subclasses, as well as anything that uses them.

Custom metaclasses

The main purpose of metaclasses is to automatically change the class at the time of creation.

This is usually done for an API when you want to create classes in accordance with the current context.

Imagine a stupid example: you decided that all classes in your module attribute names should be written in upper case. There are several ways to do this, but one of them is to set __metaclass__ at the module level.

In this case, all classes of this module will be created using the specified meclass, and we just have to force the metaclass to translate the names of all attributes into upper case.

Fortunately, __metaclass__ can be any callable object, not necessarily a formal class (I know that something with the word “class” in the name does not have to be a class, what nonsense? However, this is useful).

So we start with a simple example using a function.

 #        , #     `type` def upper_attr(future_class_name, future_class_parents, future_class_attr): """  -,        """ #   ,    '__' attrs = ((name, value) for name, value in future_class_attr.items() if not name.startswith('__')) #      uppercase_attr = dict((name.upper(), value) for name, value in attrs) #     `type` return type(future_class_name, future_class_parents, uppercase_attr) __metaclass__ = upper_attr #        class Foo(object): #    __metaclass__ ,       bar = 'bip' print hasattr(Foo, 'bar') # Out: False print hasattr(Foo, 'BAR') # Out: True f = Foo() print f.BAR # Out: 'bip' 

And now the same thing, only using the real class:

 # ,  `type`     ,  `str`  `int`, #       class UpperAttrMetaclass(type): #  __new__   __init__ #       , #     __init__   ,    . #     __new__,     , #    #       ,     , #    __new__. #    -  __init__,  . #        __call__, #      . def __new__(upperattr_metaclass, future_class_name, future_class_parents, future_class_attr): attrs = ((name, value) for name, value in future_class_attr.items() if not name.startswith('__')) uppercase_attr = dict((name.upper(), value) for name, value in attrs) return type(future_class_name, future_class_parents, uppercase_attr) 

But this is not exactly OOP. We directly call type and do not overload the call to the __new__ parent. Let's do that:

 class UpperAttrMetaclass(type): def __new__(upperattr_metaclass, future_class_name, future_class_parents, future_class_attr): attrs = ((name, value) for name, value in future_class_attr.items() if not name.startswith('__')) uppercase_attr = dict((name.upper(), value) for name, value in attrs) #   type.__new__ #  ,   return type.__new__(upperattr_metaclass, future_class_name, future_class_parents, uppercase_attr) 

You may have noticed the additional argument upperattr_metaclass . There is nothing special about it: the method always receives the current instance as the first argument. In the same way as you use self in normal methods.

Of course, the names I used here are so long for clarity, but like self , there is an agreement on the naming of all these arguments. So the real metaclass looks something like this:

 class UpperAttrMetaclass(type): def __new__(cls, name, bases, dct): attrs = ((name, value) for name, value in dct.items() if not name.startswith('__')) uppercase_attr = dict((name.upper(), value) for name, value in attrs) return type.__new__(cls, name, bases, uppercase_attr) 

You can do even better by using super , which will cause inheritance (since, of course, you can create a metaclass inherited from a metaclass inherited from type ):

 class UpperAttrMetaclass(type): def __new__(cls, name, bases, dct): attrs = ((name, value) for name, value in dct.items() if not name.startswith('__')) uppercase_attr = dict((name.upper(), value) for name, value in attrs) return super(UpperAttrMetaclass, cls).__new__(cls, name, bases, uppercase_attr) 

That's all. There is nothing more to say about metaclasses.

The reason for the complexity of code using metaclasses is not in the metaclasses themselves. It is that metaclasses are usually used for all sorts of sophisticated things based on introspection, manipulation by inheritance, variables like __dict__ and the like.

Indeed, metaclasses are especially useful for all "black magic", and, therefore, complex pieces. But by themselves they are simple:

• intercept class creation
• change class
• return the modified

Why use metaclasses instead of functions?

Since __metaclass__ accepts any callable object, why would you suddenly use a class if it’s obviously more difficult?

There are several reasons for this:

• The destination is clearer. When you see UpperAttrMetaclass(type) , you immediately know what will happen next.
• You can use OOP. Metaclasses can be inherited from metaclasses, overloading parent methods.
• Better structured code. You will not use metaclasses for such simple things as in the example above. Usually it is something complicated. The ability to create multiple methods and group them in one class is very useful to make the code more readable.
• You can use __new__ , __init__ and __call__ . Of course, you can usually do everything in __new__ , but some are more comfortable using __init__
• They are called meta classes , damn it! That should mean something!

Why use metaclasses at all?

Finally, the main question. Why would anyone use some incomprehensible (and error-prone) feature?

Well, usually do not use:

Metaclasses are deep magic, about which 99% of users do not even need to think. If you think whether you need to use them - you do not need (people who really need them, know exactly why they need them, and do not need an explanation of why).
~ Guru Python Tim Peters

The main use of metaclasses is to create an API. A typical example is Django ORM.

It allows you to write something like this:

  class Person(models.Model): name = models.CharField(max_length=30) age = models.IntegerField() 

However, if you run the following code:

  guy = Person(name='bob', age='35') print guy.age 

you get not IntegerField , but int , and the value can be obtained directly from the database.

This is possible because models.Model defines __metaclass__ , which __metaclass__ some magic and turns the Person class that we just defined with a simple expression into a complex database binding.

Django makes something complex looking simple by exposing a simple API and using metaclasses, recreating the code from the API and doing all the work imperceptibly.

At last

First, you learned that classes are objects that can instantiate.

In fact, classes are instances too. Instances of metaclasses.

  >>> class Foo(object): pass >>> id(Foo) 142630324 

Anything is an object in Python: an instance of a class or an instance of a metaclass.

Except type .

type is its own metaclass. It cannot be reproduced on pure Python and is done with a small cheat at the implementation level.

Secondly, metaclasses are complex. You do not need to use them to easily change classes. This can be done in two different ways:

• by hands
• class decorators

In 99% of cases, when you need to change the class, it is better to use these two.

But in 99% of cases you don’t need to change classes at all :-)