Pseudo-practical example of closures and decorators

When I first started learning Python, I was impressed by route decorators in the famous flask framework. Of course, I guessed how they could be implemented, but as always the desire to write (and not read) surpassed the need to look at the source code of the flask, and I had to invent something that could look as lapidary as the above-mentioned decorators from flask . An exercise on closures, decorators and scope in Python could look like this:

def do_something(p): return p @implements(do_something, lambda: not p % 2) def do_mod2_something(p): return p / 2 @implements(do_something, lambda: not p % 3) def do_mod3_something(p): return p / 3 do_something(10) # returns 5 do_something(9) # returns 3 do_something(11) # returns 11 

How to implement the @implements decorator? Whether such an implementation can be used somewhere in real projects is a question that we seldom take into account when inventing exercises for ourselves to understand how certain programs work. It seemed to me that this looks like a kind of override (override) of a function that occurs in other programming languages.

Override

In languages ​​with static data typing, there is such a technique as the substitution of the function implementation. With the help of the signature at the time of compilation is selected suitable for the call function. In C ++ and Java, for example, this technique is often used to have multiple function implementations for arguments of different data types. In order to fully understand what is being discussed, below is an almost canonical example of replacing a function with C ++:
')

 #include <iostream> int sum(int a, int b) { std::cout << "int" << std::endl; return a + b; } double sum(double a, double b) { std::cout << "double" << std::endl; return a + b; } int main(void) { std::cout << sum(1, 2) << std::endl; std::cout << sum(1.1, 3.0) << std::endl; return 0; } 

In programming languages ​​with dynamic typing, there is practically no need to support implementations for different data types. However, what if we have the opportunity to run different implementations of the function depending on the values ​​of the arguments? For example, in the FSM, where for each step you need to check the current state and make the transition to another. Or in the implementation of any very platform-specific functions. Can we, in any way, without using chains from if-then-else, implement this in Python?

It seems that in Python you can implement almost everything. Of course, not without potential losses in performance, but the presence of such powerful tools as closures and decorators opens up scope for the implementation of their own bikes and unhealthy fantasies.

Functions

Functions are first class objects. This is written in every book on programming in the Python language. This makes it possible to create functions at runtime, change their attributes, and generally treat them as ordinary objects.

A lot has been written about decorators not only on this resource, so I don’t want to go deep into this topic. Closures are function objects that store an environment with them. In essence, each decorated function is a closure, carrying with it not only the function code, but also the whole environment that existed inside the decorator during the function definition:

 In [1]: def m(p): ...: def s(): ...: return p ...: return s ...: In [2]: x = m(10) In [3]: x.func_closure Out[3]: (<cell at 0x10cd547f8: int object at 0x7f89ab505860>,) 

This example shows that the x () function contains information about an integer object. This object will exist as long as the function x () exists.

In addition, the function contains information about the environment in which it was defined. To do this, use the func_globals attribute, represented by a dictionary that is amenable to change. These features will be used to implement the @implements decorator.

@implements

 def implements(orig_obj, requirements=lambda: False): ... 

The decorator declares the implementation of the orig_obj object being decorated if the requirements are met during the call. An example of use was given at the beginning of the article. The implementation of the decorator does not allow calling the orig_obj function from the implementation, but this is easily solved by adding additional attributes to functions and checking them during the call to the function being decorated.

In a nutshell, how the decorator works. When called, the decorator searches for orig_obj in the global namespace using the globals () function. This is necessary to replace the original function call with the orig_wrapper handler.

Next, it is checked whether the object found by name is a wrapper for the original function by checking for the presence of the __orig_wrapper__ attribute. If this attribute is absent, then substitution is performed. The __impl__ attribute is added to the replacement function to store implementations and conditions (requirements).

As soon as the first decorator was called, do_something changes its behavior in such a way that, before executing its own implementation, it checks all requirements, and if any condition is met, the decoded function will be called. The implementation uses the above function attribute func_globals to ensure that the lambda expression is executed in the required context.

Conclusion

I'm not sure that this approach to organizing various implementations can be convenient and “ideologically” correct, but studying and working on this example was a good exercise for me to understand how closures and scope work in Python.