Frame object in Python. What can and cannot be done with it (in production and in another decent place)

About Python on Habré there were many good articles. Both about features of implementation, and about the applied features which are absent in other mainstream languages. However, I was surprised to find (correct, if not right) that there is one important topic that has not been disclosed either on Habré or on the Russian-language Internet in general. This article will focus on the stack frame. Most likely, it will not say anything, well, or, perhaps, taking into account the last point, almost nothing new to experienced python developers, but it will be useful for beginners (and maybe harmful, but all the examples are below).

I tried to write an article so that it would be convenient to read it by opening parallel repl and thoughtlessly copying the code there by experimenting. Therefore, whenever possible, most of the examples have the form of "one-liners in the interpreter".

We will start a little from a distance, with the fact that we note that Traceback is also an object, and then we will find where the stack frame is and get down to business.

>>> import requests >>> requests.get(42) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/lib/python3/dist-packages/requests/api.py", line 55, in get return request('get', url, **kwargs) File "/usr/lib/python3/dist-packages/requests/api.py", line 44, in request return session.request(method=method, url=url, **kwargs) File "/usr/lib/python3/dist-packages/requests/sessions.py", line 421, in request prep = self.prepare_request(req) File "/usr/lib/python3/dist-packages/requests/sessions.py", line 359, in prepare_request hooks=merge_hooks(request.hooks, self.hooks), File "/usr/lib/python3/dist-packages/requests/models.py", line 287, in prepare self.prepare_url(url, params) File "/usr/lib/python3/dist-packages/requests/models.py", line 338, in prepare_url "Perhaps you meant http://{0}?".format(url)) requests.exceptions.MissingSchema: Invalid URL '42': No schema supplied. Perhaps you meant http://42? 

')
Above, we see Traceback, which contains a chain of calls to various functions.

 >>> import sys >>> sys.last_traceback <traceback object at 0x7f8c37378608> 

 >>> dir(sys.last_traceback) ['tb_frame', 'tb_lasti', 'tb_lineno', 'tb_next'] 

So, above we see an object containing the last unhandled exception. It can be investigated using the debager to see what happened. You can also use the Traceback module. I will not dwell on this, it is somewhat off topic.

We will be interested in the tb_frame attribute. This is actually the stack frame. Its essence is not fundamentally different from that in C. When a function is called, its arguments go on the stack, after which it is already being executed.

However, there are two fundamental differences from C.

Firstly, due to the fact that Python is an interpretable language, the stack frame is stored in it explicitly, and secondly, due to the fact that it is object-oriented, it is also an object.

In fact, this is the main source of introspection in Python — the inspect module is in part just a wrapper around it. (and partly over other functions from sys).

A detailed description of all the attributes of the object can be read in another place, so we turn to examples of application.

0) How to get a link to the stack frame

Of course, you can get it from the traceback. But fortunately there are more ways to work with it than to throw exceptions and watch the traceback /

 >>> sys._getframe() <frame object at 0x7fda9bffa388> 

Returns the current object from the call stack. You can also pass a depth to it as a parameter to get an object that is higher on the stack. However, this can be done using the f_back attribute.

Unfortunately, if we run repl, we are on top of the stack, so when we call sys._getframe (). F_back, it returns None, and sys._getframe (1) does throw an exception.

 >>> sys._getframe().f_back >>> sys._getframe(1) Traceback (most recent call last): File "<stdin>", line 1, in <module> ValueError: call stack is not deep enough 

However, the lambda functions will solve this problem:

 >>> (lambda: sys._getframe().f_back == sys._getframe(1))() True   (   ) >>> (lambda: sys._getframe().f_back is sys._getframe(1))() True 

There is also sys._current_frames (), but it makes sense in the case of multithreading. It will not be mentioned further, but for completeness:

 >>> import threading >>> sys._current_frames() {140576946280256: <frame object at 0x7fda9bffa6c8>} >>> threading.Thread(target=lambda: print(sys._current_frames())).start() {140576883275520: <frame object at 0x7fda9b337048>, 140576946280256: <frame object at 0x228fbc8>} 

Well, it's time to quote the documentation for these functions:

This function should be used for internal and specialized purposes only. It is not guaranteed to exist in all implementations of Python.

So it is hardly necessary to use for the mercenary purposes what will be lower.

1) How to determine who called the function

Such a task can easily emerge during debug, especially when using many third-party and intricate frameworks. But adherents of black magic can use such a trick and its variations for much less innocuous things.

After the last subclause, it should be generally understood how to do this. In fact, getting the frame in which the function was called is very simple: sys._getframe (1). The question remains how to extract the necessary information from the frame.

For example, like this:

 >>> threading.Thread(target=lambda: print(sys._getframe(1).f_code.co_name)).run() run 

Discussion Code object is beyond the scope of the article, maybe some other time. But the frame object has an attribute f_code, which is a code object, among the attributes of which is including the name of the method of the corresponding code object.

By the way, in the case of inspect, the same thing happens in fact, but it looks nicer.

 >>> threading.Thread(target=lambda: print(inspect.stack()[1][3])).run() run 

Opportunities to abuse the method obtained by the reader will find himself.
Another example pushing for such ideas:

 >>> threading.Thread(target=lambda: print('called from', sys._getframe(1).f_globals['__name__'])).run() called from threading 

Now, having received the name of the module, you can find the module in the list of loaded modules by that name, and then take something from there. And no Hierarchy of Classes will save from such blasphemy (although a good one will probably save from the desire to do this)

2) Change the locals of the frame

The information in this sub-note is not too new and should be known to advanced pythonists, but nonetheless.

At this point, you will have to move away from one-liners and write a full-fledged example.

Suppose we have the following code:

 import sys class Example(object): def __init__(self): pass def check_noosphere_connection(): print('OK') def proof(example): a = 2 b = 2 example.check_noosphere_connection() print('2 + 2 = {}'.format(str(a + b))) def broken_evil_force(): def corrupted_noosphere(pseudo_self): print('OK') frame = sys._getframe() frame.f_back.f_locals['a'] = 3 FakeExample = type('FakeExample', (), {'check_noosphere_connection': corrupted_noosphere}) proof(FakeExample()) 

 % python habr_example.py OK 2 + 2 = 4 

Alas, the locals cannot be changed. More precisely, it is possible, but this result will not remain globally.

Here are two links to this topic, in one of them the author achieves the desired result by modifying the bytecode, and in the other, the source code itself changes and compiles.

code.activestate.com/recipes/577283-decorator-to-expose-local-variables-of-a-function-
stackoverflow.com/a/4257352

However, there is a way! pydev.blogspot.ru/2014/02/changing-locals-of-frame-frameflocals.html

Calling cctypes.pythonapi.PyFrame_LocalsToFast (ctypes.py_object (frame), ctypes.c_int (0)) helps to save changes to locals.

 import sys import ctypes class Example(object): def __init__(self): pass def check_noosphere_connection(): print('OK') def proof(example): a = 2 b = 2 example.check_noosphere_connection() print('2 + 2 = {}'.format(str(a + b))) def broken_evil_force(): def corrupted_noosphere(pseudo_self): print('OK') frame = sys._getframe() frame.f_back.f_locals['a'] = 3 FakeExample = type('FakeExample', (), {'check_noosphere_connection': corrupted_noosphere}) proof(FakeExample()) def evil_force(): def corrupted_noosphere(pseudo_self): print('OK') frame = sys._getframe() frame.f_back.f_locals['a'] = 3 ctypes.pythonapi.PyFrame_LocalsToFast(ctypes.py_object(frame.f_back), ctypes.c_int(0)) FakeExample = type('FakeExample', (), {'check_noosphere_connection': corrupted_noosphere}) proof(FakeExample()) if __name__ == '__main__': broken_evil_force() evil_force() 

Run and get:

 % python habr_example.py OK 2 + 2 = 4 OK 2 + 2 = 5 

On this, perhaps, everything. Is that I add: in the case of traceback there is an opportunity to walk on the frame not only up, but also down, although it should already be clear.

I will be glad to hear comments and observations, as well as interesting examples of the use of frames.

Useful links (part was in the article)

1) docs.python.org/3.4/library/sys.html
2) docs.python.org/3/library/inspect.html
3) code.activestate.com/recipes/577283-decorator-to-expose-local-variables-of-a-function-
4) stackoverflow.com/questions/4214936/how-can-i-get-the-values-of-the-locals-of-a-function-after-it-has-been-executed/4257352#4257352
5) pydev.blogspot.ru/2014/02/changing-locals-of-frame-frameflocals.html