Coroutines in python

I propose to discuss this interesting, but little used feature of python, as coroutines (coroutines).
Coroutines in python are based on generators (they, in fact, are).
Therefore, I propose to start it with the generators, in the general sense. And then let's figure out how to write your coroutine.

Generators

Any more or less decent Python programmer knows that there is such a wonderful thing as function generators. Their main feature is the state preservation between calls.

I mean that you know how they work, so I’ll just remind you how it looks.
Take this function:

def read_file_line_by_line(file_name): with open(file_name, 'r') as f: while True: line = f.readline() if not line: break yield line 

')
This function accepts a file name as input and returns it line by line without loading it entirely into memory, which may be necessary when reading large files.
Such a technique is called lazy (lazy) reading, implying that we do not do "work" unnecessarily.

In general, working with generators is as follows:

 In [78]: lines_generator = read_file_line_by_line("data.csv") In [79]: type(lines_generator) Out[79]: generator In [83]: lines_generator.next() Out[83]: 'time,host,event\n' In [84]: lines_generator.next() Out[84]: '1374039728,localhost,reboot\n' In [85]: lines_generator.next() Out[85]: '1374039730,localhost,start\n' In [86]: lines_generator.next() --------------------------------------------------------------------------- StopIteration Traceback (most recent call last) <ipython-input-86-65df1a2cb71b> in <module>() ----> 1 lines_generator.next() StopIteration: #       3  #     ,   StopIteration,      . 

Naturally, more often we read the values ​​from the generator in a loop, and not line by line.
In this simple way we can get all the unique lines from an arbitrarily large file:

 uniq = [] for line in lines_generator: if line not in uniq: uniq.append(line) 

A short generator record is also possible:

 In [92]: gen = (x for x in xrange(0, 100*10000)) In [93]: gen.next() Out[93]: 0 In [94]: gen.next() Out[94]: 1 In [95]: gen.next() Out[95]: 2 In [96]: gen.next() Out[96]: 3 In [97]: gen.next() Out[97]: 4 

Looks like list expressions, right? Only does not require the creation of the entire range(0, 100*10000) list in memory, the return value is "calculated" every time it is accessed.

Coroutines as a special case of generators

And now, for the sake of what it was, in fact, started. It turns out that the generator can not only return values, but also take them to the input.

About the standard can be read here PEP 342 .

I suggest to start immediately with an example. We write a simple implementation of a generator that can add two arguments, store a history of results, and output a history.

 def calc(): history = [] while True: x, y = (yield) if x == 'h': print history continue result = x + y print result history.append(result) c = calc() print type(c) # <type 'generator'> c.next() #  .   c.send(None) c.send((1,2)) #  3 c.send((100, 30)) #  130 c.send((666, 0)) #  666 c.send(('h',0)) #  [3, 130, 666] c.close() #   

Those. we created a generator, initialized it, and provided it with input data.
He, in turn, processes this data and maintains its state between calls until we close it . After each call, the generator returns control to where it was called from . This is the most important property of generators we will use.

So, with how it works, sort of, sorted out.
Let us now rid ourselves of the need to initialize the generator each time.
We will solve this in a typical way for a python using a decorator.

 def coroutine(f): def wrap(*args,**kwargs): gen = f(*args,**kwargs) gen.send(None) return gen return wrap @coroutine def calc(): history = [] while True: x, y = (yield) if x == 'h': print history continue result = x + y print result history.append(result) 

In this example, you can understand how to write your more complex (and useful) coroutines.

Conclusion

Although the problems that can be solved with this tool affect very many areas (such as asynchronous programming), many developers prefer the more familiar OOP tools. But at the same time, coroutines can be a very useful tool in your arsenal, since they are fairly intuitive, and creating functions is a cheaper operation compared to creating an object of a class.

Yes, and a certain academic interest, they are, I think.

Here is such a first article.

UPD:
Fixed in the example of a short recording of the range generator on xrange .
In version 2, python range() creates the entire list at once; to create a generator, you must use xrange() , in version 3, range == xrange (that is, it returns a generator).