Control flow abstraction

Any programmer, even if he does not look at his work in this way, is constantly engaged in building abstractions. Most often, we abstract calculations (write functions) or behavior (procedures and classes), but apart from these two cases, there are many repetitive patterns in our work, especially when handling errors, managing resources, writing standard handlers and optimizations.

What does abstraction of control flow or “control flow” mean, as our overseas friends put it? In the case when no one shows off, the flow of control structures involved. Sometimes these control structures are not enough and we add our own, abstracting the behavior of the program we need. This is easy in languages ​​like lisp, ruby ​​or perl, but in other languages ​​this is possible, for example, using higher order functions.

Abstractions

Start over. What needs to be done to build a new abstraction?

1. Select a piece of functionality or behavior.
2. Give him a name.
3. Implement it.
4. Hide implementation behind the selected name.

I must say that the third point is not always fulfilled. The ability to implement depends heavily on the flexibility of the language and on what you are abstracting.
')
What if your language is not flexible enough? It's okay, instead of implementation, you can simply describe your technique in detail, make it popular and, thus, generate a new “design pattern”. Or simply switch to a more powerful language, if creating patterns does not tempt you.

But enough theory, let's do business ...

Life example

The usual code on python (taken from a real project with minimal changes):

urls = ... photos = [] for url in urls: for attempt in range(DOWNLOAD_TRIES): try: photos.append(download_image(url)) break except ImageTooSmall: pass #     except (urllib2.URLError, httplib.BadStatusLine, socket.error), e: if attempt + 1 == DOWNLOAD_TRIES: raise 

This code has many aspects: iterating through the url list, downloading images, collecting downloaded images in photos, skipping small images, and repeated download attempts when network errors occur. All these aspects are confused in a single piece of code, although many of them would be useful in their own right, if only we could isolate them.

In particular, the iteration + collection of results is implemented in the built-in map function:

 photos = map(download_image, urls) 

Let's try to catch the other aspects. Let's start with skipping small images, it could look like this:

 @contextmanager def skip(error): try: yield except error: pass for url in urls: with skip(ImageTooSmall): photos.append(download_image(url)) 

Not bad, but there is a drawback - I had to abandon the use of the map . Let's leave this problem for now and move on to the aspect of resistance to network errors. Similarly, the previous abstraction could be written:

 with retry(DOWNLOAD_TRIES, (urllib2.URLError, httplib.BadStatusLine, socket.error)): # ... do stuff 

Only it will not work, with in python cannot execute its block of code more than once. We have stumbled upon the limitations of the language and now have to either minimize and use alternative solutions, or generate another “pattern”. Noticing such situations is important if you want to understand the differences in languages, and than one can be more powerful than the other, despite the fact that they are all turing. In ruby ​​and with less convenience in perl we could continue to manipulate the blocks, in the Lisp - blocks or code (the latter in this case, apparently, to nothing), in the python we have to use an alternative option.

Let us return to the functions of a higher order, or rather to their special variety - decorators:

 @decorator def retry(call, tries, errors=Exception): for attempt in range(tries): try: return call() except errors: if attempt + 1 == tries: raise http_retry = retry(DOWNLOAD_TRIES, (urllib2.URLError, httplib.BadStatusLine, socket.error)) harder_download_image = http_retry(download_image) photos = map(harder_download_image, urls) 

As we can see, this approach fits well with the use of map , we also got a couple of things that we will someday come in handy - retry and http_retry .

Rewrite skip in the same style:

 @decorator def skip(call, errors=Exception): try: return call() except errors: return None skip_small = skip(ImageTooSmall) http_retry = retry(DOWNLOAD_TRIES, (urllib2.URLError, httplib.BadStatusLine, socket.error)) download = http_retry(skip_small(download_image)) photos = filter(None, map(download, urls)) 

filter needed to skip the dropped pictures. In fact, the filter(None, map(f, seq)) pattern filter(None, map(f, seq)) so common that in some languages ​​there is a built-in function for such a case .

We can also implement this:

 def keep(f, seq): return filter(None, map(f, seq)) photos = keep(download, urls) 

What is the result? Now all aspects of our code are visible, easily distinguishable, changeable, replaceable and removable. And as a bonus, we received a set of abstractions that can be used in the future. And also, I hope, I made someone see a new way to make my code better.

PS The implementation of @decorator can be taken here .

PPS Other examples of abstraction of control flow: manipulations with functions in underscore.js , list and generator expressions, function overloading , caching wrappers for functions, and much more.

PPPS Seriously, you need to come up with a better translation for the expression “control flow”.