"Simple" python programming

functools (this is such a dump for all unnecessary things to me :-).
- Guido van Rossum

It may seem like an article on OP, but I'm not going to discuss the paradigm. It will be about reuse and simplification of the code - I will try to prove that you are writing too much code, so it is difficult and hard to test, but most importantly: read and change it for a long time.

The article borrows examples and / or concepts from the funcy library. Firstly, it is cool, and secondly, you can immediately start using it. And yes, we need the OP.

Briefly about OP

• pure functions
• higher order functions
• a sense of superiority over those who write non-functional (optional)

AF also has the following techniques:

• partial application
• compositing (there are still decorators in python)
• lazy calculations

If you already know all this, go directly to the examples.

Pure functions

Pure functions depend only on their parameters and return only their result. The following function called several times with the same argument will produce a different result (although the same object, in this case%).

We write a filter function that returns a list of elements with labor values.

pred = bool result = [] def filter_bool(seq): for x in seq: if pred(x): result.append(x) return result 

Make it clean:

 pred = bool def filter_bool(seq): result = [] for x in seq: if pred(x): result.append(x) return result 

Now you can call her lard once in a row and the result will be the same.

Higher order functions

These are functions that take other functions as arguments or return another function as a result.

 def my_filter(pred, seq): result = [] for x in seq: if pred(x): result.append(x) return result 

I had to rename the function because it is now much more useful:

 above_zero = my_filter(bool, seq) only_odd = my_filter(is_odd, seq) only_even = my_filter(is_even, seq) 

Notice, one function and already does a lot of things. In fact, she should be lazy, do:

 def my_filter(pred, seq): for x in seq: if pred(x): yield x 

Have you noticed that we removed the code, and only it became better? This is just the beginning, soon we will write functions only on holidays. Look here:

 my_filter = filter 

The built-in capabilities of python are almost enough for a full life, you just need to correctly assemble them.

Partial application

This is the process of fixing part of the function's arguments, which creates another function, less arity. Translated into ours is functools.partial .

 filter_bool = partial(filter, bool) filter_odd = partial(filter, is_odd) filter_even = partial(filter, is_even) 

I understand that this is all the basics of the OP, but I want to note that we have not written anything new: we have taken ready-made functions and done others. The basis of the new ones is very small, simple, easily testable functions, we can safely use them to create more complex ones.

Compositing

There is no such simple, cool and necessary thing in python. It can be written independently, but it would be desirable sane implementation: (

 def compose(*fns): init, *rest = reversed(fns) return lambda *a, **kw: reduce(lambda a, b: b(a), rest, init(*a, **kw)) 

Now we can do all sorts of things (execution goes from right to left):

 mapv = compose(list, map) filterv = compose(list, filter) 

These are previous versions of map and filter from the second version of python. Now, if you need a non-greasy map , you can call mapv . Or in the old manner to write a little more code. Everytime.

The compose and partial functions are excellent because they allow you to reuse ready-made, tested functions. But most importantly, if you understand the advantage of this approach, then over time you will immediately write them ready for composition.

This is a very important point - the function should solve one simple task, then:

• she will be small
• it will be easier to test
• easy to compose
• just read and change
• hard to break

Example

Task: to drop None from the sequence.
The solution is in the old manner (most often it is not even written as a function):

 no_none = (x for x in seq if x is not None) 

Note: no matter what the variable in the expression is called. It is so unimportant that most programmers stupidly write x , so as not to bother. Everyone writes this meaningless code over and over again. Every time censorship : for , in , if and several times x - because for a framework you need scope and it has its own syntax. We write: for each iteration of the cycle, assign a value to a variable. And it is assigned and the condition is checked.

Every time we write this boilerplate and write tests for this boilerplate. What for?

Let's rewrite:

 from operator import is_ from itertools import filterfalse from functools import partial is_none = partial(is_, None) filter_none = partial(filterfalse, is_none) #  no_none = filter_none(seq) #  all_none = compose(all, partial(map, is_none)) 

Everything. No extra code. I am pleased to read this, because this code ( no_none = filter_none(seq) ) is very simple. The way this function works, I need to read exactly once in the project. You have to read the composition every time to understand exactly what it does. Well, or shove it into the function, without a difference, but do not forget about the tests.

Example 2

Quite a frequent task is to get the values ​​by key from the array of dictionaries.

 names = (x['name'] for x in users) 

By the way, it works very quickly, but we again wrote a bunch of unnecessary garbage. Rewrite to work even faster:

 from operator import itemgetter def pluck(key, seq): return map(itemgetter(key), seq) #  names = pluck('name', users) 

And how often will we do this?

 get_names = partial(pluck, 'name') get_ages = partial(pluck, 'age') #  get_names_ages = partial(pluck, ('name', 'age')) users_by_age = compose(dict, get_names_ages) ages = users_by_ages(users) # {x['name']: x['age'] for x in users} 

And if we have objects? PF, parameterize it:

 from operator import itemgetter, attrgetter def plucker(getter, key, seq): return map(getter(key), seq) pluck = partial(plucker, itemgetter) apluck = partial(plucker, attrgetter) #  names = pluck('name', users) # (x['name'] for x in users) object_names = apluck('name', users) # (x.name for x in users) #      object_data = apluck(('name', 'age', 'gender'), users) # ((x.name, x.age, x.gender) for x in users) 

Example 3

Imagine a simple generator:

 def dumb_gen(seq): result = [] for x in seq: #  - c result.append(x) return result 

There is a lot of boilerplate here: we create an empty list, then we write a cycle, add an element to the list, give it away. It seems that I literally listed the whole body of the function :(

The correct solution would be to use filter(pred, seq) or map(func, seq) , but sometimes you need to do something more complicated, i.e. generator really need to write. And if the result is always needed in the form of a list or a tap? Yes Easy:

 @post_processing(list) def dumb_gen(seq): for x in seq: ... yield x 

This is a parametric decorator, it works like this:

 result = post_processing(list)(dumb_gen)(seq) 

Those. the result of the first call will be a new function, which will take the function as an argument and return another function. It sounds harder than it is:

 def post_processing(post): return lambda func: compose(post, func) 

Pay attention, I used already existing compose . The result is a new feature that nobody has written .
And now the verses:

 post_list = post_processing(list) post_tuple = post_processing(tuple) post_set = post_processing(set) post_dict = post_processing(dict) join_comma = post_processing(', '.join) @post_list def dumb_gen(pred, seq): for x in seq: ... yield x 

A bunch of new features for the price of one! And I removed the boiler plate, the function became smaller and much prettier.

Total

Looking through the data with reinforced concrete functions (clean, higher), we maintain the simplicity of implementation and ensure the stability of the program, which is easier to test:

• write clean functions, they will ensure the stability of the program
• write higher order functions, the code will be much more compact and reliable
• compose, decorate, partially apply, reuse code
• use sishnye liby, they will give speed to your software

As soon as you write your set of tools, a new code will be created with the knowledge that you have a piece that can solve part of the problem. So the software will be smaller and easier.

Where to begin?

• Be sure to check out itertools , functools , operator , collections , especially the examples at the end.
• Check out the documentation for funcy or other php files, read the source code.
• write your funcy , you don’t need it all at once, but the experience is very useful

Credits

In my case, the use of OP is familiarized with clojure - this thing is thoroughly reversing the brains, I strongly recommend that you look at least vidos on YouTube .

Clojure is somehow so arranged that you have to write easier, without the things we are used to: without variables, without the favorite style of the "novel", where we first reveal the personality of the hero, then we embark on his heart problems. In clojure, you have to think of%) There are only basic data types and "no syntax" (c) . And this “simple” concept, it turns out, can be ported to python.

UPD

It seems that readers have the impression that I am writing a solid OP. I want to reassure everyone: I use the functional approach exclusively in the places where the code I wrote is written. In my opinion, repeating the "working" techniques every time is stupid and pointless, so I translate such pieces into functions and reuse them. A working example can be found in the comments .