Not another article about functional programming.

For several years now, functional programming has gained popularity. This, of course, does not mean that people abandon their old languages ​​and OOP and massively switch to Haskell, Lisp or Erlang. Not. The functional paradigm penetrates our code through loopholes of multi-paradigm languages, and the aforementioned languages ​​more often serve as flags in this attack than are used directly.

I was going to continue in the same vein and in the second part of the article to present my library , adding a couple of functional tricks in python, but then I realized that the focus of my library is not on functional programming, but on practicality. I will focus on this, I will give a few life examples of the usefulness of funcy.

Development of funcy began with an attempt to put together a few utilities for manipulating data and less often functions, so most of my examples will focus on this. Some (or many) examples may seem trivial, but it is surprising how much time these simple functions can save and how much more expressive they can make your code.

I will go through several typical tasks that are encountered in the practice of drinking and, despite their simplicity, raise constant questions. So let's go.
')

Easy data manipulation

1. Combine the list of lists. Traditionally, I did it this way:

from operator import concat reduce(concat, list_of_lists) #  : sum(list_of_lists, []) #  : from itertools import chain list(chain.from_iterable(list_of_lists)) 

All of them are not bad, but they require either extra gestures: imports and additional calls, or impose restrictions: you can combine only lists with lists and tupl with tups, for the amount you still need to know in advance which type will come. In funcy, this is done like this:

 from funcy import cat cat(list_of_lists) 

cat() combines a list of lists, tuples, iterators, and indeed any iterated into one list. If you want to combine lists of the results of a function call, you can use mapcat() , for example:

 from funcy import mapcat mapcat(str.splitlines, bunch_of_texts) 

will sort all the lines in the texts into one flat list. For both functions there are lazy versions: icat() and imapcat() .

2. Add a few dictionaries. In python, there are several clunky ways to combine dictionaries:

 d1.update(d2) #  d1 dict(d1, **d2) #   > 2  d = d1.copy() d.update(d2) 

I always wondered why they can not just lay down? But we have what we have. In any case, with funcy this is done easily:

 from funcy import merge, join merge(d1, d2) merge(d1, d2, d3) join(sequence_of_dicts) 

But merge() and join() can combine not only dictionaries, they work for almost any collections: dictionaries, ordered dictionaries, sets, lists, tuples, iterators, and even strings.

3. Capturing a substring using a regular expression. This is usually done like this:

 m = re.search(some_re, s) if m: actual_match = m.group() #  m.group(i),  m.groups() ... 

With funcy, it turns into:

 from funcy import re_find actual_match = re_find(some_re, s) 

If this doesn't seem impressive enough, take a look at this:

 from funcy import re_finder, re_all, partial, mapcat #      map(re_finder('\d+'), words) #  ini  (re_finder()      > 1 ) dict(imap(re_finder('(\w+)=(\w+)'), ini.splitlines())) #     (    )      mapcat(partial(re_all, r'\d+'), bunch_of_strings) 

A retreat about imports and practicality

As you can see, I import functions directly from funcy, without using any subpackages. The reason I stopped at this interface is practical; it would be pretty boring to require all users of my library to remember where you need to import walk () from funcy.colls or funcy.seqs, in addition, multi-line imports at the beginning of each file and without me there is someone to stuff.

An additional advantage of this solution is the ability to simply write:

 from funcy import * 

And enjoy all the functional delights and convenience that brings funcy, no longer returning to the beginning of the file for the addition. Well, now that you know where all the good is, I will no longer explicitly point out the imports from funcy. We continue.

Some more functional things.

We have already seen a couple of examples of using higher order functions, re_finder() and partial() . It should be added that the re_finder() function itself is a partial use of re_find() created for ease of use in map() and the like. And naturally, with filter() it is convenient to use re_tester() :

 #      is_private = re_tester('^_') filter(is_private, dir(some_obj)) 

Great, we can set several predicates, such as is_private() , and filter object attributes by them:

 is_special = re_tester('^__.+__$') is_const = re_tester('^[A-Z_]+$') filter(...) 

But, what if we want to get a list of public attributes or private constants, something involving a combination of predicates? Easy:

 is_public = complement(is_private) is_private_const = all_fn(is_private, is_const) either_const_or_public = any_fn(is_const, is_public) 

For convenience, there is also a function that complements filter() :

 remove(is_private, ...) #  ,  filter(is_public) 

I hope everyone quenched their functional appetite, so it's time to move on to something less abstract.

Work with collections

In addition to tools for working with sequences, of which there are many more than I described here, funcy also helps to work with collections. The basis is the walk() and select() functions, which are similar to map() and filter() , but retain the type of collection being processed:

 walk(inc, {1, 2, 3}) # -> {2, 3, 4} walk(inc, (1, 2, 3)) # -> (2, 3, 4) #        - swap = lambda (k, v): (v, k) walk(swap, {1: 10, 2: 20}) # -> {10: 1, 20: 2} select(even, {1, 2, 3, 10, 20}) # -> {2, 10, 20} select(lambda (k, v): k == v, {1: 1, 2: 3}) # -> {1: 1} 

This pair of functions is supported by a set for working with dictionaries: walk_keys(), walk_values(), select_keys(), select_values() :

 #       select_keys(is_public, instance.__dict__) #      select_values(bool, some_dict) 

The last example in this series will use several new functions at once: silent() - silences all exceptions thrown by the wrapped function, returning None ; compact() - removes None values ​​from the collection; walk_values() - bypasses the values ​​of the passed dictionary by constructing a new dictionary with the values ​​converted by the passed function. In general, this line selects a dictionary of integer parameters from the query parameters:

 compact(walk_values(silent(int), request_dict)) 

Data manipulation

ABOUT! We got to the most interesting, I included some of the examples here simply because they seem cool to me. Although, to be honest, I did it above. Now we will divide and group:

 #   URL   absolute, relative = split(re_tester(r'^http://'), urls) #     group_by(lambda post: post.category, posts) 

Collect flat data in nested structures:

 #       dict(partition(2, flat_list_of_pairs)) #     {id: (name, password) for id, name, password in partition(3, users)} # ,     assert all(prev + 1 == next for prev, next in partition(2, 1, versions)): #    for chunk in chunks(CHUNK_SIZE, lots_of_data): process(chunk) 

And a couple more examples, just to the heap:

 #     for line, prev in with_prev(text.splitlines()): if not prev: print ' ', print line #     1611  where(plays, author="Shakespeare", year=1611) # => [{"title": "Cymbeline", "author": "Shakespeare", "year": 1611}, # {"title": "The Tempest", "author": "Shakespeare", "year": 1611}] 

Not just a library

Perhaps some of you have met familiar functions from Clojure and Underscore.js (by the way, the example of Shakespeare is brazenly torn from the documentation of the latter) - no wonder, I drew a lot of inspiration from these sources. At the same time, I tried to follow the style of the library, preserve the consistency of the library and never sacrifice practicality, so not all functions fully correspond to their prototypes, they rather correspond to each other and the standard library.

And one more thought. We used to call programming languages ​​languages, and we rarely realize that syntactic constructs and standard functions are the words of these languages. We can add our words by defining functions, but usually such words are too specific to fall into the everyday language dictionary. Utilities from funcy, on the contrary, are sharpened for a wide range of applications, so this library can be perceived as an extension of python, as well as underscore or jQuery — an extension of JavaScript. So, everyone who wants to replenish their vocabulary is welcome .