Geekly Articles each Day
📜 ⬆️ ⬇️

F library for functional programming in Python

Hello colleagues!


I will tell about the library for Python with the laconic name f . This is a small package with functions and classes for solving problems in a functional style.


- What, another functional lib for Python? Author, are you aware that there is fn.py and in general these functional crafts a million?


- Yes, in the know.


The reasons for the appearance of the library


I have been doing Python for a long time, but a couple of years ago I seriously got carried away with functional programming and Klozhey in particular. Some of the approaches taken in the OP made me such a strong impression that I wanted to transfer them to everyday development.


I emphasize that I do not accept the approach when the patterns of one language are roughly introduced into another without taking into account its principles and coding agreements. No matter how much I love AF, the piling up of the MAP and Lambda in an attempt to impersonate it as a functional style annoys me.


Therefore, I tried to arrange my functions so as not to meet the resistance of colleagues. For example, use standard cycles with conditions instead of maps and reducts inside to make it easier for those who are not familiar with AF to understand.


As a result, some of the parts of the library were in combat projects and, possibly, still in service. First, I copied them from project to project, then I started a file-dump of functions and snippets, and finally I designed everything with a library, a package in Pypi and documentation.


General information


The library is written on pure Python and works on any OS, incl. on windose. Both branches of Python are supported. Specifically, I checked on versions 2.6, 2.7 and 3.5. If there are difficulties with other versions, let me know. The only dependency is the six pack for agile development under both branches at once.


The library is installed in the standard way via pip:


 pip install f

All functions and classes are available in the head module. It means no need to memorize
paths to entities:


 import f f.pcall(...) f.maybe(...) f.io_wraps(...) fL[1, 2, 3]

The package carries on board the following subsystems:



In the sections below, I will provide code examples with comments.


Functions


The first function I transferred to Python from another ecosystem was pcall from Lua. I programmed it several years ago, and although the language is not functional, I was delighted with it.


The pcall function (protected call, protected call) takes another function and returns a pair (err, result) , where either err is an error and result empty, or vice versa. This approach is familiar to us from other languages, such as Javascript or Go.


 import f f.pcall(lambda a, b: a / b, 4, 2) >>> (None, 2) f.pcall(lambda a, b: a / b, 4, 0) >>> (ZeroDivisionError('integer division or modulo by zero'), None)

The function is convenient to use as a decorator for already written functions that throw exceptions:


 @f.pcall_wraps def func(a, b): return a / b func(4, 2) >>> (None, 2) func(4, 0) >>> (ZeroDivisionError('integer division or modulo by zero'), None)

Using the destructive syntax, you can unpack the result at the signature level:


 def process((err, result)): if err: logger.exception(err) return 0 return result + 42 process(func(4, 2))

Unfortunately, the destructive syntax was cut in the third Python. It is necessary to unpack manually.


It is interesting that the use of the pair (err, result) is nothing else but the Either monad, which we will talk about later.


Here is a more realistic pcall example. Often it is necessary to do HTTP requests and retrieve data structures from Jason. A lot of errors may occur during the request:



Wrapping a call in try with catching four exceptions means making the code completely unreadable. Sooner or later you will forget to intercept something, and the program will fall. Here is an example of almost real code. It retrieves the user from the local rest-service. The result will always be a pair:


 @f.pcall_wraps def get_user(use_id): resp = requests.get("http://local.auth.server", params={"id": user_id}, timeout=3) if not resp.ok: raise IOError("<log HTTP code and body here>") data = resp.json() if "error" in data: raise BusinesException("<log here data>") return data

Consider other library functions. I would like to highlight f.achain and f.ichain . Both are designed to safely retrieve data from objects in a chain.


Suppose you have Django with the following models:


 Order => Office => Department => Chief

In this case, all the fields are not null and you without fear go through the adjacent fields:


 order = Order.objects.get(id=42) boss_name = order.office.department.chief.name

Yes, I know about select_related , but it does not matter. The situation is valid not only for ORM, but also for any other class structure.


So it was in our project, until one customer asked for some links to be empty, because these are the features of his business. We made the fields in the nullable base and were glad that we got off easy. Of course, because of the rush, we did not write unit tests for models with empty links, and in the old tests the models were filled in correctly. The client started working with updated models and received errors.


The f.achain function safely traverses the attribute chain:


 f.achain(model, 'office', 'department', 'chief', 'name') >>> John

If the chain is broken (the field is None, does not exist), the result will be None.


The analog function f.ichain runs through the chain of indices. She works with dictionaries, lists and tuples. The function is convenient for working with data obtained from Jason:


 data = json.loads('''{"result": [{"kids": [{"age": 7, "name": "Leo"}, {"age": 1, "name": "Ann"}], "name": "Ivan"}, {"kids": null, "name": "Juan"}]}''') f.ichain(data, 'result', 0, 'kids', 0, 'age') >>> 7 f.ichain(data, 'result', 0, 'kids', 42, 'dunno') >> None

I took both of the functions from Klozha, where their ancestor is called get-in . Convenience is that in microserver architecture, the structure of the response is constantly changing and may not correspond to common sense.


For example, in the answer there is a field-object "user" with nested fields. However, if there is no user for some reason, the field will not be an empty object, but None. In the code, ugly constructions like:


 data.get('user', {]}).get('address', {}).get('street', '<unknown>')

Our version is easier to read:


 f.ichain(data, 'user', 'address', 'street') or '<unknown>'

From Thread in the library f , two threading macros passed: -> and ->> . In the library, they are called f.arr1 and f.arr2 . Both pass the original value through the functional forms . This term in Lisp means an expression that is evaluated later.


In other words, a form is either a func function, or a tuple of the form (func, arg1, arg2, ...) . You can pass such a form somewhere as a frozen expression and calculate it later with the changes. It turns out something like macros in Lisp, only very poor.


f.arr1 substitutes the value (and further result) as the first
form argument:


 f.arr1( -42, #   (lambda a, b: a + b, 2), #  abs, #  str, #  ) >>> "40"

f.arr2 does the same thing, but puts the value at the end of the form:


 f.arr2( -2, abs, (lambda a, b: a + b, 2), str, ("000".replace, "0") ) >>> "444"

Next, the f.comp function returns a composition of functions:


 comp = f.comp(abs, (lambda x: x * 2), str) comp(-42) >>> "84"

f.every_pred builds a super predicate. This is a predicate that is true only if all internal predicates are true.


 pred1 = f.p_gt(0) #   pred2 = f.p_even #  pred3 = f.p_not_eq(666) #   666 every = f.every_pred(pred1, pred2, pred3) result = filter(every, (-1, 1, -2, 2, 3, 4, 666, -3, 1, 2)) tuple(result) >>> (2, 4, 2)

The super predicate is lazy: it breaks the chain of calculations at the first false value. In the example above, predicates from the predicate.py module are used, which we will talk about later.


The f.transduce function is a naive attempt to implement the transducer pattern (transducer) from Klozi. In short, transducer is a combination of the map and reduce functions. Their superposition yields a conversion "from anything to anything without intermediate data":


 f.transduce( (lambda x: x + 1), (lambda res, item: res + str(item)), (1, 2, 3), "" ) >>> "234"

The module of functions f.nth and its synonyms: f.first , f.second and f.third for safe access to the elements of the collections:


 f.first((1, 2, 3)) >>> 1 f.second((1, 2, 3)) >>> 2 f.third((1, 2, 3)) >>> 3 f.nth(0, [1, 2, 3]) >>> 1 f.nth(9, [1, 2, 3]) >>> None

Predicates


is an expression that returns true or false. Predicates are used in mathematics, logic, and functional programming. Often, a predicate is passed as a variable in a higher order function.


I added some of the most useful predicates to the library. Predicates can be unary (without parameters) and binary (or parametric), when the behavior of the predicate depends on the first argument.


Consider examples with unary predicates:


 f.p_str("test") >>> True f.p_str(0) >>> False f.p_str(u"test") >>> True #  ,    int  float  f.p_num(1), f.p_num(1.0) >>> True, True f.p_list([]) >>> True f.p_truth(1) >>> True f.p_truth(None) >>> False f.p_none(None) >>> True

Now binary. Create a new predicate that claims something greater than zero. What exactly? While unknown, this is an abstraction.


 p = f.p_gt(0)

Now, having a predicate, check any value:


 p(1), p(100), p(0), p(-1) >>> True, True, False, False

Similarly:


 # -    : p = f.p_gte(0) p(0), p(1), p(-1) >>> True, True, False #    : p = f.p_eq(42) p(42), p(False) >>> True, False #    : ob1 = object() p = f.p_is(ob1) p(object()) >>> False p(ob1) >>> True #      : p = f.p_in((1, 2, 3)) p(1), p(3) >>> True, True p(4) >>> False

I will not give examples of all predicates, it is tiresome and long. Predicates work well with the composition functions of f.comp , the f.comp super-predicate, the built-in filter function and the generic, which is discussed below.


Generics


A generic (generic, generic) is a callable object that has several strategies for calculating the result. The choice of strategy is determined based on the incoming parameters: their composition, type or value. Generic allows for a default strategy when no other is found for the passed parameters.


In Python, there are no generics out of the box, and especially they are not needed. Python is flexible enough to build its system of selection functions for incoming values. And yet, I was so pleased with the implementation of generics in Common Lisp, that of sports interest, I decided to do something similar in my library.


It looks like this. First, create a generic instance:


 gen = f.Generic()

Now we will expand it with specific handlers. The .extend decorator accepts a set of predicates for this handler, one per argument.


 @gen.extend(f.p_int, f.p_str) def handler1(x, y): return str(x) + y @gen.extend(f.p_int, f.p_int) def handler2(x, y): return x + y @gen.extend(f.p_str, f.p_str) def handler3(x, y): return x + y + x + y @gen.extend(f.p_str) def handler4(x): return "-".join(reversed(x)) @gen.extend() def handler5(): return 42

The logic under the hood is simple: the decorator stitches a function into the internal dictionary along with the predicates assigned to it. Now the generic can be called with arbitrary arguments. When calling, a function with the same number of predictors is searched. If each predicate returns true for the corresponding argument, it is considered that the strategy is found. The result of calling the function found is returned:


 gen(1, "2") >>> "12" gen(1, 2) >>> 3 gen("fiz", "baz") >>> "fizbazfizbaz" gen("hello") >>> "olleh" gen() >>> 42

What happens if no strategy comes up? It depends on whether the default handler was set. Such a handler must be ready to meet an arbitrary number of arguments:


 gen(1, 2, 3, 4) >>> TypeError exception goes here... @gen.default def default_handler(*args): return "default" gen(1, 2, 3, 4) >>> "default"

After decorating, the function becomes a generic instance. An interesting technique - you can transfer the execution of one strategy to another. Multiple-body functions are obtained, almost as in Kluch, Erlang or Haskell.


The handler below will be called if you pass None . However, inside he redirects us to another handler with two intas, this is handler2 . Which, in turn, returns the sum of the arguments:


 @gen.extend(f.p_none) def handler6(x): return gen(1, 2) gen(None) >>> 3

Collections


The library provides "enhanced" collections based on list, tuple, dictionary, and set. By improvements, I mean additional methods and some features in the behavior of each of the collections.


Enhanced collections are created either from ordinary class calls, or special syntax with square brackets:


 fL[1, 2, 3] #  f.List([1, 2, 3]) >>> List[1, 2, 3] fT[1, 2, 3] #  f.Tuple([1, 2, 3]) >>> Tuple(1, 2, 3) fS[1, 2, 3] #  f.Set((1, 2, 3)) >>> Set{1, 2, 3} fD[1: 2, 2: 3] >>> Dict{1: 2, 2: 3} #  f.Dict({1: 2, 2: 3})

Collections have methods .join , .foreach , .map , .filter , .reduce , .sum .


The list and the tuple additionally implement .reversed , .sorted , .group , .distinct and .apply .


Methods allow you to get the result by calling it from the collection without passing it to the function:


 l1 = fL[1, 2, 3] l1.map(str).join("-") >>> "1-2-3" 

 result = [] def collect(x, delta=0): result.append(x + delta) l1.foreach(collect, delta=1) result == [2, 3, 4] >>> True 

 l1.group(2) >>> List[List[1, 2], List[3]]

I will not bore a listing on each method, anyone can see the source code with comments.


It is important that the methods return a new instance of the same collection. This reduces the likelihood of accidental change. Operation .map or any other on the list will return a list, on a tuple - a tuple, and so on:


 fL[1, 2, 3].filter(f.p_even) >>> List[2] 

 fS[1, 2, 3].filter(f.p_even) >>> Set{2}

The dictionary is iterated over pairs (, ) , which I have always dreamed of:


 fD[1: 1, 2: 2, 0: 2].filter(lambda (k, v): k + v == 2) >>> Dict{0: 2, 1: 1}

Improved collections can be added to any other collection. The result will be a new collection of this (left) type:


 #   fD(a=1, b=2, c=3) + {"d": 4, "e": 5, "f": 5} >>> Dict{'a': 1, 'c': 3, 'b': 2, 'e': 5, 'd': 4, 'f': 5} #  +   fS[1, 2, 3] + ["a", 1, "b", 3, "c"] >>> Set{'a', 1, 2, 3, 'c', 'b'} #     fL[1, 2, 3] + (4, ) List[1, 2, 3, 4]

Any collection can be switched to another:


 fL["a", 1, "b", 2].group(2).D() >>> Dict{"a": 1, "b": 2} fL[1, 2, 3, 3, 2, 1].S().T() >>> Tuple[1, 2, 3]

Combo!


 fL("abc").map(ord).map(str).reversed().join("-") >>> "99-98-97" 

 def pred(pair): k, v = pair return k == "1" and v == "2" fL[4, 3, 2, 1].map(str).reversed() \ .group(2).Dict().filter(pred) >>> Dict{"1": "2"}

Monads


The latest and most difficult section in the library. After reading a series of articles about monads, I ventured to add them to the library too. At the same time he allowed himself the following deviations:



Maybe


The Maybe monad is (possibly) also known as Option. This monad class is represented by two instances: Just (or Some) is a repository of a positive result, in which we are interested. Nothing (in other languages ​​- None) - empty result.


A simple example. Define a monad constructor - an object that will convert scalar (flat) values ​​to monadic ones :


 MaybeInt = f.maybe(f.p_int)

In another way it is called unit , or monad unit. Now we get the monad values:


 MaybeInt(2) >>> Just[2] MaybeInt("not an int") >>> Nothing

We see that a good result is only what is being tested for int. Now we will try in business a monadny pipeline (monadic pipeline) :


 MaybeInt(2) >> (lambda x: MaybeInt(x + 2)) >>> Just[4] MaybeInt(2) >> (lambda x: f.Nothing()) >> (lambda x: MaybeInt(x + 2)) >>> Nothing

From the example it can be seen that Nothing interrupts the execution of the chain. If to be absolutely accurate, the chain does not break, but passes to the end, only at each step Nothing returned.


Any function can be covered with a monad decorator to get monadic representations of scalars from it. In the example below, the decorator ensures that only the return of the inta is considered a success - this value will go to Just , all the rest - to Nothing :


 @f.maybe_wraps(f.p_num) def mdiv(a, b): if b: return a / b else: return None mdiv(4, 2) >>> Just[2] mdiv(4, 0) >>> Nothing

The >> operator is otherwise called a monadic binding or a pipeline (monadic binding) and is called by the .bind method:


 MaybeInt(2).bind(lambda x: MaybeInt(x + 1)) >>> Just[3]

Both ways >> and .bind can take on not only a function, but also a functional form , which I have already written above:


 MaybeInt(6) >> (mdiv, 2) >>> Just[3] MaybeInt(6).bind(mdiv, 2) >>> Just[3]

To free a scalar value from a monad, use the .get method. It is important to remember that it is not included in the classical definition of monads and is a kind of concession. The .get method should be strictly at the end of the pipeline:


 m = MaybeInt(2) >> (lambda x: MaybeInt(x + 2)) m.get() >>> 3

Either


This monad extends the previous one. Maybe the problem is that the negative result is discarded, while we always want to know the reason. Either consists of the Left and Right subtypes, left and right values. The left value is responsible for the negative case, and the right - for the positive.


The rule is easy to remember from the phrase "our cause is right (i.e. right)". The word right in English also means "true."


And here is a flashback from the past: do you agree, reminds a couple (err, result) from the beginning of the article? Callbacks in Javascript? Call results in Go (only in a different order)?


That's the same thing. All these are monads, just not decorated in containers and without a mathematical apparatus.


The Either Monad is mainly used to catch errors. An erroneous value goes to the left and becomes the result of a conveyor. The correct result is sped to the right for the following calculations.


The Either monadic constructor takes two predicates: for the left value and for the right one. In the example below, the string values ​​go to the left value, the numeric values ​​to the right.


 EitherStrNum = f.either(f.p_str, f.p_num) EitherStrNum("error") >>> Left[error] EitherStrNum(42) >>> Right[42]

Check conveyor:


 EitherStrNum(1) >> (lambda x: EitherStrNum(x + 1)) >>> Right[2] EitherStrNum(1) >> (lambda x: EitherStrNum("error")) \ >> (lambda x: EitherStrNum(x + 1)) >>> Left[error]

Decorator f.either_wraps makes a monad constructor from a function:


 @f.either_wraps(f.p_str, f.p_num) def ediv(a, b): if b == 0: return "Div by zero: %s / %s" % (a, b) else: return a / b @f.either_wraps(f.p_str, f.p_num) def esqrt(a): if a < 0: return "Negative number: %s" % a else: return math.sqrt(a) EitherStrNum(16) >> (ediv, 4) >> esqrt >>> Right[2.0] EitherStrNum(16) >> (ediv, 0) >> esqrt >>> Left[Div by zero: 16 / 0]

Io


A monad of IO (I / O) isolates I / O data, such as reading a file, typing, typing on a screen. For example, we need to ask for a username. Without the monad, we would just raw_input , but this reduces the abstraction and clogs the code with a side effect.


Here's how to isolate keyboard input:


 IoPrompt = f.io(lambda prompt: raw_input(prompt)) IoPrompt("Your name: ") #  .   "Ivan"   RET >>> IO[Ivan]

Since we got a monad, it can be forwarded further along the conveyor. In the example below, we will enter a name and then display it. The f.io_wraps decorator turns a function into a monadic constructor:


 import sys @f.io_wraps def input(msg): return raw_input(msg) @f.io_wraps def write(text, chan): chan.write(text) input("name: ") >> (write, sys.stdout) >>> name: Ivan #   >>> Ivan #   >>> IO[None] #

Error


Monad Error, it is Try (Error, Attempt) is extremely useful from a practical point of view. It isolates exceptions, ensuring that the result of the calculation is either a Success instance with the correct value inside, or Failture with a wired exception.


As with Maybe and Either, the monad pipeline is executed only for a positive result.


The monadic constructor accepts a function whose behavior is considered unsafe. Success , Failture :


 Error = f.error(lambda a, b: a / b) Error(4, 2) >>> Success[2] Error(4, 0) >>> Failture[integer division or modulo by zero]

.get Failture . ? .recover :


 Error(4, 0).get() ZeroDivisionError: integer division or modulo by zero # value variant Error(4, 0).recover(ZeroDivisionError, 42) Success[2]

( ), . Success . . , Success . :


 def handler(e): logger.exception(e) return 0 Error(4, 0).recover((ZeroDivisionError, TypeError), handler) >>> Success[0]

. :


 @f.error_wraps def tdiv(a, b): return a / b @f.error_wraps def tsqrt(a): return math.sqrt(a) tdiv(16, 4) >> tsqrt >>> Success[2.0] tsqrt(16).bind(tdiv, 2) >>> Success[2.0]


, . , , ? ?


do-, . :


 def mfunc1(a): return f.Just(a) def mfunc2(a): return f.Just(a + 1) def mfunc3(a, b): return f.Just(a + b) mfunc1(1) >> (lambda x: mfunc2(x) >> (lambda y: mfunc3(x, y))) # 1 2 1 2 >>> Just[3]

, mfunc3 , . x y . .


Conclusion


, f . , . , . — .


. — . Pypi .


, .


. Thanks for attention.


')

Source: https://habr.com/ru/post/305750/

More articles:


All Articles