Applied introduction to monad transformers, from problem to solution

Imagine you are sitting at your desk, drinking coffee and getting ready to write code on Scala. Functional programming was not as scary as it is painted, life is beautiful, you sit back, concentrate and start writing a new functionality that you need to pass this week.

Everything is as usual: a few laconic one-line expressions (yes, baby, this is Scala!), A few strange compiler errors (oh no, Scala, no!), A slight regret that you wrote such a tangled code ... And suddenly you come across strange problem: the for expression does not compile. “It's okay,” you think: “Now I’ll look at StackOverflow,” as you do every day. How we all do it daily.

But today seems to be a bad day.

At first it seems to you that the best answer is too abstruse. Usually it is enough to scroll down, find a simpler solution and forget about the explanation using the theory of categories, monads, and all that without much remorse.

However, this time the second answer is similar to the first, and the third, and the fourth. What's happening?

Monads. Transformers.

Even the names sound scary. Let's take a closer look, what is your problem?

First you wrote the functions:

 def findUserById(id: Long): Future[User] = ??? def findAddressByUser(user: User): Future[Address] = ??? 

It looked elegant: the Future class represents asynchronous computing, and it has a flatMap method, which means you can put it in a for expression. Super!

 def findAddressByUserId(id: Long): Future[Address] = for { user <- findUserById(id) address <- findAddressByUser(user) } yield address 

Then you suddenly realized that a user does not exist for every identifier. What to do if the user is not found? Well, well, the Option class serves just this purpose:

 def findUserById(id: Long): Future[Option[User]] = ??? 

And, for that matter, some users may not have an address:

 def findAddressByUser(user: User): Future[Option[Address]] = ??? 

But when you returned to this code, a compilation error appeared:

 def findAddressByUserId(id: Long): Future[Address] = for { user <- findUserById(id) address <- findAddressByUser(user) } yield address 

Yes, that's right, because the return type is now Future[Option[Address]] :

 def findAddressByUserId(id: Long): Future[Option[Address]] = for { user <- findUserById(id) address <- findAddressByUser(user) } yield address 

The compiler must be satisfied. But what does he write?

 error: type mismatch; found : Option[User] required: User address <- findAddressByUser(user) 

Not good. Thinking a little, you remembered that <- is just a convenient way to call the flatMap method, and if you called it on an object of the Future[Option[User]] , you got Option[User] , although you need a User object ...

You tried this and that, but it's not that. The best you could think of was as follows:

 def findAddressByUserId(id: Long): Future[Option[Address]] = findUserById(id).flatMap { case Some(user) => findAddressByUser(user) case None => Future.successful(None) } 

Ugly or at least not as beautiful as it was before. Ideally, you would like something like:

 def findAddressByUserId(id: Long): Future[Option[Address]] = for { user <- userOption <- findUserById(id) address <- addressOption <- findAddressByUser(user) } yield address 

Such a flatMap method, which immediately extracts the value from both Option and Future . But on StackOverflow, no one mentioned it ...

Still, what's the catch? Why is there no flatMap supermethod that works with objects of the Future[Option[X]] ?

Dear reader, take a deep breath: we are going to mention a few things from theory, but don't despair. Here is all you need to know to read further:

1. Functor is a class with a map function.
2. Monad is a class with the flatMap function.
   It's all. I promise.

This basic knowledge of category theory helps to solve the riddle.

If you have two functors A and B (that is, you can call the map method on objects of class A[X] and on objects of class B[X] ), you can group them without knowing anything else about these classes. You can take the class A[B[X]] and get the Functor[A[B[X]] by assembling Functor[B[X]] and Functor[A[X]] .

In other words, if you know how to display inside A[X] and inside B[X] , you can also display inside A[B[X]] . Automatically. Easily.

For monads, this is not true: the ability to perform flatMap on A[X] and B[X] does not automatically give you the ability to perform flatMap on A[B[X]] .

It turns out that this is a well-known fact: monads are not arranged , at least in the general case.

Well, monads are not compiled in the general case , but you need flatMap and map methods that work on objects of the Future[Option[A]] class.

We can definitely do that. Let's write a wrapper for Future[Option[A]] with the map and flatMap :

 case class FutOpt[A](value: Future[Option[A]]) { def map[B](f: A => B): FutOpt[B] = FutOpt(value.map(optA => optA.map(f))) def flatMap[B](f: A => FutOpt[B]): FutOpt[B] = FutOpt(value.flatMap(opt => opt match { case Some(a) => f(a).value case None => Future.successful(None) })) } 

Not bad! Let's use it!

 def findAddressByUserId(id: Long): Future[Option[Address]] = (for { user <- FutOpt(findUserById(id)) address <- FutOpt(findAddressByUser(user)) } yield address).value 

Works!

Well, this is if you have an object of type Future[Option[A]] . But what if you have, say, List[Option[A]] ? Maybe another wrapper will help? Let's try:

 case class ListOpt[A](value: List[Option[A]]) { def map[B](f: A => B): ListOpt[B] = ListOpt(value.map(optA => optA.map(f))) def flatMap[B](f: A => ListOpt[B]): ListOpt[B] = ListOpt(value.flatMap(opt => opt match { case Some(a) => f(a).value case None => List(None) })) } 

Yeah, she looks like FutOpt , right?

If you look closely, it is clear that we do not need to know anything about the "external" monad ( Future or List from the previous examples). As long as we can do map and flatMap , everything is fine. On the other hand, remember how we analyzed the Option object? You need to know the specifics of the "inner" monad (in this case, Option ) that we have.

So, we can write a general data structure that wraps any monad M around the Option class.

Awesome news: we accidentally came up with a monad transformer , which is usually called OptionT !

OptionT has two parameters F and A , where F is the wrapping monad and A is the type inside Option . In other words, OptionT[F, A] is a flat version of F[Option[A]] and has map and flatMap .

class OptionT [F, A] is a flat version of class F [Option [A]], and is itself a monad

Notice that the OptionT class OptionT also a monad, so we can use it in a for expression (after all, we have a city or a garden for this).

If you use libraries like cats , many monad transformers ( OptionT , EitherT , ...) already exist in them.

Let's return to our original example:

 import cats.data.OptionT, cats.std.future._ def findAddressByUserId(id: Long): Future[Option[Address]] = (for { user <- OptionT(findUserById(id)) address <- OptionT(findAddressByUser(user)) } yield address).value 

Works!

Can we improve on something else? Perhaps if we use wrappers all the time, it’s worth returning OptionT[F, A] of these methods:

 def findUserById(id: Long): OptionT[Future, User] = OptionT { ??? } def findAddressByUser(user: User): OptionT[Future, Address] = OptionT { ??? } def findAddressByUserId(id: Long): OptionT[Future, Address] = for { user <- findUserById(id) address <- findAddressByUser(user) } yield address 

And this is very similar to our original code. And when we need an actual value of type Future[Option[Address]] , we can simply call value .

Before completing the article, a little caveat:

• Monad transformers work well in some common cases (as in this), but don't get too carried away: I do not advise investing more than two monads into each other , otherwise the code will become complicated. For a sad example, see this README: https://github.com/djspiewak/emm ;
• Monad transformers are not given for free in terms of memory allocation. They use a lot of wrappers, so if you are worried about the speed of work, think twice and run the performance tests;
• Since they are not standard in the language (there are several implementations in the cats , scalaz libraries, and possibly others) do not expose them in your API . Call value on the transformers and return just the class A[B[X]] . It does not impose any restrictions on your users, and also allows you to change the internal implementation without making changes to the API.

I will add that monad transformers are just one of the ways to get rid of nested monads. They are suitable if you have a simple problem and you do not want to change the code much, but if you are ready for more, pay attention to the Eff library .

So, I repeat, monad transformers help us in working with nested monads, providing a flat view of two nested monads, and are themselves monads.

I hope I have proved that they are not as scary as they are called, and that you could come up with them yourself (or maybe they came up with them to some degree or another).

There are standard application cases for them, but do not abuse them.

If you want to know more on this topic, I talked about monad transformers at the Scala Italy conference last year: https://vimeo.com/170461662

Happy (functional) programming!