How we switched from Scala to Go

Translation of the article by CrowdStrike’s technical director, how and why they switched from Scala to Go as the company grew from 5 to 200+ people.

Scala has long been part of our CrowdStrike company's stack, in fact, even the main language. I helped introduce Scala when we started developing our software in 2012. In fact, it was even one of the main reasons for my transition to CrowdStrike. Several major developers were interested in implementing Scala, so it was a good option for everyone.

I came from the company Gravity, which is quite active in using Scala. It was the main language in the company. I got used to it, I liked it, I saw its power and I was sure that I could prevent some of the difficulties that I saw in Scala as CrowdStrike grows. We did high-load analytics, batch-tasks on Hadoop and our Chief Architect (hello, Bissel!) Used lambda-architecture long before it became fashionable.
')
A recent quote from one of our senior-developers made me write this post, describing why we transferred most of our stack to Go, and why we write to Go by default.

Instead of making the reader wait until the end of the post, I will immediately say that Scala does not leave our stack completely. In fact, it remains an addition to Go in those places where it does not shine. Scala is an important link in our machine learning and analytics stack. It works in conjunction with the Java products we use, and its ability to provide a pleasant DSL for our analysts, making Scala still a good choice. But it has become a more specialized tool instead of the default language.

I will tell you this story through the prism of the Technical Director. The prism in which you need to expand the company from its early days and 5 developers to 200+ programmers as the business grows. It's all about having a supported code base in which people can easily move from project to project and hire new people easily and quickly.

I remember the first time I saw the potential problems with Scala's scalability in Gravity back in 2009/2010. It was the end of the working day when we received information about a serious problem in production from one of our big clients. Several people began to investigate the problem, and we were able to localize the code in which the crash occurred. But the real problem turned out to be that we had no idea what this code was doing. We have rested against a strange symbol that we have never met in our projects before. This is the spaceship operator <| * |>. Someone said out loud "what the fuck is ** nh?". There was some kind of implicit magic that was completely unobvious. Cmd-B to go to the method did not help, since our IDE could not find the symbol (the IDE has since improved). Fast googling for "<| * |>" also did not bring results. We were driven into a dead end.


The programmer who wrote this code was unavailable because he was on vacation, so we needed to figure it out on our own. We found a recently added new library called scalaz, and a few hours later we found this mystical symbol and figured out what he was doing, rolled out the fix, and everyone was happy. But this moment turned the fix, which was supposed to take a couple of minutes, into a long chase. This was the moment after which I began to see a split in our development team.

Scala is a very powerful language, born in an academic environment that gives you enough flexibility to easily start writing “write-once” code. Usually Scala developers are divided into two camps - “This is the best version of Java” -laser and “I (heart) Applicative Functors” -lager.


In the camp “This is the best version of Java” they love the conciseness of Scala and the standard features that make Scala as a whole more enjoyable than Java. In their code, there is functional programming, but without fanaticism. In the camp of “I (the heart) Applicative Functors,” the functional world flourishes for the whole, and the people begin to deepen their knowledge, bringing their already functional background from languages ​​like Haskell.

In my experience, I divided these two camps, and in each were excellent programmers, so I could not say that one camp is worse or better than the other. In the semi-functional camp, there were perhaps potentially more more versatile developers working with different languages, or those who did not want to learn lambda calculus in order to work on an API server.

As an example, here is a piece of code from one project written by one of our experienced Scala developers:

import scalaz._
import scalaz.std.list._
import scalaz.syntax.monad._
import scalaz.syntax.monoid._
import scalaz.syntax.traverse.{ToFunctorOps => _, _}

class Foo[F[+_] : Monad, A, B](val execute: Foo.Request[A] => F[B], val joins: Foo.Request[A] => B => List[Foo.Request[A]])(implicit J: Foo.Join[A, B]) {

def bar: Foo[({type l[+a]=WriterT[F, Log[A, B], a]})#l, A, B] = {
type TraceW[FF[+_], +AA] = WriterT[FF, Log[A, B], AA]
def execute(request: Request[A]): WriterT[F, Log[A, B], B] =
self.execute(request).liftM[TraceW] :++>> (repr => List(request -> request.response(repr, self.joins(request)(repr))))
----- REDACTED -------

- , , - — WTF? . , , . , — , , , .

, . Scala , , SBT, IDE, , JAR… ScalaZ, . , . SBT . , , , , , SBT .

- , . Twitter , , , , Scala, . , . , Scala , 50 .

Go. Go , , . Go . , , . , 3 , 3 . , (, Sean Berry), 3 , Go , Scala .

, , , , , , , ? , ! Go, , , , Go, Go , . Go , . Scala? , , , Go.

Go , . , Go . Scala JVM, Java , JVM .

, . Go, Go Go : «, , . Scala- 4 , . , Go». senior-, web- . Go , .

Golang.

Scala ScalaZ( ValidationNel!), 7- . - Scala, Scalding. Scala, , , . Go -.

Go .