If you want to figure out something, learn right away from the best. Today, my questions are answered by the god Korutin and concurrency, Roma elizarov Elizarov. We talked not only about Kotlin, as you might think, but also about a bunch of related topics:

• Golang and gorutiny;
• JavaScript and its applicability for serious projects;
• Java and Project Loom;
• Olympiad programming on Kotlin;
• how to learn programming;
• and other exciting things.

Kotlin - well done!

Hey. First give a few words about yourself. Have you been doing Kotlin for a long time?

I have a long history with Kotlin. In 2010, Kotlin started as a project in JetBrains, where I was not working at the time. But Max Shafirov (he was working on Kotlin then and was one of the initiators of this movement inside JetBrains) invited me to become an external expert and look at the design, comment. Initially, the language was designed to solve its problems, because JetBrains has its own large code base in Java, with understandable problems that the code always has, and I wanted to make a language for myself to write my code more pleasantly, more efficiently, with fewer errors. Just spend at upgrading. Naturally, this quickly developed into the idea that once we have such problems, it means that others also have such problems, and they needed confirmation from other people that they were on the right path.

I was invited as an expert, so that I looked and checked what was happening with what was needed. About nullability - I insisted on doing this, because at that moment it was obvious to me that if you write Java, there are a lot of problems, but nullability is the main trouble that you constantly encounter.
')
I did not participate in the work of the team, I just occasionally glanced, participated in competitions at the Kotlin (Kotlin Cup). I have been engaged in competitions all my life, but I did not actively participate even then. For example, I would not reach the final of competitions like the Facebook Hacker Cup, the form is not due to the fact that I no longer participate in competitions on a permanent basis. And I took part in the Kotlin Cup, and since it did not gather a wide audience, I easily reached the final.

At that time (2012-2013), Kotlin was a sad sight from the point of view of tuling, because everything was slowing down there. Since then, the team has done a great job. I joined the team two years ago, right after the release of 1.0 and before Google officially recognized the language. In the team, I took up all kinds of asynchrony and corutines, just because it turned out that I had the right experience, I worked a lot in DevExperts with all sorts of large enterprise systems, and there was a lot of asynchrony and communication. Therefore, I was well aware of the problem areas - what needs to be repaired and what hurts people. It is very well laid down for the needs of Kotlin, because it hurts not only for us. It hurts everyone. Even in the JVM, Project Loom took over, which seemed to hint that it hurts everyone. I still do the Kotlin libraries, and our main focus is on all kinds of connected applications and asynchrony.

That is, you are mainly engaged in libraries, not a compiler, and that's it?

No, I do the compiler as far as possible. I communicate with the guys, and our library team dogfudit everything that they do in the compiler. We are also customers, we create a lot of features, when we run into some flaws, and we are testers of the first line of everything new that rolls out.

It turns out, if you go to YouTrack, filter on you, you can find a lot of interesting things.

Yes, you can find a bunch of all sorts of tasks, because I constantly bump into something.

You mentioned Project Loom. He was made by the guy who made Quasar. From the side it looks very funny, I just wanted to write an article about Loom on Habra. Can you tell me something about him?

I saw the presentation, the idea is clear. Korutiny and asynchronous programming is needed by all. For example, in the past JPoint, the guys from Alibaba told how they hacked the JVM and screwed the hotspot fayber themselves, just rolling a patch there, which even they didn't write, but some guys before them. They already then filed for themselves. Wonderful report. Highly recommend.

Do you recommend doing this?

So do in the enterprise account . Each large enterprise, above a certain size, when several thousand people start working for you (and for some less), keep your OpenJDK hack. And of course, if you have business-critical yuzkeys, then why not hack something for yourself, I do not see any big problem in this. Not that I recommend it, but I have to. If there are no lightweight streams in HotSpot, what to do? This, in fact, suggests that people need what is overdue. And the feedback that we receive on Korutin also suggests that yes, it is overdue, people need lightweight flows, people have a wagon for lightweight cases. The fact that they should somehow be supported in the JDK is long overdue, and in this sense I have no doubt that when Loom comes to production sooner or later, it will be in demand. There are people who need it. There are people who even HotSpot patches for this.

I saw a frequent problem - you have some kind of web server, a lot of people are knocking at it, and it starts blocking on threads.

This is a fairly typical problem. And a web server, and application-server, and backend. If you look at the same Alibaba presentation, why this business was needed, then they don’t have a web server, they have a classic enterprise architecture, they have all sorts of services written on the backend for Java, these services are under load. I worked with the same in DevExperts: services under load, you receive requests that you do not handle, because in the modern world you have everything connected. And you don’t process this request yourself, and you also call 100,500 other services and wait for them to respond. And if these services slow down, then you have a lot of threads waiting. You cannot afford to have tens of thousands of these waiting streams. And you get it just because of some nonsense the following: one service that you use slows down, and a lot of threads are waiting for you. And now it is a very big problem.

One of the reasons why people migrate en masse to Go is not because the language is good, but because there are lightweight streams out of the box, and there is no such problem anymore: the gorutines can wait, and they are worth nothing. In the same Alibaba, the decision that they have implemented is generally stupid of all the stupid ones. They are not very lightweight in the sense that they allocate one large stack of 2 megabytes each, hacking HotSpot, so that these stacks can be switched. They save physical flow, but do not save stacks. And for them the solution works - it is, incidentally, very simple, they have a HotSpot patch, as far as I understand it, not very big. The guys from Loom started something more global. They decided to save not only on physical flows, but also on the stack, so as not to spend 2 megabytes per stream. In the prototype, the current stack passes through HotSpot, it is copied into a small hip structure. And they can further reuse this physical stack for other purposes.

But there is such a clever hack: when you come back to the performance, they copy it not all, but only the very top.

Yes, there are hack cars and optimizations. What is the result of this is very difficult to say. Because using the example of the copying approach, the following problem immediately arises: what to do with native calls? From inside a native call, you can no longer copy a stack of a native call. Alibaba’s approach is no such problem. Native, not native - what's the difference, you just uncoupled that stack completely and left it alone, picked up another stack, everything works. And here it’s too early to say what happens or fails, sometimes you can live with this native stack, sometimes you don’t — it’s too early to say at this stage. For example, how it is implemented in Go - there is a completely different mechanism. As long as you execute the gosh code, small gosh stacks are used. Accordingly, when the ghost runtime calls a function, it looks at how much the stack needs. If the current stack is not enough, it re-allocates - increases the size of the selected stack. If, accordingly, you make a native call, then they already take some big native stack from some pool and use it.

And for the code too?

Never mind. They can simply switch to a large native stack, if you need to call some external function, for which it is unclear how much stack is needed. And when you execute a gosh code, you know how much stack you need, so we can run it on a small stack. There is a completely different approach. We do not copy, but immediately execute on a small stack. In fact, there is not much difference between these approaches as long as you occasionally fall asleep.

We are constantly asked the question: “What is faster? What fits? How do you do this in Korutin? ”We in Korutin do not hack JVM. Our task is to make it work under a regular JVM. And to work on Android too. There is an ART, which also knows nothing about korutinakh. And therefore, naturally, we have to use handles to generate bytecode, which does something very similar to copying the stack, which Loom does, only we do it in bytecode. We take it when he already zassenditsya. We take the stack, unwind and copy to the hip. We are not on the runtime that would do this for us, we have generated bytecode that does it. It saves and restores the state of cortina. Due to the fact that we are not doing runtime, naturally, we have more of an overhead projector. In runtime, you can do everything faster. On the other hand, if you use Korutiny for asynchronous programming, then you need to fall asleep, if you left to expect a response from some service, and send a request to some service is so expensive that the entire overhead on copying the stack doesn't bother anyone at all - slow you have it or fast - it generally becomes unimportant. Yes, if you use it for asynchronous programming. With us on Korutin in Kotlin, this is remarkably scaled, as shown in the prototype Project Loom.

Another difference is that since we in Kotlin are forced to do this in bytecode, we have such an interesting side effect. On the one hand, it seems to be unfortunate, but on the other - on the contrary. It consists in the following: it is impossible to put down an arbitrary function. You need functions that can fall asleep, mark with the suspend modifier - clearly mark that the function can pause and wait for something, that it is long. On the one hand, you don’t need it in Loom, because runtime can put anything to sleep. Alibaba’s solution is the same - you can remove a stack from any thread. Or in Go - there everything can be zassependit, any code can fall asleep. Flood another gorutin and do. On the one hand, this approach is very similar to programming with threads. You kind of program as before, only now threads are called faybers and have become very cheap. If you carefully look at the presentation of the same Loom, it turns out that the faders and threads are different things after all. How to make the old code, which is written with threads, run right out of the box completely on the faybery - is not obvious, and that they will succeed - no one knows. The problems begin there: what to do with the deadlocks, what to do with the code that is optimized for thread locals, again, some local hashes have their own or, with thread ID, some cleverly performance optimization does. And in Go, the same problem - when hard thread thread IDs do not run, writing some high performance algorithm becomes nontrivial.

Is there no such thing in Kotlin?

In Kotlin, we are not trying to pretend that thread and fayber are one and the same. We don’t even try to run the old code, we don’t need such a task in principle. We say: "Sorry, since we are not runtime, we cannot arbitrarily take the old java code and start switching something there." And we will not even try. We have another task. We say that we have a language feature that falls asleep to functions, you can write asynchronous code with them, and this is a new language feature. And from this problem (“how to run the old code”) we thus completely distance ourselves, we say: “Here is the new code, good, Orthodox, it can be put to sleep.” To some extent, this makes life easier, because you don’t need to float your head or people, and what happens if some old govnokod, who didn’t know that they are going to run it on the computers, will suddenly be launched.

We have no old code in our model, only a new one, which is initially prepared for the fact that today it is on one thread, tomorrow on another, and if, for example, it needs to know what thread it is, it will know it. Yes, we need thread local, but he can recognize them. However, he must be prepared for the fact that today the thread locals are one, and tomorrow - the others. If he wants these locales to travel with him, there is another mechanism for this, a corundum context, where he can store his things, which will travel along with the coruntine from thread to thread. This, in a sense, makes life easier for us, because we are not trying to maintain the old code.

On the other hand, we force a person to clearly think about his API, to say: here I am writing a function on Kotlin with Korutins. If earlier I look at some method in my code, getWell , it is not clear, this method works quickly and returns immediately or goes to the network and can work for an hour - I can only read the documentation and understand how quickly it will work. Or maybe now it works fast, and tomorrow programmer Vasya Pupkin will come and make it so that he now goes to the network. With Kotlin-Korutin we give a language-guaranteed mechanism with the suspend modifier. When I myself work with Korutin, I look at some function, if I don’t see the suspend modifier, it means that it works quickly, it does everything locally. There is a suspend modifier, which means that this function is somehow asynchronous, it will go on the network for a long time. And it helps to do a self-documenting API, so that we can immediately see what awaits us. This helps to immediately avoid stupid mistakes when I was somewhere forgotten and somewhere in the code I caused something long without knowing it.

In practice, this turns out to be very good. This is the need to explicitly mark these sleeping functions. In Go, for example, this is not, I am not obliged to mark anything there. It turns out that this side effect of our implementation (which should be marked with the suspend modifier) ​​helps you to make the architecture correctly, helps you control that you will not cause some random wildly long asynchronous game in the place where you initially expected everything to happen quickly.

But there are some things that are difficult to forbid, for example, some kind of network IO, file.

No, network IO just ban quite easily. Here file IO - difficult. But here’s another subtle point: for most applications, file IO is a fast thing, and therefore it’s perfectly normal that it works synchronously. A very rare application works so much with IO that it becomes a problem that it takes so long. And here we give a person the opportunity to choose: you can directly make an IO file with us and not bathe, because it will block what is happening (because usually it is fast). But if it’s specifically in your case, it’s just some kind of very long calculation, it’s not asynchronous, but it’s just a lot of CPU time, and you don’t want to block any of your other thread-pools, we provide a simple clear mechanism: you start a separate thread pool for its heavy calculations, and instead of writing a regular function that is fun computeSomething () , and writing in the documentation “Dudes, neatly, this function can work for a very long time, so attention - do not use it anywhere, do not use it in UI ", we offer a simpler way Khanizm. You simply write this function as suspend fun computeSomething () , and to implement it you use the special library function withContext , which transfers the calculation to the special thread pool you specified. This is very convenient: the user does not need to soar the brain anymore: he immediately sees suspend, knows that this thread does not block him with a call, and he can quite easily call it from the UI thread and so on.

It will switch to the desired stream inside, and its stream will not be blocked. This is the correct separation of concern: the user does not soar, as it is implemented, and the one who implements can be thrown right into the pool to which it is needed, and the computational resources in the application are properly distributed. In practice, this is very convenient in terms of programming style. It is necessary to write less documentation, the compiler will check and correct more.

I think how safe it is. Can someone break the thread pool or break into other people's data?

Naturally, everything is possible. From the curves of the hands is hard to protect. It is clear that no matter how much we write to the compiler all sorts of type systems and checks, everything can always be broken. The question is that the compiler should help write the correct code. Unfortunately, the dream to forbid writing bad code is utopian. We specifically do not include any features in the language. - Java, , . , , . , . -. .

Kotlin, . , .

?

, : raw types? .

.

Java?

Yes.

- , - . List, .

, List. List , . . raw List, platform type, , , List . raw type, , Java , . Java, , , , , , raw type. , — , . , — . — , , raw types. , migration story.

platform type ?

flexible. nullable-. , String. , String nullable String — . , String — nullable -nullable. Kotlin , , . String, nullable String. , - Java, .

?

, , , , . Flexible , dynamic. Flexible — , Java, String, nullable String, int.

- - , .

, , . . read only mutable. List MutableList. Java List, , Java- , , - List MutableList. , Java. , Java - , . , , . , , , List, , , MutableList, List . List, String, .

- , ? - ?

, . , . It just works. Java . nullability- , , nullable . , . , , , . , , , , - . , -. - JavaFx, - , JavaFx Java, , . , - , . . , , .

, .

Of course. , . Kotlin Native. : , seamless C- , , - .

, Native ?

, , Kotlin JavaScript, - - . «Write once, run anywhere», , . : , Common Kotlin Portable Kotlin, , . , - - , , . , , - . , .

JVM , Java, JS , JS, Native . , , , , . - JS, . -. - : double, String. JVM- 0 «0.0», JS . . JS , , — ? , . , , JS- . , -. , . - — , , — . , , , , JVM, JS, Native — , , - . .

?

Yes. , , .

, …

, . , . Loom . - JVM. , .

, , Java?

, . . — . , . , - . , for ( i in 0..10 ) , for (int i = first . , , . . , .

- ?

, … , . . JVM JS.

Node.js?

! . , JS-. , , JS- . , . , - — , JS-, - — . , . .

« ». , ?

Of course. , , . JVM , Java, . JVM . legacy, enterprise, . , , . , JVM. , — , . , . , — JVM « ». . API , . , . , . , , . , , .

TechTrain « ?» . . , .

JS , Kotlin Java?

Java . «Kotlin in Action», , Java-. , Java- , , Kotlin-. , «by design». , Java- . . JPoint , Kotlin . , . , 60-70% Java. Java, , - . Java .

- — , , -. as-is. , , . Kotlin , , «», «». , «» — . while — . 100500 . But why? , Java-. - , « Kotlin», , .

, . . , , . , .

, -, ?

, . . Java- . Android- — , . . , . — .

- , ?

, . , , . , , . — , , . Kotlin . , . - , , , . : , . C++, Java, Python. . Java - , Java, . , , puiblic static void main… -, . Java, , . ?

Kotlin : , , Python, . C++ , C++ . , , . , . , , . Kotlin — . — Python. . . , . , . . — , , , , — . , .

, — ?

, . must have. , — . . , . — . , , , .

?

… , , . . JS- — . — . JS, , . - JS, type checkers, Flow TypeScript. - JS . Python. - DSL, . Django , . , . , , , . . Django, , CRUD-. , - — . -, - , , . . . Kotlin, , Java, . core belief Kotlin, .

, - , Kotlin ?

Of course! , , - CRUD-. : , , — , . -, — , . , TypeScript Flow — JS.

Kotlin , Kotlin JS TypeScript , . TS Kotlin/JS, , TS , JS-, . Kotlin/JS , . , .

, — double…

, . , , .

- .

.

?

:-) , .

, Java .

. , , . Java C++ — , , — , .

Java , - ?

. — , . - , , — , , Java , . . , , HotSpot . — , Java , HotSpot — .

, , Java . (, ) — , Java . : Java , . - Java Python — - , , Python. . , , . , - Facebook Hacker Cup, , , — . . , — . , . , , . Java — , , Java. ++ — . , , C++-, .

— Kotlin ?

, , . ICPC, , , . ICPC Kotlin . : C/C++, Java, Python Kotlin. , , , : . , .

- Kotlin?

- . , . , « 0» .

- ?

, Kotlin , . — . , . . , , , ? .

, , , , , .

Not. , . . . — . , — , . , . , , . - , , — ? , , , ?

?

, . -. , , — . - , , . - . - , . Kotlin , , , .

. , , , , . , . , , . , , . ?

, .

Of course not! , , .

, — …

. , . . — , , . — , .

- , .

, . . , , . Kotlin, Kotlin.

, JetBrains , .

JetBrains — , . , , .

, : - ? - , - ?

:-) , ? , : , . , , , . - , , Kotlin, , . , . . , , . , — , , . , .

Kotlin , , , Kotlin , Java. , Java. , , ( ) . , , , , pure, , — unsafe. , , Kotlin. Kotlin , StackOverflow most loved languages. , , Rust. Rust , Rust — . . , Rust Kotlin. , Kotlin , . Rust , . , -. . - . . - , .

— Java , — .

, Kotlin . . — - . Kotlin nullable-. , , , -, , exception — NullPointerException. , — . nullability. And so on.If you design a language not in the abstract, not as an academic exercise, but you try to solve the problems of people with whom they often come across, the language turns out to be good. Why love him? Because he solves their problems.