“To get above average, we need some kind of motivation beyond money” - an interview with Ruslan Cheremin

Friends, before you the next issue of "No slides" - a program, video cassette, podcast, where I interview with people interesting to me. The guest of this release was Ruslan Cheremin aka cheremin , an expert on Java and Concurrency. We talked about the Java Memory Model, technical blogging, experiment culture, basic education, and more.


')
As always - under the cut decoding interview.

Who needs memory models and multi-threaded bikes

- Today we are talking about technology. Ruslan, three years ago you told about the memory model - you said that very few people understand what it is, how it works and works. In particular, he said that you see it at the interviews. Has anything changed in recent years?

- Changed: I stopped conducting interviews! Therefore, I lost the thread a little and I do not observe it directly. Judging by what I read, and not by personal impressions - yes, now there is much more information, but I am not sure that there is more understanding. Rather, everyone began to ask certain questions about the memory model for the interview, respectively, the other party had memorized the answers to them. This is also not bad, because some of the stocks were remembered - that it is not necessary to do this, but it should be done like this. No need to understand the memory model of all.

- Among your colleagues with whom you encounter at work, is there a need for this knowledge?

- In fact, it turns out that at a deep level, few need it. If you have one person in your team who can prompt you in a difficult situation, then this is enough.
Yet they say that it is difficult - and it is really difficult. Making your own concurrent algorithms is not something you do every day, because it usually does not pay off. As a rule, you collect something ready from the constructor, somewhere you will make it simple in the simple case, and that is enough.

- Are these multi-threaded design patterns like “Java oncurrency in Practice”?

- Yes. If you know this, more than 99% of the tasks will be solved. It is more important to understand where your border lies: so that you understand when you step on those half percent, where you cannot competently do it yourself. Then you go with the question.

- Where is your border?

- In addition to the memory model, as an abstract model, there are many more performance details, where everything is much deeper. With an abstract model of memory, I more or less figured out. This does not mean that I understand everything - there is a large part about Data Races, and it is unclear whether you need to understand it for something else, except to complete the formal model.
And there is another task - ok, you made the correct algorithm, but will it be faster than something primitive, assembled from a ready-made designer? This is a big question, because people who write elements of a ready-made constructor, who write java.util.concurrent, know much more than you. They have more resources to make a good decision.

- On the other hand, if you in a large company develop high-load solutions, then you know more about performance than the people who do it in theory - you have a lot of production servers at hand, and you come across this every day.

- It's true. But you come across in a particular scenario, and it turns out that a niche for self-made solutions is when you have a specific hardware, specific version or range of Java versions, specific software. Here, of course, to beat Dag Lee is not so difficult, if you have it all recorded. Because the genius of Dag Lee is that he is trying to make a good algorithm that works on a whole range of platforms and virtual machines and for a whole range of usage scenarios. And all this works at 70-80% of the optimum. This is very cool, in order to do this, you need to keep a lot of things in your head. And when you have a specific hardware, there is a specific version of Java, and write benchmarks - here you also need some kind of expertise, but not transcendental.

- Was there any experience in your practice when you wrote something self-made, and it began to work faster at times?

- There was such an experience. Another thing is that the acceleration of the algorithm many times does not guarantee that the system will also start working several times faster. You have speeded up a specific piece, for example, three times, which gives a 5% increase, and that is good, if it happened so much. I have had the experience of speeding up a particular place at times - it just doesn’t have significant weight on the system-wide scale.

- Usually you move to another narrow place?

- Yes. You open this bottleneck, and when it becomes wide, you come across the next, as it usually happens.

- It turns out that the overall performance gain is not very large, and the investment does not pay for itself?

- Not always, but, as a rule, ultra-optimization of a separate small piece rarely pays off. Perhaps that is why there are not so many cool specialists who know this whole thing. I think that this will change further as there will be more cores, and a growing percentage of algorithms will have to be done concurrent to ensure sane performance - this is my vision.

About tech blogging

- We met five years ago. Then you were seriously interested in this whole topic, and by concurrency you had one of the most popular blogs in runet. Now you are writing again, but still a little about something else - that is, you also talk about performance, but already about Escape Analysis . Are you finished with this or will you develop it further?

- I do not understand. I do not have planning for the year ahead - they say, write a series of posts on Escape Analysis. If there is something interesting, write about it.

- Where did you get the desire to share your experience with the world?

- There is a great value in some clarity. First, you make something clear for yourself, and then the question arises - maybe this is an illusion of understanding, because you have loaded yourself with a lot of information? Can you state it in such a way that the other person can understand? It's like in mathematics - you can find the idea of ​​proof, but then it must be formalized as proof competent and formal. I think that this idea came from mathematics to me - I am writing to a blog and doing similar things.
Of course, there is a motivation to write for others, to share experience, to boast, in the end - as much worked. But at the same time, there is, so to speak, its own motivation to clearly state the idea, to make it understandable and readable. At such a moment, I understand that the topic is closed, and this gives a feeling of some finality.



- Has your own understanding of the area of ​​concurrency changed in recent years? Were there sharp turns, or was it just overgrown with "meat"?

- Four years ago I was fond of formal models - it was interesting how this can be formalized, how this complex topic can be squeezed into the framework of a formal model. And during this time I realized that the formal model is not the most difficult. The formal model is just such a contract.

Unlike physical models, this is a much less fundamental contract. This is not a contract between nature and man - it is a contract between some people and others. You just agreed to do so and try to do just that.

Therefore, it is not very interesting for me to study formal models too deeply. The fact is that there is some engineering practice behind them: the decisions of the engineers who make the processors, the decisions of other engineers who write the programming languages, and all of them have to agree among themselves. Mathematical rigor is a kind of compromise between them, not very fundamental.

- Do you follow what is happening in oncurrency-interest?

- I watch. But, for example, what is happening now with JMM 9, a new memory model, while I do not follow - I wait until they agree to something that I can understand. There is a certain percentage of specialists who know this topic much better than me, and when I try to understand what they are saying ... well, I’m waiting.

- Let's talk about escape analysis. How did you get the idea to do this?

- This is a cool idea and far from young, I heard about it for the first time 5-7 ago. It seemed that it could be something revolutionary if it was well executed in Java.

Because there are many people who complain - they say, there are few opportunities; it is not clear how and where memory is allocated and when it is deployed. But this, however, is really not enough - for example, I definitely need to know that the Nth object lies from this to this. And why should I take the garbage in the Garbage Collector. And it's great when they can do all this for me - I can relax and enjoy life.

And when this idea appeared - Sun seemed to report what it would be for them - I expected that there would be an explosion of articles on this issue and that they would do this work for me. I'm a lazy person and better wait for everyone to write. A small percentage of basic articles appeared, but the impression was that it simply did not fire - as if it somehow seemed to work, but it was impossible to rely on it. I occasionally tried to return to this topic, but I was still lazy, googled and waited for someone to do this job for me.

But no one did this work. And then I read someone's hot code — whether it was the Aeron code that did everything there, or Peter Laurie’s Chronicle Map.

And I saw that a person in a hot place, where there should clearly be no allocation, directly returns the object simply from a method, like a tuple. That is, he clearly expects that there will be no allocation here. Yes, I understand that escape analysis should remove this, but I see that a person already trusts it. It turns out that the analysis on the sly has matured to the stage when it can be relied on, even in simple situations.

This prompted me to stop braking, take it and try it. So little by little everything grew.

- In short, what is the result of your work?

- It turns out that in a large percentage of scenarios under certain conditions, it really works and works quite reliably. But there are several unobvious "but" flaws that if you don’t know, you won’t guess what they should be. There are more obvious "but", by which it is clear where they come from and how to get around them.

- If somewhere Escape Analysis machine can not do, then it lacks some information, or it rests on some kind of trade-offs. What usually information is missing and what trade-offs usually appear in this place?

- Oracle is now not too much pressure on Escape Analysis, including because Value Objects are already in RoadMap, which, on the other hand, will partially solve this problem. The algorithm for which escape analysis works is mutated, but its bottlenecks are the same as they were in 2010 - to change them dramatically means to cut a large piece and insert some other in its place. Not sure what HotSpot will do. There is another algorithm that is implemented in Graal. They have a different approach and, in many cases, it may be more intelligent.

- This thing is strongly related to how JIT works specifically?

- Yes, of course, the escape analysis is fairly glued into the JIT, but it also has algorithms.

- As I understand it, from the point of view of IR (the internal representation of the compiler), Graal and C2 are not very different. That is, it seems like the model with which the compiler works is very similar. Due to what and why is Graal done differently?

- I'm still not a compiler developer, but my understanding is that after 7 years from the moment when you first saw the algorithm, it’s not so easy to change one algorithm for another, very different. Yes, it has already spread all over the code.
I have an idea that Graal just emerged as a project that, perhaps, will eventually allow to get rid of a certain amount of historical layers of OpenJDK. Because there are no fundamental fundamental restrictions, but simply the amount of work that needs to be done is very large.

- Have you experimented with Graal?

- Rather, while I read that they write about him.

Practice diverges from theory

- Have you thought about how you got into this story? Why from the variety of technologies do you do what you do? Is your daily professional activity strongly connected with performance and its optimization?


- Approximately 10% of my work is related to performance. For the rest, I do what most of the senior developers do - juggling classics, “classic here, classic there.”

- And how often do you manage to manage with simple solutions? And how many have to sit and wrestle?

- As a rule, I have to break my head rather over the design, not over the performance. In those projects over which I work, his deep understanding is not necessary.

- Does your practice often disagree with theory? How often is the reality you expect to see, is it different from what you actually see?


- Practice disagrees with the theory. I do expectations on the basis of something, my intuition. For example, I read about the algorithm, as it was 10-15 years ago, and I understand - yes, it should work. In this case, more often it turns out that it works better than you expect. Still, engineers do not eat their bread for nothing. There are other cases, but according to statistics, more often it works better than expectations than worse. And if you compare it with the OpenJDK code, then this is a very difficult thing.

- Especially, apparently, C2 compiler.

- Yes. Therefore, it is difficult to draw conclusions there, and one can be guided only by assumptions.

About education

- Remind me who you are by education?

- I am a theorist, I graduated from the Physics Department of Moscow State University - I was engaged in chaotic processes there.

- I also engaged in chaos in my time, however, in mathematics. Do you think the physics that you dedicated to for many years changed you? What does your physical education give you as an engineer?

- One of the important things that gives a fundamental education - the ability to figure everything out, if you make enough effort. I often find myself looking at some task, and I get the feeling that it is for some special people, and I won’t figure it out at all. In fact, I can, if I spend on it a certain amount of strength. But it's easier to wait until someone writes a simple article that you read, you will understand everything and be satisfied.

The second important point - it is difficult to clearly draw the boundary of education. Each person has his own way through the education system, each creeps his own way. I had not only the Physical Department, there was also a boarding school - SUNTs MSU. There I also studied physics, but more in the mathematical sciences. And it is difficult for me to say who I am in terms of ideology, which is more influenced by physical education or boarding school. After all, I mastered IT independently, and an independent learning process is different from when they teach you - another, different view appears.

- What did you do for a few years between how you started doing seriously performance and concurrency, and the time when you graduated from the Physical Department?

- Earned money. I also worked on the physics department at programming, at first not with loads - the usual enterprise of a small scale. Then he worked for some time in education - in 1C there is a department dealing with products for education, training courses. At first I did a mathematical constructor, which is “Live Geometry”, then I got to physical ones. Mathematical released five versions and continue to release. It was an interesting experience - a kind of hybrid of programming and education, but still a little work on the scale. Client-side software, a lot of interface tasks, and the balance between complex algorithms and the interface is often shifted into "let's more buttons".

Then there was Yandex, where we were engaged in simulations, and there for the first time I had to engage in tight performance - we had the task to model the entire Yandex network on a limited amount of iron, with some significant simplifications, of course. But all this had to be pushed into one server, so there was a lot to play with there, without going beyond common sense. I didn't do super tricks and just juggled with what I had. Disruptor just grew up from there and quite by accident - I was looking for something else and came across it, and for some reason it interested me. Then I met Vova Dolzhenko - he turned around this topic and worked at Deutsche Bank. So I found myself at Deutsche.

"Poking nature with a stick"

- The culture of physics is a culture of experiment, which people lack now. People do not understand what it is and how the model is built, how to find out some properties of the model, how to draw a conclusion back to reality.

- It seems to me that, after all, in the natural sciences there is a definite view on how “to stick a stick at nature”, it has been formed for centuries. There are a significant number of graduates of the same Physics Department, who did not have time to understand this idea, and indeed I myself am a theoretician, to be honest. I don’t like to work with my hands. It is grafted with Feynman lectures or something similar. Just different motivations from people who experiment in IT.

There is a natural-science motivation, that is, to poke a stick into nature and understand what is happening. And at the first stage you do not have a practical task - first I will understand what is happening, then I will return the task and try to understand whether the result can be applied or not. It turns out that not all models go to practice right away.

- Doesn’t it happen that a specific case flies to you, where something slows down, and you need to overclock it three times?

- Yes, but there is another approach when a specific case comes to you and all you need is to solve it. This is not even a risk to call an engineering, rather an artisan approach. It is not bad, but they are different in that you either get a certain model, or you solve (or it seems to you that you are solving) a specific task.

- That is, you can take, without understanding the model, try 10 different ways?

- Yes, and one of them will shoot. To poke, and if the shot is - great, close the task. This is not necessarily bad if some objective metrics show that the problem is solved, maybe it is good. A business is not always ready to pay for scientific research lasting for months, and it is not always necessary. Still, payback is an important thing.

But these are different approaches, and somewhere one is better, somewhere else. And it is bad when a person does not understand the boundary between applicability. Yes, it is clear that it is necessary to present both approaches and distinguish between them. And when a person accustomed to the approach of “just solving a specific task” starts to make benchmarks and pull out some models from them, then it usually turns out to be garbage.

- There is a whole school on this subject - in Scandinavia, it seems. There, one guy did a cool thing: you run a certain set of tests, and he builds a model of how your processor is based on benchmark results. That is, he is trying to drive them in different ways and at the expense of this he makes a suggestion about which cache is common or not, how many cores, and so on. And this, it turns out, is an approach that, on the basis of benchmarks, tries to understand what is in reality there.

“But you have to understand how much time a person picked up this set of benchmarks, and how many models he had to go through - he also has a certain set of models from which he makes a choice. This is not at all the same thing as poking a finger blindly. Here there is a set of hypotheses and there is a set of benchmarks that allow you to make a choice from these hypotheses.

- It's difficult. If 10 years ago it was possible to say that we have an Intel processor, it has the MESI cache coherence protocol, now we only know that there is a MESI cache coherence protocol, but I’m not even sure that it is in a spec. In my opinion, this is classified information. How to live in such a world, when you are not too lazy to learn something, but simply do not provide information?

- It is difficult to live, of course, in such a world. In this case, you have to rely either on the fact that the JVM developers agree with the Intel developers and tell them everything quietly, or that they will figure everything out - not a bad option, for a large percentage of cases it works. JVM itself will do the work that you need. IT goes in this direction - we are moving further and further away from iron.

- This is about the division of the level of abstraction. On the other hand, the whole thing is getting harder. And the point is not in increasing the number of layers, but in the fact that the complexity of each layer increases. I am afraid that after some time attempts to build a model will be not only difficult but impossible - the complexity will be so high.

- Yes, it is possible and it will be difficult with such an approach. But you also need to understand why you need it and whether you need it at all. A good model is needed by people who write JVM.

- I’m rather talking about the fact that people who take 10 standard methods and try to somehow combine - maybe they are right?

- It is necessary to understand well why you climb into this area - where standard methods are not combined or it seems that they are not combined. And you can just get out of interest, but then the practical value of the results in question.

Interest is a normal motivation, it is an important thing. Hence the love of his work, without which nothing will work. If you want to play, that's great. But if you are useful for the sake of interest, you should understand this. And if it is useful for a specific purpose and I am sure that simpler ways will not give you any results, be ready to meet with great difficulty.

How to find time for self-development

- Do you love your job?

- Yes. I think I was lucky, because at least a few days a week I love her.

- On duty, I communicate a lot with different people, and often complain: “I would like to write an article, make a report or something for myself, but the administration does not give time to properly search. Since I can not guarantee a serious increase, I usually say: "Go through 10 ways that you know, combine something with them, and let's go further, because we have no time for research, we need to cut the features!". " How do you solve this issue?

- I also often hear about it, what kind of a good work you have, that you have time to explore all this. In fact, this is a kind of vicious circle - if you cannot find time for yourself to educate yourself, you are not doing enough valuable or competent employee to give you time to be a reasonable investment.

If you remain at the level of some Middl all the time, you are unlikely to dig up anything useful, and in this sense the employer is statistically right. He wants to get an accurate result, but it’s not clear about the employee’s digging if they will give a result. Of course, at first it is necessary to do it in due time, in the evenings, but this is normal. If you have no interest in this, it is generally not clear why you are working in this area.

- Well, the region can be chosen for reasonable money. , . , . , — .

— , — , . , - , , , . .

, . , , , , , , , . .

— , , — . ?

— , . Deutsche Bank , , , .

— , , — - - ?

- Yes. , , , , . , , .

— IT-, ? , , ?

— , , . : , IT .

— . — ?

— , .

— , ?

— , , .

— , , ?

— , , . , — , - , . , , , . , , .

— , , , . , , , - , : . . ?

— JMM , , , , .

— .

— , , , : «, oncurrency-interest , !» feedback loop . , - , , - .

— ?

— , Martin Thompson . , — . : «. ?» — , , . , .

- Yes. , — , .

— . — , .

— , . 10 , — , , , . , .

— , , — , . , .

— , ?

Nitsan Wakart , . , .

— , .

— — . , , « , ».

---

useful links

• Disruptor
• Java Memory Model