“Of course, it’s risky, but you can use it with proper precautions”: Chris Talinger on Graal

“Living in Hawaii, working on a super popular service, introducing an experimental Java technology there in production, which others are still looking at with caution” - sounds like a description of a made-up Java career that you can only dream of. But there is a person for whom this is all harsh everyday life, and we talked with him.



Thanks to Chris Talinger , Twitter is already using the new Graal compiler, and not just in the name of the love of innovation: it helps the company to save significant amounts. Chris has already shared his Twitter experience in St. Petersburg at the Joker conference, and now has prepared a new report designed to show ordinary Java developers how to approach Graal. While waiting for this report, we asked him about the fundamentals of Graal, how Twitter is now going even further, and how Chris organized the Java conference LavaOne in Hawaii.

')



Ruslan ARG89 Akhmetzyanov: Let's start from the very beginning: what is Graal in general?



- It is pronounced / gral /, because it is not an English word / gr æ l /, it is German / gral /. Graal is a just-in-time compiler for the JVM. There are different JVMs, but I think that most people know and use HotSpot, which is part of OpenJDK. There is also, for example, J9 from IBM. But I have been working with HotSpot for a long time.



HotSpot has long been working with the C1 and C2 compilers. And Graal is another JIT compiler that you can use if you wish, which we do, for example, we are on Twitter. This was my previous report.



Ruslan: And here's more about the basics. If you start to google Graal, it is always mentioned in conjunction with Truffle, and Truffle was not mentioned in your report ...



- It was recently renamed. A couple of years ago, the name Graal referred simply to the JIT compiler. It was called Graal, and there was another Truffle technology based on Graal. And recently, Oracle Labs re-branded and combined the Graal, Truffle, Substrate VM JIT compiler and some other things under the same name Graal VM. And this kind of confuses people. When I say "Graal", I always mean the JIT compiler, not Truffle, Substrate VM or something like that. The compiler is called Graal. Truffle is a framework for developing runtimes for programming languages.



Ruslan: Okay, we decided, we are talking only about JIT. The first question is: why does anyone even need a JIT compiler written in Java if we already have a perfectly working compiler on “syach”?



- There are two reasons. As I said, HotSpot already had two compilers, C1 and C2, written in C ++. And C2 is the top-tier compiler. This means that it generates code for peak performance. The goal of Graal is to replace C2 and HotSpot. And the main reason is that C2 is a very old technology.



Yes, as you said, she copes with her tasks. But it is very old and complex, so it’s very difficult to understand, it’s difficult for new people to understand the code and work on it. It is very difficult to make changes. The compiler works, but we really can’t bring new optimizations to it. And new language features appear: the projects I worked on at Oracle, Valhalla, Panama, and the like ... Implementing all this in C2 is difficult, and in Graal it is much easier. This is the first reason.



And the second reason is that under certain conditions, let's say, Graal produces a far better native code than C2 is capable of - especially for companies like us, Twitter. I told about this in a report on Joker 2017. In production, we save 8% of CPU utilization simply by replacing C2 with Graal. And on the scale of Twitter, it's a lot of money.



Oleg olegchir Chirukhin: Sounds reasonable. And what about backward compatibility? All these innovations do not accidentally break our old code?



- No, because the compiler simply reads Java bytecode, the input language for the compiler is Java bytecode, we just parse it. Graal does not interfere with the bytecode; it just reads it, just like C2. But the fact that he then turns out on the basis of bytecode - native code - is not too backward compatible, yes.



Oleg: And when people base their code on all sorts of weird and unreliable things like Unsafe, is this something in Graal that doesn't break?



- I would say that problems are very unlikely. I will not guarantee anything, people are probably doing very strange crap, but the probability is very low. Until now, I have not come across a code that would not run on Graal. And I tried everything. Today at Graal all the main services work for us.



Ruslan: Did you remember any particularly insane code from all that fed Graal, about which you were worried “everything won't go smoothly here”?



- I do not even know. I have been doing this for a long time and have always tried to try a very different code. When I was still working in Oracle, I just downloaded random software and tried to run it on Graal and see what I got. Then I found a few bugs, and we fixed them all. Oh, what software I sometimes come back to check if it still works: Runescape game, now it is considered old.



Oleg: Oh, I know her.



- She was super popular 10 years ago. There is still. It seems that everything has already changed there and now you can download the native client, but at that time it was written in Java and launched in the browser using a Java plug-in, if I remember correctly. And you could run a separate Java client. It is still installed on my laptop, it still works and I can run it.



The reason I use it is because it downloads a lot of things. You have a small Java client on your desktop that downloads a bundle of JAR files and runs them. And all the code there obfustsirovan. Obfuscation usually makes changes to the code that neither the compiler nor the VM expect. This is still a valid code, there are no errors in it. But bytecode is different from what Javac would issue. And for the compiler it is, let's say, interesting! I still occasionally appeal to this game, but I think because of it, nothing ever fell.



Oleg: By the way, can we use Graal for obfuscation? Replace fragments? We can use it for the ahead-of-time compilation, can we obfuscate in this case by getting the native code?



- Are you talking about AOT compilation, which appeared in Java 9?



Oleg: Yes!



- How does AOT work in Java 9? Creates a native library, but does not create a static executable file. You take a .class file, a JAR file or a module and compile it into native code. But to run it, you still need the JVM anyway, and she still needs to read the class file, just then she sees: “Oh, for this I already have native code, I don’t have to pull the compiler!” Do you understand? This is not exe-shnik, it is just precompiled code.



I may be mistaken, but the Substrate VM (part of the whole Graal VM story I was talking about) acts a little differently. Substrate VM creates an executable file that already contains a VM. Here I’m not completely sure, but inside there is still required data class files. And in this case, all obfuscation is lost.







Ruslan: Let's talk about using Graal on Twitter. With the fact that you can reduce the load on the processor, of course. But is it not risky to use such experimental technology in a service of this magnitude?



- There is always a risk of anything to do. You have an application, you have Java code. When you change this code - at least add a line, at least refactor - it is always risky. Yes, Java is well tested, and you can be 99.999% sure of success, but everything may fall because of some bug.



You are right, of course, it is risky, because Graal is not as well tested as C2, which has existed for 20 years and which the whole world works on. The reasons why we do it: firstly, money (it is always good), and secondly, we test very, very well before sending it to production. We do not just take service in production, switch it and cross our fingers, hoping for success.



We tested different scenarios in the very first trial versions. At first, it was checked in experimental environments just to see if it worked or not, and then rolled out very slowly. Only one instance in the production was started and they looked, everything is in order. And then hundreds have already been launched. We slowly changed everything, and almost a year passed before we switched over entirely. Yes, it is risky, but if you take proper precautions, you can.



Ruslan: A year from the start of technology testing or a year from the launch of the first instance?



- A year since I started working on Twitter. And from the first instance up to 100% on Graal 6-8 months passed.



Ruslan: You said that since the beginning of using Graal on Twitter, we have encountered only 4-5 bugs in this regard.



- Something like that, yes.



Ruslan: Is this related to the mentioned thoroughness of testing? Or do you have any special life hacking "like with a huge architectural transition to an experimental technology not to sting bugs"?



- There are no special life hacks, they just connected Graal and that's it, they used the experimental environment at first. I did nothing special, did not modify Graal for use with us. These were the usual bugs, they were sorted out, they were all upstream. I already mentioned this in the report, but since then even more time has passed: the last time a bug was discovered more than a year ago.



Ruslan: If you google “Twitter Graal crashes”, there are related questions on Stack Overflow?



- We fixed everything on GitHub. The source code for Graal is on GitHub and I’ve got bugs there, so you can open the project. Unfortunately, Oracle Labs changed the repositories, now they seem to be in the third. In general, I started up bugs in some old versions, in the old repository. I think if you open the slides of my report , you can find us links to bug reports .



Oleg: And where to see bugs and feature requests? I opened the page on GitHub, there are few of them.



- Well, technically, this should all happen on GitHub. Something happens in the mailing list. And what, you want to understand how to issue feature-request?



Oleg: No, I want to understand how to monitor the activity in this project.



- In fact, everything is on GitHub. If you go there and open Graal - now there seems to be a project there called Graal - and subscribe to it, you will receive notifications about everything going on. And you should also subscribe to the OpenJDK Graal mailing list, discussions also take place there.



Ruslan: Speaking of GitHub and checking your bug reports. What are you doing now in connection with Graal? Since you have been running it for more than a year now, it looks like your system is already working normally. Now do some fixes or new features?



- I started one thing that I need to complete, especially now that Java 10 is coming soon. Although for 10, I think it's too late. In general, I began to add support, although this is not quite the right word ... to transfer the optimization of Compact Strings from C2 to Graal.



I started this work, Oleg here nods, because he knows that his name will pop up now. Not everything has been done there, some parts have remained, but the main ones have been made. Now you need to test and the like. And I did not have time to go back to this and finish it. And then Oleg said, “Maybe I'll do it?” And he did not!



Oleg: I am very ashamed, really. What about processors and architectures other than Intel?



- There is a SPARC-port Graal, I started this long ago, and then someone else finished. He works. And there was also a guy who forgot his name, which made the ARM 64-port as a thesis at the university, and this port lay idle. But then I persuaded him to donate this code to the Graal OpenJDK project, and when this happened and Oracle decided to open the ARM, Red Hat picked up this development. So now, in fact, all three major platforms are supported. Perhaps I shouldn’t call SPARC “main” anymore, but the two main ones are supported. By the way, Graal does not have 32-bit support. All platforms it supports are 64-bit.



Ruslan: Right, who needs these 32 today?



- Here is what I say! I do not know, there are some crazy people who want to run it on ARM, which can even be understood because it is used in embedded systems, so there is some sense. In general, this is one of the things that I do.



The other one we are doing on Twitter ... Nowadays the fashion for the word “cloud” has already passed, the new buzzword is “machine learning”. Everyone is talking about ML. There have been reports about this, this is not something new, we have been doing this for 2-3 years. We have an ML framework, let's call it that, which was designed to customize the values ​​of JVM parameters, primarily GC. So you start a service, or an application, or whatever you have there, and tell the framework: "So, I want to adjust my GC parameters for the best performance."



This framework is called Autotune, and we use it to configure inlaying parameters in Graal for services using Graal. This is the next step. We are doing this now, testing it, and have already found a good set of parameter values ​​that improve performance. But we haven't tried production yet, I only have experimental numbers, so don't trust them too much. I think we managed to squeeze out another 8% in the experimental environment. Production usually turns out to be less - perhaps we will end up with 5%, it will be great, again saved money.



Oleg: Autotune is not in the open source?



- No Unfortunately. I don’t know what the status is there - I’m not sure that it will ever be revealed, because we are based on a different technology there, so I don’t know if it can be opened in principle. It will be necessary to clarify.



Oleg: Are these inaling parameters of Graal being configured statically, or can I be changed directly on the fly during the performance?



- Good question. I think at the moment there is no support for changing them on the fly. But technically, I guess it is possible. In Graal, it’s just as good as C2 that its parameters are simply “-Dgraal.tototam”, you can change them on the fly. I have never tried it, but I think it is possible.



Oleg: But then deoptimization and re-optimization with these parameters is required, right?



“I’ll not say right now, I’ll have to look at the code, but how the parameters are defined ... I think that it’s not static static or something like that, but in this case the compiler probably just reads them from memory, and you can change them . I do not think that they are inlined in the native code.



Oleg: I think it should be interesting for the conditions of dynamic cloud environments. All constants can change on the fly.



- This is interesting, yes. And it creates a whole class of new problems. When you do this, and then you want to restart, it's always tricky.



Ruslan: On the question of the openness of the Grail: how much does it take to be competent to contribute to it? And then, it seems, Oleg now can not wait for something to go there!



- I hope, pododit! This is a good question, it is connected with the main reason for which I am drowning for this technology. I have already said that C2 is very complex. They need to spend 2-3 years only to study the code and understand what is happening. And then you will find yourself ready to write a new code. I saw this many times in Oracle, when we hired new people and fixed bugs for a year or two, they were annoyed that they did not understand the code as quickly as they would like. So the whole point of replacing C2 with Graal is that it is much easier to understand - because of the way it is written, it is well structured and divided into modules. I don't mean the Java 9 modules, but there is a division into parts, this is a very modular structure. And in this project, measures are taken so that the participant does not create circular dependencies and does not add dependencies that would not follow. In general, everything is much clearer.



And besides, it is also written in Java, which is much clearer, especially when it comes to very low-level system code. With C ++ you can do a bunch of weird shit that no one can understand. And in Java, it's hard to do what people can't understand. Unless you start going crazy with Unsafe, but in Graal they don’t.



The idea is that you can recruit college graduates and show them the Graal. Especially cool is that you can just open Graal in your favorite IDE and climb the code there. The goal is to attract graduates to work on Graal, and within a few months they will understand and be able to be productive. This is what I would really like to see.



Oleg: Maybe you should write a book for newbies, bringing them up to date?



- In fact, for now this is still a research project. It becomes more serious and we already use it in production, so in the strict sense you can’t call it research, but it developed very quickly, the code changed constantly. Now everything will calm down a little, then it may make sense in composing a book. But writing a book is a big job.







Ruslan: I think one more question about Graal and Java, and then switch to LavaOne ...



- Do you try Graal yourself?



Ruslan: I don’t, I’m a marketer and I write rarely in Java. But Oleg, we have written technical blog posts and likes to dig in this. I suspect he has already tried.



Oleg: Two years ago, generally speaking.



- Yes, he is an old-timer!



Oleg: In general, the question is this. In Java 9, we received modules, in the next version we are waiting for value types. Do you think Graal will benefit from these Java innovations in the future? And what features are the most useful for you?



- Well, value types are not “in the next version” because of the new release cycle. Java 10 will be released in March, so the value types will not get there. But what gets into it is ... There is a JEP, I don’t remember the number [we checked: JEP 317 ] , adding Graal as an experimental JIT compiler. So we are moving in the right direction. And in Java 9, in fact, Graal is already present, but there it is not advertised as an “experimental JIT compiler,” it will happen in the top ten. This is great, there is progress, this is what I want to see.



And returning to value types, unfortunately, not included in the "top ten": they will be very important for Graal. The reason is that one of the drawbacks of Graal compared to C2 is that the graphs of the objects of the compiler itself grow very large, and when compiling, the traversal of the graph turns out to be the second one.In C ++, an object can be almost a structure, it is tightly packed, and a Java object is another matter. And if you have a graph, it should be a Java object, you cannot get anywhere. And a Java object leads to a significant overhead, with which a lot of things come immediately. It uses more memory to represent graphs. Using value types for this will save some memory. We are waiting for this. And as soon as the value types get into the JDK, I'm sure that Graal will use them immediately.



Oleg: As far as I remember, there is about 40 megabytes per instance of the Java process, right? Can be reduced to 10. In the report you said so.



- No, this is different. The 40 megabytes mentioned in my report are metadata. There are different things. In the case of Graal, more Metaspace is used. I think about 20-30 megabytes. He also uses the memory in the old generation, it is 40 megabytes. The static state is stored there, so it is in the old gen. This is stored all the time while the JVM is running.



. Java- compiler graph, , . , . . . , , . ( , ). . C2, , . C2, malloc. C2 , C2 . - . . , . C2 , Graal — Java-.



: , , , Java- escape analysis?



— , .



: , .



— Escape analysis - , . : , . , , .



: , , .



— , . . .



Ruslan: Oleg, do you have any more questions on Graal?



Oleg: What if I want to JIT-compile one method using Graal, and everything else with C2? Is it possible?



- Well, in principle, it is possible. But there is not something that you can just drive into the command line: "So, compile this method with Graal, the rest with C2". The broker who distributes the code in HotSpot does not support this. But what is possible ... We mentioned Truffle earlier, and with its help it is possible. The VM has a JVMCI, you can enable it by getting the ability to compile the method with the JVMCI compiler. Graal is a JVMCI compiler. Truffle uses C1 and C2, and it also parses the code. So when parsing a method, you can send it to Graal.



Oleg: Have you ever used such tricks?



— , , . Graal 100% , C2, - .



: , Graal , .



— !



: LavaOne. 2018-?



— .



: , : , Call for Papers , ?



— , Java-, . , , Java- — , Java Users Group. : CFP, , , , , Java-.



, CFP, « , ». , . . , . , — - , - . , .



: 12 , NightHacking ?



— , 12, , .



: , «, , ...», . ?



— , , .



: , Java User Group, , ?



— . , . ! * JUG*



: !



- Yes.In general, the main problem was in the audience. As I have already said, there was no difficulty in collecting speakers, and it was unclear with the audience without a community. We actively used Twitter, but I have a feeling that few IT people in Hawaii are tweeting. They use LinkedIn and Facebook. I realized this when I was planning on LavaOne and trying to reach people. I learned a lot of this. I lecture at the university, so I tried to attract students. They did not need to buy a ticket, for them the visit was free. In general, this was difficult, but I hope that next year it will be easier.



Ruslan: Did you try to get support from Oracle?



- I tried, and a number of speakers were from Oracle. They promised to help me next year. See if it helps.



All the spectators were local, nobody flew for the conference from somewhere, but I didn’t expect it. But there was a speaker who flew in from Japan, it was great. And Russia is not far from Japan, so come too next year!



Ruslan: Well, as “close”, we have a border with Japan, but Moscow and St. Petersburg are thousands of kilometers away.



- I heard that you can get there by train. Sit on it!



Ruslan: Yes, in a week, the Scala application will be compiled during this time! Here's something else I want to ask. Over the past year we have received a lot of feedback that it is worth increasing the number of women among the speakers. But for Java conferences it is difficult, because in principle there are few women among Java seniors. Your list of speakers seems to be male too, did you think about this problem?



— ! , . — , . JCP (Java community manager Oracle), IT, LavaOne . , , .



. , , . . JDuchess .



: ! , - Java- ? Graal?



— , ! , «Graal: JIT- JVM ». , . , , , . . , , «, !» , .



, . , , , . , , Java- , , … . , , . : . , . , , .



: ! , , .



— ! , , Joker . , .



: , , « , ».



— , , ? , … . , , . .



: , ?



— . , , 8:30 , 23 .



: , , -29. ! , . , !



— ! JBreak!

: , JBreak 2018 , . , , — !