⬆️ ⬇️

The best reports of the JPoint 2018: Java / JVM and its performance, Kotlin, Spring, Docker

We have already posted videos of JPoint 2018 reports on YouTube and, specifically for the Java hub on Habré, we have made a traditional selection of the best of them according to conference visitors.



As usual, the top “junior” reports, in the end - with the highest rating. Of course, this does not mean that one report is much worse than another: if you change the calculation method, places can easily change. In reality, we changed it, now the “soft quorum” version of the rating is used, taking into account the number of participants present at the report. This approach has its drawbacks (for example, more people come to the keyout than to the regular report, simply because the audience has no choice), but in general it gives a better picture of what happened.



Under the cut - and videos of the best reports, and links to their presentations, and short descriptions, and a link to the full playlist.





Full playlist



The full playlist with all the videos that will be discussed below is available here .



10. Linux container performance tools for JVM applications



Download slides





The container revolution has overwhelmed everyone - the world of Java has not been without it. Nevertheless, Java has always had some problems and peculiarities with it, which will be discussed in the report of Sasha Goldstein.



Sasha is a serial creator of performance hardcore. Last year at JPoint, he gave a great talk about using the Berkeley Packet Filter for JVM (I desperately recommend watching the recording on YouTube), and it was only a matter of time before he gets to the detailed containerization parsing. The world goes to the clouds and dockers, which in turn brings us many new problems. As you can see, most low-level debugging and profiling systems after their application to containers are overgrown with various features and jambs. Sasha skated around the main scenarios (CPU loading, IO responsiveness, access to shared databases, etc.) through the prism of using modern tools on the GNU / Linux platform, including BCC and perf.



This specificity is needed not by all. This is one of the reasons why the report was only in tenth place - it was attended by about two hundred people (compare with more than a thousand from Tolkachev and Borisov), and our rating calculation algorithm is very sensitive to the size of the audience.



For those who are somehow connected with containers, this report is a must-see. Save a lot of time on an independent search for all the details using the method of complete enumeration of the countless number of utilities and ways to use them.



I would also like to note that Sasha’s report out of all the top 10 is almost the densest in terms of the amount of practical information per unit of time. Almost every slide you can copy yourself some useful things. In order not to engage in a detailed retelling, I will show only the surface structure of the presentation:



Report outline
  • How are containers arranged?

    • Control groups: CPU, memory, block I / O;
    • Namespaces: pid namespace, mount namespace, network namespace;
  • The difference between the problems:

    • On the host;
    • In the container;
    • Examples of problems with solutions:

      • Connect to JVM?
      • Performance data JVM?
    • Demo of JVM tools on the host;
  • Resource information tools:

    • sidecar for monitoring;
    • docker stats;
    • systemd-cgtop;
    • htop + cgroup ID;
    • nsenter or docker exec;
    • Container resource demo;
  • CPU container profiling:

    • perf on the host: -G, symbol mapping, PID mapping, sharing perf map (perf-map-agent);
    • Demo with flamegraphs; oos;
    • perf in a container in unprivileged mode;
    • honest-profiler, async-profiler, perf vs async-profiler;
    • Demo async-profiler;
    • BCC is a profiling tool (it solves a number of problems on its own, but it only works on Linux 4.9+);
    • Throttling;
  • Other ways to monitor:

    • cAdvisor, SysDig, New Relic, DataDog ...


9. Once a year, gardens bloom: debriefing exactly-once semantics by Apache Kafka



Download slides





Victor Gamov has a completely unfair advantage over the other speakers. This is the man who released nearly two hundred issues of the podcast Debriefing, the author of a heap of reports, texts, posts and even the book “Enterprise Web Development” of O'Reilly. The very presence of Gamow makes anything better. I am sure he could, with the same intonation, tell not about Kafka, but about the growth of geranium on the window sill, and it would still be fascinating.



Here we have a special case: initially holivornaya topic about the semantics of "exactly once". Kafka was rushed to use everything on an extraordinary scale, which required reworking the traditional semantics of message delivery. Kafka didn’t put a finger in his mouth - they pereobulsya on the go under the correct protocol and message format and did everything you need. Throughout the report, Victor tells how it works inside and what it affects.



Two news, good and bad. Good: in Kafka, everything is configured. The bad: in Kafka everything is set up. To live with this, you need to understand how it works and where to climb with your dirty hands.



You could expect some kind of smoothie from Viktor, but then you got a desperate hardcore, and something like this:



Report outline
  • Introduction to Kafka:

    • Topics
    • Partitions
    • Logs and offsets
    • Replicas and Leaders
    • Customers: Producers and Consumers
  • Producer

    • How it works on examples
    • Producer protocol
  • Consumer

    • How it works on examples
    • Consumer Protocol
  • Processing model

    • Read, count, write
    • Types of failures (everything broke, zombies)
  • Processing semantics

    • At-least-once, at-most-once, exactly-once
  • The semantics "exactly once"

    • With examples
    • Weakness

      • unsafe replays
      • non-atomic record with offset
      • zombie
  • We fix the first weakness: idempotent producer
  • We fix the second weakness:

    • Chandy Lampport Snapshots

      • Principle
      • How to live malfunctioning?

        • Hard: crash during processing
        • Soft: fail before or during snapshot removal
    • Transactions in Kafka

      • Two types of markers (COMMIT, ABORT)
      • Atomic writing to multiple partitions (Including! _Consumer_offsets)
  • Fix third weakness: zombie fencing
  • Isolation of reading at the bureau
  • end-to-end eos

    • Kafka Connect Source
    • Kafka streams
    • Kafka Connect Sink


8. Fighting Russian Hackers with Kafka Streams and Firehose API



Download slides





Again Kafka! Again Gamov! Yes, and with Baruch Sadogursky ( jbaruch ) now. Nevertheless, this report is not a smoothie, but a very concrete practical thing about how on the Bintray platform (of which Baruch is the god) using Apache Kafka (for which Victor is responsible) and Firehose analyze behavioral patterns, process large amounts of data.



If the previous report about Kafka's insides had 245 slides, then this one is only 17. That's because you need to watch it! This is primarily a lively dialogue between the leading and live demos. Do not linger, open vidosik and look.



7. Hardware Transactional Memory in Java



Download slides





Nikita Koval ( ndkoval ) is a researcher at JetBrains on the Kotlin team and a PhD student at IST Austria (at the time of the report he was at Devexperts). His report contrasts strongly with the “Russian hackers”, because it’s not a light entertaining read, but a story about the intricate insides of a VM. If you look at the slides , you can find 150 sheets, most of which are code.



If you were on JBreak in the beginning of 2018, you could find Nikita’s story about completely different things - writing a fast multi-threaded hash table using the power of modern multi-core architectures and special algorithms. Agree, Nikita has a style . I immediately recall Shipilev.



This time we will talk about transactional memory, which gradually appears in modern processors, but which is still not clear how to use an ordinary person from the JVM world. Outside the JVM, of course, everything is easier. But imagine how you tell the usual Spring web developer: "Yes, you just edit the vmstructs, load your intrinsics, rebuild OpenJDK and you're done!" And he said: "Well, of course, I do it every day!". Nikita just very clearly talks about how to use it, what optimization already exists in OpenJDK and how to perform transactions directly from Java code.



Report outline
  • Introduction: Why we need multithreading
  • Approaches to the construction of algorithms

    • Rough lock
    • Slim lock
    • Non-blocking sync
    • All three types are illustrated on an example task about a toy bank.
  • Multithreading is hard. What to do?

    • Transactions in a perfect world. Just write atomic!
    • Where to get atomic:

      • Software Transactional Memory (STM). Scala STM, NOrec, Corutin.
      • Hardware Transactional Memory (HTM). Haswell, Power 8.
      • Hybrid Transactional Memory
    • Intel RTM examples

      • XBEGIN, XEND, XABORT, XTEST
      • Intel RTM + Java

        • Lock elision
        • java.util.concurrent.RTMSupport
      • Intrinsics: interpreter, C1, C2
      • Coarse-grained / Lock-free + RTMSupport on charts


6. We profile up to microseconds and processor instructions



Download slides





Sergey Melnikov ( RainM ) from Raiffeisenbank brought us the second profiling report. Interestingly, before taking up the low-latency code in Java, he worked at Intel as a performance engineer of compilers for C / C ++ / FORTRAN languages. This report also has a perf! :-) There is also about the hardware features of the processors and Intel Processor Trace technology, which allows you to take the next step in the accuracy of profiling and to reconstruct the execution of the program section. There are quite a few such reports (for example, you can find the Andi Kleen report at the Tracing Summit 2015), they usually leave a lot of questions and do not shine with practicality in relation to Java. Here we not only have a person who has visited both worlds (both Intel and Java in the bank), but also who is able to clearly explain complex topics.



Report outline
  • What is it and why is it necessary

    • Domain - low latency of the application
    • Moscow Exchange Example
  • Choose a profiler

    • How to profile? Sampling and Instrumenting Profilers
    • async-profiler
  • We learn the profiler to collect a detailed profile

    • How to run perf
    • How to look in a profile, in a call stack
    • perf-map-agent, sysctl, dmesg ...
  • Use PMU / PEBS events in perf
  • Intel Processor Trace - what it is and how to profile Java

    • Requirements: packages, hardware, operating system
    • How to run on Skylake-X (Xeon and i9)


5. VMStructs: Why does an application need to know about JVM internals?



Download slides





Andrey ( apangin ) is a person who collects the most profound and powerful reports over and over again. On the past Joker, he had a little less than a thousand people - a record in terms of audience size on a regular non-keyout report. In this he is helped by a decade of experience in virtual machines and the ability to explain technical hardcore so that it can be repeated in practice.



Many people do not quite understand why digging into a virtual machine if you have a regular application on a computer, with a database and everything is as it should be. In this report, there is a good argument at the level of "how to understand which query will pull a lot of data?" If you try to instrument the code with the help of JMX, then something strange is happening with the performance. First, it is possible to understand why this is happening, and secondly, what can be done with it. In my opinion, this report is valuable not so much as a set of tools (although there is a little about helfy), as a demonstration of the correct mindsets developer OpenJDK and how to behave in difficult situations. It says the existence and explanation of the specific things like TLAB, Code Cache, Constant Pool, etc.



4. Korutin in Kotlin



Download slides





Only the lazy one does not know about Kotlin now, and you, the reader, once finished reading up to the fourth paragraph - obviously not one of the lazy. Roman ( elizarov ) - in the past, the developer of high-performance trading software, and now - Timlid in Kotlin libraries. We have already done an interview with Roma about Korutin , and it may be worth rereading it before or after viewing the report. Korutins are a very old concept, from the time of Simula, but not all mainstream languages ​​support them, they will not appear in Java soon. But in Kotlin they are already in a stable version.



This report answers all topical questions about Korutin, that is, all topical issues of our time in general :-)



Report outline
  • General picture of the development of languages
  • Asynchronous programming with callbacks
  • Futures / Promises / Rx
  • Korutin in Kotlin

    • regular loops, exception handling, higher-order functions
    • custom higher-order functions
    • everything looks like in blocking code!
  • How it works?

    • suspending functions
  • Integration

    • Retrofit async
    • Callbacks and Continuations (call / cc from Scheme!)
    • contlinx-coroutines-core

      • jdk, guava, nio, reactor, rx1, rx2
  • How to run a korutiny?
  • async / await

    • Why is there no await keyword in Kotlin?
    • Concurrency is hard. You need to do this explicitly.
    • Kotlin approach to async
  • The concept of korutin: very light threads
  • Interop with Java
  • Synchronous corutines - generate / yield

    • Fibonacci example
  • Communicating Sequential Processes (CSP)
  • Library vs Language

    • The core language should be small!
    • Keywords: async / await, generate / yield
    • Modifiers: suspend
    • kotlinx-coroutines: launch, async, runBlocking, future, delay, Job, Deferred, ...


3. On the shoulders of giants: the languages ​​in which Kotlin studied



Download slides





A report about Kotlin from one of the creators of the language - what else is needed for happiness? The essence and structure of the presentation is not at all the same as in the previous report of Elizarov. Roman talked about specific things - what, how and why in the design of korutin, and how to use it, and I need it to improve my programming skills on Kotlin. Here, Andrew ( abreslav ) tells things that improve knowledge of life in general and give an understanding of their place in the world. If you have seen all these languages ​​with your own eyes - Java, C #, Scala, Groovy, Python, Gosu - this is even more interesting, because there is a reason for discussion (conference visitors probably could have taken the opportunity and really discussed their understanding of things live in the discussion area ). This is “seven languages ​​in seven weeks,” but only in one hour.



By the way, we recently did a separate interview with Andrey, but it was not only about Kotlin, but about a lot of different things.



2. The Adventures of Senor Holmes and Junior Watson in the world of software development



Download slides





Making reports in which more than one speaker participates is unusually difficult. Often it looks like this: half of the time one of them stands on the stage and is bored, and it looks very sad. What can be said about the joint performance of Baruch Sadogursky and Evgeny Borisov ( EvgenyBorisov ) - on the contrary, it was made magnificently, it is a work of art. The stars came together in the right order, and two top speakers appeared on the scene with a wealth of experience and practice in conducting paired reports to discuss an interesting topic for both of them. Why I emphasize it this way - usually, viewers have no idea what the creation of such a presentation is worth, and take everything for granted.



But the result justified the investment. Judge for yourself, this is an evening keyout, the most recent report of the conference, during which people are already exhausted by two days of tin, they want to sleep, someone needs to fly home right now and so on. And yet, in the hall there are more than 600 people for sure.



This is not a reference report, but a report-show, you need to watch it. In it, Holmes and Watson will uncover several riddles that you have encountered, encountered or will encounter in everyday development. There will be no gutters of garbage collectors and bytecode, but there will be tools, libraries and frameworks that perplex ordinary developers in their daily routines, leading to downtime, dead footing and prolonged depressions. In fact, in this report, Sherlock and Watson save your forehead from the facepalm and the rake, which someone has already attacked.



Therefore, the summary content here again will not be - just start the video and watch.



1. Boot yourself, Spring is coming



Download slides







Baruch + Gamov, Baruch + Borisov, who is missing among the authors of the popular pair reports? That's right, Borisov + Tolkachev ( tolkkv ). Another cool report that, not being a keyout, gathered a record number of participants - more than a thousand. The material turned out to be so much, and it is so interesting that they allocated as many as two slots in the conference program - and you’ll have to watch two whole videos on YouTube, respectively.



Many years ago, Java programmers used “new” to create services. They did a lot of manual actions and mixed configuration with business logic. They even used copy-paste techniques. It was written many lines of poor code, which at times even worked.



Then Spring appeared. With him, much has changed ... We got a lot of "magic" from the magic cylinder of Spring, and our code has become more clean, simple and maintained.



And there was Spring Boot. On the one hand, it solves thousands of previously existing problems: version conflicts, configuration tasks, work with infrastructure bins, the problem of setting up the environment and, of course, launching or deploying the application, including building jar / war archives. On the other hand, Spring Boot added even more magic to our magic cylinder. As a result, there are two scenarios:





This report reveals the basic secrets of Spring Boot magic. You understand the basic principles and conventions by which typical Spring Boot applications work. The task of Cyril and Eugene is that all the magic from the cylinder turns into a much more transparent process for you and you can not only have fun when everything works, but also understand the essence of the problems that arise and can solve them without using ambulance and other services available urgent calls.



Conclusion



It took me a few days to watch all the videos from the list. It was not easy, but it is clear that it was worth it. In reality, there were much more reports at the conference, and watching the top ten of them is a good start for a much larger way. If you also watch these reports - do not be lazy to write your reviews in the comments on Habré!



Meanwhile, it is already possible to purchase tickets for the next JPoint. It will be held April 5-6, 2019 at the WTC Congress Center. Until January 1, there is still an opportunity to purchase tickets at low prices !


')

Source: https://habr.com/ru/post/433858/



All Articles