Java conference with English roots. XXXL size overview

Friends, kent2171 , my colleague from CleverDATA, and I visited one of the largest conferences in Europe - Jax London 2017, dedicated to the Java ecosystem and everything that surrounds it in the modern world - microservice architecture, Continuous Delivery, and DevOps culture and practices.

The amount of information received in four days was so huge that the review had to be divided into two parts. Today we bring to your attention the first part, in which we will tell about Chaos Engineering the approach to building sustainable distributed systems, as well as how the containerization of java applications affects the development process and what advantages besides syntax give lambda expressions. These are the highlights. The rest is under the cut.

In the same place at the same hour

JAX always, according to my observations, collected a large number of participants. Success was reinforced by world-famous speakers (for example, Henny Kevlin, Daniel Bryant, Stephen Coleburn), with whom you could chat live by asking questions from the audience, as well as during breaks and during coffee breaks.
')
2017 was no exception, the conference lasted four days and gathered an unprecedented number of visitors from more than 40 countries of the world. Workshops were organized on the first and last day, and the second and third days were devoted to reports.

This year, however, as always, the conference was held in central London at the Business Design Center.


And here is another, historical facade of this building, known to Londoners as the Royal Agricultural Hall.



There are three ways to get to the conference from Heathrow Airport. The cheapest, but at the same time quite convenient and comfortable - London Underground, from all terminals there is a direct line to the station Kings Cross. The route takes about an hour on average. We were curious to try this type of transport. Express and taxis are the second and third alternatives, but they are much more expensive.

The first day. Workshop Russ Miles

Production Microservices. Chaos from Order

“Production hates you. Machines, network, each user of your service hate you. This is our reality, and this is what turns production into a bloody battlefield, ”the Ras Miles workshop about Chaos Engineering began on how to increase application stability and identify potential problems before they happen.

Chaos Engineering is a practice that empirically identifies and eventually eliminates the weaknesses of the systems and applications being developed, without allowing changes to further regress into applications.

Chaos Engineering. Where to start?

Architecture and design, ready for Chaos Engineering

Any complex distributed system consists of many components, and even when each individual component functions properly, the interaction between components often causes problems that cause the system as a whole to behave unpredictably. When designing a system, it is necessary to take into account the SLA of all the components that make up the system; you should strive to minimize the number of external dependent systems that directly affect the availability of the system as a whole.

Hypotheses of steady state system

The concept of a steady state system is determined from the system requirements:

• SLI (Service level indicators) - the quantitative characteristic (metric) of one of the aspects of the level of service provided, for example, throughput, latency, error rate
  
• SLO (Service level objectives) - the range of acceptable values ​​for the level of service provided, bounded on two sides by SLI.
  

Having an idea of ​​the steady state of the system, we can talk about possible deviations and about the actions that need to be taken in this or that case.

The essence of Chaos Engineering - having the values ​​of the metrics of the system's work as the results of Chaos experiments, make sure that the system is not working, rather than trying to check how it works.

Chaos Experiment Scripting

Having an idea of ​​the steady state of the system, the next step is to determine all possible stress factors that can take the system out of this state. For example, problems associated with limited computing resources (network, CPU, memory, disk space, etc.), as well as problems that occur at the system level (inaccessibility of one of the dependent services, database, etc.).

Run Chaos experiments

It is recommended to begin with “debugging” Chaos scripts, collecting the results of the system behavior depending on the environment in which the system is located.

Automation of Chaos Experiments

Finally, we are all set to automate the process: a set of scenarios consisting of stress factors and the expected behavior of the system in the form of an SLO. Chaos Toolkit , a free tool that is great for conducting Chaos experiments with most platforms and applications, was considered as a tool to automate stress scenarios.

Second day. Lambda, Bugs and Continuous Delivery

On this day, at the official opening of the conference, Sebastian Meyen made a welcoming speech, and the first report was devoted to the responsibility of developers for their code and for users suffering from various bugs.

Henney Kevlin. The Error of Our Ways

Henney cites the consequences of seemingly insignificant problems, leading to failures of a different nature - from insignificant oddities and information leaks to very substantial material losses, and sometimes to disasters. Of course, most failures and catastrophes can be avoided just by following simple and well-known practices.



And what if you do not follow?

Knight Capital Group, 2012

As part of the system upgrade, it was intended to de-add (that is, essentially eliminate) the old functionality that has not been used for more than eight years. Due to the human factor, one of the eight SMARS servers was not updated and continued to work. For 45 minutes, the company lost $ 460 million.

Any process in which the human factor is present carries risks due to the possibility of incorrect explanation, perception or performance of the required actions.

Cluster (spacecraft), 1996

A crash occurred while launching the Cluster spacecraft due to a software error. The essence of the error was inadequate protection against integer overflow. The unit test for boundary values ​​and overflows cost the company $ 370 million.

Analysis of failures in distributed systems (Cassandra, HBase, HDFS, MapReduce, Redis), conducted by the USENIX Association (Advanced Computing Technical Association), showed the following interesting results.

• Almost all catastrophic failures occur as a result of incorrect handling of non-critical application errors that are clearly manifested.
  
• The cause of 58% of disasters can be detected only by covering the application error handling code with tests. For example, 35% of errors from this category can be easily identified using static code analysis or review code.
  
• 77% of problems can be easily reproduced using unit tests.
  

Simon Ritter (Azul Systems). Lambdas: It's Java Jim

Although Java 9 has already been released at the time of this report, it is sometimes useful to look back and look at lambda expressions — an incredibly useful and powerful tool that appeared in the previous release of the Java platform.

With the help of lambda expressions you can solve any problem, although it is not always recommended (each tool must solve the problem for which it was created).



Simon Ritter examined the advantages of using lambda expressions over anonymous classes.

To answer this question, it is necessary to consider what the code is compiled in in both cases:

• lambda expressions:

myList.forEach(w -> System.out.println(w)); 

• and anonymous class:

 myList.forEach(new Consumer<String>() {   @Override   public void accept(String w) {       System.out.println(w);   } }); 

From the name “anonymous class” it follows that we are still dealing with a class. Therefore, the compiler will generate a class.

Disadvantages of this approach:

• a large number of types are generated than the application actually needs, which leads to type pollution;
  
• an anonymous class is still a class, respectively, in runtime it needs to be loaded and initialized;
  
• instantiating a class leads to a greater load on memory and on the garbage collector;
  
• a non-static anonymous class takes up more space due to the storage of a reference to an instance of the class to which it refers.
  

Lambda expressions could potentially be syntactic sugar and compiled in a similar way as anonymous classes are compiled, but with the advent of the invokedynamic instruction in jdk7 and the beginning of its use by the compiler in jdk-8, things are very different.

• Lambda expression is converted to a method:
  
  - non-exciting lambda expressions and lambda expressions that capture static variables are converted to the static method of the class in which they are used.
  - lambda, exciting link to a copy of the class in which they are declared, is converted into a method of this class.
  
• Calling lambda expressions is compiled into an invokedynamic statement, with which the LambdaMetafactor meta-factory associates CallSite — factories of “functional objects” that implement one or more interfaces by delegating to the specified MethodHandle .
  

It is important to note the advantages of using lambda.

• Creating instances of functional objects is lazy: if lambda is not used, then there is no overhead.
  
• Converting lambda expressions into methods is more efficient from the point of view of memory consumption than anonymous classes, especially in the case of capturing a reference to an instance of a class.
  
• Lambda context capture effectively works in a multi-threaded environment
  

Finally, as an example of how far you can go using only one functional interface and lambda expressions, Simon Ritter demonstrated the calculation of the 2 + 2 expression. On the video of the report you can follow the entire chain of an entertaining process.

Continuous Delivery with Containers: The Good, the Bad, and the Ugly

Daniel Bryant, a consultant, CTO at SpectoLabs, plunged into the field of operations a few years ago. In this report, he shares his experience about Continuous delivery of java applications in containers.

Continuous delivery is a set of practices and disciplines whose goal is the continuous release of new software versions in short periods of time. The basis of this process is the build-pipeline, the task of which is to prove that the changes in the code meet all the requirements necessary for the release of the version. The main value of the build-pipeline is instant feedback in the form of metrics, reflecting the result of the implementation of each phase of the pipeline.

Bryant spoke about how the build pipeline is subject to changes during containerization of Java applications, and lists the best practices applicable to each phase separately.

Application Development

Assembly and packaging of applications in containers and the execution of tests occurs locally on the developer's machine.

Highlights:

• Contents of Docker files (OS selection, configuration, opening of ports, JDK vs JRE, whether Python Virtual env is needed).
  
• Details of the work of applications / middleware in linux containers. For this, it is ideal to involve the operations team to make the best decisions and practices. Their knowledge base can be very useful at this moment.
  
• For local tests, you need to use production container images to eliminate the effect of the test container configuration on the operation of the application.
  

Continuous integration (execution of unit and integration tests)

Publishing Java artifacts (JAR, WAR, etc.) replaces the publication of container images in repositories such as, for example, Docker-hub.

Highlights:

• use of ready-made tools for assembling and publishing images, for example, Jenkins plugin ;
  
• you should avoid using the latest tag as a version of the image, be sure to use image versioning;
  
• using metadata to describe a container image:
  
  - application metadata (version / git sha);
  - assembly metadata (build date, image name, vendor, etc.);
  - QA metadata;
  there is a convention for the metadata schema;
  
• the use of labels (label) as metadata leads to the creation of a new image, which is not always permissible, as a workaround it is proposed to store metadata separately from the image . Jfrog Artifactory + Docker, as well as NexussOSS, support separate metadata storage schemes.
  

Component Testing (Acceptance and Load Tests)

Highlights:

• using tools to describe and launch multicarrier applications (for example, Docker Compose, Ephemeral Kubernetes Clusters );
  
• use of ready-made tools to run component tests such as Pipeline as Code with Jenkins .
  

On this I take a pause. My colleague kent2171 will tell you about the reports of the last two days of the London conference. But it will be next week. Subscribe to the blog not to miss.



UPD: the second part of the review is published here .