Notes on reactive programming. Technological landscape
Hello dear readers! Today we bring to your attention the promised article with an overview of the possibilities of reactive programming. Pleasant and fruitful reading.
Reactive programming (again) is of interest, there is a lot of hype associated with it, which is not so easy for an uninitiated person or a simple Java enterprise developer to understand - such as the author of this article. Perhaps this article will help you understand what is happening in this area. I tried to adhere to the maximum specifics, and there is not a word about “denotational semantics”. If you are interested in a more academic text and a lot of Haskell code - the Internet will help you, but you probably should not read this article.
Reactive programming is often mixed with competitive and high-performance programming, and so much so that these concepts are difficult to separate, whereas in principle they are completely different. This inevitably leads to confusion. In addition, reactive programming is often identified or comes close to functional reactive programming (FER), here we will use these terms as synonyms. Some people think that there is nothing new in reactivity, and that they somehow always use it (mostly JavaScript specialists). Others believe that reactive programming is a gift from Microsoft (the company recently drew serious attention to such programming by releasing a number of C # extensions). Recently, discussion of the topic has been stepped up in the Enterprise Java space (see for example the Reactive Streams initiative ), and, as always with bright new technologies, it is easy to make mistakes in this area, not fully understanding when and where the technology can and should be used.
Reactive programming is a micro-architectural style associated with intelligent routing and the consumption of events, which in sum allows you to influence behavior. It sounds a bit abstract, in addition, many other definitions of reactive programming can be found on the Web. Below we will try to more specifically describe what “being reactive” is, or why it might be important.
')
The origins of reactive programming arise in the 1970s or even earlier, so the idea is not new. However, it does find a serious response in some aspects of modern enterprise development. The resonance (not by chance) coincided with the flourishing of microservices and the ubiquitous spread of multi-core processors.
Here are a couple of helpful short definitions from other sources:
and…
(taken from Terminology Question on Stackoverflow )
FRP has a lot in common with high performance, competitiveness, asynchronous operations and non-blocking I / O. However, it does not hurt to proceed from the hypothesis that the FER has nothing to do with any of the above. Indeed, it is quite natural to delegate the handling of all these problems to the caller when working on the reactive model. But, with regard to the essence or efficiency of handling these problems, the result is completely dependent on the specific implementation (therefore, the implementation should be considered as critical as possible). It is also possible to implement a completely logical and useful FRP framework in a synchronous single-threaded manner, but this is unlikely to help you if you want to practice with any new tools and libraries.
Apparently, the newcomer is the most difficult to answer the question: "Why do I need reactive programming." Below are a few examples from the enterprise-environment, illustrating the general principles of using such a paradigm.
Calling external services . Currently, many database services are implemented in the REST style (that is, they work over HTTP). Therefore, their basic protocol is essentially blocking and synchronous. It may not be the most obvious area for exercises with an FER, but in fact this is a very fruitful ground, since the implementation of such services often involves calling other services, and then following services, depending on the results of the first calls. If, with such active I / O, you waited for the completion of one call, and only then sent the next request, then your poor client would simply get exhausted before you could generate a response. So, the challenges of external services, and especially the complex orchestration of dependencies between calls, are those things that are really worth optimizing. The FRP should provide the “composability” of the logic underlying these operations, so that the work of the developer of the calling service is simplified.
Consumers highly competitive messages . Processing messages, in particular, highly competitive ones, is a typical task in an enterprise environment. In the field of reactive frameworks like microbenchmarks, they boast about how many messages per second can be processed on the JVM. The results are truly staggering (performance is easily achieved in tens of millions of messages every second), but perhaps they are somewhat artificial - not so impressive when it turns out that control points were set in a simple for loop. However, you should not quickly write off such work; it is easy to see that when performance is critical, any contribution to it is as found. Reactive patterns are naturally combined with message processing (since the event is beautifully translated into a message), so this way to process even more messages faster deserves attention.
Tables Perhaps this is not quite an enterprise-example, but just one that everyone in the field of enterprise may well use. In addition, this example well captures the philosophy and complexity of the embodiment of the FER. If cell B depends on cell A, and cell C depends on both A and B, then how to propagate from A and ensure that C is updated before sending any change events to B? If you have to base yourself on a truly active framework, then our answer is “nothing, all you have to do is to declare dependencies,” and the whole essence of the true power of the tables is in it. In addition, this example emphasizes the difference between FRP and ordinary event-oriented programming - it turns “routing” into “intelligent routing”.
Abstraction (a) synchronous processing . This example is more abstract - one of those I was going to avoid. In addition, it largely overlaps with the more specific cases already mentioned, but I hope it will still be useful to discuss it. The main thesis is known (and) justified - if the developer is ready to do an additional level of abstraction, then you can simply not bother about whether the synchronous code has to be called or asynchronous. Since asynchronous programming is a heavy load on your delicate gray cells, some useful ideas can be drawn from reactive programming. Reactive programming is not the only approach to solving such problems, but some authors of the FER have so strongly reflected on this topic that they have turned out really effective tools for its solution. This netflix
If you have not lived in a cave since the 1970s, you have probably come across other concepts that are important for reactive programming. You noticed what problems people are trying to solve with the help of such concepts. Here are some of these problems - I prioritized the subjective priority.
The event machine in ruby . Event Machine is an abstraction of concurrent programming (usually associated with non-blocking I / O). Rubists suffered for a long time trying to turn the language, sharpened for single-threaded scripting, into a tool for writing server applications that a) will work 2) will work quickly and 3) will survive under load. In Ruby, streams have been around for quite some time, but they were not very actively used and had notoriety, as they did not always work normally. The alternative that has now spread everywhere — so much so that Ruby 1.9 was proposed to be included in the core of the language — called Fibers (yes, “fibers”). A fiber programming model resembles working with co-procedures (see below), where a separate native thread is used to process a large number of concurrent requests (often associated with I / O). This programming model itself is a bit abstract, and it is difficult to talk about it, so most programmers use a wrapper, which is most often taken as an event machine. An event machine does not necessarily use fibers (it abstracts the problems associated with them), but it is easy to find code examples from web applications in Ruby, where the event machine is used with fibers (see this article by Ilya Grigorik or an example with fibers from em-http-request ). This is done mainly for the sake of scalability, which is achieved by using the event machine in applications with active I / O, without the need for those ugly software models that are associated with many nested callbacks.
Actor model . The actor model, like object-oriented programming, is an ancient branch of computer science, whose origins date back to the 1970s. Actors abstract calculations (and not data and not behaviors) and, therefore, ensure competitiveness. Therefore, in practice, actors can serve as the basis for a competitive system. Actors exchange messages, so in a sense they are reactive, therefore actor and reactive programming models largely overlap. Often, the distinction is made at the implementation level (for example,
Deferred results (Futures) . In Java 1.5, a whole host of new libraries appeared, including the
Map-reduce and fork-join Parallel processing abstractions are useful, and you don’t need to go far for examples. Map-reduce and fork-join have recently developed in the Java world under the influence of the massively distributed parallel processing tools ( MapReduce and Hadoop ), as well as within the framework of the JDK itself in version 1.7 ( Fork-Join ). These abstractions are useful, but (like pending results) they are too small. An FER can also be used as an abstraction of simple parallel processing, but its capabilities are far from being limited to composability and declarative communication.
Soprocedury “ Soprocedura ” is the generalization of “subprocedure”. There is an entry point and exit point in it, as in the subprocedure, but after the exit, the coprocessor transfers control to another coprocedural (not necessarily the one that called it), and the acquired state is saved and stored until the next call. Co-procedures can be used as "building blocks" for higher-level entities, such as actors or streams. One of the goals of reactive programming is to provide such an abstraction for the communication of concurrently working agents, so co-procedures (if available) are very useful. There are various options for co-procedures, some of which are generally stricter than others, but more flexible than simplified co-procedures. Fibers (we discussed them above in connection with the event machine) - one of these varieties, the other variety is called “generators” (found in Scala and Python).
Java cannot be called “reactive language”, as it does not natively support co-procedures. There are other languages ​​for JVM (Scala and Clojure) that more naturally support reactive models, but in Java itself this feature was not available until version 9. However, Java is a real engine of enterprise development, and recently a lot of work has been done on creating reactive levels over the JDK. Here we will only briefly review some of them.
Reactive streams are a very low-level contract, expressed in the form of several Java interfaces (plus TCK), also applicable in other languages. Interfaces provide the basic simplest elements of Publisher and Subscriber with pronounced back pressure, thereby forming a common language for interoperable libraries. Reactive threads were implemented in the JDK as java.util.concurrent.Flow in version 9.
RxJava : Netflix used reactive patterns for a while at the corporate level, and released tools that were used under the free software license called Netflix / RxJava (then rebranding: “ReactiveX / RxJava”). There is a transition to Reactive Streams. RxJava is the “second generation” library according to David Karnok’s classification of Generations of Reactive .
Reactor is a Java framework from the Pivotal development team (the one that Spring created). It is built directly on Reactive Streams, so no transition is required. The Reactor IO project provides wrappers for low-level network execution environments like Netty and Aeron. Reactor is the fourth generation library according to David Karnok’s classification of Generations of Reactive .
The Spring Framework 5.0 (the first results were announced in June 2016) is equipped with reactive capabilities — for example, it has tools for creating HTTP servers and clients. Current Spring users on the web tier will find a lot of familiarity in the programming model, where annotations are used to decorate controller methods to handle HTTP requests. At the same time, dispatching of reactive requests and back pressure is mainly transmitted to the level of the framework. Spring is built on the basis of Reactor, but also provides an API that allows you to express its capabilities using various libraries (for example, Reactor or RxJava). You can work with Tomcat, Jetty, Netty (via Reactor IO) and Undertow for the server network stack.
Ratpack is a collection of libraries for creating high-performance HTTP services. It is built on Netty and implements Reactive Streams for interoperability (for example, you can use other implementations of Reactive Streams higher up the stack). Spring is supported as a native component and can be used to inject dependencies with some simple helper classes. There is also some autoconfiguration that allows Spring Boot users to embed Ratpack into a Spring application, download the HTTP endpoint and listen to it, rather than using one of the built-in servers that ship directly with Spring Boot.
Akka is a toolkit for building applications using the Actor pattern on Scala or Java, with interprocess communication performed via Akka Streams, and Reactive Streams contracts are simply embedded. Akka is the “third generation” library according to the David Karnok classification of Generations of Reactive .
What caused the increased interest in reactivity in Enterprise Java? This is not just a technological fad - it would seem that people just pounced on a bag of new shiny toys. The incentive lies in the efficient use of resources, more precisely, in savings on servers and data centers. Reactive programming should help achieve more with smaller forces, namely, handle higher loads with fewer threads. This is where reactive programming and non-blocking asynchronous I / O intersect. If reactive programming is really good at solving a particular problem, then the effect is dramatic. However, otherwise the situation can only worsen.
In this article, we have considered, in the most general terms, reactive programming in the context of modern enterprise development. There are a number of reactive libraries and frameworks for the JVM, all of which are being actively developed. In many ways, they are similar, but, thanks to Reactive Streams, they interact well with each other.
Reactive programming (again) is of interest, there is a lot of hype associated with it, which is not so easy for an uninitiated person or a simple Java enterprise developer to understand - such as the author of this article. Perhaps this article will help you understand what is happening in this area. I tried to adhere to the maximum specifics, and there is not a word about “denotational semantics”. If you are interested in a more academic text and a lot of Haskell code - the Internet will help you, but you probably should not read this article.
Reactive programming is often mixed with competitive and high-performance programming, and so much so that these concepts are difficult to separate, whereas in principle they are completely different. This inevitably leads to confusion. In addition, reactive programming is often identified or comes close to functional reactive programming (FER), here we will use these terms as synonyms. Some people think that there is nothing new in reactivity, and that they somehow always use it (mostly JavaScript specialists). Others believe that reactive programming is a gift from Microsoft (the company recently drew serious attention to such programming by releasing a number of C # extensions). Recently, discussion of the topic has been stepped up in the Enterprise Java space (see for example the Reactive Streams initiative ), and, as always with bright new technologies, it is easy to make mistakes in this area, not fully understanding when and where the technology can and should be used.
What it is?
Reactive programming is a micro-architectural style associated with intelligent routing and the consumption of events, which in sum allows you to influence behavior. It sounds a bit abstract, in addition, many other definitions of reactive programming can be found on the Web. Below we will try to more specifically describe what “being reactive” is, or why it might be important.
')
The origins of reactive programming arise in the 1970s or even earlier, so the idea is not new. However, it does find a serious response in some aspects of modern enterprise development. The resonance (not by chance) coincided with the flourishing of microservices and the ubiquitous spread of multi-core processors.
Here are a couple of helpful short definitions from other sources:
The basic idea of ​​reactive programming is that some types of data represent a value “in time”. The results of calculations associated with such time-varying values ​​will also change with time.
and…
To make a first impression about this, it is convenient to compare your program with a table, and all your variables with cells. If the value in one of the cells of the table changes, then the values ​​in all other cells that refer to the first one also change. In the FER all that way. Now suppose that some cells change on their own (or rather), come from somewhere outside. In a GUI situation, a good example is moving the mouse cursor.
(taken from Terminology Question on Stackoverflow )
FRP has a lot in common with high performance, competitiveness, asynchronous operations and non-blocking I / O. However, it does not hurt to proceed from the hypothesis that the FER has nothing to do with any of the above. Indeed, it is quite natural to delegate the handling of all these problems to the caller when working on the reactive model. But, with regard to the essence or efficiency of handling these problems, the result is completely dependent on the specific implementation (therefore, the implementation should be considered as critical as possible). It is also possible to implement a completely logical and useful FRP framework in a synchronous single-threaded manner, but this is unlikely to help you if you want to practice with any new tools and libraries.
Reactive programming examples
Apparently, the newcomer is the most difficult to answer the question: "Why do I need reactive programming." Below are a few examples from the enterprise-environment, illustrating the general principles of using such a paradigm.
Calling external services . Currently, many database services are implemented in the REST style (that is, they work over HTTP). Therefore, their basic protocol is essentially blocking and synchronous. It may not be the most obvious area for exercises with an FER, but in fact this is a very fruitful ground, since the implementation of such services often involves calling other services, and then following services, depending on the results of the first calls. If, with such active I / O, you waited for the completion of one call, and only then sent the next request, then your poor client would simply get exhausted before you could generate a response. So, the challenges of external services, and especially the complex orchestration of dependencies between calls, are those things that are really worth optimizing. The FRP should provide the “composability” of the logic underlying these operations, so that the work of the developer of the calling service is simplified.
Consumers highly competitive messages . Processing messages, in particular, highly competitive ones, is a typical task in an enterprise environment. In the field of reactive frameworks like microbenchmarks, they boast about how many messages per second can be processed on the JVM. The results are truly staggering (performance is easily achieved in tens of millions of messages every second), but perhaps they are somewhat artificial - not so impressive when it turns out that control points were set in a simple for loop. However, you should not quickly write off such work; it is easy to see that when performance is critical, any contribution to it is as found. Reactive patterns are naturally combined with message processing (since the event is beautifully translated into a message), so this way to process even more messages faster deserves attention.
Tables Perhaps this is not quite an enterprise-example, but just one that everyone in the field of enterprise may well use. In addition, this example well captures the philosophy and complexity of the embodiment of the FER. If cell B depends on cell A, and cell C depends on both A and B, then how to propagate from A and ensure that C is updated before sending any change events to B? If you have to base yourself on a truly active framework, then our answer is “nothing, all you have to do is to declare dependencies,” and the whole essence of the true power of the tables is in it. In addition, this example emphasizes the difference between FRP and ordinary event-oriented programming - it turns “routing” into “intelligent routing”.
Abstraction (a) synchronous processing . This example is more abstract - one of those I was going to avoid. In addition, it largely overlaps with the more specific cases already mentioned, but I hope it will still be useful to discuss it. The main thesis is known (and) justified - if the developer is ready to do an additional level of abstraction, then you can simply not bother about whether the synchronous code has to be called or asynchronous. Since asynchronous programming is a heavy load on your delicate gray cells, some useful ideas can be drawn from reactive programming. Reactive programming is not the only approach to solving such problems, but some authors of the FER have so strongly reflected on this topic that they have turned out really effective tools for its solution. This netflix
Comparison
If you have not lived in a cave since the 1970s, you have probably come across other concepts that are important for reactive programming. You noticed what problems people are trying to solve with the help of such concepts. Here are some of these problems - I prioritized the subjective priority.
The event machine in ruby . Event Machine is an abstraction of concurrent programming (usually associated with non-blocking I / O). Rubists suffered for a long time trying to turn the language, sharpened for single-threaded scripting, into a tool for writing server applications that a) will work 2) will work quickly and 3) will survive under load. In Ruby, streams have been around for quite some time, but they were not very actively used and had notoriety, as they did not always work normally. The alternative that has now spread everywhere — so much so that Ruby 1.9 was proposed to be included in the core of the language — called Fibers (yes, “fibers”). A fiber programming model resembles working with co-procedures (see below), where a separate native thread is used to process a large number of concurrent requests (often associated with I / O). This programming model itself is a bit abstract, and it is difficult to talk about it, so most programmers use a wrapper, which is most often taken as an event machine. An event machine does not necessarily use fibers (it abstracts the problems associated with them), but it is easy to find code examples from web applications in Ruby, where the event machine is used with fibers (see this article by Ilya Grigorik or an example with fibers from em-http-request ). This is done mainly for the sake of scalability, which is achieved by using the event machine in applications with active I / O, without the need for those ugly software models that are associated with many nested callbacks.
Actor model . The actor model, like object-oriented programming, is an ancient branch of computer science, whose origins date back to the 1970s. Actors abstract calculations (and not data and not behaviors) and, therefore, ensure competitiveness. Therefore, in practice, actors can serve as the basis for a competitive system. Actors exchange messages, so in a sense they are reactive, therefore actor and reactive programming models largely overlap. Often, the distinction is made at the implementation level (for example,
Actors in Akka can be distributed between processes, which is a distinctive feature of this framework).Deferred results (Futures) . In Java 1.5, a whole host of new libraries appeared, including the
"java.util.concurrent" from Doug Lea. It was then that the concept of deferred result, encapsulated in the form of Future . This is a good example of a simple asynchronous pattern abstraction that does not force us to go for an asynchronous implementation or use any particular model of asynchronous processing. As well shown in the article Netflix Tech Blog: Functional Reactive in the Netflix API with RxJava , Futures are great when you need to competitively handle a set of similar tasks, but as soon as you are going to create dependencies between them, or want them to be executed with some conditions, you immediately fall into the “hell of callbacks”. Reactive programming is the antidote to this disaster.Map-reduce and fork-join Parallel processing abstractions are useful, and you don’t need to go far for examples. Map-reduce and fork-join have recently developed in the Java world under the influence of the massively distributed parallel processing tools ( MapReduce and Hadoop ), as well as within the framework of the JDK itself in version 1.7 ( Fork-Join ). These abstractions are useful, but (like pending results) they are too small. An FER can also be used as an abstraction of simple parallel processing, but its capabilities are far from being limited to composability and declarative communication.
Soprocedury “ Soprocedura ” is the generalization of “subprocedure”. There is an entry point and exit point in it, as in the subprocedure, but after the exit, the coprocessor transfers control to another coprocedural (not necessarily the one that called it), and the acquired state is saved and stored until the next call. Co-procedures can be used as "building blocks" for higher-level entities, such as actors or streams. One of the goals of reactive programming is to provide such an abstraction for the communication of concurrently working agents, so co-procedures (if available) are very useful. There are various options for co-procedures, some of which are generally stricter than others, but more flexible than simplified co-procedures. Fibers (we discussed them above in connection with the event machine) - one of these varieties, the other variety is called “generators” (found in Scala and Python).
Reactive Java programming
Java cannot be called “reactive language”, as it does not natively support co-procedures. There are other languages ​​for JVM (Scala and Clojure) that more naturally support reactive models, but in Java itself this feature was not available until version 9. However, Java is a real engine of enterprise development, and recently a lot of work has been done on creating reactive levels over the JDK. Here we will only briefly review some of them.
Reactive streams are a very low-level contract, expressed in the form of several Java interfaces (plus TCK), also applicable in other languages. Interfaces provide the basic simplest elements of Publisher and Subscriber with pronounced back pressure, thereby forming a common language for interoperable libraries. Reactive threads were implemented in the JDK as java.util.concurrent.Flow in version 9.
RxJava : Netflix used reactive patterns for a while at the corporate level, and released tools that were used under the free software license called Netflix / RxJava (then rebranding: “ReactiveX / RxJava”). There is a transition to Reactive Streams. RxJava is the “second generation” library according to David Karnok’s classification of Generations of Reactive .
Reactor is a Java framework from the Pivotal development team (the one that Spring created). It is built directly on Reactive Streams, so no transition is required. The Reactor IO project provides wrappers for low-level network execution environments like Netty and Aeron. Reactor is the fourth generation library according to David Karnok’s classification of Generations of Reactive .
The Spring Framework 5.0 (the first results were announced in June 2016) is equipped with reactive capabilities — for example, it has tools for creating HTTP servers and clients. Current Spring users on the web tier will find a lot of familiarity in the programming model, where annotations are used to decorate controller methods to handle HTTP requests. At the same time, dispatching of reactive requests and back pressure is mainly transmitted to the level of the framework. Spring is built on the basis of Reactor, but also provides an API that allows you to express its capabilities using various libraries (for example, Reactor or RxJava). You can work with Tomcat, Jetty, Netty (via Reactor IO) and Undertow for the server network stack.
Ratpack is a collection of libraries for creating high-performance HTTP services. It is built on Netty and implements Reactive Streams for interoperability (for example, you can use other implementations of Reactive Streams higher up the stack). Spring is supported as a native component and can be used to inject dependencies with some simple helper classes. There is also some autoconfiguration that allows Spring Boot users to embed Ratpack into a Spring application, download the HTTP endpoint and listen to it, rather than using one of the built-in servers that ship directly with Spring Boot.
Akka is a toolkit for building applications using the Actor pattern on Scala or Java, with interprocess communication performed via Akka Streams, and Reactive Streams contracts are simply embedded. Akka is the “third generation” library according to the David Karnok classification of Generations of Reactive .
Why now?
What caused the increased interest in reactivity in Enterprise Java? This is not just a technological fad - it would seem that people just pounced on a bag of new shiny toys. The incentive lies in the efficient use of resources, more precisely, in savings on servers and data centers. Reactive programming should help achieve more with smaller forces, namely, handle higher loads with fewer threads. This is where reactive programming and non-blocking asynchronous I / O intersect. If reactive programming is really good at solving a particular problem, then the effect is dramatic. However, otherwise the situation can only worsen.
Conclusion
In this article, we have considered, in the most general terms, reactive programming in the context of modern enterprise development. There are a number of reactive libraries and frameworks for the JVM, all of which are being actively developed. In many ways, they are similar, but, thanks to Reactive Streams, they interact well with each other.
Source: https://habr.com/ru/post/307658/
All Articles