JCoro - asynchronous on coroutines in Java
I was inspired to research in this area by the article Asynchrony: back to the future . In it, the author describes the idea of how, using coroutines , you can simplify asynchronous code so that it looks just like regular synchronous, but retains the buns that are given by the use of asynchronous operations. In short, the essence of the approach is as follows: if we have a mechanism to save and restore the execution context (coroutine support), then the code on callback chains
we can rewrite it like this:
Here, readSocket () and writeFile () are coroutines in which asynchronous operations are invoked as follows:
')
The yield () and resume () methods save and restore the execution context, with all frames and local variables. The following happens: when we call readSocket (), we plan an asynchronous operation by calling startReadSocket () and execute yield (). Yield () saves the execution context and the thread ends (returns to the pool). When the asynchronous operation is completed, we will call resume () before exiting the callback, and thereby resume the execution of the code. The control will again get the main function that will call writeFile (). writeFile () is similar, and everything will be repeated.
By doing this transformation once for all used asynchronous operations and placing the functions in the library, we get a tool that allows us to write asynchronous code as if it were normal synchronous code. We get the opportunity to combine the advantages of synchronous code (readability, convenient error handling) and asynchronous (performance). Paying for this convenience is the need to somehow save and restore the execution context. In the article, the author describes the implementation in C ++, but I wanted to have something like this in Java. This will be discussed.
First of all, it was necessary to find a coroutine implementation for the JVM. Among several options, the javaflow library turned out to be the most suitable. It would be quite suitable for the experiment, but, unfortunately, the project has long been abandoned. Having poked the wand (decompiler) into the code generated by it, I found out that there are several serious problems in javaflow:
Despite all this, javaflow helped to figure out how to implement state save and restore. Then there were 2 options: try to support javaflow or write your own implementation. For obvious reasons (fatal flaw), the second method was chosen.
Coroutines added to a language where they were not expand it. To write applications that fully take advantage of the proposed approach, and not to curse at the same time, you need to make them comfortable. When reading the code, we must immediately see that this function is a coroutine and performs an asynchronous operation, and therefore it must be run within a context that supports saving and restoring the stack. In C #, the keywords are async and await. In Java, unfortunately, adding your keywords does not seem real, but you can use annotations! All this looks, of course, cumbersome, but what to do. Maybe something else will come up. For now:
The presence of the @Async annotation tells jcoro to instruct the bytecode of this method, making it a coroutine. Recovery point signatures are specified by @Await annotations. All calls inside the coroutine whose signatures are in the list of @ Await annotations become restore points. A coroutine in jcoro is a method marked with the @Async annotation and having at least one restore point. If there is no recovery point in the method, it will not be instrumented. A recovery point is a Coro.yield () call or any (coroutine) call that can ultimately lead to a Coro.yield () call.
Suppose we need to write a server application that, in response to a request, accesses the database, then does something with the data, then applies it to the template, and returns a piece of markup. Classic web server application. Almost at all stages we can use asynchronous operations. Establishing a connection, reading data from a socket when a request is received, all network operations with a database, reading a file when loading a template, sending the result to a socket. In such a scenario, the CPU should be occupied only with the planning of asynchronous operations, the logic of data preprocessing and templating. The rest of the time, the processor can rest. Let's try to figure out how this could be organized in code. Sketches the server:
Some try-catch blocks necessary for correct compilation are omitted in the code, this is done to make it easier to read the code.
Inside the handle, you can add any logic. For example, the definition of "controller" and call it through reflection, the introduction of dependencies. But you need to be careful with calls to the code containing the recovery points, through reflection or non-instrumented libraries. About this below.
From the point of view of recycling, it works as follows. There is a pool of worker threads, and there is a system pool of threads that the JVM reserves to perform asynchronous callbacks. When an asynchronous operation completes, one of the threads starts a callback. It first restores the state of the coroutine, then the coroutine continues execution, either reaching completion, or the next asynchronous operation. After the coroutine completes (or suspends execution after scheduling the next asynchronous operation), the thread returns to the pool. Thus, one request in turn can be processed by different threads, and this imposes some restrictions on our code. So, for example, we cannot use thread local-variables if there is no certainty that the coroutine's execution will not be interrupted between put and get. On the other hand, the scheme looks close to optimal, and promises good performance.
Unlike javaflow, jcoro does not instrument all methods and all calls within them. Subroutines are subject to instrumenting only - those methods that have at least one recovery point. A restore point is a call that, when executed, can ultimately lead to a call to yield (). That is, this does not have to happen with every call, a theoretical enough is enough. How do you code the code? How can I save and restore the execution status of the whole thread? It turns out that this is not difficult at all. It is enough to turn each method that claims the proud title of coroutine into a small state-machine. To do this, a bytecode is added at the beginning of the method, which does nothing if it does not need to be restored, and if necessary, executes a switch (state) and proceeds to the call of the restore point on which execution was suspended. This is sufficient, because state saving can occur only at the moment of calling the restore point (and the yield () call itself is also a restore point). Well, plus to this, you need to remember to restore the local variables and the frame stack. Since in JVM the state of the frame is uniquely identified by this set (the state of the stack, local variables and the current instruction), after that it can be argued that everything works correctly for us. Similarly, it works out save-restore on the entire execution stack.
Returning to our example, let's look at what it will turn into:
This coroutine does nothing useful, but only suspends its work, and then returns the value. In bytecode it looks like this:
After instrumentation, we will see this (the result is given in the form of unified diff, unfortunately, habr does not support line highlighting):
At the beginning of the method, the code defining the recovery point was added, and before and after the recovery point, the code for recovery and saving. If the recovery points were greater, then at the beginning, instead of a simple transition, we would see a switch. There is another nuance. Since we use parallel stacks to save and restore frames, we must follow the order of adding and receiving objects. If we first put object A on the stack, and then B, then we should receive them in the reverse order. Therefore, if we save the local variables first, and then the frame stack, then the recovery should be the opposite. Plus, the processing of the call object reference (this) fits in perfectly. When saved, it is put on the extreme stack, and when restored it is taken first (unless, of course, the restore point is a non-static method). In the given example there are no local variables, but with them the code would be almost the same.
Unfortunately, the described strategy of saving and restoring the stack only works if it is possible to instrument all coroutines. If there is a method that contains a restore point, we cannot instrument it, this strategy will not work. This is possible if we call the code via reflex or a library that cannot be instructed. And if you can think of something else with the libraries, then you can't do anything without reflexion. All programmers want to use DI-containers, proxies and AOP. However, it can be noted that most often such calls are completely stateless, that is, how many do not call them, they essentially do nothing except to transfer control further. And when resuming a coroutine, you can call this method again, simply passing the same arguments to it. And already in the code that he calls, continue to restore the state. And to support this mechanism, only the second state preservation strategy is needed, in which the arguments are saved before the call, not after. This strategy is now supported in jcoro, and to use it, you just need to mark recovery points as @Await (patchable = false).
You can find information on what method calls using each of the strategies turn to on the wiki .
Lambdas are supported, but crooked. There are two problems. One of them is that it is difficult to hang annotations on lambda in java, and even harder to read them. The only solution I found is based on the Type Annotations that have appeared recently and looks like this:
When the compiler sees this, it adds an annotation to the class file and links it with the invokedynamic instruction. And it works, but, unfortunately, not always. Sometimes the compiler associates such an annotation not with this instruction, but with the previous one (most likely, it is a bug), and sometimes it does not write the annotation into the class file at all. For example, this happens when compiling such code:
In the class file, only the invokedynamic instructions for the outer two lambda will be annotated. And the compiler ignores annotations for internal two lambda. This is also most likely a bug, I sent it to Oracle, but I haven’t received confirmation yet. I hope that this will work out.
The second problem is connected with the fact that lambdas are rather strange creatures in the world of Java. They are called as instance methods, but in reality they are static methods. And this wave-particle duality creates a conceptual problem for the conservation-recovery mechanism. The fact is that for the optimal recovery strategy, we should save this in the body of the instance method (see diagram ). But only the calling code has a link to the instance of the functional interface! Ultimately, we come to the need to use argument preservation before executing the lambda, that is, the same variant with patchable = false (which was intended to bypass reflection problems). And he works slower. Although, perhaps, this is not critical compared to the inconvenience that the need to prescribe patchable = false on each lambda coroutine.
Summing up these two problems, a disappointing conclusion can be made: it is not recommended to use lambda coroutines for the time being.
The project is available at https://github.com/elw00d/jcoro . Now available engine, a set of tests for it and a few examples . To bring the technology to the mind, you must do the following:
If you want to help or try jcoro in business, welcome! For public communication, it's probably the easiest way to use Github Issues .
startReadSocket((data) -> { startWriteFile(data, (result) -> { if (result == ok) ... }); }); we can rewrite it like this:
data = readSocket(); result = writeFile(data); if (result == ok) ... Here, readSocket () and writeFile () are coroutines in which asynchronous operations are invoked as follows:
')
byte[] readSocket() { byte[] result = null; startReadSocket((data) -> { result = data; resume(); }); yield(); return result; } The yield () and resume () methods save and restore the execution context, with all frames and local variables. The following happens: when we call readSocket (), we plan an asynchronous operation by calling startReadSocket () and execute yield (). Yield () saves the execution context and the thread ends (returns to the pool). When the asynchronous operation is completed, we will call resume () before exiting the callback, and thereby resume the execution of the code. The control will again get the main function that will call writeFile (). writeFile () is similar, and everything will be repeated.
By doing this transformation once for all used asynchronous operations and placing the functions in the library, we get a tool that allows us to write asynchronous code as if it were normal synchronous code. We get the opportunity to combine the advantages of synchronous code (readability, convenient error handling) and asynchronous (performance). Paying for this convenience is the need to somehow save and restore the execution context. In the article, the author describes the implementation in C ++, but I wanted to have something like this in Java. This will be discussed.
javaflow
First of all, it was necessary to find a coroutine implementation for the JVM. Among several options, the javaflow library turned out to be the most suitable. It would be quite suitable for the experiment, but, unfortunately, the project has long been abandoned. Having poked the wand (decompiler) into the code generated by it, I found out that there are several serious problems in javaflow:
- Lambdas are not supported at all. This is not surprising, given the fact that the latest release of the library was in 2008.
- The code is extremely non-optimally instrumented - all calls inside the method are instrumented, although most of them never lead to the suspend () call. As a result, the bytecode swells up a lot, and in real life such an approach would be unacceptably slow to work.
- No support for reflection. If during the execution of the code some method can be called through reflection, javaflow cannot save and restore the execution context in this place. And this is critical in everyday programming, and the point is not even in principle, but in the fact that almost everyone now uses DI containers, which work through reflection. Therefore, it is impossible to forbid reflection, it is too strong a restriction for programmers.
Despite all this, javaflow helped to figure out how to implement state save and restore. Then there were 2 options: try to support javaflow or write your own implementation. For obvious reasons (fatal flaw), the second method was chosen.
jcoro
Coroutines added to a language where they were not expand it. To write applications that fully take advantage of the proposed approach, and not to curse at the same time, you need to make them comfortable. When reading the code, we must immediately see that this function is a coroutine and performs an asynchronous operation, and therefore it must be run within a context that supports saving and restoring the stack. In C #, the keywords are async and await. In Java, unfortunately, adding your keywords does not seem real, but you can use annotations! All this looks, of course, cumbersome, but what to do. Maybe something else will come up. For now:
Coro coro = Coro.initSuspended(new ICoroRunnable() { @Override @Async({@Await("foo")}) public void run() { int i = 5; double f = 10; final String argStr = foo(i, f, "argStr"); } @Async(@Await("yield")) private String foo(int x, double y, String m) { Coro c = Coro.get(); c.yield(); return "returnedStr"; } }); coro.start(); coro.resume(); The presence of the @Async annotation tells jcoro to instruct the bytecode of this method, making it a coroutine. Recovery point signatures are specified by @Await annotations. All calls inside the coroutine whose signatures are in the list of @ Await annotations become restore points. A coroutine in jcoro is a method marked with the @Async annotation and having at least one restore point. If there is no recovery point in the method, it will not be instrumented. A recovery point is a Coro.yield () call or any (coroutine) call that can ultimately lead to a Coro.yield () call.
What happens in this example?
First, an instance of Coro is created - it is an object that stores the saved state of the coroutine and can start, save and restore it. Initially, the coroutine is only initialized, but not launched. When you call start (), the control is obtained by the run () method, which first checks whether it is necessary to restore the state. So far we have just started the coroutine, and run () just starts to execute its code. Method executes code, calls foo (). Inside foo (), the same check is performed - is it necessary to restore the state? The answer is negative, and, similarly, the method code starts from the beginning. But when you call yield (), the following happens. The yield () call itself only sets the “isYielding” flag and does nothing more, but the code after the call, upon seeing this flag, does not continue execution, but maintains its state and immediately ends, returning null. The same happens a level above. And then the start () method returns control. What do we have at this moment? The code before the yield () call is executed, the execution state is stored in the Coro instance. Next, we call resume (). This causes the run () method to be called again. And, like the first time, the method checks whether it is necessary to restore the state. This time it really needs to be done, and the method, remembering that it stopped on the foo () call, restores its local variables and stack and proceeds directly to the foo () call, without executing the code that was in front of it. In the foo () method, the same thing happens - it restores the stack and local variables, and then goes straight to calling yield (). A call to yield () by itself does nothing except resetting the internal flag. After it, the foo () method completes execution, returning the string "returnedStr". There remains the run () method, which also terminates safely, returning control to the code that calls resume (). At the exit, we have a fully completed coroutine, the implementation of which we have divided into two parts.
How does this help us in writing asynchronous applications?
Suppose we need to write a server application that, in response to a request, accesses the database, then does something with the data, then applies it to the template, and returns a piece of markup. Classic web server application. Almost at all stages we can use asynchronous operations. Establishing a connection, reading data from a socket when a request is received, all network operations with a database, reading a file when loading a template, sending the result to a socket. In such a scenario, the CPU should be occupied only with the planning of asynchronous operations, the logic of data preprocessing and templating. The rest of the time, the processor can rest. Let's try to figure out how this could be organized in code. Sketches the server:
public static void main(String[] args) { Coro.initSuspended(new ICoroRunnable() { @Async({@Await("accept")}) public void run() { final AsynchronousServerSocketChannel listener = bind(new InetSocketAddress(5000)); // , ExecutorService executorService = Executors.newFixedThreadPool(Runtime.getRuntime().availableProcessors()); while (true) { AsynchronousSocketChannel channel = accept(listener); // executorService.submit(new Runnable() { @Override public void run() { Coro.initSuspended(new ICoroRunnable() { @Async({@Await("handle")}) public void run() { // - , // handle(channel); } }).start(); } }); } } }).start(); } @Async({@Await("read"), @Await("write")}) public static void handle(AsynchronousSocketChannel channel) { ByteBuffer buffer = ByteBuffer.allocate(10 * 1024); Integer read = read(channel, buffer); // write(channel, outBuffer); // channel.close(); } Some try-catch blocks necessary for correct compilation are omitted in the code, this is done to make it easier to read the code.
Inside the handle, you can add any logic. For example, the definition of "controller" and call it through reflection, the introduction of dependencies. But you need to be careful with calls to the code containing the recovery points, through reflection or non-instrumented libraries. About this below.
From the point of view of recycling, it works as follows. There is a pool of worker threads, and there is a system pool of threads that the JVM reserves to perform asynchronous callbacks. When an asynchronous operation completes, one of the threads starts a callback. It first restores the state of the coroutine, then the coroutine continues execution, either reaching completion, or the next asynchronous operation. After the coroutine completes (or suspends execution after scheduling the next asynchronous operation), the thread returns to the pool. Thus, one request in turn can be processed by different threads, and this imposes some restrictions on our code. So, for example, we cannot use thread local-variables if there is no certainty that the coroutine's execution will not be interrupted between put and get. On the other hand, the scheme looks close to optimal, and promises good performance.
Implementation
Unlike javaflow, jcoro does not instrument all methods and all calls within them. Subroutines are subject to instrumenting only - those methods that have at least one recovery point. A restore point is a call that, when executed, can ultimately lead to a call to yield (). That is, this does not have to happen with every call, a theoretical enough is enough. How do you code the code? How can I save and restore the execution status of the whole thread? It turns out that this is not difficult at all. It is enough to turn each method that claims the proud title of coroutine into a small state-machine. To do this, a bytecode is added at the beginning of the method, which does nothing if it does not need to be restored, and if necessary, executes a switch (state) and proceeds to the call of the restore point on which execution was suspended. This is sufficient, because state saving can occur only at the moment of calling the restore point (and the yield () call itself is also a restore point). Well, plus to this, you need to remember to restore the local variables and the frame stack. Since in JVM the state of the frame is uniquely identified by this set (the state of the stack, local variables and the current instruction), after that it can be argued that everything works correctly for us. Similarly, it works out save-restore on the entire execution stack.
Returning to our example, let's look at what it will turn into:
@Async(@Await("yield")) private String foo(int a, double b, String c) { Coro c = Coro.get(); c.yield(); return "returnedStr"; } This coroutine does nothing useful, but only suspends its work, and then returns the value. In bytecode it looks like this:
private java.lang.String foo(int, double, java.lang.String); descriptor: (IDLjava/lang/String;)Ljava/lang/String; flags: ACC_PRIVATE Code: stack=1, locals=6, args_size=4 0000: invokestatic org/jcoro/Coro.get:()Lorg/jcoro/Coro; 0003: astore 5 0005: aload 5 0007: invokevirtual org/jcoro/Coro.yield:()V 0010: ldc "returnedStr" 0012: areturn After instrumentation, we will see this (the result is given in the form of unified diff, unfortunately, habr does not support line highlighting):
private java.lang.String foo(int, double, java.lang.String); descriptor: (IDLjava/lang/String;)Ljava/lang/String; flags: ACC_PRIVATE Code: - stack=1, locals=6, args_size=4 + stack=2, locals=6, args_size=4 + 0: invokestatic org/jcoro/Coro.getSafe:()Lorg/jcoro/Coro; // + 3: ifnull 0000 // - + 6: invokestatic org/jcoro/Coro.popState:()Ljava/lang/Integer; // popState() null, + 9: dup + 10: ifnull 32 // - + 13: invokestatic org/jcoro/Coro.isUnpatchableCall:()Z // + 16: ifeq 23 + 19: invokestatic org/jcoro/Coro.popRef:()Ljava/lang/Object; + 22: pop + 23: ldc 0 + 25: invokestatic org/jcoro/Coro.setUnpatchableCall:(Z)V + 28: pop + 29: goto 43 + 32: pop 0000: invokestatic org/jcoro/Coro.get:()Lorg/jcoro/Coro; // 0003: astore 5 0005: aload 5 + 40: goto 0007 // - + 43: invokestatic org/jcoro/Coro.popRef:()Ljava/lang/Object; + 46: checkcast "Lorg/jcoro/Coro;" + 49: astore 5 + 51: invokestatic org/jcoro/Coro.popRef:()Ljava/lang/Object; + 54: checkcast "Ljava/lang/String;" + 57: astore 4 + 59: invokestatic org/jcoro/Coro.popDouble:()D + 62: dstore_2 + 63: invokestatic org/jcoro/Coro.popInt:()I + 66: istore_1 + 67: invokestatic org/jcoro/Coro.popRef:()Ljava/lang/Object; + 70: checkcast "Lorg/jcoro/tests/SimpleTest$1;" + 73: astore_0 + 74: invokestatic org/jcoro/Coro.popRef:()Ljava/lang/Object; + 77: checkcast "Lorg/jcoro/Coro;" 0007: invokevirtual org/jcoro/Coro.yield:()V // + 83: invokestatic org/jcoro/Coro.isYielding:()Z // - + 86: ifeq 0010 + 89: aload_0 + 90: invokestatic org/jcoro/Coro.pushRef:(Ljava/lang/Object;)V + 93: iload_1 + 94: invokestatic org/jcoro/Coro.pushInt:(I)V + 97: dload_2 + 98: invokestatic org/jcoro/Coro.pushDouble:(D)V + 101: aload 4 + 103: invokestatic org/jcoro/Coro.pushRef:(Ljava/lang/Object;)V + 106: aload 5 + 108: invokestatic org/jcoro/Coro.pushRef:(Ljava/lang/Object;)V + 111: aload_0 + 112: invokestatic org/jcoro/Coro.pushRef:(Ljava/lang/Object;)V + 115: ldc 0 + 117: invokestatic org/jcoro/Coro.pushState:(I)V + 120: aconst_null // null, + 121: areturn 0010: ldc "returnedStr" // 0012: areturn At the beginning of the method, the code defining the recovery point was added, and before and after the recovery point, the code for recovery and saving. If the recovery points were greater, then at the beginning, instead of a simple transition, we would see a switch. There is another nuance. Since we use parallel stacks to save and restore frames, we must follow the order of adding and receiving objects. If we first put object A on the stack, and then B, then we should receive them in the reverse order. Therefore, if we save the local variables first, and then the frame stack, then the recovery should be the opposite. Plus, the processing of the call object reference (this) fits in perfectly. When saved, it is put on the extreme stack, and when restored it is taken first (unless, of course, the restore point is a non-static method). In the given example there are no local variables, but with them the code would be almost the same.
Unpatchable code
Unfortunately, the described strategy of saving and restoring the stack only works if it is possible to instrument all coroutines. If there is a method that contains a restore point, we cannot instrument it, this strategy will not work. This is possible if we call the code via reflex or a library that cannot be instructed. And if you can think of something else with the libraries, then you can't do anything without reflexion. All programmers want to use DI-containers, proxies and AOP. However, it can be noted that most often such calls are completely stateless, that is, how many do not call them, they essentially do nothing except to transfer control further. And when resuming a coroutine, you can call this method again, simply passing the same arguments to it. And already in the code that he calls, continue to restore the state. And to support this mechanism, only the second state preservation strategy is needed, in which the arguments are saved before the call, not after. This strategy is now supported in jcoro, and to use it, you just need to mark recovery points as @Await (patchable = false).
You can find information on what method calls using each of the strategies turn to on the wiki .
Lambda support
Lambdas are supported, but crooked. There are two problems. One of them is that it is difficult to hang annotations on lambda in java, and even harder to read them. The only solution I found is based on the Type Annotations that have appeared recently and looks like this:
Coro coro = Coro.initSuspended((@Async({@Await(value = "yield")}) ICoroRunnable) () -> { Coro.get().yield(); }); When the compiler sees this, it adds an annotation to the class file and links it with the invokedynamic instruction. And it works, but, unfortunately, not always. Sometimes the compiler associates such an annotation not with this instruction, but with the previous one (most likely, it is a bug), and sometimes it does not write the annotation into the class file at all. For example, this happens when compiling such code:
public static void main(String[] args) { Runnable one = (@TypeAnn("1") Runnable) () -> { Runnable two = (@TypeAnn("2") Runnable) () -> { Runnable three = (@TypeAnn("3") Runnable) () -> { Runnable four = (@TypeAnn("4") Runnable) () -> { }; }; }; }; } In the class file, only the invokedynamic instructions for the outer two lambda will be annotated. And the compiler ignores annotations for internal two lambda. This is also most likely a bug, I sent it to Oracle, but I haven’t received confirmation yet. I hope that this will work out.
The second problem is connected with the fact that lambdas are rather strange creatures in the world of Java. They are called as instance methods, but in reality they are static methods. And this wave-particle duality creates a conceptual problem for the conservation-recovery mechanism. The fact is that for the optimal recovery strategy, we should save this in the body of the instance method (see diagram ). But only the calling code has a link to the instance of the functional interface! Ultimately, we come to the need to use argument preservation before executing the lambda, that is, the same variant with patchable = false (which was intended to bypass reflection problems). And he works slower. Although, perhaps, this is not critical compared to the inconvenience that the need to prescribe patchable = false on each lambda coroutine.
Summing up these two problems, a disappointing conclusion can be made: it is not recommended to use lambda coroutines for the time being.
Current status and plans
The project is available at https://github.com/elw00d/jcoro . Now available engine, a set of tests for it and a few examples . To bring the technology to the mind, you must do the following:
- As part of the engine improvements - to optimize the generation of stack map frames and solve problems with lambdas
- Write maven and gradle plugins to instruct specified jar-nicknames or sets of class files
- Perform performance testing by writing 3 servers with the same functionality. One will use the blocking model, the second - asynchronous on callbacks (normal nio, without jcoro), and the third - asynchronous using jcoro. It is necessary to estimate how much the conservation-restoration of the context eats compared to the code that does not. I really hope that this will not be too much.
- . . . , , jdbc. - «» jdbc, — mysql, postgresql, mssql. — jcoro, . — - -.
- IntelliJ IDEA, . - , ( @Await , @Async) .
- , User Guide .
If you want to help or try jcoro in business, welcome! For public communication, it's probably the easiest way to use Github Issues .
Source: https://habr.com/ru/post/269021/
All Articles