Asynchronous API and Deferred object in detail
Most modern programming languages allow the use of asynchronously executed blocks of code. Along with the flexibility gained by using the asynchronous approach, the one who has risked applying it also gets code that is more difficult to understand and maintain. However, any complication that programmers encounter, as a rule, finds a practical solution in the form of a new approach or an increase in the level of abstraction. In the case of asynchronous programming, this means is an object of the type pending result or deferred (English deferred - deferred, delayed).
The article will cover basic approaches to returning asynchronous results, callback functions, deferred objects, and their capabilities. Examples in the JavaScript language will be given, and a typical deferred object will be parsed. The article will be useful to programmers who begin to comprehend asynchronous programming, as well as those familiar with it, but not owning the deferred object.
Any function can be described in synchronous and asynchronous form. Suppose we have a
In the case of the usual, "synchronous" approach, the result of the calculation will be transmitted through the return value, that is, the result will be available immediately after the function is executed, and can be used in another calculation.
')
The code is executed strictly sequentially, the result obtained on one line can be used on the next. This is reminiscent of the proof of the theorem, when subsequent statements logically follow from previous ones.
In the case of an asynchronous call, we cannot get the result in place. By calling the
The example shows that the function now has a signature:
Since
Agree, such a code has already become somewhat more difficult for perception, with the same semantic load as its “straight-line” synchronous analogue. What is the benefit of using the asynchronous approach? First of all - in the rational use of system resources. For example, if
Generally speaking, most of the APIs, by their nature, are asynchronous, but can mimic as synchronous: access to remote resources, queries to the database, even the file API is asynchronous. If the API “pretends” to be synchronous, then the success of such “pretense” is associated with delays in the result: the lower the delay, the better. The same file API, working with the local machine, shows small delays and is often implemented as synchronous. Work with remote resources and access to the database is increasingly being implemented asynchronously.
The difficulties of the asynchronous approach become more noticeable when it is necessary not only to make an asynchronous call, but, having received its result, to do something with it and use it in another asynchronous call. Obviously, the synchronous approach in several consecutively executed lines of code does not fit here:
The code will take the following form:
Firstly, this code has become “multi-level”, although, in its actions it is similar to synchronous. Secondly, in the signatures of the functions
This code can be simplified by defining callback functions separately and passing them by name, however, this is not a solution to all problems. This requires a new level of abstraction, namely, we can abstract an asynchronous result to abstraction.
What is such a result? In essence, this is an object containing information that the result will ever come or has already arrived. The result is subscribed to through the same callback, but now it is encapsulated in this object and does not oblige asynchronous functions to implement callbacks as input parameters.
In essence, the result object requires three things: to implement the ability to subscribe to the result, the ability to indicate the arrival of the result (this will be used by the asynchronous function itself, not the API client), and the storage of this result.
An important distinctive feature of such an object is also the specificity of its states. Such an object can be in two states: 1) there is no result and 2) there is a result. Moreover, the transition is possible only from the first state to the second. When the result is obtained, it is no longer possible to go into a state of its absence or a state with a different result.
Consider the following simple interface for this object:
The
The
The object with this behavior is called deferred (and also known as promise and future ). It has several advantages over simple callbacks:
1. Abstraction of the asynchronous function from the result: now each asynchronous function is not required to provide callback parameters. Subscription to the result remains for the client code. For example, you can don’t subscribe to the result at all if we don’t need it (similar to passing the noop function as a callback). The asynchronous function interface becomes cleaner: it has only significant input parameters, it becomes possible to more reliably use functions with an indefinite number of parameters, an options parameter, and so on.
2. Abstraction from the state of the result: the client of the code does not need to check the current state of the result, he simply signs the handler and does not think about whether the result has arrived or not.
3. The possibility of multiple subscriptions: you can sign more than one handler and they will all be called when the result arrives. In a scheme with callbacks, we would have to create a function that calls a group of functions, for example.
4. A number of additional amenities, including, for example, the “algebra” of deferred objects, which allows you to define relationships between them, run them in a chain or after the successful completion of a group of such objects.
Consider the following example. Let there be an asynchronous function
Rewrite it using
In this case, the code is practically not complicated; rather, the approach has simply changed. The result is now passed through the return value of the function as a deferred. However, as will become noticeable further, in more complex cases the use of deferred gives some advantage in the readability of the code.
A reasonable question would be about handling errors when using such objects. In synchronous code, an exception mechanism is widely used, which in case of an error allows to transfer control to higher blocks of code, where all errors can be caught and processed, without complicating the “local” code, freeing the programmer from having to write checks for every code.
In asynchronous code (and in any scheme with callbacks), there is some difficulty in using exceptions, because an exception will come asynchronously, as well as the result, and therefore it cannot be easily caught by framing the asynchronous function call in
Such a result, as well as a successful one, is realized in the form of a kollbek (sometimes called errback , a game of words from error and back ).
Let's strengthen our learning object
As a first parameter, you can pass the value of an enumerated type with two values, for example,
The
The first parameter is the state to which the
So, let our
The code will look like this:
Consider a more realistic example, namely, take the typical fs.readFile method from the Node.js standard library. This method is used to read the file. At the beginning of the article it was mentioned that almost any function can be written either in synchronous or in asynchronous style. In the standard library Node.js file API is defined in both styles, each function has its own synchronous equivalent.
For example, we use the asynchronous version of readFile and adapt it to use Deferred.
Such a function is somewhat easier to use, because it allows you to register functions for success and for error separately.
The described functionality is quite enough for the overwhelming majority of cases, but deferred has more potential, as will be discussed below.
1. Unlimited number of result options. In the example, a
2. Ability to subscribe to all variants of the result. This can be used for all sorts of generalized handlers (this has the most meaning, along with paragraph 1.).
3. Creating a sub-object promise. From the interface of the
4. Passing state from one deferred to another, optionally, subjecting the result to a transform. This is very useful for multi-level calls.
5. Creating deferred, which depends on the result of a set of other deferred. The essence of this improvement is to subscribe to the result of a group of asynchronous operations.
Suppose we need to read two files and do something interesting with both. We use our
It is worth noting that in JavaScript there is no multithreading. If a callback has been set by
In JavaScript, functions can be called with any number of parameters, as well as with any context. This allows you to send as many parameters to callbacks as needed. For example, if an asynchronous function returns a pair of values
Calling a callback in this case can take the following form:
JavaScript uses links, and memory release is controlled by the garbage collector. The deferred object is needed both inside the asynchronous function (to signal the arrival of the result), and outside (to get the result), in languages with more strict models of working with memory, care should be taken to properly handle the lifetime of such an object.
1. In jQuery, there is a
All library ajax methods return a promise of such an object.
2. The q library implements deferred objects, it is possible to make chains of asynchronous functions, you can register deferred by the result of the deferred list.
3. The async.js library allows you to apply filter / map / reduce on asynchronous calls, create chains and groups of asynchronous calls.
4. The when.js library also allows the use of deferred.
5. The Dojo Toolkit contains a Deferred object ( documentation ).
6. In the fraternal Python language, in the event-driven framework Twisted there is a Deferred object ( documentation ). This implementation is very old and can claim the right to be the founder of the idea of pending results.
Supports subscription to success, error, and both results. You can pause an object.
7. Overcome with interest in Deferred, I wrote my own version of this object ( documentation , source code , tests ). A number of features described in this article are supported.
That's all, thank you for your attention.
The article will cover basic approaches to returning asynchronous results, callback functions, deferred objects, and their capabilities. Examples in the JavaScript language will be given, and a typical deferred object will be parsed. The article will be useful to programmers who begin to comprehend asynchronous programming, as well as those familiar with it, but not owning the deferred object.
Synchronous and asynchronous calls
Any function can be described in synchronous and asynchronous form. Suppose we have a
calc function that performs some calculation.In the case of the usual, "synchronous" approach, the result of the calculation will be transmitted through the return value, that is, the result will be available immediately after the function is executed, and can be used in another calculation.
')
var result = calc(); another_calc(result * 2); The code is executed strictly sequentially, the result obtained on one line can be used on the next. This is reminiscent of the proof of the theorem, when subsequent statements logically follow from previous ones.
In the case of an asynchronous call, we cannot get the result in place. By calling the
calc function, we only point out the need to perform a calculation and get its result. In this case, the next line will begin to run without waiting for the previous one. Nevertheless, we somehow need to get the result, and here a callback comes to the rescue - a function that will be called by the system upon the arrival of the result of the calculation. The result will be passed to this function as an argument. calc(function (result) { another_calc(result * 2); }); no_cares_about_result(); The example shows that the function now has a signature:
calc(callback) , and callback takes the result as the first parameter.Since
calc is performed asynchronously, the no_cares_about_result function no_cares_about_result not access its result, and, generally speaking, it can be executed earlier than callback (if we are talking specifically about JavaScript, if the function being called is truly asynchronous, instead of taking data from the cache, for example it is guaranteed that it will always be executed after the execution of the calling code, that is, the remaining code will always be executed before the callback (this will be discussed below).Agree, such a code has already become somewhat more difficult for perception, with the same semantic load as its “straight-line” synchronous analogue. What is the benefit of using the asynchronous approach? First of all - in the rational use of system resources. For example, if
calc is a time-consuming calculation that can be time consuming, or uses some external resource, the use of which imposes a certain delay, then with a synchronous approach, all subsequent code will be forced to wait for the result and will not be executed until calc executed. Using the asynchronous approach, you can explicitly specify which part of the code depends on a certain result, and which part is indifferent to the result. In the example, no_cares_about_result does not explicitly use the result, and therefore it does not need to expect it. The part of the code inside the callback will be executed only after receiving the result.Generally speaking, most of the APIs, by their nature, are asynchronous, but can mimic as synchronous: access to remote resources, queries to the database, even the file API is asynchronous. If the API “pretends” to be synchronous, then the success of such “pretense” is associated with delays in the result: the lower the delay, the better. The same file API, working with the local machine, shows small delays and is often implemented as synchronous. Work with remote resources and access to the database is increasingly being implemented asynchronously.
Multi-level calls
The difficulties of the asynchronous approach become more noticeable when it is necessary not only to make an asynchronous call, but, having received its result, to do something with it and use it in another asynchronous call. Obviously, the synchronous approach in several consecutively executed lines of code does not fit here:
var result = calc_one(); result = calc_two(result * 2); result = calc_three(result + 42); // using result The code will take the following form:
calc_one(function (result) { calc_two(result * 2, function (result) { calc_three(result + 42, function (result) { // using result }); }); }); Firstly, this code has become “multi-level”, although, in its actions it is similar to synchronous. Secondly, in the signatures of the functions
calc_two , calc_three input parameters and the callback are mixed, which, in essence, is the place where the result is returned, that is, the output parameter. Thirdly, each function may fail with an error and the result will not be obtained.This code can be simplified by defining callback functions separately and passing them by name, however, this is not a solution to all problems. This requires a new level of abstraction, namely, we can abstract an asynchronous result to abstraction.
Asynchronous result
What is such a result? In essence, this is an object containing information that the result will ever come or has already arrived. The result is subscribed to through the same callback, but now it is encapsulated in this object and does not oblige asynchronous functions to implement callbacks as input parameters.
In essence, the result object requires three things: to implement the ability to subscribe to the result, the ability to indicate the arrival of the result (this will be used by the asynchronous function itself, not the API client), and the storage of this result.
An important distinctive feature of such an object is also the specificity of its states. Such an object can be in two states: 1) there is no result and 2) there is a result. Moreover, the transition is possible only from the first state to the second. When the result is obtained, it is no longer possible to go into a state of its absence or a state with a different result.
Consider the following simple interface for this object:
function Deferred () // constructor function on (callback) function resolve (result) The
on method accepts callback. Callback will be called as soon as the result is available and will be passed as a parameter. Here is a complete analogy with the usual callback passed as a parameter. At the time of registration of the kollbek, the object can be in the state with the result and without. If there is no result yet, the callback will be called upon his arrival. In case the result is already there, the callback will be called immediately. In both cases, the callback is called once and gets the result.The
resolve method allows you to translate (split) an object into a state with a result and specify this result. This method is idempotent , that is, repeated calls to resolve will not change the object. Upon transition to the state with the result, all registered callbacks will be called, and all callbacks that will be registered after the call to resolve will be called up instantly. In both cases (registration before and after calling resolve ), callbacks will get the result, due to the fact that the object stores it.The object with this behavior is called deferred (and also known as promise and future ). It has several advantages over simple callbacks:
1. Abstraction of the asynchronous function from the result: now each asynchronous function is not required to provide callback parameters. Subscription to the result remains for the client code. For example, you can don’t subscribe to the result at all if we don’t need it (similar to passing the noop function as a callback). The asynchronous function interface becomes cleaner: it has only significant input parameters, it becomes possible to more reliably use functions with an indefinite number of parameters, an options parameter, and so on.
2. Abstraction from the state of the result: the client of the code does not need to check the current state of the result, he simply signs the handler and does not think about whether the result has arrived or not.
3. The possibility of multiple subscriptions: you can sign more than one handler and they will all be called when the result arrives. In a scheme with callbacks, we would have to create a function that calls a group of functions, for example.
4. A number of additional amenities, including, for example, the “algebra” of deferred objects, which allows you to define relationships between them, run them in a chain or after the successful completion of a group of such objects.
Consider the following example. Let there be an asynchronous function
getData(id, onSuccess) , which takes two parameters: the id of some element that we want to receive and the callback to get the result. A typical usage code would look like this: getData(id, function (item) { // do some actions with item }); Rewrite it using
Deferred . The function now has a signature getData(id) and is used as follows: getData(id).on(function (item) { // do some actions with item }); In this case, the code is practically not complicated; rather, the approach has simply changed. The result is now passed through the return value of the function as a deferred. However, as will become noticeable further, in more complex cases the use of deferred gives some advantage in the readability of the code.
Error processing
A reasonable question would be about handling errors when using such objects. In synchronous code, an exception mechanism is widely used, which in case of an error allows to transfer control to higher blocks of code, where all errors can be caught and processed, without complicating the “local” code, freeing the programmer from having to write checks for every code.
In asynchronous code (and in any scheme with callbacks), there is some difficulty in using exceptions, because an exception will come asynchronously, as well as the result, and therefore it cannot be easily caught by framing the asynchronous function call in
try . If we consider an error, then in fact, this is only a different result of the function (negative, we can say, but also a result), with the object of error (exception) being the return value.Such a result, as well as a successful one, is realized in the form of a kollbek (sometimes called errback , a game of words from error and back ).
Let's strengthen our learning object
Deferred so that it can provide a subscription separately for success and for failure, namely, rework the methods on and resolve . function on (state, callback) As a first parameter, you can pass the value of an enumerated type with two values, for example,
E_SUCCESS , E_ERROR . For readability, we will use simple string values in the examples: 'success', 'error'. Also, we will strengthen this method, obliging it to return the Deferred object itself . This will allow the use of subscription chains (very specific to JavaScript).The
resolve method changes accordingly: function resolve (state, result) The first parameter is the state to which the
Deferred (error, success) object should go, and the second is the result. The rule of states still extends to such a modified object: after transition to the state with the result, the object cannot change its state to another. This means that if an object has passed, for example, to the state of success, then all the handlers registered for the error will never work, and vice versa.So, let our
getData function may end up with some error (no data, incorrect input data, failure, etc.).The code will look like this:
getData(id) .on('success', function (item) { // do some actions with item }) .on('error', function (err_code) { // deal with error }); Consider a more realistic example, namely, take the typical fs.readFile method from the Node.js standard library. This method is used to read the file. At the beginning of the article it was mentioned that almost any function can be written either in synchronous or in asynchronous style. In the standard library Node.js file API is defined in both styles, each function has its own synchronous equivalent.
For example, we use the asynchronous version of readFile and adapt it to use Deferred.
function readFileDeferred (filename, options) { var result = new Deferred; fs.readFile(filename, options, function (err, data) { if (err) { result.resolve('error', err); } else { result.resolve('success', data); } }); return result; } Such a function is somewhat easier to use, because it allows you to register functions for success and for error separately.
The described functionality is quite enough for the overwhelming majority of cases, but deferred has more potential, as will be discussed below.
Advanced features of Deferred objects
1. Unlimited number of result options. In the example, a
Deferred object was used with two possible results: success and error. Nothing prevents the use of any other (custom) options. Fortunately, we used the string value as a state, this allows us to define any set of results without changing any enumerated type.2. Ability to subscribe to all variants of the result. This can be used for all sorts of generalized handlers (this has the most meaning, along with paragraph 1.).
3. Creating a sub-object promise. From the interface of the
Deferred object, it is clear that the client code has access to the resolve method, although, in fact, it only needs the ability to subscribe. The essence of this improvement is the introduction of the promise method, which returns a “subset” of the Deferred object, from which only a subscription is available, but not a result setting.4. Passing state from one deferred to another, optionally, subjecting the result to a transform. This is very useful for multi-level calls.
5. Creating deferred, which depends on the result of a set of other deferred. The essence of this improvement is to subscribe to the result of a group of asynchronous operations.
Suppose we need to read two files and do something interesting with both. We use our
readFileDeferred function for this: var r1 = readFileDeferred('./session.data'), r2 = readFileDeferred('./data/user.data'); var r3 = Deferred.all(r1, r2); r3.on('success', function (session, user) { session = JSON.parse(session); user = JSON.parse(user); console.log('All data recieved', session, user); }).on('error', function (err_code) { console.error('Error occured', err_code); }); Deferred.all creates a new Deferred object that will go to the success state if all the arguments passed go to this state. In doing so, he will also get the results of all deferred as arguments. If at least one argument goes to the error state, then the result of Deferred.all will also go to this state, and get the result of the argument that has passed to the error state as the result.Features deferred in javascript
It is worth noting that in JavaScript there is no multithreading. If a callback has been set by
setInterval / setTimeout or by events, it cannot interrupt the execution of the current code, or run in parallel with it. This means that even if the result of the asynchronous function comes instantly, it will still be obtained only after the completion of the current code.In JavaScript, functions can be called with any number of parameters, as well as with any context. This allows you to send as many parameters to callbacks as needed. For example, if an asynchronous function returns a pair of values
(X, Y) , then they can be passed as an object with two fields, or a list with two values (an improvised analogue of a tuple), or you can use the first two arguments of the callback for this purpose.Calling a callback in this case can take the following form:
callback.call(this, X, Y); JavaScript uses links, and memory release is controlled by the garbage collector. The deferred object is needed both inside the asynchronous function (to signal the arrival of the result), and outside (to get the result), in languages with more strict models of working with memory, care should be taken to properly handle the lifetime of such an object.
Existing deferred
1. In jQuery, there is a
$.Deferred object ( documentation ). Supports subscription to success, error, also supports progress-notifications: intermediate events generated before the arrival of the result; you can transfer the state to another Deferred ( then method), you can register Deferred according to the result of the Deferred list ( $.when ), you can create a promise .All library ajax methods return a promise of such an object.
2. The q library implements deferred objects, it is possible to make chains of asynchronous functions, you can register deferred by the result of the deferred list.
3. The async.js library allows you to apply filter / map / reduce on asynchronous calls, create chains and groups of asynchronous calls.
4. The when.js library also allows the use of deferred.
5. The Dojo Toolkit contains a Deferred object ( documentation ).
6. In the fraternal Python language, in the event-driven framework Twisted there is a Deferred object ( documentation ). This implementation is very old and can claim the right to be the founder of the idea of pending results.
Supports subscription to success, error, and both results. You can pause an object.
7. Overcome with interest in Deferred, I wrote my own version of this object ( documentation , source code , tests ). A number of features described in this article are supported.
That's all, thank you for your attention.
Source: https://habr.com/ru/post/175947/
All Articles