
The post contains a translation of the article
“Error Handling in Node.js” , which was prepared by
Joyent employees. The article was published on March 28, 2014 on the company's website. Dave Pacheco
explains that the article is intended to eliminate the confusion among developers regarding the best practices for working with errors in Node.js, as well as to answer questions that often arise for novice developers.
Error Handling in Node.js
As you master Node.js, you can write programs for quite a long time, without paying due attention to correct error handling. However, the development of serious projects on Node.js requires a conscious approach to this problem.
Beginner developers often have the following questions:
')
- Is it possible to use
throw
to return an error from a function, or should a callback function be called passing the error object as an argument? When do I need to generate an 'error'
event on an object of class EventEmitter? - Do I need to check the arguments of the passed functions? What if incorrect arguments are passed to the function? In this case, is it necessary to generate an exception or call a callback function, passing an error to it?
- Is it possible to programmatically distinguish errors by type, so that the application can properly handle errors according to their type (for example, “Bad Request” or “Service Unavailable”)?
- How can a function inform the program in the most informative way about the occurrence of an error, so that it can process it correctly?
- Do I need to handle errors caused by "bugs" in the program?
This article consists of seven parts:
- Introduction The fact that the reader should know before reading the article.
- Software errors and programmer errors. Familiarizing with the types of errors.
- Function writing templates. Fundamental principles of writing functions that implement the correct work with errors.
- Rules for writing functions. A list of instructions to follow when writing functions.
- Example. An example of writing a function.
- Summary. A brief presentation of the main provisions discussed in the article.
- Application. Common field names for error objects.
1. Introduction
It is assumed that the reader:
- is familiar with the term “exception” in JavaScript, Java, Python, C ++, or another similar language, and understands the principle of operation of the
try
/ catch
construct; - familiar with the development on Node.js and mastered the principles of asynchronous programming.
The reader should understand why interception of exceptions does not work in the code below, despite the presence of a
try
/
catch
construct.
one function myFunc(callback) { try { doSomeAsyncOperation(function (err) { if (err) { throw (err); } }); } catch (ex) { callback(ex); } }
The reader should be aware that in Node.js there are 3 main ways a function can return an error:
- Throwing a
throw
error (throwing an exception). - Call the callback function with the error object as the first argument.
- Generating an
'error'
event for an object of class EventEmitter.
It is assumed that the reader is not familiar with the
domains in Node.js.
The reader must understand the difference between the error and the exception in JavaScript. Error is any object of class
Error
. The error can be created by the class constructor and returned from the function or thrown using the ThrowStatement instruction. When an error object is thrown, an exception is thrown. The following is an example of throwing an error (generating an exception):
2 throw new Error(' ');
An example where the error is passed to the callback function:
callback(new Error(' '));
The second option is more common in Node.js, due to the asynchrony of most operations performed. As a rule, the first option is used only when data is deserialized (for example,
JSON.parse
), while the thrown exception is caught using the
try
/
catch
construct. This distinguishes Node.js from Java or C ++ and other languages where you have to work with exceptions more often.
2. Software errors and programmer errors
Errors can be divided into two types:
3- Software errors are conflicts that arise during the normal operation of a program. They are not "bugs". Usually, they are not directly related to the program: system errors (for example, memory overflow), configuration errors (for example, the remote server address is incorrect), Internet connection errors or errors that occurred on the remote server.
Examples of software errors:
- user entered incorrect data
- timeout for request response (timeout) expired,
- server responded to request with error code 500,
- disconnection
- allocated memory consumed.
- Programmer errors are code defects that lead to incorrect program operation. Errors of this type cannot be processed correctly, since the very fact of their presence indicates incorrectness of the written code. Errors of this type can be eliminated by changing the program code. Programmer errors include:
- attempt to access a field with a value of
undefined
, - call an asynchronous function without a callback function,
- function call with invalid arguments.
Developers use the term “error” for both types of errors, despite their fundamental differences. “File not found” is a program error, its occurrence may mean that the program needs to create the desired file. Thus, the occurrence of this error is not the incorrect behavior of the program. Programmer errors, on the contrary, were not intended by the developer. Perhaps the developer made a mistake in the name of the variable or incorrectly described the verification of data entered by the user. This type of error cannot be processed.
There may be cases when, for the same reason, both a program error and a programmer's error occur. Suppose an HTTP server attempts to read a field with a value of
undefined
, which is a programmer’s error. As a result, the server goes down. The client, in
ECONNRESET
, receives an
ECONNRESET
error, usually described by Node.js as: “socket hang-up”, as a response to its request. For a client, this is a software error and a correctly written client program will handle the error accordingly and continue to work.
The absence of a program error handler is a programmer error. Suppose that a client program, when establishing a connection with a server, encounters an ECONNREFUSED error, as a result, the connection object generates an
'error'
event, but no handler function is registered for this event, for this reason the program fails. In this case, the connection error is a software error, however, the absence of a handler for the 'error' event of the connection object is a programmer's error.
It is important to understand the differences between programmer errors and software errors. Therefore, before continuing to read the article, make sure that you understand these concepts.
Error handling
Handling software errors, as well as issues of security or application performance, does not relate to the type of tasks that can be solved by implementing a module - it is impossible to solve all problems related to error handling in one place of the source code. To solve the problem of error handling requires a decentralized approach. For all areas of the program where an error is possible (access to the file system, connection to a remote server, creation of a child process, etc.), it is necessary to prescribe appropriate processing scripts for each possible type of error. This means that it is necessary not only to highlight the problem areas, but also to understand what types of errors can occur in them.
In some cases, it is necessary to transfer the error object from the function in which it originated through the callback function to a higher level, and from it even higher, thus the error “pops up” until it reaches the logical level of the application that is responsible for processing this type of error. At the responsible level, the program can decide: whether to launch the problem operation again, report the error to the user, or write the error information to the log file, etc. You should not always rely on this scheme and transmit errors to higher levels of the hierarchy, since callback functions at high levels do not know anything about the context in which the error transmitted to them occurred. As a result, a situation may arise where at the selected logical level it will be difficult to describe the processing logic corresponding to the error that has occurred.
Highlight possible error handling scenarios:
- Elimination of errors. Sometimes, the error can be eliminated. Suppose an ENOENT error occurred while attempting to write information to a log file. This may mean that the program has been launched for the first time and a log file has not yet been created. In this case, the handler can resolve the error by creating the desired file. Let us give a more interesting example: the program needs to constantly maintain a connection with a certain north (for example, with a database), but during the work a disconnection appeared. In this case, the error handler may reconnect to the database.
- Informing the user and stop processing the request. If you cannot solve the problem, the easiest way is to interrupt the current operation and inform the user about the error. This scenario is applicable in cases where it is known that the reason for the error does not disappear over time. For example, if an error occurred while trying to deserialize the JSON data transmitted by the client, there is no point in trying again with the same data.
- Repeat operation. In case of network related errors, restarting the operation can help. Suppose the program received an error 503 (Service Unavailable error) in response to a request to the remote service, in which case it may be worth repeating the request after a few seconds. It is important to determine the final number of repetitions, as well as the frequency with which attempts should be performed. But do not always rely on this scenario. Suppose a user made a request to a certain service that needed to contact your program to process a request, and your program, in turn, makes a request to another service that responded with error 503. In this case, the best solution would be not to retry , and immediately give the opportunity to handle the error of the original service with which the user works. If each service participating in the request chain makes repeated attempts, the user will wait for the response to his request longer than if only the original service performed them.
- Termination of the program. If an unexpected situation has occurred, the appearance of which is impossible during the normal operation of the program, you should record the error information in the corresponding log file and stop working. This scenario can be used if your program consumes available memory (however, if your program received an ENOMEM error from a child process, then the error can be processed and the program cannot be terminated). Also, this scenario can be applied if your program does not have access rights to the files necessary for the work.
- Record errors in the log file and the continuation of the work. In some cases, there is no need to stop the program even if the error that occurred is unrecoverable. An example is the situation when your program periodically calls a group of remote services through the DNS system, and one of the services “dropped out” of the DNS. In this situation, the program can continue to work with the remaining services. But, nevertheless, it is necessary to record the error in the log file. (There are always exceptions for any rule, if an error occurs a thousand times a second, and you cannot do anything with it, then you do not need to write to the log each time, however, you should periodically log in.)
Programmer Error Handling
There is no right way to handle programmer errors. By definition, if such an error occurs, the program code is incorrect. You can fix the problem only by correcting the code.
There are programmers who believe that in some cases it is possible to restore a program after an error has occurred in such a way that the current operation is interrupted, but the program, nevertheless, continues to work and process other requests. This is not recommended. Taking into account that the programmer’s error puts the program in an unstable state, can you be sure that the error that occurred does not disrupt other requests? If the requests work with the same entities (for example, a server, a socket, connections to a database, etc.), all that remains is to hope that subsequent requests will be processed correctly.
Consider a REST service (implemented, for example, using the
restify module). Suppose that one of the request handlers threw a
RefferenceError
exception because the programmer made a typo in the variable name. If you do not immediately stop the service, there may be a number of problems that can be difficult to track:
- If an entity as a result of a typo turns out to be
null
or undefined
, then subsequent requests, turning to it, will also throw exceptions and will not be processed. - If the function that threw the exception worked with the database, a connection could leak. Each time a similar error occurs, the number of connections using which the service can work with the database will decrease.
- A more complicated situation can occur if postgres is used as the database and the connection is left unclosed during the execution of a transaction. In this case, the "hung" transaction will not allow to clear the old versions of the records that are visible to it. The transaction may remain open for weeks. The size that the table occupies in memory will grow without restrictions, which will cause the processing of subsequent requests to slow down. 4 Of course, this example is quite specific and concerns only postgres, however, it perfectly illustrates that it is dangerous to continue the work of a program that is in an unstable state.
- A connection to a remote service may remain with an unclosed session, as a result of which, the following request may be processed on behalf of the wrong user.
- May remain unclosed socket. By default, Node.js will close an inactive socket in two minutes, but this behavior can be overridden, and if the error repeats, then the number of possible sockets will be exhausted. If you leave the default configuration, it will be hard to track and fix the problem, since an error about an inactive socket occurs with a delay of two minutes.
- There may be a memory leak, which will lead to its overflow and program failure. Or even worse - a leak can complicate the garbage collection process, which is why program performance will suffer. Finding the cause of the problem in this case will be especially difficult.
Given the above, in such situations, the best solution would be to interrupt the program. You can restart your program after it has been interrupted - this approach will automatically restore the stable operation of your service after any errors occur.
The only, but significant, disadvantage of this approach is that all users working with the service will be disabled at the time of restart. Keep in mind the following:
- Crashes caused by a programmer error enter the application in an unstable state. We must strive to ensure that such errors do not occur, their elimination has the highest priority.
- After a restart, requests can be executed correctly, and again lead to an error. It may happen that requests are processed incorrectly, but it is difficult to track down the problem.
- In a well-designed system, regardless of whether an error is caused by a problem with an Internet connection or an error occurred in Node.js, the client program must be able to handle server errors (reconnect, perform repeated requests).
If the program is restarted very often, you should debug the code and fix the errors. The best way to debug is to
save and analyze the kernel snapshot . This approach works both in GNU / Linux-based systems and in illumos-systems, and allows you to view not only the sequence of functions that led to the error, but also the arguments passed to them, as well as the state of other objects that are visible through closures.
3. Patterns of writing functions
Firstly, it is worth noting that it is very important to document your functions in detail. It is necessary to describe what the function returns, what arguments it takes and what errors may occur during the execution of the function. If you do not define the types of possible errors and formulate what they mean, then you will not be able to correctly write a handler.
Throw, callback or EventEmitter?
There are three main ways to return an error from a function:
throw
returns an error synchronously. This means that an exception will occur in the same context in which the function was called. If try / catch is used, the exception will be caught. Otherwise, the program will fail (if, of course, the exception does not catch the domain or the event handler of the global object of the process 'uncaughtException'
, this option will be discussed later).- callback- . callback-
callback(err, results)
, null
. 'error'
EventEmitter, , 'error'
. :
- , . . EventEmitter
'row'
— , "end"
— 'error'
— . - , . ,
'connect'
, 'end'
, 'timeout'
, 'drain'
'close'
. , 'error'
. , , .
Using callback functions and generating events are related to asynchronous ways to return errors. If an asynchronous operation is performed, then one of these methods is implemented, but both are never used at once.So, when to use throw, and when to use callback-functions or events? It depends on two factors:- error type (programmer error or program error),
- the type of function in which the error occurred (asynchronous or synchronous).
Software errors are more characteristic of asynchronous functions. Asynchronous functions take a callback function as an argument, when an error occurs, it is called with an error object as an argument. This approach has proven itself well and is widely used. As an example, see the Node.js module fs
. The event approach is also used, but in more complex cases.Software errors in synchronous functions can occur, as a rule, if the function works with data entered by the user (for example, JSON.parse). In such functions, an error is thrown when an error occurs, more rarely an error object is returned by the return operator.If a function has at least one of the possible errors asynchronous, then all possible errors should be returned from the function using the asynchronous approach. Even if the error occurred in the same context in which the function was called, the error object should be returned asynchronously.There is an important rule: for returning errors in the same function, either a synchronous or asynchronous approach can be implemented, but never both . Then, in order to accept an error from a function, you will need to use either a callback function (or an event handler function 'error'
) or a try / catch construct, but never both. The documentation for the function should indicate which method is applicable to it.Verification of the input arguments as a rule allows you to prevent many errors that programmers make. It often happens that when you call an asynchronous function, they forget to transfer the callback function to it, as a result, in order to understand where an error occurs, the developer has to at least look at the stack of called functions. Therefore, if the function is asynchronous, then first of all, it is important to check whether the callback function is transmitted. If not passed, an exception must be generated. In addition, at the beginning of the function, you should check the types of the arguments passed to it, and also generate an exception if they are incorrect.Recall that programmer errors are not part of the normal process of the program. They should not be caught and processed. Therefore, these recommendations on the immediate throwing of exceptions in case of programmer errors do not contradict the rule formulated above that the same function should not implement both a synchronous and asynchronous approach for returning errors.Considered recommendations are presented in the table:Function example | Type of function | Mistake | Error type | How to return | How to handle |
fs.stat | asynchronous | file not found | software | callback | handler function |
JSON.parse | synchronous | Input Error | software | throw | try / catch |
fs.stat | asynchronous | missing required argument | programmer error | throw | not processed (termination of work) |
The first entry presents the most common example - the asynchronous function. In the second line - an example for a synchronous function, this option is less common. In the third line - a programmer's error, it is desirable that such cases take place only in the process of developing the program.Input Error: Programmer Error or Software Error?
How to distinguish programmer errors from software errors? It is up to you to decide which data the transferred functions are correct and which are not. If arguments that do not meet your requirements are passed to the function, this is a programmer's mistake. If the arguments are correct, but the function cannot currently work with them, then this is a software error.You have to decide how rigorously the argument is tested. Imagine a function connect
that takes an IP address and a callback function as arguments. Suppose that this function was called with an argument that is different in format from the IP address, for example: "bob". Consider what can happen in this case:- , IPv4 , . .
- , , IP- «bob».
Both options satisfy the recommendations considered and it is up to you to decide how strictly to check. The Date.parse function, for example, accepts arguments of various formats, but for good reason. However, for most functions, it is recommended to strictly check the arguments passed. The more vague the criteria for checking arguments, the more difficult the process of debugging the code becomes. As a rule, the stricter the check, the better. And even if in future versions of the program you suddenly soften the criteria for checking inside some function, then you do not risk breaking your code.If the transferred value does not meet the requirements (for example,undefined
or the string has the wrong format), the function should report that the value passed is incorrect and terminate the program. Stopping the work of the program by reporting incorrect arguments, you simplify the process of debugging code for yourself and other programmers.Domains and process.on ('uncaughtException')
Software errors can always be caught by a specific mechanism: through try
/ catch
, in the callback function or event handler 'error'
. Domains and events of a global object process
'uncaughtException'
are often used to reinsure against unforeseen errors that a programmer could make. Given the above considerations, this approach is strongly discouraged.4. Rules for writing functions
When writing functions, follow these rules:
. :
- , .
:
- Error ( ) .
Error , . name
message
, stack
.
- , .
, propertyName propertyValue. remoteIp, . syscall
, , , errno
, . .
:
name
: .message
: . , , .stack
: . V8 , , .
, , message . , .
- , .
, , , callback- , , , . «». . verror .
fetchConfig
, . fetchConfig
. .
1.
1.1
1.1.1 DNS
1.1.2 TCP
1.1.3
1.2.
1.3.
1.4.
2.
, 1.1.2 . fetchConfig
, :
myserver: Error: connect ECONNREFUSED
.
, :
myserver: failed to start up: failed to load configuration: failed to connect to database server: failed to connect to 127.0.0.1 port 1234: connect ECONNREFUSED
, :
myserver: failed to load configuration: connection refused from database at 127.0.0.1 port 1234.
, , — .
, :
- , .
name
, . , , , .- The field
message
during the wrapper can also be changed, but you should not change the message of the original object. Do not perform any actions with the field stack
, as mentioned above, V8 forms an object stack
only when accessing it, and this is a fairly resource-intensive process that can lead to a significant decrease in the performance of your program.
In Joyent, we use the verror module for error wrapping, since it has a minimalistic syntax. At the time of this writing, some of the recommendations reviewed have not been implemented in the module, but it will be finalized.
5. Example
Consider as an example the function that creates a TCP connection at the specified IPv4 address. function connect(ip4addr, tcpPort, timeout, callback) { assert.equal(typeof (ip4addr), 'string', " 'ip4addr' "); assert.ok(net.isIPv4(ip4addr), " 'ip4addr' IPv4 "); assert.equal(typeof (tcpPort), 'number', " 'tcpPort' "); assert.ok(!isNaN(tcpPort) && tcpPort > 0 && tcpPort < 65536, " 'tcpPort' 1 65535"); assert.equal(typeof (timeout), 'number', " 'timeout' "); assert.ok(!isNaN(timeout) && timeout > 0, " 'timeout' "); assert.equal(typeof (callback), 'function'); }
This example is rather primitive, but it illustrates many of the recommendations reviewed:- Arguments, their types, and requirements for them are documented in detail.
- The function checks the arguments passed to it and throws an exception if the arguments do not satisfy the criteria.
- The types of possible errors are documented, as well as the fields they contain.
- The way the function returns errors is specified.
- The returned errors have the “remoteIp” and “remotePort” fields, which will allow the handler to form an error message based on this information.
- The state of the connections after the occurrence of the error is documented: “after the occurrence of the error, the sockets that were opened by the function will be closed”.
It may seem that in the presented example a lot of extra work has been done, however, ten minutes spent on writing documentation can save you or other developers a few hours.6. Summary
- .
- , . , .
- (,
throw
) (callback- ), . , , , callback-, try/catch, . - When writing functions, document in detail the arguments, their types, the requirements imposed on them, as well as the types of possible errors and how the function returns errors (synchronously, using
throw
, or asynchronously, using a callback function or event approach). - The returned error must be an object of the class Error or a class of successor. Expand the error object with new fields to include the necessary error information in the object. If possible, use the common field names provided in the application.
7. Appendix: common error field names
It is strongly recommended that the field names in the table be used to extend error objects. The presented names are used in standard Node.js modules, they should be used in error handlers, as well as when generating error messages. | |
localHostname | DNS- (, , ) |
localIp | IP- (, , ) |
localPort | TCP (, , ) |
remoteHostname | DNS- (, , ) |
remoteIp | IP- (, , ) |
remotePort | (, , ) |
path | , (IPC-) (, , ) |
srcpath | (, ) |
dstpath | (, ) |
hostname | DNS (, , IP-) |
ip | IP- (, , DNS-) |
propertyName | (, , ) |
propertyValue | (, , ) |
syscall | |
errno | errno (, "ENOENT" ) |
1 Novice developers often make a similar mistake. In this example, try
/ catch
and the call to the function that throws the exception will be executed in different contexts due to the asynchrony of the function doSomeAsyncOperation
, so the exception will not be caught.2 In JavaScript, it throw
can work with values of other types, but it is recommended to use objects of the Error class. If other values are used in ThrowStatement, it will be impossible to get the call stack, which led to an error, which complicates debugging of the code.3 These concepts emerged long before Node.js. In Java, an analogue can be considered checked and unchecked exceptions. Assertions are provided in C for working with programmer errors .four The above example may seem too substantive, this is because it is not fictional, we really encountered this problem, it was unpleasant.