Node.js Part 10 tutorial: standard modules, streams, databases, NODE_ENV

This material completes the Node.js tutorial translation series. Today we will talk about the modules os, events and http, discuss work with threads and databases, touch upon the use of Node.js in application development and production.

Node.js os module

The os module provides access to many functions that can be used to obtain information about the operating system and the hardware of the computer running Node.js. This is a standard module, it is not necessary to install it, to work with it from the code it is enough to connect it:
')

 const os = require('os') 

There are several useful features that, in particular, can be useful when working with files.

So, the os.EOL property allows os.EOL to find out the line separator used in the system (end of line). On Linux and macOS, this is \n , on Windows - \r\n .

It should be noted that by mentioning “Linux and macOS” here, we are talking about POSIX-compatible platforms. For the sake of brevity, less popular platforms are not mentioned here.

The os.constants.signals property provides information about constants used to process process signals like SIGHUP , SIGKILL , and so on. Here you can find details about them.

The os.constants.errno property contains constants used for error messages - like EADDRINUSE , EOVERFLOW .

Now consider the basic methods of the os module.

▍os.arch ()

This method returns a string identifying the system architecture, for example - arm , x64 , arm64 .

▍os.cpus ()

Returns information about the processors available in the system. For example, this information might look like this:

 [ { model: 'Intel(R) Core(TM)2 Duo CPU     P8600 @ 2.40GHz',   speed: 2400,   times:    { user: 281685380,      nice: 0,      sys: 187986530,      idle: 685833750,      irq: 0 } }, { model: 'Intel(R) Core(TM)2 Duo CPU     P8600 @ 2.40GHz',   speed: 2400,   times:    { user: 282348700,      nice: 0,      sys: 161800480,      idle: 703509470,      irq: 0 } } ] 

▍os.endianness ()

Returns BE or LE depending on which byte order (Big Engian or Little Endian) was used to compile the Node.js binary file.

▍os.freemem ()

Returns the amount of free system memory in bytes.

▍os.homedir ()

Returns the path to the current user's home directory. For example - '/Users/flavio' .

▍os.hostname ()

Returns the host name.

▍os.loadavg ()

Returns, as an array, data on average load values ​​calculated by the operating system. This information makes sense only in Linux and macOS. It can look like this:

 [ 3.68798828125, 4.00244140625, 11.1181640625 ] 

▍os.networkInterfaces ()

Returns information about the network interfaces available on the system. For example:

 { lo0:  [ { address: '127.0.0.1',      netmask: '255.0.0.0',      family: 'IPv4',      mac: 'fe:82:00:00:00:00',      internal: true },    { address: '::1',      netmask: 'ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff',      family: 'IPv6',      mac: 'fe:82:00:00:00:00',      scopeid: 0,      internal: true },    { address: 'fe80::1',      netmask: 'ffff:ffff:ffff:ffff::',      family: 'IPv6',      mac: 'fe:82:00:00:00:00',      scopeid: 1,      internal: true } ], en1:  [ { address: 'fe82::9b:8282:d7e6:496e',      netmask: 'ffff:ffff:ffff:ffff::',      family: 'IPv6',      mac: '06:00:00:02:0e:00',      scopeid: 5,      internal: false },    { address: '192.168.1.38',      netmask: '255.255.255.0',      family: 'IPv4',      mac: '06:00:00:02:0e:00',      internal: false } ], utun0:  [ { address: 'fe80::2513:72bc:f405:61d0',      netmask: 'ffff:ffff:ffff:ffff::',      family: 'IPv6',      mac: 'fe:80:00:20:00:00',      scopeid: 8,      internal: false } ] } 

▍os.platform ()

Returns information about the platform for which Node.js was compiled. Here are some of the possible return values:

• darwin
• freebsd
• linux
• openbsd
• win32

▍os.release ()

Returns a string identifying the operating system release number.

▍os.tmpdir ()

Returns the path to the directory specified in the system for storing temporary files.

▍os.totalmem ()

Returns the total amount of system memory in bytes.

▍os.type ()

Returns information that identifies the operating system. For example:

• Linux is Linux.
• Darwin - macOS.
• Windows_NT - Windows.

▍os.uptime ()

Returns the system time in seconds since the last restart.

Node.js events module

The events module provides us with the EventEmitter class, which is designed to work with events on the Node.js platform. We already talked a little about this module in the seventh part of this series of materials. Here is the documentation for it. Here we consider the API of this module. Recall that to use it in the code you need, as is usually the case with standard modules, to connect it. After that you need to create a new EventEmitter object. It looks like this:

 const EventEmitter = require('events') const door = new EventEmitter() 

The object of the EventEmitter class uses standard mechanisms, in particular, the following events:

• newListener - This event is newListener when an event handler is added.
• removeListener - called when removing a handler.

Consider the most useful methods of objects of the EventEmitter class (a similar object in the method names is designated as emitter ).

▍emitter.addListener ()

The alias for the emitter.on() method.

▍emitter.emit ()

Generates an event. It synchronously calls all event handlers in the order in which they were registered.

▍emitter.eventNames ()

Returns an array that contains registered events.

▍emitter.getMaxListeners ()

Returns the maximum number of handlers that can be added to an object of the EventEmitter class. The default is 10. If necessary, this parameter can be increased or decreased using the setMaxListeners() method.

▍emitter.listenerCount ()

Returns the number of event handlers whose name is passed to this method as a parameter:

 door.listenerCount('open') 

▍emitter.listeners ()

Returns an array of event handlers for the corresponding event, whose name is passed to this method:

 door.listeners('open') 

▍emitter.off ()

The alias for the emitter.removeListener() method, which appeared in Node 10.

▍emitter.on ()

Register a callback that is called when an event is generated. Here's how to use it:

 door.on('open', () => { console.log('Door was opened') }) 

▍emitter.once ()

It registers a callback that is called only once - at the first occurrence of the event for which this callback is registered. For example:

 const EventEmitter = require('events') const ee = new EventEmitter() ee.once('my-event', () => { //         }) 

▍emitter.prependListener ()

When registering a handler using the on() or addListener() methods, this handler is added to the end of the handler queue and called to process the corresponding event last. When using the prependListener() method, the handler is added to the top of the queue, which causes it to be called first to handle the event.

▍emitter.prependOnceListener ()

This method is similar to the previous one. Namely, when a handler intended for a single call is registered using the once() method, it is the last in the handler queue and the last is called. The prependOnceListener() method allows you to add such a handler to the front of the queue.

▍emitter.removeAllListeners ()

This method removes all handlers for the specified event registered in the corresponding object. Use it like this:

 door.removeAllListeners('open') 

▍emitter.removeListener ()

Removes the specified handler to pass to this method. In order to save a handler for later deletion, the corresponding callback can be assigned to a variable. It looks like this:

 const doSomething = () => {} door.on('open', doSomething) door.removeListener('open', doSomething) 

▍emitter.setMaxListeners ()

This method allows you to specify the maximum number of handlers that can be added to a single event in an instance of the EventEmitter class. By default, as already mentioned, you can add up to 10 handlers for a specific event. This value can be changed. Use this method as follows:

 door.setMaxListeners(50) 

Node.js http module

In the eighth part of this series of materials, we already talked about the standard Node.js http module. It provides the developer with mechanisms for creating HTTP servers. It is the main module used for solving network communication problems in Node.js. You can connect it in code like this:

 const http = require('http') 

It includes properties, methods and classes. Let's talk about them.

▍Properties

http.METHODS

This property lists all supported HTTP methods:

 > require('http').METHODS [ 'ACL', 'BIND', 'CHECKOUT', 'CONNECT', 'COPY', 'DELETE', 'GET', 'HEAD', 'LINK', 'LOCK', 'M-SEARCH', 'MERGE', 'MKACTIVITY', 'MKCALENDAR', 'MKCOL', 'MOVE', 'NOTIFY', 'OPTIONS', 'PATCH', 'POST', 'PROPFIND', 'PROPPATCH', 'PURGE', 'PUT', 'REBIND', 'REPORT', 'SEARCH', 'SUBSCRIBE', 'TRACE', 'UNBIND', 'UNLINK', 'UNLOCK', 'UNSUBSCRIBE' ] 

http.STATUS_CODES

This contains the HTTP status codes and their descriptions:

 > require('http').STATUS_CODES { '100': 'Continue', '101': 'Switching Protocols', '102': 'Processing', '200': 'OK', '201': 'Created', '202': 'Accepted', '203': 'Non-Authoritative Information', '204': 'No Content', '205': 'Reset Content', '206': 'Partial Content', '207': 'Multi-Status', '208': 'Already Reported', '226': 'IM Used', '300': 'Multiple Choices', '301': 'Moved Permanently', '302': 'Found', '303': 'See Other', '304': 'Not Modified', '305': 'Use Proxy', '307': 'Temporary Redirect', '308': 'Permanent Redirect', '400': 'Bad Request', '401': 'Unauthorized', '402': 'Payment Required', '403': 'Forbidden', '404': 'Not Found', '405': 'Method Not Allowed', '406': 'Not Acceptable', '407': 'Proxy Authentication Required', '408': 'Request Timeout', '409': 'Conflict', '410': 'Gone', '411': 'Length Required', '412': 'Precondition Failed', '413': 'Payload Too Large', '414': 'URI Too Long', '415': 'Unsupported Media Type', '416': 'Range Not Satisfiable', '417': 'Expectation Failed', '418': 'I\'ma teapot', '421': 'Misdirected Request', '422': 'Unprocessable Entity', '423': 'Locked', '424': 'Failed Dependency', '425': 'Unordered Collection', '426': 'Upgrade Required', '428': 'Precondition Required', '429': 'Too Many Requests', '431': 'Request Header Fields Too Large', '451': 'Unavailable For Legal Reasons', '500': 'Internal Server Error', '501': 'Not Implemented', '502': 'Bad Gateway', '503': 'Service Unavailable', '504': 'Gateway Timeout', '505': 'HTTP Version Not Supported', '506': 'Variant Also Negotiates', '507': 'Insufficient Storage', '508': 'Loop Detected', '509': 'Bandwidth Limit Exceeded', '510': 'Not Extended', '511': 'Network Authentication Required' } 

http.globalAgent

This property points to a global instance of the http.Agent class. It is used to manage connections. It can be considered a key component of the Node.js HTTP subsystem. We'll talk more about the http.Agent class below.

▍ Methods

http.createServer ()

Returns a new instance of the http.Server class. Here is how to use this method to create an HTTP server:

 const server = http.createServer((req, res) => { //      }) 

http.request ()

Allows you to perform an HTTP request to the server, creating an instance of the class http.ClientRequest .

http.get ()

This method is similar to http.request() , but it automatically sets the HTTP method to GET and automatically calls a command like req.end() .

▍Classes

The HTTP module provides 5 classes — Agent , ClientRequest , Server , ServerResponse and IncomingMessage . Consider them.

http.Agent

The global instance of the http.Agent class created by Node.js is used to manage connections. It is used as the default for all HTTP requests and provides for queuing requests and reusing sockets. In addition, it supports a pool of sockets, which ensures high performance of the network subsystem Node.js. If necessary, you can create your own http.Agent object.

http.ClientRequest

An object of the http.ClientRequest class, which is a running request, is created when the http.request() or http.get() methods are called. When a response to a request is received, a response event is triggered, in which the response is sent — an instance of http.IncomingMessage . The data obtained after the execution of the request can be processed in two ways:

• You can call the response.read() method.
• In the response event handler, you can configure a listener for the data event, which allows you to work with streaming data.

http.Server

Instances of this class are used to create servers using the http.createServer() command. After we have a server object, we can use its methods:

• The listen() method is used to start the server and organize waiting and processing incoming requests.
• The close() method stops the server.

http.ServerResponse

This object is created by the http.Server class and is passed as the second parameter to the request event when it occurs. Usually, similar objects in the code are assigned the name res :

 const server = http.createServer((req, res) => { //res -   http.ServerResponse }) 

In such handlers, after the server’s response is ready to be sent to the client, the end() method is called, which completes the response. This method must be called after the completion of the formation of each response.

Here are the methods that are used to work with HTTP headers:

• getHeaderNames() - returns a list of names of installed headers.
• getHeaders() - returns a copy of the installed HTTP headers.
• setHeader('headername', value) - sets the value for the specified header.
• getHeader('headername') - returns the installed header.
• removeHeader('headername') - removes the installed header.
• hasHeader('headername') - returns true if the response already has a header, the name of which is transferred to this method.
• headersSent() - returns true if headers have already been sent to the client.

After processing the headers, they can be sent to the client by calling the response.writeHead() method, which, as the first parameter, accepts a status code. As the second and third parameters, you can send a message corresponding to the status code, and headers.

To send data to the client in the response body use the write() method. It sends buffered data to the HTTP response stream.

If headers have not yet been set with the response.writeHead() command, the headers with the status code and the message specified in the request will be sent first. You can set their values ​​by setting values ​​for the statusCode and statusMessage :

 response.statusCode = 500 response.statusMessage = 'Internal Server Error' 

http.IncomingMessage

The object class http.IncomingMessage is created in the course of the following mechanisms:

• http.Server - while handling the request event.
• http.ClientRequest - while handling the response event.

It can be used to work with response data. Namely:

• The statusCode and statusMessage properties are used to find out the response status code and the corresponding message.
• The response headers can be viewed by referring to the headers or rawHearders (for a list of raw headers).
• The request method can be found using the method property.
• You can httpVersion version of HTTP you are using with the httpVersion property.
• To get the URL, use the url property.
• The socket property allows you to get the net.Socket object associated with the connection.

Response data is presented as a stream as the http.IncomingMessage object implements the Readable Stream interface.

Work with streams in Node.js

Threads are one of the fundamental concepts used in Node.js applications. Flows are tools that allow you to read and write files, organize networking of systems, and, in general, effectively implement data exchange operations.

The concept of threads is not unique to Node.js. They appeared in the OS of the Unix family decades ago. In particular, programs can interact with each other by transmitting data streams using pipelines (using the pipeline symbol - | ).

If we imagine, say, reading a file without using streams, then, in the course of executing the corresponding command, the contents of the file will be completely read into memory, after which it will be possible to work with these contents.

Through the use of a streaming mechanism, files can be read and processed in parts, which eliminates the need to store large amounts of data in memory.

The Node.js stream module is the foundation on which all APIs that support streaming are built.

▍About Strengths Strengths

Streams, in comparison with other data processing methods, have the following advantages:

• Efficient use of memory. Working with a stream does not involve storing in memory large amounts of data loaded there in advance, before it becomes possible to process them.
• Save time. The data obtained from the stream can be processed much faster than in the case when, in order to start processing them, you have to wait for them to be fully loaded.

▍Example work with threads

A traditional example of working with streams demonstrates reading a file from disk.

First, consider the code in which the threads are not used. The standard module Node.js fs allows you to read the file, after which it can be sent via the HTTP protocol in response to a request received by the HTTP server:

 const http = require('http') const fs = require('fs') const server = http.createServer(function (req, res) { fs.readFile(__dirname + '/data.txt', (err, data) => {   res.end(data) }) }) server.listen(3000) 

The readFile() method used here allows you to read the entire file. When the reading is completed, it calls the appropriate callback.

The res.end(data) method called in the callback sends the contents of the file to the client.

If the file size is large, then this operation will take a lot of time. Here is the same example rewritten using streams:

 const http = require('http') const fs = require('fs') const server = http.createServer((req, res) => { const stream = fs.createReadStream(__dirname + '/data.txt') stream.pipe(res) }) server.listen(3000) 

Instead of waiting for the moment when the file is completely read, we begin to transfer its data to the client immediately after the first portion of this data is ready to be sent.

PipeMethod pipe ()

In the previous example, we used a construction of the form stream.pipe(res) , in which the file stream method pipe() called. This method takes data from its source and sends it to its destination.

It is called for the stream representing the data source. In this case, it is the file stream that is sent to the HTTP response.

The return value of the pipe() method is the target stream. This is very convenient because it allows you to chain multiple calls to the pipe() method:

 src.pipe(dest1).pipe(dest2) 

This is equivalent to this design:

 src.pipe(dest1) dest1.pipe(dest2) 

▍API Node.js using threads

Threads are a useful mechanism, with the result that many Node.js core modules provide standard threading capabilities. Here are some of them:

• process.stdin - returns the stream connected to stdin .
• process.stdout - returns the stream connected to stdout .
• process.stderr - returns the stream connected to stderr .
• fs.createReadStream() - creates a readable stream for working with a file.
• fs.createWriteStream() - creates a writable stream for working with a file.
• net.connect() - initiates a connection based on the stream.
• http.request() - returns an instance of the class http.ClientRequest , which provides access to the recorded stream.
• zlib.createGzip() - compresses data using the gzip algorithm and sends them to the stream.
• zlib.createGunzip() - performs decompression of the gzip stream.
• zlib.createDeflate() - compresses data using the deflate algorithm and sends them to the stream.
• zlib.createInflate() - performs decompression of the deflate stream.

▍Different stream types

There are four types of threads:

• A readable stream is a stream from which data can be read. You cannot write data to such a stream. When data flows into such a stream, they are buffered until the data consumer starts reading them.
• Writable Stream ( Writable ) is a stream to which data can be sent. You cannot read data from it.
• Duplex stream ( Duplex ) - in such a stream, you can and send data and read them from it. Essentially, it is a combination of a stream for reading and a stream for writing.
• Transforming stream ( Transform ) - such streams are similar to duplex streams, the difference is that what comes to the input of these streams converts what can be read from them.

▍Create a stream to read

The stream for reading can be created and initialized using the stream module features:

 const Stream = require('stream') const readableStream = new Stream.Readable() 

, :

 readableStream.push('hi!') readableStream.push('ho!') 

▍

Writable _write() . :

 const Stream = require('stream') const writableStream = new Stream.Writable() 

_write() :

 writableStream._write = (chunk, encoding, next) => {   console.log(chunk.toString())   next() } 

, :

 process.stdin.pipe(writableStream) 

▍

, , :

 const Stream = require('stream') const readableStream = new Stream.Readable() const writableStream = new Stream.Writable() writableStream._write = (chunk, encoding, next) => {   console.log(chunk.toString())   next() } readableStream.pipe(writableStream) readableStream.push('hi!') readableStream.push('ho!') readableStream.push(null) 

readableStream.push(null) .

, readable :

 readableStream.on('readable', () => { console.log(readableStream.read()) }) 

▍

write() :

 writableStream.write('hey!\n') 

▍ ,

, , end() :

 writableStream.end() 

. , , .

MySQL Node.js

MySQL . Node.js , MySQL-, — , .

mysqljs/mysql . , , 12000 GitHub. , MySQL-.

▍

:

 npm install mysql 

▍

:

 const mysql = require('mysql') 

:

 const options = { user: 'the_mysql_user_name', password: 'the_mysql_user_password', database: 'the_mysql_database_name' } const connection = mysql.createConnection(options) 

:

 connection.connect(err => { if (err) {   console.error('An error occurred while connecting to the DB')   throw err } } 

▍

options :

 const options = { user: 'the_mysql_user_name', password: 'the_mysql_user_password', database: 'the_mysql_database_name' } 

. — :

• host — , MySQL-, — localhost .
• port — , — 3306 .
• socketPath — Unix .
• debug — , .
• trace — , .
• ssl — SSL- .

▍ SELECT

SQL- . query , . — , . . :

 connection.query('SELECT * FROM todos', (error, todos, fields) => { if (error) {   console.error('An error occurred while executing the query')   throw error } console.log(todos) }) 

, :

 const id = 223 connection.query('SELECT * FROM todos WHERE id = ?', [id], (error, todos, fields) => { if (error) {   console.error('An error occurred while executing the query')   throw error } console.log(todos) }) 

, , :

 const id = 223 const author = 'Flavio' connection.query('SELECT * FROM todos WHERE id = ? AND author = ?', [id, author], (error, todos, fields) => { if (error) {   console.error('An error occurred while executing the query')   throw error } console.log(todos) }) 

▍ INSERT

INSERT . , :

 const todo = { thing: 'Buy the milk' author: 'Flavio' } connection.query('INSERT INTO todos SET ?', todo, (error, results, fields) => { if (error) {   console.error('An error occurred while executing the query')   throw error } }) 

, , auto_increment , results.insertId :

 const todo = { thing: 'Buy the milk' author: 'Flavio' } connection.query('INSERT INTO todos SET ?', todo, (error, results, fields) => { if (error) {   console.error('An error occurred while executing the query')   throw error }} const id = results.resultId console.log(id) ) 

▍

— end() :

 connection.end() 

.

-

Node.js -.

Node.js . , -, NODE_ENV :

 NODE_ENV=production 

. Linux, , :

 export NODE_ENV=production 

, , .bash_profile ( Bash), .

, :

 NODE_ENV=production node app.js 

Node.js. NODE_ENV production :

• .
• .

, Pug — , Express, , NODE_ENV production . Express, , . - . .

Express . , , NODE_ENV :

 app.configure('development', () => { //... }) app.configure('production', () => { //... }) app.configure('production', 'staging', () => { //... }) 

, :

 app.configure('development', () => { app.use(express.errorHandler({ dumpExceptions: true, showStack: true })); }) app.configure('production', () => { app.use(express.errorHandler()) }) 

▍

, , Node.js , . , , , Node.js, - , Node.js.

Dear readers! , Node.js, , .