Expressive javascript: Node.js

Content

The student asked: “Programmers used only simple computers and programmed without languages, but they made excellent programs. Why do we use complex computers and programming languages? ” Fu-Tzu replied: "The builders used only sticks and clay beforehand, but they made beautiful huts."

Master Yuan-Ma, "Programming Book"
')
Currently you have learned JavaScript and used it in a single environment: in the browser. In this and the next chapter, we will briefly introduce Node.js, a program that allows you to apply JavaScript skills outside the browser. With it, you can write everything from command line utilities to dynamic HTTP servers.

These chapters are about learning the important ideas that make up Node.js and are designed to give you enough information so you can write useful programs in this environment. They are not trying to be comprehensive Node directories.

You could write and execute the code from the previous chapters directly in the browser, but the code from this chapter is written for Node and will not work in the browser.

If you want to immediately run the code from this chapter, start by installing Node from the nodejs.org site for your OS. Also on this site you will find documentation on Node and its built-in modules.

Introduction

One of the most difficult problems when writing systems that communicate over a network is input and output processing. Reading and writing data to and from the network, to disk, and other devices. Moving data takes time, and proper planning of these actions can greatly affect the system response time for a user or network requests.

In the traditional input and output processing method, it is assumed that a function, for example, readFile, starts reading a file and returns only when the file is completely read. This is called synchronous I / O, input / output.

Node was designed to facilitate and simplify the use of asynchronous I / O. We have already met with asynchronous interfaces, such as the XMLHttpRequest object of the browser, discussed in Chapter 17. This interface allows the script to continue working while the interface is doing its own, and calls the end callback function. So the whole I / O works in Node.

JavaScript fits easily into a Node type system. This is one of the few languages in which the I / O system is not embedded. Therefore, JavaScript is easily integrated into a rather eccentric approach to I / O in Node and, as a result, does not generate two different input and output systems. In 2009, when developing Node, people already used I / O in a browser based on callbacks, so the community around the language was familiar with the asynchronous programming style.

Asynchrony

I will try to illustrate the difference in synchronous and asynchronous approaches in I / O with a small example where the program should get two resources from the Internet, and then do something with the data.

In a synchronous environment, the obvious way to solve a problem is to make requests sequentially. This method has a minus - the second request will begin only after the end of the first. The total time will be no less than the sum of the time to process the two requests. This is an inefficient use of a computer that will be idle most of the time while data is being transmitted over the network.

The solution to the problem in the synchronous system is the launch of additional program execution control flows (we have already discussed them in Chapter 14). The second thread can start the second query, and then both threads will wait for the return of the result, after which they will be synchronized again to bring work into one result.

In the diagram, the bold lines indicate the time of normal program operation, and the thin lines indicate the I / O waiting time. In a synchronous model, the time spent on I / O is included in the time schedule of each of the streams. In asynchronous, starting an I / O action causes a timeline to branch. The thread that started the I / O continues to execute, and the I / O is executed parallel to it, at the end of the work making a callback to the function.

Program flow for synchronous and asynchronous I / O

Another way to express this difference: in a synchronous model, waiting for the end of an I / O is implicit, and in asynchronous, it is explicit, and is under our direct control. But asynchrony works both ways. Using it, it is easier to express programs that do not work on the principle of a straight line, but it becomes more difficult to express straight-line programs.

In chapter 17, I already touched on the fact that callbacks add a lot of noise and make the program less orderly. Whether such an approach is generally a good idea is a controversial issue. In any case, it takes time to get used to it.

But for a javascript-based system, I would say that using asynchrony with callbacks makes sense. One of the strengths of JavaScript is simplicity, and attempts to add several threads to the program would lead to a strong complication. Although callbacks do not make the code simple, their idea is very simple and at the same time strong enough to write high-performance web servers.

Node command

When Node.js is installed on your system, you have a program called node that runs the JavaScript files. Suppose you have a hello.js file with the following code:

var message = "Hello world"; console.log(message);

You can execute your program from the command line:

 $ node hello.js Hello world

The console.log method in Node works the same as in the browser. Displays a piece of text. But in Node, the text is output to standard output, not to the JavaScript console in the browser.

If you run node without a file, it will give you a query string in which you can write JavaScript code and get the result.

 $ node > 1 + 1 2 > [-1, -2, -3].map(Math.abs) [1, 2, 3] > process.exit(0) $

The process variable, like the console, is available globally in Node. It provides several ways to inspect and manipulate the program. The exit method terminates the process, and you can transfer to it the code of the end-of-program status, which informs the program that launched the node (in this case, the program shell) whether the program completed successfully (zero code) or with an error (any other number).

To access the command line arguments passed to the program, you can read an array of process.argv strings. It also includes the name of the node command and the name of your script, so the argument list starts at index 2. If the showargv.js file contains only the console.log statement (process.argv), you can run it like this:

 $ node showargv.js one --and two ["node", "/home/marijn/showargv.js", "one", "--and", "two"]

All standard JavaScript global variables - Array, Math, JSON, are also in the Node environment. But there is no functionality associated with the browser, for example, document or alert.

The global scope object, which in the browser is called window, in Node has a more meaningful name global.

Modules

In addition to the few variables mentioned, like console and process, Node holds little functionality in the global scope. To access the rest of the built-in capabilities, you need to contact the module system.

The CommonJS system, based on the require function, was described in Chapter 10. Such a system is built into Node and is used to load everything from embedded modules and downloaded libraries to files that are part of your program.

When calling the require Node you need to convert the specified string to the file name. Paths beginning with "/", "./" or "../" are converted to paths relative to the current one. "./" means the current directory, "../" is the directory above, and "/" is the root directory of the file system. If you request "./world/world" from the file /home/marijn/elife/run.js, Node will try to download the file /home/marijn/elife/world/world.js. The .js extension can be omitted.

When a string is passed that does not look like a relative or absolute path, it is assumed that this is either a built-in module or a module installed in the node_modules directory. For example, require ("fs") will give you a built-in module for working with the file system, and require ("elife") will try to load the library from node_modules / elife /. The typical method of installing libraries is with NPM, which I will return to later.

To demonstrate, let's make a simple project of two files. The first one is called main.js, and it will define a script called from the command line, designed to distort strings.

 var garble = require("./garble"); //   2        var argument = process.argv[2]; console.log(garble(argument));

The file garble.js defines string distortion, which can be used both by the command line program specified earlier and by other scripts that need direct access to the garble function.

 module.exports = function(string) { return string.split("").map(function(ch) { return String.fromCharCode(ch.charCodeAt(0) + 5); }).join(""); };

Replacing module.exports instead of adding properties to it allows us to export a specific value from a module. In this case, the result of the query of our module is the distortion function itself.

The function splits a string into characters using split with an empty string, and then replaces all characters with others, with a code 5 units higher. It then connects the result back to the string.

Now we can call our tool:

 $ node main.js JavaScript Of{fXhwnuy

Installation via NPM

NPM, incidentally mentioned in Chapter 10, is an online repository of JavaScript modules, many of which are written specifically for Node. When you install Node on a computer, you get the npm program, which gives a convenient interface to this repository.

For example, one of the NPM modules is called a figlet, and it converts the text into “ASCII art”, drawings made up of text characters. Here is how to install it:

 $ npm install figlet npm GET https://registry.npmjs.org/figlet npm 200 https://registry.npmjs.org/figlet npm GET https://registry.npmjs.org/figlet/-/figlet-1.0.9.tgz npm 200 https://registry.npmjs.org/figlet/-/figlet-1.0.9.tgz figlet@1.0.9 node_modules/figlet $ node > var figlet = require("figlet"); > figlet.text("Hello world!", function(error, data) { if (error) console.error(error); else console.log(data); }); _ _ _ _ _ _ _ | | | | ___| | | ___ __ _____ _ __| | __| | | | |_| |/ _ \ | |/ _ \ \ \ /\ / / _ \| '__| |/ _` | | | _ | __/ | | (_) | \ VV / (_) | | | | (_| |_| |_| |_|\___|_|_|\___/ \_/\_/ \___/|_| |_|\__,_(_)

After running npm install, NPM will create the node_modules directory. Inside it will be the figlet directory containing the library. When we run a node and call require ("figlet"), the library is loaded and we can call its text method to print large beautiful letters.

Interestingly, instead of simply returning a string that contains large letters, figlet.text takes a callback function, to which it passes the result. It also passes there another argument, error, which in the case of an error will contain an error object, and in case of success, null.

This principle of operation adopted in Node. To create letters, figlet must read a file from a disk containing letters. Reading a file is an asynchronous operation in Node, so figlet.text cannot return the result immediately. Asynchrony is contagious - any function that calls asynchronous itself becomes asynchronous.

NPM is more than just npm install. It reads package.json files containing information in JSON format about a program or library, in particular, on which libraries it is based. Running npm install in the directory containing the file automatically installs all dependencies, and in turn their dependencies. The npm tool is also used to host libraries in the NPM online storage so that other people can find, download, and use them.

We will no longer go into the details of using NPM. Contact npmjs.org for documentation on finding libraries.

Module file system

One of the most popular built-in Node modules is the “fs” module, which means “file system”. The module provides functionality for working with files and directories.

For example, there is a readFile function that reads a file and makes a callback with the contents of the file.

 var fs = require("fs"); fs.readFile("file.txt", "utf8", function(error, text) { if (error) throw error; console.log("    :", text); });

The second argument readFile sets the character encoding in which to convert the contents of the file into a string. Text can be converted to binary data in different ways, but the newest one is UTF-8. If you have no reason to believe that the file contains text in a different encoding, you can safely pass the "utf8" parameter. If you did not specify a character encoding, Node will give you binary data in the form of a Buffer object, not a string. This is a massive object containing the bytes from the file.

 var fs = require("fs"); fs.readFile("file.txt", function(error, buffer) { if (error) throw error; console.log("   ", buffer.length, " .", " :", buffer[0]); });

A similar function, writeFile, is used to write a file to disk.

 var fs = require("fs"); fs.writeFile("graffiti.txt", "  Node ", function(err) { if (err) console.log("  ,   :", err); else console.log(" .  ."); });

You do not need to set the encoding here, because writeFile assumes that if it was given a string for writing, and not a Buffer object, then it should be output as text with the default encoding UTF-8.

The “fs” module contains a lot of useful information: the readdir function returns a list of directory files as an array of strings, stat returns information about the file, rename renames the file, unlink deletes, and so on. See the nodejs.org documentation

Many fs functions have both a synchronous and asynchronous version. For example, there is a synchronous version of the readFile function called readFileSync.

 var fs = require("fs"); console.log(fs.readFileSync("file.txt", "utf8"));

Synchronous functions are easier and more useful for simple scripts, where the additional speed of the asynchronous method is not important. But note - for the duration of the synchronous operation, your program stops completely. If she needs to respond to user input or other programs over the network, shutting down synchronous I / O leads to annoying delays.

HTTP module

Another main module is “http”. It provides functionality for creating HTTP servers and HTTP requests.

Here is all you need to run a simple HTTP server:

 var http = require("http"); var server = http.createServer(function(request, response) { response.writeHead(200, {"Content-Type": "text/html"}); response.write("<h1>!</h1><p>  <code>" + request.url + "</code></p>"); response.end(); }); server.listen(8000);

By running the script on your machine, you can direct the browser to localhost : 8000 / hello, thus creating a request to the server. It will respond with a small HTML page.

The function passed as an argument to createServer is called each time you try to connect to the server. The request and response variables are objects that represent input and output data. The first contains information on the request, for example, the url property contains the URL of the request.

To send something back, use the methods of the response object. The first, writeHead, writes response headers (see chapter 17). You give it a status code (in this case, 200 for “OK”) and an object containing header values. Here we inform the client that he must wait for the HTML document.

Then comes the response body (the document itself), sent via response.write. This method can be called several times if you want to send a response in chunks, for example, sending stream data as it arrives. Finally, response.end signals the end of the response.

The server.listen call forces the server to listen to requests on port 8000. Therefore, you need to go to localhost: 8000 in the browser, and not just to localhost (where the default port is 80).

To stop such a script Node, which does not end automatically, because it is waiting for the following events (in this case, connections), you must press Ctrl-C.

A real web server does much more than this example. He looks at the request method (the method property) to understand what action the client is trying to perform, and at the request URL to see which resource this action should be performed on. Next you will see a more advanced version of the server.

To make an HTTP client, we can use the “http” request module function.

 var http = require("http"); var request = http.request({ hostname: "eloquentjavascript.net", path: "/20_node.html", method: "GET", headers: {Accept: "text/html"} }, function(response) { console.log("    ", response.statusCode); }); request.end();

The first request argument configures the request, explaining to Node which server we will communicate with, which way the request will have, which method to use, etc. The second is the function. which will need to call at the end of the request. It is passed the response object, which contains all the information on the response - for example, a status code.

Like the server's response object, the object returned by request allows you to transfer data using the write method and end the request with the end method. The example does not use write, because GET requests should not contain data in the body.

For requests for secure URLs (HTTPS), Node offers an https module, which has its own request function, similar to http.request.

Streams

We saw two examples of streams in the HTTP examples — a response object to which the server can write, and a request object, which is returned from http.request

Writeable streams are a popular concept in Node interfaces. All threads have a write method that can be passed a string or a Buffer object. The end method closes the stream, and if there is an argument, will output a piece of data before closing. Both methods can be given a callback function through an additional argument, which they will call at the end of the recording or closing the stream.

It is possible to create a stream showing a file using the fs.createWriteStream function. You can then use the write method to write to the file bit by bit, and not entirely, as in fs.writeFile.

Readable threads will be a bit more complicated. Both the request variable passed to the callback function in the HTTP server and the response variable passed to the HTTP client are readable threads. (The server reads the request and then writes responses, and the client writes the request and reads the response). Reading from the stream is done through event handlers, not through methods.

Objects that create events in Node have an on method that is similar to the addEventListener browser method. You give it an event name and a function, and it registers this function so that it will be called immediately when an event occurs.

Reading threads have “data” and “end” events. The first occurs when the data is received, the second - at the end. This model is suitable for streaming data that can be processed immediately, even if not the entire document is received. The file can be read as a stream through fs.createReadStream.

The following code creates a server that reads the request bodies and sends them back in response as a text in capital letters.

 var http = require("http"); http.createServer(function(request, response) { response.writeHead(200, {"Content-Type": "text/plain"}); request.on("data", function(chunk) { response.write(chunk.toString().toUpperCase()); }); request.on("end", function() { response.end(); }); }).listen(8000);

The chunk variable passed to the data handler will be a binary Buffer, which can be converted to a string by calling its toString method, which decodes it from the default encoding (UTF-8).

The following code, running simultaneously with the server, sends a request to the server and displays the response received:

 var http = require("http"); var request = http.request({ hostname: "localhost", port: 8000, method: "POST" }, function(response) { response.on("data", function(chunk) { process.stdout.write(chunk.toString()); }); }); request.end("Hello server");

The example is written in process.stdout (the standard output of the process, which is a writable stream), and not in console.log. We cannot use console.log, since it adds an extra line feed after each piece of code - this is not necessary here.

Simple file server

Let's combine our new knowledge about HTTP servers and working with the file system, and put a bridge between them: an HTTP server that provides remote access to files. This server has many options for use. It allows web applications to store and share data, or can give a group of people access to a set of files.

When we treat files as HTTP resources, the GET, PUT, and DELETE methods can be used to read, write, and delete files. We will interpret the path in the request as the path to the file.

We do not need to open access to the entire file system, so we will interpret these paths as given relative to the root directory, and this will be the script startup directory. If I start the server from / home / marijn / public / (or C: \ Users \ marijn \ public \ on Windows), then the request for /file.txt should point to /home/marijn/public/file.txt (or C : \ Users \ marijn \ public \ file.txt).

We will build the program gradually using the methods object to store functions that process different HTTP methods.

 var http = require("http"), fs = require("fs"); var methods = Object.create(null); http.createServer(function(request, response) { function respond(code, body, type) { if (!type) type = "text/plain"; response.writeHead(code, {"Content-Type": type}); if (body && body.pipe) body.pipe(response); else response.end(body); } if (request.method in methods) methods[request.method](urlToPath(request.url), respond, request); else respond(405, "Method " + request.method + " not allowed."); }).listen(8000);

This code will launch a server returning 405 errors — this code is used to indicate that the requested method is not supported by the server.

The respond function is passed to functions that handle different methods, and it acts as a callback to end the request. It accepts the HTTP status code, body, and possibly the content type. If the transferred body is a readable stream, it will have a pipe method that is used to pass the readable stream to the recordable one. If not, it is assumed that this is either null (the body is empty), or a string, and then it is passed directly to the end method of the response.

To get the path from the URL in the request, the urlToPath function, using the built-in Node module “url”, parses the URL. It takes a pathname, something like /file.txt, decodes it to remove% 20 escape codes, and inserts a dot at the beginning to get a path relative to the current directory.

 function urlToPath(url) { var path = require("url").parse(url).pathname; return "." + decodeURIComponent(path); }

Does it seem to you that the urlToPath function is insecure? You're right. Let's return to this issue in the exercises.

We will arrange the GET method so that it returns a list of files when reading a directory, and the contents of a file when reading a file.

Backfill question - what type of Content-Type header should we return when reading a file. Since there can be anything in the file, the server cannot simply return the same type for all. But NPM can help with that. The mime module (file type content indicators like text / plain are also called MIME types) knows the correct type for a huge number of file extensions.

By running the following npm command in the directory where the server script lives, you can use require ("mime") to query the type library.

 $ npm install mime npm http GET https://registry.npmjs.org/mime npm http 304 https://registry.npmjs.org/mime mime@1.2.11 node_modules/mime

When the requested file does not exist, the correct error code for this case is 404. We will use fs.stat to return information on the file to find out if there is such a file, and if this is not a directory.

 methods.GET = function(path, respond) { fs.stat(path, function(error, stats) { if (error && error.code == "ENOENT") respond(404, "File not found"); else if (error) respond(500, error.toString()); else if (stats.isDirectory()) fs.readdir(path, function(error, files) { if (error) respond(500, error.toString()); else respond(200, files.join("\n")); }); else respond(200, fs.createReadStream(path), require("mime").lookup(path)); }); };

Because disk requests take time, fs.stat works asynchronously. When the file does not exist, fs.stat will pass an error object with the "ENOENT" code property to the callback function. It would be great if Node defined different types of errors for different errors, but there is no such thing. Instead, it produces tangled Unix-style codes.

We will issue all unexpected errors with code 500, denoting that there is a problem on the server - in contrast to codes starting with 4, which indicate a problem with the request. In some situations it will not be quite neat, but for a small sample program this will be enough.

The stats object returned by fs.stat tells us everything about the file. For example, size - file size, mtime - modification date. Here we need to find out if this is a directory or a regular file — the isDirectory method will tell us.

fs.readdir, , . fs.createReadStream , , “mime” .

DELETE :

 methods.DELETE = function(path, respond) { fs.stat(path, function(error, stats) { if (error && error.code == "ENOENT") respond(204); else if (error) respond(500, error.toString()); else if (stats.isDirectory()) fs.rmdir(path, respondErrorOrNothing(respond)); else fs.unlink(path, respondErrorOrNothing(respond)); }); };

, , 204 . , , , . HTTP – , .

 function respondErrorOrNothing(respond) { return function(error) { if (error) respond(500, error.toString()); else respond(204); }; }

When the HTTP response contains no data, you can use the status code 204 (“no content”). Since we need to provide callback functions that either report errors or return a 204 response in different situations, I wrote a special function respondErrorOrNothing that creates such a callback.

Here is the PUT request handler:

 methods.PUT = function(path, respond, request) { var outStream = fs.createWriteStream(path); outStream.on("error", function(error) { respond(500, error.toString()); }); outStream.on("finish", function() { respond(204); }); request.pipe(outStream); };

– , . pipe , – . , “error”, . , pipe , “finish”. 204.

: eloquentjavascript.net/code/file_server.js. Node. , .

curl, unix-, HTTP . . –X , –d .

 $ curl http://localhost:8000/file.txt File not found $ curl -X PUT -d hello http://localhost:8000/file.txt $ curl http://localhost:8000/file.txt hello $ curl -X DELETE http://localhost:8000/file.txt $ curl http://localhost:8000/file.txt File not found

file.txt , . PUT , - – . DELETE .

Error processing

, , , . , , . , , .

, - ? try, . Node ( ) .

Therefore, our server will crash if there are problems in the code - unlike asynchronous problems, which will be passed as arguments to the call functions. If we need to handle all the exceptions that occur during the processing of a request, so that we can send an exact response, we need to add try / catch blocks in each callback.

This is bad. Node , , , .

– , 17. , . Node promise . Node , . “promise” NPM denodeify, fs.readFile , .

 var Promise = require("promise"); var fs = require("fs"); var readFile = Promise.denodeify(fs.readFile); readFile("file.txt", "utf8").then(function(content) { console.log("The file contained: " + content); }, function(error) { console.log("Failed to read file: " + error); });

For comparison, I wrote another version of the file server using promises, which can be found at eloquentjavascript.net/code/file_server_promises.js . It is cleaner because functions can now return results rather than assign callbacks, and exceptions are implicitly passed.

Let me give you a few lines from there to show the difference in styles.

The fsp object used in the code contains variants of the fs functions with promises wrapped with Promise.denodeify. The object returned from the method handler, with the code and body properties, becomes the final result of the promise chain, and is used to determine which response to send to the client.

 methods.GET = function(path) { return inspectPath(path).then(function(stats) { if (!stats) // Does not exist return {code: 404, body: "File not found"}; else if (stats.isDirectory()) return fsp.readdir(path).then(function(files) { return {code: 200, body: files.join("\n")}; }); else return {code: 200, type: require("mime").lookup(path), body: fs.createReadStream(path)}; }); }; function inspectPath(path) { return fsp.stat(path).then(null, function(error) { if (error.code == "ENOENT") return null; else throw error; }); }

The inspectPath function is a simple wrapper around fs.stat that handles the case when the file is not found. In this case, we replace the error with success, returning null. All other errors can be transmitted. When the promise returned from these handlers fails, the server responds with a code of 500.

Total

Node – , JavaScript . , . , JavaScript, Node .

NPM , ( -, ), . Node , “fs” , “http” HTTP HTTP .

Node , , fs.readFileSync. , Node , I/O .

Exercises

17 eloquentjavascript.net/author, Accept.

, Node http.request. , , text/plain, text/html application/json. , headers, http.request.

.

, /home/marijn/public. , - , . What happened?

, urlToPath, :

 function urlToPath(url) { var path = require("url").parse(url).pathname; return "." + decodeURIComponent(path); }

Now remember that the paths passed to the “fs” function can be relative. They may contain the path “../” in the top directory. What happens if a client sends requests to a URL like the following:

myhostname : 8000 /../. Config / config / google-chrome / Default / Web% 20Data
myhostname : 8000 /../. Ssh / id_dsa
myhostname : 8000 / .. /../../etc/passwd

Change the urlToPath function to fix this problem. Note that Windows Node allows both forward and backward slashes to specify paths.

In addition, meditate on the fact that once you put a raw system on the Internet, errors in the system can be used against you and your computer.

Creating directories

Although the DELETE method works when deleting directories (via fs.rmdir), the server does not provide the ability to create directories.

Add support for the MKCOL method, which should create a directory via fs.mkdir. MKCOL is not the main HTTP method, but it exists precisely for this in the WebDAV standard, which contains HTTP extensions to use to write resources, not just to read them.

Public network space

Since the file server produces any files and even returns the correct Content-Type header, it can be used to maintain the website. Since it allows everyone to delete and replace files, it would be an interesting site - which can be changed, corrupted and deleted by anyone who can create the correct HTTP request. But it would still be a website.

Write a simple HTML page with a simple javascript file. Place them in a directory maintained by the server and open in a browser.

Then, as an advanced exercise, combine all the knowledge gained from the book to build a more user-friendly interface to modify the website from within the site itself.

HTML ( 18) , , HTTP-, 17.

, . , . , .

– , . , , .

, firewall, , . whatismyip.com, IP :8000 . , .

Source: https://habr.com/ru/post/245775/

All Articles