Expressive javascript: http

Content

The dream for which the Network was created is a common information space in which we communicate, sharing information. Its versatility is its integral part: a hypertext link can lead anywhere, be it personal, local or global information, draft or verified text.

Tim Bernes-Lee, The World Wide Web: A Very Short Personal Story

Protocol

If you type eloquentjavascript.net/17_http.html in the address bar of your browser, the browser first recognizes the server address associated with the name eloquentjavascript.net and tries to open a TCP connection on port 80 - the default HTTP port. If the server exists and accepts the connection, the browser sends something like:
')
GET /17_http.html HTTP / 1.1
Host: eloquentjavascript.net
User-Agent: Browser Name

The server responds by the same connection:

HTTP / 1.1 200 OK
Content-Length: 65585
Content-Type: text / html
Last-Modified: Wed, 09 Apr 2014 10:48:09 GMT

<! doctype html>
... the remainder of the document

The browser takes the part that follows the answer after the empty line and displays it as an HTML document.

The information sent by the client is called a request. It starts with the line:

GET /17_http.html HTTP / 1.1

The first word is the query method. GET means we need to get a certain resource. Other common methods are DELETE to delete, PUT to replace, and POST to send information. Note that the server is not required to fulfill every request received. If you choose a random site and tell it to delete the main page, it will most likely refuse.

The part after the method name is the path to the resource to which the request has been sent. In the simplest case, the resource is just a file on the server, but the protocol is not limited to this feature. A resource can be anything that can be transferred as a file. Many servers create responses on the fly. For example, if you open twitter.com/marijnjh, the server will look at the user’s database marijnjh, and if it finds it, it will create a profile page for this user.

After the path to the resource, the first line of the request mentions HTTP / 1.1 to report the version of the HTTP protocol it uses.

The server's response also begins with the protocol version, followed by the status of the response — first a three-digit code, then a line.

HTTP / 1.1 200 OK

Status codes starting with 2 indicate successful requests. Codes starting with 4 mean something went wrong. 404 is the most famous HTTP status, indicating that the requested resource was not found. Codes starting with 5 indicate that an error has occurred on the server, but not the fault of the request.

The first line of the request or response can be any number of header lines. These are strings in the form “name: value” that denote additional information about the request or response. These headers were included in the example:

Content-Length: 65585
Content-Type: text / html
Last-Modified: Wed, 09 Apr 2014 10:48:09 GMT

This determines the size and type of the document received in response. In this case, this is an HTML document of 65'585 bytes. It also indicates when the document was last modified.

For the most part, the client or server determines which headers to include in the request or response, although some headers are required. For example, Host, denoting the host name, should be included in the request, because one server can serve many host names at one ip-address, and without this header the server will not know which host the client is trying to communicate with.

After the headers, both the request and the response can indicate an empty string followed by a body containing the data to be transmitted. GET and DELETE requests do not send additional data, and PUT and POST send. Some responses, such as error messages, do not require a body.

Browser and HTTP

As we saw in the example, the browser sends the request when we enter the URL in the address bar. When the resulting HTML document contains references to other files, such as images or JavaScript files, they are also requested from the server.

A medium-sized website can easily contain from 10 to 200 resources. In order to be able to request them quickly, browsers make several requests at the same time, rather than waiting for the end of requests one by one. Such documents are always requested through GET requests.

HTML pages can have forms that allow users to enter information and send it to the server. Here is an example of a form:

<form method="GET" action="example/message.html"> <p>: <input type="text" name="name"></p> <p>:<br><textarea name="message"></textarea></p> <p><button type="submit"> </button></p> </form>

The code describes a form with two fields: a small one asks for a name, and a large one - a message. When you click the “Send” button, the information from these fields will be encoded into a query string (query string). When the element's method attribute is GET, or when it is not specified at all, the query string is placed in the URL from the action field, and the browser makes a GET request with this URL.

GET /example/message.html?name=Jean&message=Yes%3F HTTP / 1.1

The beginning of the query string is indicated by a question mark. After that, there are pairs of names and values corresponding to the name attribute of the form fields and the contents of these fields. Ampersand (&) is used to separate them.

The message sent in the example contains the string “Yes?”, Although the question mark is replaced by some strange code. Some characters in the query string must be escaped (escape). The question mark is included, and it is represented by code% 3F. There is some unwritten rule that every format must have a way to escape characters. This rule, called URL coding, uses a percentage followed by two hexadecimal digits that represent the character code. 3F in decimal will be 63, and this is the code of the question mark. JavaScript has encodeURIComponent and decodeURIComponent functions for encoding and decoding.

 console.log(encodeURIComponent("Hello & goodbye")); // → Hello%20%26%20goodbye console.log(decodeURIComponent("Hello%20%26%20goodbye")); // → Hello & goodbye

If we change the method attribute in the form in the previous example to POST, the HTTP request with the form will be sent using the POST method, which sends the query string in the request body, instead of adding it to the URL.

POST /example/message.html HTTP / 1.1
Content-length: 24
Content-type: application / x-www-form-urlencoded

name = Jean & message = Yes% 3F

By convention, the GET method is used for queries that have no side effects, such as searching. Requests that change something on the server — create a new account or post a message — should be sent using the POST method. Browser-type client programs know that it is not necessary to simply make POST requests, and sometimes GET requests are made seamlessly by the user — for example, to download content in advance that the user may soon need.

In the next chapter, we’ll go back to the forms and talk about how we can do them with JavaScript.

XMLHttpRequest

The interface through which JavaScript in the browser can make HTTP requests is called XMLHttpRequest (notice how the size of the letters jumps). It was developed by Microsoft for the Internet Explorer browser in the late 1990s. At this time, the XML format was very popular in the world of business software - and in this world, Microsoft always felt at home. It was so popular that the XML abbreviation was attached to the interface name for working with HTTP, although the latter is not related to XML at all.

And yet the name is not completely meaningless. The interface allows you to parse your answers as if they were XML documents. To mix two different things (request and answer analysis) into one is, of course, a disgusting design, but what can you do.

When the XMLHttpRequest interface was added to Internet Explorer, it became possible to do things that were previously very difficult to do. For example, sites began to show lists of prompts, while the user enters something in the text field. The script sends the text to the server via HTTP simultaneously with the user typing. The server, which has a database for possible input options, searches for suitable entries among the entries and returns them back for display. It looked very cool - people used to wait for the rest of the page to reload after each interaction with the site.

Another important browser of the time, Mozilla (later Firefox), did not want to lag behind. To allow doing similar things, Mozilla copied the interface along with the name. The next generation of browsers followed suit, and today XMLHttpRequest is a de facto standard.

Submit request

To send a simple request, we create a request object with the XMLHttpRequest constructor and call the open and send methods.

 var req = new XMLHttpRequest(); req.open("GET", "example/data.txt", false); req.send(null); console.log(req.responseText); // → This is the content of data.txt

The open method customizes the query. In our case, we decided to make a GET request for the file example / data.txt. URLs that do not begin with the protocol name (for example, http :) are referred to as relative, that is, they are interpreted relative to the current document. When they begin with a slash (/), they replace the current path - the part after the server name. Otherwise, a portion of the current path, up to the last slash, is placed before the relative URL.

After opening the request, we can send it by the send method. The argument is the request body. For GET requests, use null. If the third argument for open was false, then send will return only after the response to our request has been received. To get the response body, we can read the responseText property of the request object.

Other information can be obtained from the response object. The status code is available in the status property, and the status text is available in statusText. Headers can be read from getResponseHeader.

 var req = new XMLHttpRequest(); req.open("GET", "example/data.txt", false); req.send(null); console.log(req.status, req.statusText); // → 200 OK console.log(req.getResponseHeader("content-type")); // → text/plain

Title titles are not case sensitive. They are usually capitalized at the beginning of each word, for example “Content-Type”, but the “content-type” or “cOnTeNt-TyPe” will describe the same title.

The browser itself will add some headers, such as “Host” and others that are needed by the server to calculate the size of the body. But you can add your own headers with the setRequestHeader method. This is necessary for special cases and requires the assistance of the server to which you are applying - it is free to ignore headers that it cannot handle.

Asynchronous requests

In the example, the request was completed when the send call ends. This is convenient because properties like responseText are immediately available. But this means that our program will wait until the browser and server communicate with each other. With a bad connection, a weak server or a large file, this can take a long time. This is also bad because no event handlers will work while the program is in standby mode - the document will stop responding to user actions.

If the third argument to open is true, the request will be asynchronous. This means that when you call send, the request is queued for sending. The program continues to work, and the browser takes care of sending and receiving data in the background.

But while the request is being processed, we will not receive an answer. We need a notification mechanism that the data has arrived and is ready. To do this, we will need to listen to the “load” event.

 var req = new XMLHttpRequest(); req.open("GET", "example/data.txt", true); req.addEventListener("load", function() { console.log("Done:", req.status); }); req.send(null);

Just like calling requestAnimationFrame in Chapter 15, this code forces us to use an asynchronous programming style, wrapping the code that needs to be executed after the request and placing the call to this function at the right time into a function. We will come back to this later.

Getting XML data

When the resource returned by the XMLHttpRequest object is an XML document, the responseXML property will contain a parsed view of the document. It works in a manner similar to DOM, except that it does not have any inherent HTML functionality like the style property. The object contained in responseXML corresponds to the document object. Its documentElement property refers to the external tag of the XML document. In the following document (example / fruit.xml), this tag will be:

 <fruits> <fruit name="banana" color="yellow"/> <fruit name="lemon" color="yellow"/> <fruit name="cherry" color="red"/> </fruits>

We can get this file like this:

 var req = new XMLHttpRequest(); req.open("GET", "example/fruit.xml", false); req.send(null); console.log(req.responseXML.querySelectorAll("fruit").length); // → 3

XML documents can be used to exchange structured information with the server. Their form — nested tags — is good for storing most data, or at least better than text files. The DOM interface is clumsy in terms of extracting information, and XML documents are quite verbose. It is usually best to communicate using data in JSON format that is easier to read and write, both to programs and to people.

 var req = new XMLHttpRequest(); req.open("GET", "example/fruit.json", false); req.send(null); console.log(JSON.parse(req.responseText)); // → {banana: "yellow", lemon: "yellow", cherry: "red"}

HTTP sandbox

HTTP requests from a webpage raise security concerns. The person controlling the script may have interests different from those of the user on whose computer it is running. Specifically, if I went to themafia.org, I don’t want their scripts to make inquiries to mybank.com, using the information from my browser as an identifier, and giving the command to send all my money to some mafia account.

Websites can protect themselves from such attacks, but this requires some effort, and many sites can not cope with this. Because of this, browsers protect them by prohibiting scripts from making requests to other domains (names like themafia.org and mybank.com).

This may interfere with the development of systems that need to have access to different domains for a good reason. Fortunately, the server may include the following header in the response, explaining to browsers that the request may come from other domains:

Access-Control-Allow-Origin: *

We abstract requests

In Chapter 10, in our implementation of the modular AMD system, we used the hypothetical backgroundReadFile function. She accepted the file name and function, and called this function after reading the contents of the file. Here is a simple implementation of this function:

 function backgroundReadFile(url, callback) { var req = new XMLHttpRequest(); req.open("GET", url, true); req.addEventListener("load", function() { if (req.status < 400) callback(req.responseText); }); req.send(null); }

Simple abstraction simplifies the use of XMLHttpRequest for simple GET requests. If you are writing a program that makes HTTP requests, it will be a good idea to use a helper function so that you do not have to repeat the ugly XMLHttpRequest template all the time.

The callback argument is a term often used to describe such functions. The callback function is transferred to another code so that it can call us back later.

It's easy to write your own HTTP helper function, tailored specifically for your program. The previous one does only GET requests, and does not give us control over the headers or the body of the request. You can write another option for the POST request, or more general, supporting different requests. Many JavaScript libraries offer wrappers for XMLHttpRequest.

The main problem with the given wrapper is error handling. When a request returns a status code indicating an error (between 400 and above), it does nothing. In some cases, this is normal, but imagine that we put a loading indicator on the page that indicates that we are receiving information. If the request did not succeed, because the server fell or the connection was interrupted, the page will pretend that it is busy with something. The user will wait a bit, then he will get bored and he decides that the site is some kind of stupid.

We need an option in which we receive a warning about a failed request so that we can take action. For example, we can remove the download message and inform the user that something has gone wrong.

Error handling in asynchronous code is even more complicated than in synchronous. Since we often have to separate part of the work and place it in a callback function, the scope of the try block is meaningless. In the following code, the exception will not be caught, because the call to backgroundReadFile is returned immediately. Then the control leaves the try block, and the function will not be called from it.

 try { backgroundReadFile("example/data.txt", function(text) { if (text != "expected") throw new Error("That was unexpected"); }); } catch (e) { console.log("Hello from the catch block"); }

In order to handle unsuccessful requests, you will have to pass an additional function to our wrapper, and call it in case of problems. Another option is to use the convention that if the request fails, an additional argument is passed to the callback function describing the problem. Example:

 function getURL(url, callback) { var req = new XMLHttpRequest(); req.open("GET", url, true); req.addEventListener("load", function() { if (req.status < 400) callback(req.responseText); else callback(null, new Error("Request failed: " + req.statusText)); }); req.addEventListener("error", function() { callback(null, new Error("Network error")); }); req.send(null); }

We added an error event handler that will work if there is a problem with the call. We also call the callback function with an error argument when the request completes with a status that indicates an error.

Code using getURL should check if the error is returned and process it if it exists.

 getURL("data/nonsense.txt", function(content, error) { if (error != null) console.log("Failed to fetch nonsense.txt: " + error); else console.log("nonsense.txt: " + content); });

With exceptions, this does not help. When we perform several asynchronous actions in succession, an exception at any point in the chain in any case (unless you wrapped each handler in your try / catch block) will fall out at the top level and interrupt the entire chain.

Promises

It is difficult to write asynchronous code for complex projects in the form of simple callbacks. It is very easy to forget error checking or to allow an unexpected exception to abruptly interrupt the program. In addition, organizing proper error handling and passing errors through several consecutive callbacks is very tedious.

Many attempts have been made to solve this problem with additional abstractions. One of the most successful attempts is called promises. Promises wrap an asynchronous action into an object that can be transmitted and that needs to do some things when the action ends or fails. This interface has already become part of the current version of JavaScript, and for older versions it can be used as a library.

The promise interface is not particularly intuitive, but powerful. In this chapter we will only partially describe it. More information can be found at www.promisejs.org

To create a promises object, we call the Promise constructor, giving it an initialization function for asynchronous action. The constructor calls this function and passes it two arguments, which are also functions themselves. The first should be called in a successful case, the other - in an unsuccessful one.

And here is our GET request wrapper, which this time returns a promise. Now we just call it get.

 function get(url) { return new Promise(function(succeed, fail) { var req = new XMLHttpRequest(); req.open("GET", url, true); req.addEventListener("load", function() { if (req.status < 400) succeed(req.responseText); else fail(new Error("Request failed: " + req.statusText)); }); req.addEventListener("error", function() { fail(new Error("Network error")); }); req.send(null); }); }

Notice that the interface to the function itself is simplified. We give her the URL, and she returns the promise. It works as a handler for the request output. It has a then method that is invoked with two functions: one to handle success, the other to fail.

 get("example/data.txt").then(function(text) { console.log("data.txt: " + text); }, function(error) { console.log("Failed to fetch data.txt: " + error); });

So far this is still one of the ways to express what we have already done. Only when you have a chain of events does a noticeable difference become visible.

The then call produces a new promise, whose result (the value passed to the handlers of successful results) depends on the value returned by the first function we pass to then. This function can return another promise, indicating that additional asynchronous work is being done. In this case, the promise returned by then itself will wait for the promise returned by the handler function, and success or failure will occur with the same value. When a handler function returns a value that is not a promise, the promise returned by then becomes successful, using this value as the result.

So you can use then to change the result of the promise. For example, the following function returns a promise whose result is the content from the given URL, parsed as JSON:

 function getJSON(url) { return get(url).then(JSON.parse); }

The last call to then did not identify the failure handler. It is permissible. The error will be transmitted to the promise returned through then, and this is what we need - getJSON does not know what to do when something goes wrong, but there is hope that its caller knows it.

As an example showing the use of promises, we will write a program that receives the number of JSON files from the server and shows the word “download” during the execution of the request. The files contain information about people and links to other files with information about other people in properties such as father, mother, spouse.

We need to get the name of the spouse's mother from example / bert.json. In case of problems, we need to remove the text "download" and show an error message. Here's how to do this with promises:

 <script> function showMessage(msg) { var elt = document.createElement("div"); elt.textContent = msg; return document.body.appendChild(elt); } var loading = showMessage("..."); getJSON("example/bert.json").then(function(bert) { return getJSON(bert.spouse); }).then(function(spouse) { return getJSON(spouse.mother); }).then(function(mother) { showMessage(" - " + mother.name); }).catch(function(error) { showMessage(String(error)); }).then(function() { document.body.removeChild(loading); }); </script>

The final program is relatively compact and readable. The catch method is similar to then, but it expects only the handler for the unsuccessful result and, if successful, passes on the unchanged result. Program execution will continue as usual after catching an exception - just as in the case of try / catch. Thus, the last then deleting the boot message is executed anyway, even in case of failure.

You can imagine that the promise interface is a separate language for asynchronous processing of program execution. The additional calls to the methods and functions that are needed for its operation give the code a somewhat strange look, but not as inconvenient as handling all errors manually.

Appreciate http

When creating a system in which a JavaScript program in the browser (client) communicates with the server program, you can use several options for modeling such communication.

A common method is remote procedure calls. In this model, communication follows the pattern of normal function calls, only these functions are performed on another computer. The challenge is to create a request to the server, which includes the function name and arguments.The response to the request includes the return value.

When using remote calls, HTTP procedures serve only as a transport for communication, and you will most likely write an abstraction layer that will hide it completely.

Another approach is to build your communication system on the concept of HTTP resources and methods. Instead of a remote procedure call called addUser, you make a PUT request to / users / larry. Instead of encoding user properties in function arguments, you specify the document format or use an existing format that will represent the user. The body of a PUT request that creates a new resource will simply be a document of this format. A resource is obtained through a GET request to its URL (/ user / larry), which returns a document representing this resource.

The second approach simplifies the use of some HTTP features, such as resource caching support (a copy of the resource is stored on the client side). It also helps create a consistent interface, because thinking in terms of resources is easier than in terms of functions.

Security and HTTPS

Data travels on the Internet in a long and dangerous way. To get to their destination, they need to jump through all sorts of places, ranging from a Wi-Fi coffee shop network to networks controlled by various organizations and states. At any point in the way they can read or even change.

If you need to keep something secret, such as passwords to an email, or data you need to arrive at your destination unchanged - for example, such as the bank account number to which you transfer money - simple HTTP is not enough.

The secure HTTP protocol, whose URLs begin with https: //, wraps HTTP traffic so that it is harder to read and change. First, the client confirms that the server is the one for whom it claims to be, demanding from the server to submit a cryptographic certificate issued by a reputable party, which is recognized by the browser. Then, all data passing through the connection is encrypted so as to prevent eavesdropping and alteration.

Thus, when everything works correctly, HTTPS prevents both cases when someone pretends to be another website with which you are communicating, as well as cases of interception of your communication. It is not perfect, and there have already been cases where HTTPS did not cope with work due to fake or stolen certificates or broken programs. However, with HTTP it is very easy to do something bad, and hacking HTTPS requires such efforts that only state structures or very serious criminal organizations can apply (and there are sometimes no differences between these organizations).

Total

In this chapter, we saw that HTTP is a protocol for accessing resources on the Internet. The client sends a request containing a method (usually GET) and a path that identifies the resource. The server decides what to do with the request and responds with the status code and the response body. Requests and responses may contain headers in which additional information is transmitted.

Browsers make GET requests to get the resources needed to display the page. The page may contain forms that allow the information entered by the user to be sent in a request that is created after the form is submitted. You will learn more about this in the next chapter.

The interface through which JavaScript makes HTTP requests from a browser is called XMLHttpRequest. You can ignore the prefix “XML” (but you still need to write it). It can be used in two ways: synchronous, which blocks all work until the end of the request, and asynchronous, which requires the installation of an event handler that tracks the end of the request. In almost all cases, the asynchronous method is preferred. Creating a query looks like this:

 var req = new XMLHttpRequest(); req.open("GET", "example/data.txt", true); req.addEventListener("load", function() { console.log(req.statusCode); }); req.send(null);

Asynchronous programming is not an easy thing. Promises is an interface that makes it easier by helping to send error messages and exceptions to the right handler, and abstracting some duplicate error-prone elements.

Exercises

Content negotiation

One of the things that HTTP can do but which we have not discussed is called content negotiation. The Accept header in the request can be used to tell the server which types of documents the client wants to receive. Many servers ignore it, but when the server knows about different ways of coding a resource, it can look at the header and send the one that the client prefers.

The eloquentjavascript.net/author URL is configured to respond either in plain text or HTML or JSON, depending on the client’s request. These formats are defined by standardized content types text / plain, text / html, and application / json.

Send a request for all three formats of this resource. Use the setRequestHeader method of the XMLHttpRequest object to set the Accept header to one of the desired content types. Make sure you set the header after open, but before send.

Finally, try querying the contents of application / rainbows + unicorns type and see what happens.

Waiting for a few promises

The Promise constructor has an all method that, when it receives an array of promises, returns a promise that waits for the completion of all the promises specified in the array. It then returns a successful result and returns an array with the results. If some of the promises in the array failed, the total promise also returns failure (with the value of the failed promise from the array).

Try doing something like this by writing the function all.

Notice that after the promise is completed (when it either completed successfully or with an error), it cannot rerun the error or success, and further function calls are ignored. This can simplify error handling in your promise.

 function all(promises) { return new Promise(function(success, fail) { //  . }); } //  . all([]).then(function(array) { console.log("   []:", array); }); function soon(val) { return new Promise(function(success) { setTimeout(function() { success(val); }, Math.random() * 500); }); } all([soon(1), soon(2), soon(3)]).then(function(array) { console.log("   [1, 2, 3]:", array); }); function fail() { return new Promise(function(success, fail) { fail(new Error("")); }); } all([soon(1), fail(), soon(3)]).then(function(array) { console.log("     "); }, function(error) { if (error.message != "") console.log(" :", error); });

Source: https://habr.com/ru/post/245145/

All Articles