Beginner's Guide to Web Server Development with Node.js

For most of my web career, I worked exclusively on the client side. Designing adaptive layouts, creating visualizations from large amounts of data, creating application dashboards, etc. But I have never had to deal with routing or HTTP requests directly. Until recently.

This post is a description of how I learned more about server-side web development using Node.js, and a brief comparison of writing a simple HTTP server using 3 different environments, Express, Koa.js and Hapi.js.

Note: if you are an experienced Node.js developer, you will probably think that this is all elementary / simple. ¯ \ _ (ツ) _ / ¯.

Some network basics

When I started working in the web industry a couple of years ago, I stumbled upon the course on computer networks of Professor David Veral on Coursera. Unfortunately, it is no longer available, but the lectures are still available on the Pearson website .
')
I really liked this course, because it explained what was going on under the hood, in an understandable form, so if you can pick up the textbook “Computer Networks” , read all the details about the wonders of the network.


Here, however, I am only going to briefly describe the context. HTTP (Hypertext Transfer Protocol) is a communication protocol used in computer networks. There are many of them on the Internet, such as SMTP (Simple Mail Transfer Protocol) , FTP (File Transfer Protocol) , POP3 (Post Office Protocol 3), and so on.

These protocols allow devices with completely different hardware / software to communicate with each other because they provide well-defined message formats, rules, syntax and semantics, etc. This means that as long as the device supports a specific protocol, it can communicate with any other device. online.


From TCP / IP vs. OSI: what is the difference between the two models?

Operating systems usually come with support for network protocols, such as HTTP, out of the box, which explains why we do not need to explicitly install any additional software to access the Internet. Most network protocols support an open connection between two devices, which allows them to transfer data back and forth.

HTTP on which the network is running is different. It is known as a connectionless protocol because it is based on the request / response mode of operation. Web browsers send requests for images, fonts, content, etc. to the server, but after the request is completed, the connection between the browser and the server is broken.

Servers and Clients

The term server can be a little confusing to people who are new to the industry for the first time, because it can refer both to hardware (physical computers that host all the files and software required by websites) and to software (software that allows users to access these files on the Internet).

Today we will talk about the software side of things. But first, a few definitions. The URL denotes the Universal Resource Locator and consists of 3 parts: the protocol , the server, and the requested file .


URL structure

The HTTP protocol defines several methods that the browser can use to ask the server to perform a bunch of different actions, the most common of which are GET and POST. When a user clicks a link or enters a URL in the address bar, the browser sends a GET request to the server to get the resource specified in the URL.

The server needs to know how to handle this HTTP request in order to get the correct file, and then send it back to the browser that requested it. The most popular web server software that handles this is Apache and NGINX .


Web servers process incoming requests and respond to them accordingly.

Both are full-featured open source software packages that include features such as authentication schemes, URL rewriting, logging and proxying, and these are just a few of them. Apache and NGINX are written in C. Technically, you can write a web server in any language. Python , golang.org/pkg/net/http , Ruby , this list can last quite a long time. Some languages ​​just do certain things better than others.

Creating an HTTP server with Node.js

Node.js is a Javascript runtime built on the Chrome V8 Javascript engine . It comes with an http module that provides a set of functions and classes for building an HTTP server.

For this basic HTTP server, we will also use the file system , the path, and the URL , which are proprietary Node.js modules.

Start by importing the required modules.

const http = require('http') //   HTTP-  Node.js const fs = require('fs') //      const path = require('path') //        const url = require('url') //     URL 

We will also create a dictionary of MIME types so that we can assign the appropriate MIME type to the requested resource based on its extension. A full list of MIME types can be found at the Internet Assigned Numbers Authority (Internet Assigned Numbers Center) .

 const mimeTypes = { '.html': 'text/html', '.js': 'text/javascript', '.css': 'text/css', '.ico': 'image/x-icon', '.png': 'image/png', '.jpg': 'image/jpeg', '.gif': 'image/gif', '.svg': 'image/svg+xml', '.json': 'application/json', '.woff': 'font/woff', '.woff2': 'font/woff2' } 

Now we can create an HTTP server with the http.createServer() function that will return a new instance of http.Server .

 const server = http.createServer() 

We will pass the request handler function to createServer() with the request and response objects. This function is called once each time an HTTP request is sent to the server.

 server.on('request', (req, res) => { //     }) 

The server is started by calling the server object’s listen method with the port number we want the server to listen for, for example, 5000 .

 server.listen(5000) 

The request object is an instance of IncomingMessage and allows us to access all information about the request, such as the response status, headers and data.

The response object is an instance of ServerResponse , which is a recordable stream and provides many methods for sending data back to the client.

In the query handler, we want to do the following:

• Parse an incoming request and process it without extensions
  
  

   const parsedUrl = new URL(req.url, 'https://node-http.glitch.me/') let pathName = parsedUrl.pathname let ext = path.extname(pathName) //   URL    '/',   '/' //      URL    'Location' if (pathName !== '/' && pathName[pathName.length - 1] === '/') { res.writeHead(302, {'Location': pathName.slice(0, -1)}) res.end() return } //     ,  index.html //     «.html»       if (pathName === '/') { ext = '.html' pathName = '/index.html' } else if (!ext) { ext = '.html' pathName += ext } 

  
  
• Perform some basic checks to determine if the requested resource exists and respond accordingly.
  
  

   //     ,       const filePath = path.join(process.cwd(), '/public', pathName) // ,       fs.exists(filePath, function (exists, err) { //     ,  404 Not Found if (!exists || !mimeTypes[ext]) { console.log('  : ' + pathName) res.writeHead(404, {'Content-Type': 'text/plain'}) res.write('404 Not Found') res.end() return } //        200 OK, //       res.writeHead(200, {'Content-Type': mimeTypes[ext]}) //        const fileStream = fs.createReadStream(filePath) fileStream.pipe(res) }) 

  
  

All code is hosted on Glitch, and you can remix the project if you want.

https://glitch.com/edit/#!/node-http

Creating an HTTP server with Node.js frameworks

Node.js frameworks, such as Express , Koa.js, and Hapi.js , come with various useful middleware functions, in addition to many other handy features that save developers from writing themselves.

Personally, I feel it is better to first learn the basics without frameworks, just to understand what is happening under the hood, and then go crazy with any framework you like.

Express has its own built-in plugin for serving static files, so the code required to perform the same actions as in your own Node.js is much shorter.

 const express = require('express') const app = express() //         app.use(express.static('public')) //  index.html,      //     res.sendFile() app.get('/', (req, res) => { res.sendFile(__dirname + '/public/index.html') }) app.listen(5000) 

Koa.js does not have such a plugin inside its kernel, so any required plugin must be installed separately. The latest version of Koa.js uses asynchronous functions in favor of callbacks. To serve static files, you can use the koa-static plugin.

 const serve = require('koa-static') const koa = require('koa') const app = new koa() //         //   koa-static    index.html    app.use(serve(__dirname + '/public')) app.listen(5000) 

Hapi.js supports configuration and revolves around the configuration of the server object. It uses plugins to extend capabilities such as routing, authentication, and so on. To maintain static files, we need a plugin called inert .

 const path = require('path') const hapi = require('hapi') const inert = require('inert') //        const server = new hapi.Server({ port: 5000, routes: { files: { relativeTo: path.join(__dirname, 'public') } } }) const init = async () => { // server.register()      await server.register(inert) // inert     //       server.route({ method: 'GET', path: '/{param*}', handler: { directory: { path: '.', redirectToSlash: true, index: true } } }) await server.start() } init() 

Each of these platforms has its pros and cons, and they will be more apparent for larger applications, and not just for serving one HTML page. The choice of structure will greatly depend on the actual requirements of the project you are working on.

Completion

If the network side of things has always been a black box for you, I hope this article can serve as a useful introduction to the protocol that provides the network. I also highly recommend reading the Node.js API documentation , which is very well written and very useful for any newcomer to Node.js in general.

• HTTP by MDN
• Anatomy of an HTTP Transaction
• HTTP Server: HTTP server from scratch