We build our full stack on JavaScript: Server

The second article in the series on full-stack JS development.

JavaScript is constantly changing, it is very difficult to keep up with the latest technologies, because what was the best practice six months ago is now outdated. Such assertions are largely true, but it should be noted that this applies more to client-side JavaScript. For the server, everything is much more stable and thorough.

• Article 1: Basics
• Article 2: Server
• Article 3: Customer

The article is based on the code of the project Contoso Express .

List of resources on which you can learn more about some of the topics of the article here .

Fads JS

JavaScript is known for its quirks. For example, the famous 0.1 + 0.2 does not equal 0.3 due to the fact that there is no embedded decimal type in JavaScript.

console.log(0.1 + 0.2 === 0.3); //false 

Or in order to correctly verify that the value of a variable is a string, we need the following construction:

  if (typeof myVar === 'string' || myVar instanceof String) { console.log('That is string!'); } 

Some of these issues are fixed in the ES6 version of the language. For example, the scope of a variable declared through var is the entire function (in most other languages, the variable has a block scope). In ES6, this problem is solved by declaring variables via let / const , and var is not recommended for use at all. However, it remains in JS (forever) for backward compatibility.

To eliminate other shortcomings of the JS language, third-party libraries are used. For example, for exact decimal calculations you can use "big.js". There are many small packages, each solving one similar problem, but remembering their names and connecting one to the project is too difficult. Therefore, it is more convenient to use such universal solutions as "lodash" , providing at once a whole set of additional utility functions.

On the lodash website, there is excellent documentation with examples, and you don’t have to watch everything at once, you can quickly get acquainted with what you have and look in the future only what you specifically need.

For example, to verify that a variable has a string value, with lodash is much simpler:

  if (_.isString(myVar)) { console.log('That is string!'); } 

Note that lodash has support for calling function chains, for example:

  let arr = [2, 3, 1, 5, 88, 7, 13]; let oddSquares = _(arr) .filter(x => x % 2 === 1) //filter odd numbers .sort((a,b) => a > b) //sort by default sorts lexicographically: [1, 13, 3, 7] .map(x => x*x); console.log(oddSquares.join(',')); //1,9,25,49,169 

TypeScript on the server

The more complex the JS code, the more advantages the use of TypeScript. For complex server code, the benefits of TS quickly become apparent.

In Contoso , the TS compilation settings for the server side are in the "tsconfig.json" file.

Here are some that should be noted:

• rootDir / outDir : all code (all TS files) from rootDir are compiled into JS files into outputDir, while the folder structure is preserved. For example, the file '/server/helpers/emailHelper.ts' will be compiled into '/build/server/helpers/emailHelper.js'

  
  

• sourceMap : if this flag is set, then besides JS, source maps are created (myModule.ts -> myModule.js + myModule.js.map). This allows you to debug TS files, breakpoints are set and triggered in the ts file, while the JS code is actually executed.

  
  
• target : TS allows you to compile code for different versions of JS. But there are several features that are supported only when compiling to the latest version of ES6. One of them is async / await, which is described below. In order to use it for the server, the code is compiled in ES6 and the 6x version of Node is used.

If you have installed TypeScript globally, you can compile the project with the console command tsc (TypeScript Compiler) from the project root, optionally with the watch parameter. In watch mode, after any change in the source code, it will be recompiled.

 tsc --watch 

To run the application you need to run (via Node) "build / server / startServer.js".

If you do not want to use TS, then you can easily convert the TS code to JS. The main thing to do:

• convert ES6 imports to Node require
• remove type annotations
• use promises instead of asyn / await or connect babel for transpilation

Asynchrony in JS

Asynchronous (async) programming in JS is one of the most difficult moments when switching from other programming languages. I give a very brief overview, you can read more in the list of resources.

High performance Node due to the fact that all long-running operations do not block the main process. An example of such operations may be a query to the database or writing information to a file. After calling the operation, another code continues to be executed. When the operation is completed (in the future), either successfully or with some kind of error, you need to specify what to do next.

To work with async code, you can use several templates. These patterns changed (evolved) with the development of Node.

• Callbacks are the traditional way of organizing async in Node. Simple to understand, but has a number of problems: it is difficult to make several nested async calls. In this case, the code becomes difficult to write, support, and understand (callback hell); data return, error handling and generation are fundamentally different from how it happens in synchronous JS code.

  
  

• Promises (promises / promises) is a modern standard for async in JS. Now becoming ubiquitous. A promis is a construction that can be in one of 3 waiting / reject / completed states (pending / rejected / resolved). Initially, the state of waiting, but in the future promis undertakes (promises) to go to one of the final error states or successful execution (with the result). Promise constructs make it possible to organize call chains of async operations more linearly, which makes support easier and improves code understanding. In this case, the data is returned via return, errors are generated via throw, and one error handler can be used for several async operations.

  
  
• async / await is the future of async in JS. Now stage4 of the discussions is in the final stage, after which it will become a new approved JS add-on. Async / awiat further blurs the line between synchronous and asynchronous code, so that they blend harmoniously together. Async / await code is as easy to write and maintain as possible. Similar syntax exists in other languages, such as C # and Python. The disadvantage of async / await is that additional tools are needed for its use, such as TypeScript or Babel.

There are some more async templates, an "async" module and a "co" module using generators. These are intermediate evolutionary stages before promises and async / await. "co" can be used without additional transformations now, but in my opinion it is better to use promises if using async / await is not possible.

To work successfully with JS, you need to know the basic async templates, since from time to time will have to deal with each of them. If for example you are working with async / await, then you still need promises for some operations, such as parallel execution through Promise.all, if you work with promises, then sometimes you have to wrap promises of functions on callbacks.

Error processing

This is another area of ​​difficulty when switching to Node from other platforms. To begin with, how to handle the error depends on which asynchronous pattern you use. When using callbacks, the error is transmitted as the first parameter of the callback function, when using promises, you need to handle errors through catch in the promise chain, when synchronously calling code, and using async / await, try / catch should be used.

Next, you need to know that in JavaScript you can generate (via throw) an error passing any object, but the correct practice is to use the built-in Error object. In its simplest form, it looks like this:

  throw new Error('Param value is wrong'); 

At the same time, you can create your custom error object by inheriting it from the built-in Error object.

  function AppError(errorCode, options) { this.code = errorCode; Error.captureStackTrace(this, this.constructor); //merge data props into error object return Object.assign(this, options); } Object.setPrototypeOf(AppError.prototype, Error.prototype); ... throw new AppError('user_not_found', {userId: 5}); 

Pay attention to the call to Error.captureStackTrace, this is necessary in order to correctly add the stack trace to the error object. You can add your data to the error and define your signature for the error constructor.

In Contoso, the AppError object takes string parameters by default: error type and error code, error text is read by code from an external file.

One more thing: you should catch and log raw errors; if you do not do this, the application will shut down without clearly specifying the reason; you need to connect the handler for raw errors as early as possible.

  process.on('uncaughtException', (err) => { console.log(`Uncaught exception. ${err}`); }); 

Choosing a back-end web framework

Express is the main choice, all alternatives are ten times less popular. Express follows the philosophy of minimalism and flexibility. Minimalism in this context means that very little functionality is available without additional configuration. For example, to add support for cookies, to parse the body in HTTP requests, to access the session on the client, you need to add the corresponding module. Every feature used needs to be activated directly. Flexibility means that you have a lot of possibilities to change / add to the existing functionality.

These features of Express make it an excellent choice for both web applications and individual server APIs.

Other options can be divided into two types: completely separate frameworks and those based on Express.

Separate:

• Koa - a new framework from the creators of Express. On the one hand, this is an improved version of Express, in which some points are made better, on the other hand, nothing fundamentally different and there is no backward compatibility with Express.

  
  

• Hapi is a completely separate framework, originally created and financially supported by Wallmart, the largest supermarket chain in the United States. Unlike Express, it requires less configuration and more features are available immediately.

  
  
• Meteor - provides an opportunity to develop both server and client parts, trying to provide the developer with all that is needed right away, it is more difficult to integrate with third-party technologies.

Based on Express:

• Loopback - a sharpened framework for writing server APIs, especially for mobile devices, there is a DSK for iOS / Android, the built-in ORM is used for working with various databases.

  
  

• Sails is positioned as a good basis for the server API, especially if there is real-time interaction through socket.io.

  
  
• FeathersJs - this framework has many adapters for other packages / frameworks, the goal is to add more standardization and ease the development process.

What to choose

Start with Express, if you have the time and desire, look at other options. Personally, I did not have enough motivation to deal with other frameworks, because I quickly was able to build a structure based on Express, which suits me completely. The advantage of this approach is that it is less dependent on the specifics of the framework, although this initially requires more work.

Project structure

The structure of the project Contoso is based on MVC architecture.

Controllers (controllers) - modules in which there are route handlers, functions that accept the standard express request and response parameters. The controllers have the logic of data validation and basic request processing, but they do not directly access the database and do not perform routine operations, for this purpose repositories and helpers are used.

Repositories (repositories) - here is the data access logic (DB) and additional business logic, some business logic may be in the controller, but the repository is preferable for this.

Helpers - perform specific operations - sending emails, error handling, logging, etc.

Routers — put correspondences between routes (URLs) and their corresponding handlers from the controllers. In this case, no special distinction is made between the routes for the API methods and the routes for the web pages (views).

Tasks are utility scripts for such tasks as creating the initial database or importing data from a file.

Views (views) - the server view (HTML template) supports common templates (layout views) and partial templates (partial views). In Contoso, server-side views are used for authentication pages, the remaining HTML views are generated on the client.

In the classical MVC architecture, models are separately defined; here, the model is simply an object with data that is created in the controller and used to generate the view.

Express is a flexible framework and allows you to organize code as you like. In this case, the developer himself decides which structure is suitable for him. There is no one correct structure, but it is definitely good when it exists in principle.

Many features of Express and third-party packages are used through wrapping modules. This is convenient because it allows you to easily replace the package in the future or redefine the standard functionality.

Configuration

The configuration is usually stored in physical files in a project subdirectory. In Node, unlike other platforms, JSON is used more often than XML.

There are several packages for working with configuration values, I prefer "config" another popular option "nconf".

In "config", data is read from several configuration files according to certain rules. Default values ​​are stored in the "default.json" file, they can be overridden by values ​​from the "local.json" file. In addition to the JSON format, many other formats can be used to store the configuration, such as yaml / xml / hjson / js, etc. Read more about it here .

The default settings file "default.json" is added to the repository, and the file "local.json" in which they can be overridden is excluded from the repository via .gitignore.

To store the configuration, you can use several strategies:

• Developer default settings - configuration (default file) stores settings that are suitable for local development (can be overridden in the local file). In production, you need to override the default settings from the default file, production settings (local file or file with server name).
• Production settings by default - on the contrary, the default configuration stores production settings, and the developer locally overrides the settings for development.

The first option is preferable - firstly, adding settings for production to the repository is not very correct in terms of security, and secondly, there are often several environments for production / uat / test deployments that require different configuration values. For more security, you can avoid using the default file at all in production, by completely setting the configuration values ​​in the local file (for more information, see the article on the scan).

In Contoso, the configuration is used via the server / config wrap module.

Logging

For simple debugging, I still use console.log, but if I need to display an error in the logs or some additional information, use one of the logging libraries.

There are several popular logging packages: "winston", "bunyan" and "log4js". I use the most popular "winston" on the principle of "they aren't looking for good from the good," I managed to integrate winston without any problems and everything I needed was there.

You can compare and choose the library that you like best. In additional resources there is an article with a comparison of winston vs bunyan.

In Contoso, logging is used through the loggerHelper wrapper. Logs are stored in the '/ data / logs' log for errors and log for diagnostic messages are recorded in different files.

Data validation

Data that enters the application from the outside, you need to check for correctness. In the case of a simple web application, the main source of incoming data is web requests: HTML form data or AJAX client requests.

Validation of data is possible at different levels, in Contoso validation is performed at the controller level and it is assumed that the correct data is already being received in the repository. In more complex applications, it may make sense to additionally check the data in the repository, just before the state of the database is measured.

For more effective checking of incoming data, you can use the "joi" package. This is a hapi plugin (alternative to express) that works independently.

The data enters the controller in the request body (req.body) or in the query string parameters (req.query). This is an arbitrary JS object in which there can be anything. We expect to get a certain set of parameters, each of which has a certain type, a valid set of values, may be absent or must be, etc.

To check the data for correctness, you need to declare a Joi scheme describing the expected data and perform a check for compliance:

  let schema = { id: Joi.number(), number: Joi.number().required(), title: Joi.string().required() }; let obj = { number: 8, title: 'Gone with the Wind' }; Joi.validate(obj, schema, {/*options*/}, (err, val) => { //... }); 

In Contoso Joi, it is used by calling the loadSchema method in the controllerHelper. This method wraps Joi.validate in promis and generates a validation error that the application understands.

Emailing

To send emails, I use "nodemailer" it is popular and well supported.

You can use different modes of transport (the way email will be sent). The default is direct mode.

This mode is good for working on project prototypes, because it does not require a separate SMTP server or a third-party email service such as Amazon SES or SendGrid.

The disadvantage is that, depending on your IP address, the letters may appear in the spam directory.

More on the various modes of transport here .

To generate the very contents of emails in the form of HTML, use the "email-templates" package with "handlebars" templates. In Contoso, emails are sent via "/ helpers / emailHelper", email templates are stored in "/ data / emails".

Authentication

Passport is the most popular authentication package for Node. It supports many different authentication mechanisms called strategies. Typically, a project has a local strategy that uses the traditional system access mechanism via login / password and stores user data in the application database. Additionally, you can provide access to the system via SSO (single sign-on) providers, such as google / facebook / twitter, or use special types of authentication, such as Windows Active Directory.

For a local strategy, in addition to the form where the user enters a login / password, you must provide a form for registering a new user profile, implement a new profile activation mechanism (send an email with a link to activate), support cases when the user has forgotten the password or wants to change it.

( Auth0 Stormpath ), , .

, , ( ).

Google Firebase. . , , , Firebase.

Contoso SSO google/facebook. SSO , clientID/clientSecret , .

What's next?

JS . JS , , , . , . , , - Angular 2.0 . , !

.

Stay tuned!