Building a full MVC website on ExpressJS

NB: This is material for those who have already familiarized themselves with the theoretical basis of node.js and want, as they say, right off the bat, to quickly plunge into development using this tool. No deduction, only coding. If interested, do not be shy, we pass under the cat.


From the translator: I myself began to learn node.js more recently. Undoubtedly, on the Internet there are many good (!) Manuals for beginners, in particular, and here, in Habré (for example, here and here ). However, this manual from the author Krasimir Tsonev (I also advise you to take a look at his multifaceted blog ) seemed especially remarkable, since it describes the process of building a full-fledged node.js application based on the Express module using the MVC pattern and base in a reasonably short and understandable form. MongoDB data. Also, the author pays attention to such an important thing as testing.

Transfer

In this article we will build a full-fledged website with a client part, as well as a control panel for the site content. As you can guess, the final working version of the program contains a large number of different files. I wrote this manual step by step, fully tracking the development of the process, but I did not include every single file in it, as this would make the reading long and boring. However, the source code is available on GitHub , and I highly recommend that you watch it.
')

Introduction

Express is one of the best Node frameworks. It has excellent developer support and a bunch of useful features. There are many great articles on Express that cover all the basics of working with it. But now I want to dig a little deeper and share my experience in creating a full-fledged website. In general, this article is not only about Express itself, but also about its combination with other, no less remarkable tools that are available to developers.
I suppose that you are familiar with Node.js, installed it on your system, and that you have already created some applications on it.
At the core of the Express framework is Connect . This is a set of middleware-functions, which comes with a lot of useful things. If you are wondering what the middleware function is, here is a small example:

var connect = require('connect'), http = require('http'); var app = connect() .use(function(req, res, next) { console.log("That's my first middleware"); next(); }) .use(function(req, res, next) { console.log("That's my second middleware"); next(); }) .use(function(req, res, next) { console.log("end"); res.end("hello world"); }); http.createServer(app).listen(3000); 

A middleware function is basically a function that accepts request parameters and responce objects as well as a callback function that will be called next. Each middleware-function decides: either to answer, using a responce-object, or to transmit a stream to the next callback-function. In the example above, if you remove the next () method call in the second function, the string “hello world” will never be sent to the client. In general, exactly how Express works. It has several predefined middleware functions that will surely save you a lot of time. Such, for example, is the Body Parser, which parses the request body and supports the types of content application / json , application / x-www-form-urlencoded and multipart / form-data . Or a cookie, a parser that parses cookies and fills the req.cookies field with an object whose key is the name of the cookie.
In fact, Express wraps the Connect framework, complementing it with some functionality, such as, for example, routing logic, which makes the routing process more smooth. Below is an example of processing a GET request:

 app.get('/hello.txt', function(req, res){ var body = 'Hello World'; res.setHeader('Content-Type', 'text/plain'); res.setHeader('Content-Length', body.length); res.end(body); }); 

Installation

There are two options for installing Express. The first is to place its description in the package.json file and execute the npm install command (such a joke goes that the name of this package manager is decoded as “no problem man” :)).

 { "name": "MyWebSite", "description": "My website", "version": "0.0.1", "dependencies": { "express": "3.x" } } 

The framework code will be placed in the node_modules directory and you will have the opportunity to create an instance of it. I prefer the alternative: command line. Just install Express by running the npm install -g express command. By doing this, you will have a new CLI tool. For example, if you run:

 express --sessions --css less --hogan app 

Express will generate the configured application framework for you. The following is a list of parameters for the express (1) command:

 Usage: express [options] Options: -h, --help output usage information -V, --version output the version number -s, --sessions add session support -e, --ejs add ejs engine support (defaults to jade) -J, --jshtml add jshtml engine support (defaults to jade) -H, --hogan add hogan.js engine support -c, --css add stylesheet support (less|stylus) (defaults to plain css) -f, --force force on non-empty directory 

As you can see, the list of available parameters is small, but this is enough for me. As a rule, I use the less library as a CSS preprocessor, as well as the hogan templating engine . In our example, we need session support, and the --sessions option will solve this problem. When the command is completed, the structure of our project will look like this:

 /public /images /javascripts /stylesheets /routes /index.js /user.js /views /index.hjs /app.js /package.json 

If you look in the package.json file, you will see that all the dependencies that we need are added here. But they have not yet been established. To do this, simply execute the npm install command, after which the node_modules directory will appear.
I understand that the approach described above is not worthy of being called universal. For example, you may need to place route handlers in a different directory, or something else. But, as will be shown in the next few chapters, I will make changes to the generated structure, which is quite simple. Based on this, I advise you to use the express (1) command simply as a template generator.

FastDelivery

For this tutorial, I created a simple web site of a fictional company called FastDelivery. Here is a screenshot of the finished design:


At the end of this guide, we will have a complete web application with a working control panel. The idea is to create the ability to manage each part of the site in separate areas for each part. The layout was created in Photoshop and cut into CSS (less) and HTML (hogan) files. I will not describe here the process of cutting, as this is not the subject of discussion in this article, but if you have any questions on this part, do not hesitate to ask. Having finished cutting, we get the following structure of the application files:

 /public /images (there are several images exported from Photoshop) /javascripts /stylesheets /home.less /inner.less /style.css /style.less (imports home.less and inner.less) /routes /index.js /views /index.hjs (home page) /inner.hjs (template for every other page of the site) /app.js /package.json 

Here is a list of site elements that we will be able to administer:

• Homepage (banner in the center - title and text)
• Blog (add, delete and edit articles)
• Services page
• Quarry page (professional growth, not sand pit :))
• Contact Page

Configuration

There are a few things we need to do before we get started. Configuring is one of those things. Let's imagine that our small stey will be deployed on three different servers - on the local, on the intermediate and on the battle. Naturally, the settings of all three environments are different, and therefore we need to implement a mechanism that is flexible enough for such conditions. As you know, each nodejs script runs as a console application. This means that we can easily attribute to the command the parameters in which the current environment will be defined. I wrapped this part in a separate module so that it would be convenient to write tests for it later. File /config/index.js :

 var config = { local: { mode: 'local', port: 3000 }, staging: { mode: 'staging', port: 4000 }, production: { mode: 'production', port: 5000 } } module.exports = function(mode) { return config[mode || process.argv[2] || 'local'] || config.local; } 

At the moment we have only two parameters - mode (mode) and port (port). As you might expect, the application uses different ports on different servers. That is why we need to update the entry point of our site, in the file app.js.

 ... var config = require('./config')(); ... http.createServer(app).listen(config.port, function(){ console.log('Express server listening on port ' + config.port); }); 

To switch between configurations, simply add the name of the environment in the first parameter of the command. For example:

 node app.js staging 

Displays:

 Express server listening on port 4000 

Now all our settings are stored in one place, they are easy to manage.

Tests

I'm a big TDD fan. I will try to cover all base classes with tests that will be mentioned in this article. I agree, writing tests absolutely takes a lot of time for everything, but in general, this is how you should create your applications. One of my favorite testing frameworks is jasmine . Of course, it is also available in the npm package manager:

 npm install -g jasmine-node 

Let's create an office for storing our tests. The first thing we will test is our configuration file. Test file names must end with .spec.js , so we will name the file config.spec.js .

 describe("Configuration setup", function() { it("should load local configurations", function(next) { var config = require('../config')(); expect(config.mode).toBe('local'); next(); }); it("should load staging configurations", function(next) { var config = require('../config')('staging'); expect(config.mode).toBe('staging'); next(); }); it("should load production configurations", function(next) { var config = require('../config')('production'); expect(config.mode).toBe('production'); next(); }); }); 

Run the jasmine-node ./tests command and you will see the following:

 Finished in 0.008 seconds 3 tests, 6 assertions, 0 failures, 0 skipped 

Now I wrote the implementation first, and only then the tests. This is not a TDD approach, but in the following chapters I will do the opposite.

I highly recommend spending enough time writing tests. There is nothing better than a fully test-covered app.
A few years ago, I realized something very important, something that can help you create better applications. Every time you start writing a new class, a new module, or just a piece of logic, ask yourself:

How can I test this?

The answer to this question will help you write code more efficiently, create good APIs, and divide everything into beautifully separated blocks. You cannot write tests for spaghetti code. For example, in the configuration file mentioned above ( /config/index.js ), I added the ability to send mode (mode) to the module constructor. You may well wonder: why did I do this, because the main idea is to get the value for the mode from the command line arguments? It's simple: because I need to test it. Let's imagine that a month later I would need to test something using the configuration for the combat server, and the node script runs with a substitute parameter. And I will not have the opportunity to make a change without this little improvement. This previous small step will prevent possible problems with application deployment in the future.

Database

If we are building a dynamic website, we need a database to store information in it. I chose mongodb for this instruction. Mongo is a NoSQL database. You will find the installation instructions here , and since I am a Windows user, I followed the installation instructions for Windows . After you finish the installation, start the MongoDB daemon, which by default listens on port 27017. So, in theory, we have the opportunity to connect to this port and exchange messages with the mongodb server. To do this through the node script, we need a mongodb module / driver . If you downloaded the source files for this instruction, then the module has already been added to the package.json file. If not, just add “mongodb”: “1.3.10” to the dependencies property and execute the npm install command. Next, we will write a test that will check if the mongodb server is running.

File /tests/mongodb.spec.js:

 describe("MongoDB", function() { it("is there a server running", function(next) { var MongoClient = require('mongodb').MongoClient; MongoClient.connect('mongodb://127.0.0.1:27017/fastdelivery', function(err, db) { expect(err).toBe(null); next(); }); }); }); 

The callback function in the mongodb client method .connect receives a db object. Later we will use it to manage our data. This means that we need access to it within our models. Creating a new MongoClient object every time you query the database is not a good idea. That is why I transferred the launch of the Express server to the callback function of connecting to the database:

 MongoClient.connect('mongodb://127.0.0.1:27017/fastdelivery', function(err, db) { if(err) { console.log('Sorry, there is no mongo db server running.'); } else { var attachDB = function(req, res, next) { req.db = db; next(); }; http.createServer(app).listen(config.port, function(){ console.log('Express server listening on port ' + config.port); }); } }); 

Since we have the environment configuration settings file, it would be a good idea to put the hostname and port of the mongodb server here and change the connection URL to:

'mongodb: //' + config.mongo.host + ':' + config.mongo.port + '/ fastdelivery'

Pay particular attention to the middleware-function attachDB , which I added right before calling the function http.createServer . Thanks to this small addition, we will populate the .db property of the request object. The good news is that we can define several functions during route determination. For example:

 app.get('/', attachDB, function(req, res, next) { ... }) 

Thus, Express calls the attachDB function in advance so that it gets into our route handler. When this happens, the request object will have the .db property and we will be able to use it to access the database.

MVC

We all know the MVC pattern . The question is how to apply it in Express. One way or another, this is a matter of interpretation. In the next few chapters, I will create modules that will work as a model, view, and controller.

Model

A model is what will process our application data. It must have access to the db object returned by the MongoClient object. Our model should also have a method for expanding it, since we may need different types of models. For example, we may want to create a BlogModel or ContactsModel model. To do this, we first need to create a test: /tests/base.model.spec.js . And remember: creating these tests, we can guarantee that our models will do only what we want, and nothing else.

 var Model = require("../models/Base"), dbMockup = {}; describe("Models", function() { it("should create a new model", function(next) { var model = new Model(dbMockup); expect(model.db).toBeDefined(); expect(model.extend).toBeDefined(); next(); }); it("should be extendable", function(next) { var model = new Model(dbMockup); var OtherTypeOfModel = model.extend({ myCustomModelMethod: function() { } }); var model2 = new OtherTypeOfModel(dbMockup); expect(model2.db).toBeDefined(); expect(model2.myCustomModelMethod).toBeDefined(); next(); }) }); 

Instead of a real db object, I decided to pass a mock object. This is done for the case, if later I want to test something specific that will depend on the data coming from the database. It will be much easier to determine this data manually.
Implementing the method extension is a bit of an easy task, since we have to change the prototype of the module.exports , but leave the original constructor intact. Fortunately, we have already written a test that checks the performance of our code. The version of the above code snippet looks like this:

 module.exports = function(db) { this.db = db; }; module.exports.prototype = { extend: function(properties) { var Child = module.exports; Child.prototype = module.exports.prototype; for(var key in properties) { Child.prototype[key] = properties[key]; } return Child; }, setDB: function(db) { this.db = db; }, collection: function() { if(this._collection) return this._collection; return this._collection = this.db.collection('fastdelivery-content'); } } 

Two auxiliary methods are defined here. Setter for db object and getter for collection from our database.

Representation

The view will display information on the screen. In simple terms, a view is a class that sends a response to the browser. Express provides an easy way to do this:

 res.render('index', { title: 'Express' }); 

The response object is a wrapper that has a good API that makes our lives easier. However, I would prefer to create my own module into which its functionality will be rolled up. The standard presentation directory will be replaced with templates, and the old views directory will give us the presentation class Base .
This small change now requires another change. We must notify Express that our template files are now in a different directory:

 app.set('views', __dirname + '/templates'); 

First of all, I will determine everything I need, write tests, and then begin implementation. We need a module that meets the following requirements:

• Its constructor should receive the response object and the name of the template.
• It must have a template output (rendering) method that accepts a data object
• It must be expandable.

You might be surprised: why did I need to expand the presentation class? Doesn't he just call the response.render method? Well, in practice there are times when you need to send a different header, or you can manipulate a response object. For example, process the data that came in JSON:

 var data = {"developer": "Krasimir Tsonev"}; response.contentType('application/json'); response.send(JSON.stringify(data)); 

Instead of doing this every time, it would be great to have the HTMLView and JSONView classes . Alternatively, the XMLView class to send XML data to the browser. It will simply be better if you create a large web application that already has this functionality implemented than if you copy and paste the same code over and over again.
Below is a test for the /views/Base.js class:

 var View = require("../views/Base"); describe("Base view", function() { it("create and render new view", function(next) { var responseMockup = { render: function(template, data) { expect(data.myProperty).toBe('value'); expect(template).toBe('template-file'); next(); } } var v = new View(responseMockup, 'template-file'); v.render({myProperty: 'value'}); }); it("should be extendable", function(next) { var v = new View(); var OtherView = v.extend({ render: function(data) { expect(data.prop).toBe('yes'); next(); } }); var otherViewInstance = new OtherView(); expect(otherViewInstance.render).toBeDefined(); otherViewInstance.render({prop: 'yes'}); }); }); 

In order to test the rendering (rendering), I had to create a layout. For such a case, in the first part of the test, I created an object that simulates the response object of the Express framework. In the second part of the test, I created another View class that inherits the base class and uses its own rendering method.
Class code /views/Base.js :

 module.exports = function(response, template) { this.response = response; this.template = template; }; module.exports.prototype = { extend: function(properties) { var Child = module.exports; Child.prototype = module.exports.prototype; for(var key in properties) { Child.prototype[key] = properties[key]; } return Child; }, render: function(data) { if(this.response && this.template) { this.response.render(this.template, data); } } } 

Now we have three tests in our test directory, and if you run the jasmine-node ./tests command , the result should be as follows:

 Finished in 0.009 seconds 7 tests, 18 assertions, 0 failures, 0 skipped 

Controller

Remember, we talked about routes and how they were determined?

 app.get('/', routes.index); 

The symbol '/' after the route in the example above is the controller. This is the same middleware function that accepts request , response and next () callback functions.

 exports.index = function(req, res, next) { res.render('index', { title: 'Express' }); }; 

The above describes what your controller should look like in the context of using Express. The express (1) command creates a directory under the routes name, but in our case it would be better if it is called controllers . I renamed it to match the file structure to this scheme.
Since we are building not just a tiny tiny application, it would be advisable to create a base class that can be extended. If we ever need to add functionality to all our controllers, this base class would be an ideal place to implement add-ons. I write the test again first, so let's define what we need for the base class:

• It must have an extensible method that accepts an object and returns a new child instance.
• The child must have a run method, which is the good old middleware function
• There must be a name property that identifies the controller.
• We should be able to create independent objects inherited from the base class.

Just a few items, but we can add more functionality later.
The test will look something like this:

 var BaseController = require("../controllers/Base"); describe("Base controller", function() { it("should have a method extend which returns a child instance", function(next) { expect(BaseController.extend).toBeDefined(); var child = BaseController.extend({ name: "my child controller" }); expect(child.run).toBeDefined(); expect(child.name).toBe("my child controller"); next(); }); it("should be able to create different childs", function(next) { var childA = BaseController.extend({ name: "child A", customProperty: 'value' }); var childB = BaseController.extend({ name: "child B" }); expect(childA.name).not.toBe(childB.name); expect(childB.customProperty).not.toBeDefined(); next(); }); }); 

And below is the implementation of /controllers/Base.js :

 var _ = require("underscore"); module.exports = { name: "base", extend: function(child) { return _.extend({}, this, child); }, run: function(req, res, next) { } } 

Of course, each descendant class must have its own run method that implements its own logic.

FastDelivery website

Well, we have a good set of classes for our MVC architecture, we also covered all the modules with tests. Now we are ready to continue to work on the site for our fictional company FastDelivery. Let's imagine that the site will consist of two parts - the user part and the administration panel. The user part will be intended for the presentation of information stored in the database to the end user. The administration panel will be to manage this information. Let's start with the last one.

Control Panel

, -.
/controllers/Admin.js :

 var BaseController = require("./Base"), View = require("../views/Base"); module.exports = BaseController.extend({ name: "Admin", run: function(req, res, next) { var v = new View(res, 'admin'); v.render({ title: 'Administration', content: 'Welcome to the control panel' }); } }); 

, . View . , admin.hjs /templates . :

 <!DOCTYPE html> <html> <head> <title>{{ title }}</title> <link rel='stylesheet' href='/stylesheets/style.css' /> </head> <body> <div class="container"> <h1>{{ content }}</h1> </div> </body> </html> 

( , . , GitHub.)
, , app.js :

 var Admin = require('./controllers/Admin'); ... var attachDB = function(req, res, next) { req.db = db; next(); }; ... app.all('/admin*', attachDB, function(req, res, next) { Admin.run(req, res, next); }); 

, Admin.run middleware- . , . :

 app.all('/admin*', Admin.run); 

this Admin .

-

Each page that starts with / admin must be protected. To achieve this, we will use the Sessions built-in middleware function in Express . It simply adds an object to the request, which is called session . Now we need to change our Admin controller to do the following two things:

• He should check if the session is available. If not, display authorization form.
• It must accept the data sent from the authorization form and authorize the user if the username and password match the username and password from the database

Below is a small helper function that will satisfy the requirements described above:

 authorize: function(req) { return ( req.session && req.session.fastdelivery && req.session.fastdelivery === true ) || ( req.body && req.body.username === this.username && req.body.password === this.password ); } 

, session . , . , request.body, middleware- bodyParser . , .
run , ( ). : , , :

 run: function(req, res, next) { if(this.authorize(req)) { req.session.fastdelivery = true; req.session.save(function(err) { var v = new View(res, 'admin'); v.render({ title: 'Administration', content: 'Welcome to the control panel' }); }); } else { var v = new View(res, 'admin-login'); v.render({ title: 'Please login' }); } } 

Content Management

, , . , . : , , . . , “contacts”, . , , , . , , , , MongoDB- , , , API.

 // /models/ContentModel.js var Model = require("./Base"), crypto = require("crypto"), model = new Model(); var ContentModel = model.extend({ insert: function(data, callback) { data.ID = crypto.randomBytes(20).toString('hex'); this.collection().insert(data, {}, callback || function(){ }); }, update: function(data, callback) { this.collection().update({ID: data.ID}, data, {}, callback || function(){ }); }, getlist: function(callback, query) { this.collection().find(query || {}).toArray(callback); }, remove: function(ID, callback) { this.collection().findAndModify({ID: ID}, [], {}, {remove: true}, callback); } }); module.exports = ContentModel; 

The model takes care of generating a unique ID for each entry. We need it to update the information later.
If we need to add a new entry to the contact page, we can write the following:

 var model = new (require("../models/ContentModel")); model.insert({ title: "Contacts", text: "...", type: "contacts" }); 

, API mongodb. UI, . Admin . , / . , , — .

, , , , , , . , :

 var self = this; ... var v = new View(res, 'admin'); self.del(req, function() { self.form(req, res, function(formMarkup) { self.list(function(listMarkup) { v.render({ title: 'Administration', content: 'Welcome to the control panel', list: listMarkup, form: formMarkup }); }); }); }); 

, , , . — del , GET-, , action=delete&id=[id of the record] , . form , . , . list HTML-, . .
, :

 handleFileUpload: function(req) { if(!req.files || !req.files.picture || !req.files.picture.name) { return req.body.currentPicture || ''; } var data = fs.readFileSync(req.files.picture.path); var fileName = req.files.picture.name; var uid = crypto.randomBytes(10).toString('hex'); var dir = __dirname + "/../public/uploads/" + uid; fs.mkdirSync(dir, '0777'); fs.writeFileSync(dir + "/" + fileName, data); return '/uploads/' + uid + "/" + fileName; } 

, .files request- . HTML-:

 <input type="file" name="picture" /> 

, req.files.picture . , req.files.picture.path () . , , , URL . , readFileSync , mkdirSync writeFileSync .

. - , ContentModel , , . , . – /controllers/Home.js

 module.exports = BaseController.extend({ name: "Home", content: null, run: function(req, res, next) { model.setDB(req.db); var self = this; this.getContent(function() { var v = new View(res, 'home'); v.render(self.content); }) }, getContent: function(callback) { var self = this; this.content = {}; model.getlist(function(err, records) { ... storing data to content object model.getlist(function(err, records) { ... storing data to content object callback(); }, { type: 'blog' }); }, { type: 'home' }); } }); 

“home” “blog”.
, app.js :

 app.all('/', attachDB, function(req, res, next) { Home.run(req, res, next); }); 

db request-. , -.
, , , , , . , . . , . :

 app.all('/blog/:id', attachDB, function(req, res, next) { Blog.runArticle(req, res, next); }); app.all('/blog', attachDB, function(req, res, next) { Blog.run(req, res, next); }); 

: Blog , run . /blog/:id . URL', : /blog/4e3455635b4a6f6dccfaa1e50ee71f1cde75222b , req.params.id . , . ID (). , , .
, , . , . , type . , , run . :

 app.all('/services', attachDB, function(req, res, next) { Page.run('services', req, res, next); }); app.all('/careers', attachDB, function(req, res, next) { Page.run('careers', req, res, next); }); app.all('/contacts', attachDB, function(req, res, next) { Page.run('contacts', req, res, next); }); 

:

 module.exports = BaseController.extend({ name: "Page", content: null, run: function(type, req, res, next) { model.setDB(req.db); var self = this; this.getContent(type, function() { var v = new View(res, 'inner'); v.render(self.content); }); }, getContent: function(type, callback) { var self = this; this.content = {} model.getlist(function(err, records) { if(records.length > 0) { self.content = records[0]; } callback(); }, { type: type }); } }); 

Deployment

- Express, , Node.js:

• Node- ( )
• npm install
• node-

, Node — , , , . , forever Node.js. , :

 forever start yourapp.js 

, . . node yourapp.js , , , forever .
, node- Apache Nginx , , . , Apache 80, , localhost localhost :80, , Apache-, , , node- . , . , , , Apache-, expresscompletewebsite.dev . , , hosts.

 127.0.0.1 expresscompletewebsite.dev 

httpd-vhosts.conf, Apache-:

 # expresscompletewebsite.dev <VirtualHost *:80> ServerName expresscompletewebsite.dev ServerAlias www.expresscompletewebsite.dev ProxyRequests off <Proxy *> Order deny,allow Allow from all </Proxy> <Location /> ProxyPass http://localhost:3000/ ProxyPassReverse http://localhost:3000/ </Location> </VirtualHost> 

80, 3000, node-.
Nginx , , , Node.js. , Apache, hosts . /sites-enabled , , Nginx. :

 server { listen 80; server_name expresscompletewebsite.dev location / { proxy_pass http://127.0.0.1:3000; proxy_set_header Host $http_host; } } 

, Apache Nginx . , 80. , , . , .

Conclusion

Express — , -. , , . , middleware' .

Source

-, , GitHub – https://github.com/tutsplus/build-complete-website-expressjs . . -:

• npm install
• mongodb
• node app.js