Spirit: Node.js MVC Framework

Hi guys! From this point on, I want to start a stateful cycle with details on creating my own MVC framework for node.js , the name of which will be Spirit.

The first article will consist of four parts:
1. The idea and mission of the framework
2. Setting up the server
3. Creating framework framework
4. Creating an advanced and convenient router

I warn you right away that the article is huge, with a bunch of text and big blocks of code.
')

The idea and mission of the framework

Spirit will evolve slowly, as inspiration, mood and time (which is now a little). Although criticism and suggestions are welcome - I will develop it according to my own vision and, if the first article is warmly accepted by the community, putting the details of each logical section as an article.

The main objectives are training and warming up interest in the platform. I will be very happy if someone forks the project and makes a standing framework on its basis, you can even turn to me for moral support. However, if you want to take part in the development of Spirit with a detailed description of your actions, than to earn yourself a positive (real) karma - write to shocksilien@gmail.com, we will discuss).

All source code will be available on GitHub link github.com/theshock/spirit

Although I will try to describe the details, the style of presentation implies that the reader knows at least the basics of administration, programming in javascript and node.js in particular, and has an idea of ​​CMF. In some places, to unload the article, I cut out the pieces of code, leaving only comments, I hope you will think it out yourself. In any case - a complete and working example is on the GitHub.

In the examples, I assume that we use the Debian system, and the user's home directory is "/ home / spirit". The site will be located on spirit.example.com , unless otherwise noted.

Server Tuning

Unlike nginx and apache, node.js out of the box is not auto-running on the server and some configuration needs to be done.

First, with the help of init we create a daemon, and then with the help of monit check whether the server has crashed. This topic has long been painted, information can be found in Google , for example on the site nodejs.ru . This approach was tested and approved by me personally using the example of the abbreviation of references to node.js , which in a few hours reduced more than 15,000 references and did not even flinch, it still stands (almost 20 days).

The second step is the nginx as a frontend.

 server { listen 80; server_name spirit.example.com; access_log /home/spirit/example-app/logs/nginx.access.log; location / { proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_pass http://127.0.0.1:8124/; } location ~* \.(html|js|ico|gif|jpg|png|css|doc|xml|rar|zip|swf|avi|mpg|flv|mpeg|wmv|ogv|oga|ogg)$ { root /home/spirit/example-app/public; } } 

We see a very simple code - nginx redirects everything except static to our node.js, which will be located on port 8124 (or any other one that you specify). Statics will be given without any participation of node.js directly by nginks.

Create a skeleton framework

So, we will be committed to this directory structure:

/home/spirit/
├ lib/
│ ├ mysql-libmysqlclient/
│ ├ spirit/
│ └ MooTools.js
├ example-app/
│ ├ engine/
│ │ ├ classes/
│ │ └ init.js
│ ├ logs/
│ └ public/
└ another-example-app/
├ engine/
│ ├ classes/
│ └ init.js
├ logs/
└ public/


There are two key ideas in this scheme:
1. All libraries, incl. and Spirit will be located in separate application directories from applications - this will allow keeping several applications on one server without duplicating the list of all libraries.
2. Each application has two key directories - the engine, in which we will store the server logic and public, all the content of which we will give to the client. This will protect the server code from any encroachments outside and delimit various logic.

In the lib directory, we will drop all the libraries we need.

Spirit is based on the modified MooTools, how to connect it, I told in the topic about the abbreviation of links . For ease of connection, I hooked MooTools.More Class.Binds into the lib/MooTools.js file , which will allow passing methods of objects as arguments without losing context and slightly expanded the prototype string by adding the htmlEscape , ucfirst and lcfirst .

Application Initialization

The connection (require) of libraries in the code occurs by one of the following methods:
1. By name. For example, require( 'fs' ) . Thus, we include one of the libraries wired into the kernel and described in the documentation .
2. By relative to the current file path. For example, require( './classes/foo' ) or require( '../lib/mootools.js' )
3. On the absolute path. For example, require( '/home/spirit/lib/mootools' )

The second and third paths, in fact, are a link to the javascript file, but without the .js at the end (of course, otherwise there is a chance to catch the error). I recommend that you protect yourself from using relative paths as much as possible, since they work inconsistently, depend on the environment, and they also have a different root in each file.

 // __dirname -  ,   //  ,      var libPath = __dirname + '/../../lib'; //  : /home/spirit/lib //            // fs.realpathSync(libPath),     //    MooTools   , //      require(libPath + '/MooTools').apply(GLOBAL); //       var spirit = require(libPath + '/spirit/spirit') .createSpirit(__dirname, libPath); //     -   //    ,   require('http').listen spirit.listen("127.0.0.1:8124"); 

We write the main class

Everything that we want to pass to make it possible to export - we have to add to the exports object. That is, if we want to do var foo = new (require( './bar' ). bar ); , in the bar.js file bar.js we have to do this: exports . bar = new Class({ /* ... */ }); . First of all, I prefer to declare such variables not in a dot, but in square brackets, then in editors it is highlighted as a string (of which there is usually the least), which additionally highlights the name of the class with which we work. Secondly, at first I didn’t like this approach and I even found it inconvenient (in the example above, there is a need to require repeat the bar twice), but in the end we will turn it so that it plays into our hands - it will be comfortable and beautiful. So, the function to create the main framework class:

 exports['createSpirit'] = function (projectPath, libPath) { return new Spirit(projectPath, libPath); }; 

In the class itself, we implement the following ideas:
1. All classes are in app/classes and lib/spirit/classes
2. When loading a class by name, we first check the application directory, then the framework (unless stated otherwise). Thus, it will be possible to overload the framework classes if necessary (this detail will be described later)
3. Loaded classes are cached, thus avoiding unnecessary access to the file system (or is it already cached in node.js?)
4. The class will be the starting point for all other activities.

I will post the class code on pastin, and here I will describe only the interface

pastebin.com/0b14MEbe

 var Spirit = new Class({ Binds: ['hit', 'load'], initialize : function (String projectPath, String libPath) { }, load : function (String className, Boolean fromSpirit) { //   load    , : // spirit.load('Controllers.User'); //     ,  ,    -   //  fromSpirit    //  ,    }, loadLib : function (libName) { //     , : // spirit.loadLib('mysql-libmysqlclient') //     libPath //           }, //       hit : function (req, res) { }, listen : function (port, url) { //     //  .listen(8124, '127.0.0.1')  //  .listen('127.0.0.1:8124') } }); var load = function (path, className) { // some code is here //         //    Spirit.      ? //  ! return data[className](this); }; 

This approach leads to the following style for creating classes.

 exports['Name.Space.ClassName'] = function (spirit) { return new Class({ method : function (msg) { this.response.writeHead(200, {'Content-Type': 'text/plain'}); this.response.end(msg); } }); }; 

It is not yet clear why this is necessary?

 exports['Helper.AnotherClass'] = function (spirit) { var mysql = spirit.loadLib('mysql-libmysqlclient'); return new Class({ Extends : spirit.load('Name.Space.ClassName'), start : function (msg) { this.method('Started'); }, query : function () { // using mysql } }); }; 

How would you include Name.Space.ClassName ? require( '../Name/Space/ClassName' ); ? And if the class to move to another place - would rewrite all the way? Would a library be loaded by writing the full path?
Let's look at another example. Suppose our framework has a Router class that handles every hit:

 exports['Router'] = function (spirit) { return new Class({ // .. hit : function (request, response) { // some code }, // .. }); }; 

We want to enter hit logging. We declare the class Router in the directory of our application:

 exports['Router'] = function (spirit) { var Logger = spirit.load('Logger'); var logger = new Logger; return new Class({ Extends : spirit.load('Router', true), hit : function (request, response) { logger.log('hit!'); this.parent(request, response); }, // .. }); }; 

Thus, we can extend classes by changing the framework's behavior and not getting into the source code.

Create a router

The router will parse the requests to it and give it to the desired controller.
Routing will be as follows:
1. First, using regexps, a match is checked with one of the addresses of manually added routes. This will allow you to enter special URLs that do not fall under the default principle of the router.
2. If the route is not found in point 2, the address is broken down by slashes and the closest one is searched for among the controllers. With the url / AA / BB / CC / DD / EE address, the AA.BB.CC.DD.EE controller is first searched, then AA.BB.CC.DD and so on until the desired one is found. If there is no such controller, the Index controller is substituted. If there is no AA.BB.CC.DD controller, but AA.BB.CC.DD.Index, then it will be selected. Then the method is selected and everything else is passed as arguments.

All classes of controllers will be loaded when the application is initialized, thus avoiding unnecessary miscalculations with each request.

This is how the catalog of our application will look like:

/home/spirit/example-app/
├ engine/
│ ├ classes/
│ │ ├ controllers/
│ │ │ ├ Admin/
│ │ │ │ ├ Articles.js
│ │ │ │ └ Index.js
│ │ │ ├ Man/
│ │ │ │ ├ Index.js
│ │ │ │ └ Route.js
│ │ │ ├ Index.js
│ │ │ └ Users.js
│ │ └ Controller.js
│ └ init.js
├ logs/
└ public/


Add the following code to our init-file of the application, which will demonstrate the approach to manual routes. : A,: D,: H,: W correspond to the following patterns: [az], [0-9], [0-9a-f], [0-9a-z], respectively. <and> are responsible for ensuring that this expression is at the beginning and at the end of the address, respectively. Thus, an expression, for example, </test-:A> does not match the URL " /test-abc123 ", while the expression /test-:A matches this URL. All templates are added to an array and passed as an argument when calling the controller method. If there is an argsMap, then the Hash is passed. For example, at the address " /articles-15/page-3 " the first argument will be an array [15, 3] , but if you pass argsMap : ['id', 'page'] , the hash will be passed to the method {id:15, page:3} .

 spirit.createRouter() .addRoutes( { route : "</article-:D/page-:D" , contr : 'Man.Route:articleWithPage' , argsMap : ['id', 'page'] }, { route : "</article-:D" , contr : 'Man.Route:article' , argsMap : ['id'] }, { route : "</~:W>" , contr : 'Man.Route:user' }, { route : "</hash-:H>" , contr : 'Man.Route:hash' } ); 

But the controller that works with this code (in the diagram above it is bold):

 exports['Controllers.Man.Route'] = function (spirit) { return new Class({ Extends : spirit.load('Controller'), indexAction : function () { this.exit('(Man.Route) index action'); }, testAction : function () { this.exit('(Man.Route) test action'); }, articleWithPageAction : function (args) { this.exit('(Man.Route) article #' + args.id + ', page #' + args.page); }, articleAction : function (args) { this.exit('(Man.Route) article #' + args.id); }, hashAction : function (args) { this.exit('(Man.Route) hash: ' + args[0]); }, userAction : function (args) { this.exit('(Man.Route) user: ' + args[0]); } }); }; 

And the parent controller, which we will use in order for all children to set the exit method:

 exports['Controller'] = function (spirit) { return new Class({ exit : function (msg) { this.response.writeHead(200, {'Content-Type': 'text/plain'}); this.response.end(msg); } }); }; 

First you need to expand the Spirit class, we will shift the analysis of requests completely onto Router’s shoulders:

  createRouter : function () { var Router = this.load('Router'); var router = new Router(); router.spirit = this; this.router = router; router.init(); return router; }, hit : function (req, res) { this.router.hit(req, res); }, 

The router itself will also not be particularly heaped up and will give the main work to its deputy:

 exports['Router'] = function (spirit) { var RouterHelper = spirit.load('Router.Helper'); return new Class({ init : function () { var path = this.spirit.requirePath + 'Controllers'; this.routerHelper = new RouterHelper(this); this.routerHelper.requireAll(path); }, hit : function (request, response) { var contrData = this.routerHelper.route(request); var contr = contrData.contr; contr.spirit = this.spirit; contr.request = request; contr.response = response; if (typeof contr.before == 'function') contr.before(); contr[contrData.method](contrData.args); if (typeof contr.after == 'function') contr.after(); }, addRoutes : function () { var rh = this.routerHelper; rh.addRoutes.apply(rh, arguments); } }); }; 

The Deputy also has two subordinates - the one who is responsible for manually added routes (RouterRegexp) and who is the default way (RouterPlain). Pay attention to the requireAll method - it synchronously recursively manages the controller directory and all classes are connected. In this case, asynchronous is not necessary, since this method is called only when the project is initialized, but in real code it is advisable to write such things in an asynchronous style — the time to access the file system will not slow down the code execution process. I like Node.js because, unlike some other languages , all method names are clear, beautiful, short and follow the same style.

 var fs = require('fs'); exports['Router.Helper'] = function (spirit) { var RouterPlain = spirit.load('Router.Plain'); var RouterRegexp = spirit.load('Router.Regexp'); return new Class({ initialize : function (router) { this.router = router; this.plain = new RouterPlain(this); this.regexp = new RouterRegexp(this); }, route : function (request) { var url = request.url; return this.regexp.route(url) || this.plain.route(url); }, requireAll : function (path) { var files = fs.readdirSync(path); for (var i = 0; i < files.length; i++) { var file = path + '/' + files[i]; var stat = fs.statSync(file); if (stat.isFile()) { this.addController(file); } else if (stat.isDirectory()) { this.requireAll(file); } } this.checkAllIndexActions(); }, //       regexp  addRoutes : function (routes) {}, //  ".js"       removeExt : function (file) {}, //      indexAction checkAllIndexActions : function () {}, //       addController : function (file) {}, //     false,   createController : function (name) {} }); }; 

First of all, it is necessary to disassemble the regulars that are passed using addRoutes

 exports['Router.Regexp'] = function (spirit) { return new Class({ initialize : function (routerHelper) { this.routerHelper = routerHelper; }, route : function (url) { //            for (var i = 0; i < this.routes.length; i++) { var route = this.routes[i]; //           //      lastIndex,  //        , //   route.regexp.lastIndex = 0; var result = route.regexp.exec(url); if (result) { return { contr : this.routerHelper .createController(route.contr.name), method : route.contr.method, args : this.regexpRouteArgs(result, route.argsMap) }; } } return false; }, routes : [], addRoute : function (route, controller, argsMap) { }, addRoutes : function (hash) { }, //     ,   addRoutes //         , //  ( )) -    regexpContr : function (string) { var parts = string.split(':'); var method = parts.length > 0 ? parts[1] + 'Action' : 'indexAction'; var contr = 'Controllers.' + parts[0]; // ... }, //         //      // ,        regexpRoute : function (route) { var re = new RegExp(); re.compile(this.prepareRegexp(route), 'ig'); return re; }, replaces : { A : '([az]+)', D : '([0-9]+)', H : '([0-9a-f]+)', W : '([0-9a-z]+)', }, prepareRegexp : function (route) { return route .escapeRegExp() .replace(/>$/, '$') .replace(/^</, '^') .replace(/:([ADHW])/g, function ($0, $1) { return this.replaces[$1]; }.bind(this)); }, //  ,   regexp.exec  //    input  lastIndex  //  ,   argsMap regexpRouteArgs : function (result, argsMap) { }, }); }; 

When the router did not work according to regular expressions, we use the default router:

 var url = require('url'); exports['Router.Plain'] = function (spirit) { return new Class({ initialize : function (routerHelper) { this.routerHelper = routerHelper; }, route : function (url) { var parts = this.getControllerName(url); var controller = this.routerHelper.createController(parts.name); var method = 'indexAction'; if (parts.args.length) { var action = parts.args[0].lcfirst(); //    -      //     -  ,  -  indexAction if (typeof controller[action + 'Action'] == 'function') { method = action + 'Action'; parts.args.shift(); } } return { contr : controller, method : method, args : parts.args }; }, getControllerName : function (url) { var controllers = this.routerHelper.controllers; //  pathname       var path = this.splitUrl(url); var name, args = []; do { if (!path.length) { //     - name = 'Controllers.Index'; break; } //         name = 'Controllers.' + path.join('.') + '.Index'; if (controllers[name]) break; //  -      name = 'Controllers.' + path.join('.'); if (controllers[name]) break; //    -    args.unshift(path.pop()); } while (true); return { name : name, args : args }; }, splitUrl : function (urlForSplit) { return url .parse(urlForSplit, true) .pathname.split('/') .filter(function (item) { return !!item; }) .map(function (item) { return item.ucfirst(); }); }, }); }; 

Extension classes

UPD: In the comments I was asked to show the advanced router when in the URL you need to specify links to two different files, for example when comparing revisions in the repository. But the links should not be by identifier, but by the way, for example " shock/spirit/init.js/f81e45 ". I suggest using this link template:
http://example.com/compare/( shock/spirit/init.js/f81e45 )/( tenshi/spirit/src/init.js/d5d218 ) , in which each file is shown in brackets. But the framework tools do not allow this. No problem. In our project (without touching the framework) we create the class Router.Regexp:

 exports['Router.Regexp'] = function (spirit) { return new Class({ Extends : spirit.load('Router.Regexp', true), prepareRegexp : function (route) { return this.parent(route) .replace(/:P/g, '([0-9a-z._\\/-]+)'); } }); }; 

We have introduced a new modifier - ": P". It would probably be more correct to simply extend the Router.Regexp.replaces object, but I wanted to show the possibilities of overloading methods. Great, now add a new route to init.js:

 spirit.createRouter() .addRoutes( // ... { route : "</compare/(:P)/(:P)>" , contr : 'Man.Route:compare' } ); 

And add the method to Man.Route:

  compareAction : function (args) { this.exit('Compare "' + args[0] + '" and "' + args[1] + '"'); } 

Go to http://example.com/compare/( shock/spirit/init.js/f81e45 )/( tenshi/spirit/src/init.js/d5d218 ) and get the answer:

Compare " shock/spirit/init.js/f81e45 " and " tenshi/spirit/src/init.js/d5d218 "

Conclusion

So, we created our project, forced the server to display it on the web, learned how to include classes and, after analyzing the address, send the action to the required controller. In the following articles we will connect the View as some kind of template engine, and a Model in which information about our objects will be stored. After a couple of articles, we will try to write a blog on our framework. Interesting?