Isomorphic JavaScript Applications with Catberry.js

UPD:
Time passed ... The framework has evolved and a lot of things from this article are already out of date.
But no matter what, fresh material can be found here on these slides , and there is still a video for them.

Catberry.js is a framework for developing isomorphic JavaScript applications on node.js using modular architecture and fast rendering mechanisms. This framework allows you to write an application module once and use it both on the server for rendering pages for search robots, and in the browser for a single-page application, requesting only data for templates.
')
For the first time I personally saw the term “isomorphic applications” on the Airbnb company's blog ; you can read the translation of this article in Habré , although this term began to sound a bit earlier, for example, in the nodejitsu blog . Therefore, it is a little difficult to say who invented this, but the fact is that nowadays there is a whole class of web applications that are commonly called isomorphic. On Habré, this term was mainly mentioned in the articles about frameworks from gritzko and zag2art .

What is isomorphic applications?

Seeing this strange name, many developers get scared and try to avoid becoming acquainted with it. And it’s in vain, I think, that exactly isomorphic applications are the future (and, maybe, even now) of web applications.

No matter how scary it sounds, in fact, everything is quite simple - these are applications that can reuse the server code in the browser and behave in the same way both on the server and in the browser. In other words, you write once the code of your application and get a server back-end, which renders pages for search robots and responsive one-page application in the browser. It, in turn, may not load the HTML byte from the server anymore, but only request data for rendering in the browser. Almost a dream, is not it?

The emergence of a class of isomorphic applications is based on the desires of developers, for example:

• have a modern one-page application that works without reloading the page;
• do not sacrifice SEO, so that for search robots it would be a regular website that is loaded from the server;
• do not repeat the page rendering logic twice for the server and the browser; one code must be used for this;
• in the case of JavaScript applications, this should be a common language environment for the developer — no need to spend time on context switches;
• HTML rendering on the server should occur only at the first load, and then the browser is engaged in rendering, thus we will unload the server;
• All this should be easy for the developer.

Personally, I want all this as much as I do web development, and, obviously, I'm not the only one, as there are already many frameworks for developing isomorphic applications in the world of rainbow, unicorns and isomorphism:

• Rendr (Airbnb);
• Kraken (PayPal);
• Mojito (Yahoo);
• Meteor ;
• Derby ;
• probably another couple dozens of lesser-known solutions.

There is also an approach like MEAN (MongoDB + Express + Angular + node.js), which makes Angular applications isomorphic.

Why not just take one of them?

When I wanted to start a new web project, I began to study all the existing solutions at that time and saw a number of shortcomings:

• Some solutions used front-end frameworks for rendering on back-end. This meant virtualization and building DOM directly on the server, and then rendering it in HTML. This approach seemed extremely inefficient both in memory and in complexity, which would definitely lead to poor performance. For example, Rendr uses Backbone, Mojito - YUI, MEAN - Angular.
• Another disadvantage is dependence on a specific database. I have nothing against MongoDB and even used it myself several times, but sometimes we need reliability and transactionality, and sometimes the lack of a scheme and speed. I believe that the developer should not be limited in the choice. We are talking about Rendr, Meteor, Derby, which are tightly attached to MongoDB, and Derby still requires Redis.
• Real-time data binding is, of course, a very useful feature for applications such as SCADA (APCS) or applications for collaboration on anything. However, if you need to implement a regular website, as developers do on RoR or a PHP framework, these are unnecessary difficulties.

I want to be able to develop isomorphic and simple sites so that they work quickly and simply. And since my search ended in failure, I realized that it is possible to develop a framework that fills this niche, at least for me.

atberry.js

This framework is called Catberry.js, and now it is already in version 3.0. Not that it is old, but Semantic Versioning 2.0.0 is used for versioning, and the framework has undergone changes a couple of times without backward compatibility. Catberry is a fairly lightweight framework with its own ideology that says “Storage and data processing should not be part of the Catberry application, it should be done by a separate RESTful service”.

Familiarity with the framework should start with the approach of SMP (Service-Module-Placeholder), which replaces the usual MVC (Model-View-Controller), but I promise it will look familiar. Again, it’s worth mentioning that I’m not against MVC, it’s very good, but for a certain class of applications where real-time data updating is needed. The MVC that is used now in the development of web applications is often not what was originally thought of as MVC, but as some kind of interpretation. So I decided to call my interpretation differently, so as not to confuse my colleagues.

Service-Module-Placeholder

As the name implies, the application is built on three components.

Service

Service is an external component that is a RESTful service to which our Catberry application constantly refers. This service can be implemented on absolutely any platform: Erlang, Go, PHP, .NET, whatever. You are limited to HTTP 1.1 protocol only.

Module

A module in the concept of a Catberry application is a logic set that prepares data for a template engine and handles user events. In other words, if you need to render part of the page, Catberry finds the module responsible for this part of the page and asks him to prepare the data context for the template, taking into account the current state of the entire application (for example, the parameters in the current URL).

Placeholder

At first glance, you can say that this is just a template. In fact, it may have HTML elements that refer to other placeholders, and such links may appear dynamically during rendering, and this will insert another placeholder inside the current one. For example, the contents of the element placeholder “paw” of the module “cat”. Why id is used, you ask? And in order that placeholders can be very quickly found in the DOM, when rendering works in the browser. It really saves time.

Placeholder to Module

As mentioned, the placeholder is associated with the module, or rather, the module owns a certain set of placeholders, and no one else can belong to these placeholders. Hence the question may arise: “But how can you split the application into placeholders and modules?”. There are two fairly simple rules:

1. If a section of a web page can be displayed in one request to a RESTful service, then most likely it is a placeholder.
2. If several placeholders depend on the same parameters in the URL and if you change these parameters you need to update them at the same time, then these placeholders should be grouped into one module.

Differences from MVC

Probably, one of the readers will surely think: “What is the difference from MVC?”. If you try, you can say that Service == Model, Controller == Module and View = Placeholder. But this is exactly what I said earlier, the term MVC in our time is very distorted, and when people interpret it in their own way, they call it MVC. I found it necessary to specify a different name, because:

• Service is not part of the application, it is an external application with which the Module communicates via HTTP requests; this cannot be called a Model;
• as a result, we do not have data storage and processing in a Catberry application, which means there is no Model at all;
• According to the classic description of MVC with an active model, it should inform all interested View about its changes, as it is done, for example, in Meteor. There is nothing like that in Catberry.
• there is also a MVC with a passive model, but in this case, the Controller should track the model changes and update the View, nothing like that in the Catberry either;
• instead, updates come only from the user’s actions when he changes the URL or submits the form data. Of course, no one bothers you to additionally update Placeholder by calling an update request in the Module code, for example, by events or using long-polling. But the content of the placeholder will depend on the state of the application described in the URL. This is the meaning of the approach - the ability on the server and in the browser at the URL to fully restore the state of the application in order to render identical pages.

How it works

The Catberry application works exactly as I previously described isomorphic applications:

• First, the user makes a request to the server by URL;
• server renders the page at the specified URL;
• the browser version of the application is loaded into the browser along with the page;
• and then everything works without reloading the page as a one-page application.

For a better understanding of the picture

Stream server rendering

As can be understood from the description and schema, when rendering takes place on the server, to draw each placeholder, you usually need to make a request for a RESTful service. This may take considerable time for the user. So that the user does not look at a blank page with a spinning loader while we make requests for all placeholders, a streaming-based rendering engine has been developed. Its task is to deliver the page to the browser as soon as it is ready. In other words, as soon as the request to the service to render the page title is completed, the user will immediately see this title in his browser, without waiting for the entire page to be ready.

I myself do not like very much when when I open the page for a few seconds I see a white screen and I don’t even know if my request reached the back-end, or I’ll see “504 Gateway Timeout” now. Usually I close such sites in 3-4 seconds of a white screen.

With streaming rendering, I immediately see the response and the fact that the site works for me, tries and puffs, collecting data for rendering. Another nice thing is that streaming does not buffer the data in a large amount, which will save the memory of our server with the application. Well, the most pleasant moment is that the browser, having received the HEAD element that comes from the server almost immediately, starts parsing JavaScript and CSS, and also loads all the resources specified in the page, all this will work in parallel, as in the diagram below.

Parallel browser rendering

Streaming rendering is, of course, good, but only when we are limited to the sequential loading of a page along a stream of a TCP connection. When we already have a finished page in the browser, and the user clicks on the link, we need to rebuild part of the page for the new state of the application, here we are not limited by anything. We can execute queries to the RESTful service in parallel, and by the results immediately update the placeholders. And if inside there are references to others, then again in parallel to request data for them. This results in an incredibly fast rendering of placeholders in the browser. In addition, if one of the requests receives a response for a very long time, it will not affect the rendering of the remaining placeholders.

Tools for isomorphism

When we develop a web application, we often need to take advantage of actions that depend on the implementation in a particular environment, that is, they work differently in the browser and on the server. For this, Catberry has isomorphic implementations of such actions. Outwardly, they work identically, have the same software interface, but internally they are implemented using the tools of the current environment. Here is a list of such implementations:

• getting Location;
• getting Referrer;
• getting User Agent;
• cleaning URL fragment (hash);
• getting or setting cookies;
• Redirect;
• HTTP / HTTPS requests;
• data cache used for the last rendering of each placeholder.

It is this API that makes the application isomorphic on Catberry.js.

How does the application on the Catberry

Service Locator and DI

The framework architecture is based on the implementation of the patterns Service Locator and Dependency Injection .

For example,

var cat = catberry.create(config); //    cat.locator.register('uhr', UHR); //      cat.locator.registerInstance('uhr', new UHR()); //    cat.locator.resolve('uhr'); //   cat.locator.resolveAll('uhr'); //       cat.locator.resolveInstance(SomeConstructorDependsOnUHR); //      //   : function ModuleConstructor ($uhr, someConfigSection) { //    $uhr //     someConfigSection } 

Such dependency implementations do not break when minified, as it is done by the framework itself using UglifyJS2 .

How the module is arranged

Each module is a directory with an index.js file that the module constructor must export (the module is a constructor with a declared prototype). The module can also have a placeholders directory in which the module placeholders templates are located.

Methods

Each module can implement three groups of methods: render, handle, and submit. The naming convention is used here - if your placeholder is called some-awesome-placeholder, then you must implement the renderSomeAwesomePlaceholder method if you want to prepare data for it. You can and do not implement, nothing will break from this, and the template will be rendered with an empty context, which is also quite acceptable. This convention applies to the handle / submit methods that process events from the page.

An example of the implementation of all three methods:

 ModuleConstructor.prototype.renderSome = function () { //   return {some: data}; //  Promise }; ModuleConstructor.prototype.handleSome = function (event) { // -   // event.args // event.element // event.isEnding // event.isHashChanging //   Promise }; ModuleConstructor.prototype.submitSome = function (event) { //    // event.values // event.element }; 

Sometimes it is necessary to bind to DOM elements after the placeholder is rendered, for this are provided after methods, for example, for the renderSome method above:

 ModuleConstructor.prototype.afterRenderSome = function (dataContext) { //        }; 

You can add such methods also for the handle and submit methods.
An example of the implementation of the module can be found on Github .

Promises

As mentioned in the examples, everywhere where asynchronous calls are used, Promises are used in Catberry (there was a great article recently). And if they already exist in the browser , the native implementation will be used, otherwise the polyfill library from one of the authors of the specification will be used. Type Promise, while available globally, you do not need to connect anything, as if you are working with native promises.

Where used

Now based on the framework, the Confettin project site has already been launched, where you can experience the performance and responsiveness of the Catberry-based application. In addition, the next version of Flamp is being developed with might and main, which will see the light in the foreseeable future. What I personally look forward to.

Where to begin

If this rather brief description of the framework interests you, then you can start with these two lines in the terminal:

 npm -g install catberry-cli catberry init example 

This way you will get the code of the working example that works with the GitHub API and instructions on how to run it. This example shows typical implementations of things that often have to be done in a web application. The same CLI-utility can do much more interesting. For example, create a new project or add a module to the project.

If you do not want to install anything on your computer or there is no such possibility, there is the same finished project on Runnable , but there you can exceed the limit of requests to the GitHub API.

Detailed documentation and examples can be found on the official website .
And, of course, the repository page on GitHub and Twitter account catberryjs , in which there is always the latest news about the framework.