Design Template Projects or the complexity of managing large projects

Prehistory

Modern frontend is actively moving towards complication, and this complexity must be controlled. To begin with, I would like to note that the problem has existed for a long time, and several of its solutions have been tested and even adapted, but the concrete idea should have crystallized and shaped. What will be discussed further.

The company I am currently working on is developing a frontend for various customers, and the size of projects varies from very small to very solid (for example, Enterprise systems). We are faced with the fact that solutions that are well suited for large projects are poorly adapted for small ones, and vice versa - that in small projects it works very well does not meet the requirements of large ones.

In the spirit of the times, we can say that we need our own boilerplate for projects, which allows us to effectively structure projects of various scales so as to keep their complexity under control and not allow it to grow arbitrarily, because This can lead either to a failure of the project deadlines, or to a lot of trouble. And, of course, do not forget about the cost of support and development.

The initial implementation assumed a simple division of the project into independent parts, and the main emphasis was placed on the development of a tool to facilitate the work with the project. The project began to be a mono-repository under the management of lerna, which was formed on the basis of common modules, openly published on github, as well as specific modules that are in the project itself. An event bus was used to communicate between these modules, and in general, this implementation worked stably. However, this structure had a significant drawback: the modules that were not planned to be changed within the project still needed to be connected as part of the mono-repository, because otherwise we received different versions of the same package in different places. This, of course, is not a problem, but only if it is not a package with configs.

The second important problem was the description of the method of writing plug-ins, because, for all their modularity, they still turned out to be strongly connected, and the general concept of communication between modules was absent. We needed some kind of holistic specification for writing such plug-ins by any person, which would also solve the issue of resolving dependencies at the start level of the application.

Next, I will try to describe our version of eliminating these problems and begin by describing the ideal possible result.

Disclaimer

Written below does not pretend to be the ultimate truth and is just another opinion. We are looking for our own way and we want to make the world a little better and more convenient, at least for ourselves.

Perfect possible result

1. Modularity

Problem : monolithic application. Steps are already being taken to divide the client code into separate parts within a certain monolith, but in essence the project remains as one, and it is difficult to quickly replace or remove a certain part in it, since it does not have clear boundaries.

Solution : Break the project into physically demarcated modules with clearly defined boundaries and responsibilities. You also need the ability to configure the behavior of one module from another.

Profit : The project can be assembled as a designer. That is, the module itself ensures its performance within the existing infrastructure. A good example here is setting up the lint. The linter's config itself can be assembled in parts from different packages, but the plugins used should be connected in the project itself. The ideal module connecting the linter, along with itself, will bring all the necessary plug-ins, and, when used in the project, will itself make changes for its connection. In the case of eslint these are the commands in package.json and the .eslintrc config .eslintrc .

2. Interface of interaction

Problem : The increasing complexity of the internal structure of the application leads to difficulties in managing dependencies, project configuration, and running commands.

Solution : You need a cli developed cli application that provides a simple and intuitive interface for managing, extending and supporting a project. Teams must be concise and intuitive.

Profit : The developer spends much less time on routine operations - for example, creating standard components for an interface.

3. Automation of routine tasks

Problem : Most templates end their participation in a project immediately after generating a project or cloning a template. Sometimes a tool is added that can generate atomic parts of a project.

Solution : Each module must contain a specific set of templates of its component parts, which can be quickly added to the project.

Profit : The power of automating routine tasks is the responsibility of the project itself, not the template . That is, the more complex the project, the more complex generators it will require, but adding them will not affect, for example, another, more local project. But if necessary, it can be easily connected as a separate module.

Briefly: the set of generators in the project is determined by the set of installed modules.

4. Template update

Problem : After starting a project on a specific template, updating the packages used, as well as the project structure, remains with the project developers.

Solution : It is necessary to provide the ability to quickly update dependencies and migrate the code base if necessary.

Profit : Migration of breaking versions can be done only in one place. In the final projects, the project will simply update the dependencies of the project with additional migration of the code base, if necessary (it should be said that, since this task has not been solved yet, its definition and implementation method are vague).

How it works

It turns out that the Rispa project can be divided into several levels:

• cli app,
• infrastructure to support the modularity of the project,
• plug-ins:
  
  • general functionality
  • project-specific logic.

cli app

The main feature of this application is that the set of executable commands depends on the project itself. That is, there are two types of commands:

1. Project management teams: ris add , ris update , ris assemble and others;
2. Commands from plugins described as scripts in package.json . It is an extension of the ris run [plugin-name-or-alias] [command-name-from-scripts] in which the run can be omitted

More details about cli can be found in the description .

Modular infrastructure

The most obvious solution for organizing such an infrastructure is the use of the mono-repository format in the final project. The lerna.js tool was originally used, but it is better suited for managing library development projects, such as jest , babel and many others. The main feature used was the resolution of dependencies within the monorepository, but now, with the advent of yarn workspaces , they switched to using them, because it allows to solve the problems of organizing a project as a monorepository most effectively.

The second part of the modularity of the application can be called rispa-core . This package provides the ability to organize the interaction of plug-ins with each other, and also declares the specification for writing plug-ins.

Plugins

At the moment, a rather large body of plug-ins has been formed, however, they mainly serve the needs of projects at React, which in the future may be revised to introduce support for other frameworks or build systems. Here is a short list of them:

• rispa-config is the project configuration repository, it is always present in the project and is connected as a simple dependency in all plug-ins;
• rispa-server - is responsible for the entire routine associated with starting the server, configuring compression, supporting CORS, and some default routing. At the same time allows other plugins to expand their capabilities through the connection of routs and other middleware;
• rispa-client - directly responsible for the client application that runs in the browser. Now it is a React-specific client, so in the future there will be only a part that provides the work of the entire platform, and the React-part will move to a separate plugin;
• rispa-webpack - is responsible for the overall assembly of the project webpack, while the config is assembled in parts from existing plug-ins;
• rispa-eslint-config - is responsible for the connection and correct operation of the linter in the project, is simply a drop-in dependency;
• rispa-generator - is responsible for the work of generators in the project, the generators themselves are also assembled from all installed plug-ins in the project.

Experimental plugins:

• rispa-render-static - a plugin for rendering an existing application to a static site, currently under development;
• rispa-render-simple - a plugin for rendering pages without frameworks; in the initial implementation pug is used, probably the final version will also be customizable.

How to start

You must first install cli :

 $ yarn global add @rispa/cli 

After a successful installation, you need to run ris new , which will allow you to create a new project, answer questions, select plugins and wait for the installation of all dependencies:

 $ ris new 

 $ ris new [18:22:11] Enter project name [started] ? Enter project name: temp-project-name [18:22:22] Enter project name [completed] [18:22:22] Enter remote url [started] ? Enter remote url for project (optional): [18:22:24] Enter remote url [completed] [18:22:24] Generate project structure [started] [18:22:24] Generate project structure [completed] [18:22:24] Git init [started] Initialized empty Git repository in /opt/work/temp-project-name/.git/ [master (root-commit) f5af296] Create project 'temp-project-name' [18:22:24] Git init [completed] [18:22:24] Fetch plugins [started] [18:22:28] Fetch plugins [completed] [18:22:28] Select plugins to install [started] ? Select plugins: >(*) react-redux v4.1.3 - Rispa Redux plugin (*) webpack-javascript v4.1.3 - Rispa plugin which provide Webpack JavaScript configuration (*) babel v4.1.3 - Rispa Babel plugin (*) @rispa/react-config v4.1.3 - Rispa React Configuration (*) @rispa/routes v4.1.3 - rispa routes (*) react-server v4.1.7 - Rispa Server rendering plugin (*) react-client v4.1.3 - Rispa Client plugin (Move up and down to reveal more choices) [18:33:30] Select plugins to install [completed] [18:33:30] Install plugins [started] [18:33:30] Install plugin with name @rispa/react-config [started] ... success Saved lockfile. $ ris clean-cache [18:37:38] Read project configuration [started] [18:37:38] Read project configuration [completed] [18:37:38] Clean cache [started] [18:37:38] Clean cache [completed] Done in 240.41s. [18:37:39] Install project dependencies [completed] [18:37:39] Git commit [started] [master e3223c2] Bootstrap deps and install plugins [18:37:40] Git commit [completed] 

Now we have a freshly generated and customized project. However, there is no route in it. Need to add it. To do this, use the feature-plugin generator, which creates a new module and registers it with @rispa/routes . After answering a few questions, the plugin will be generated and connected to the project, and the dependencies will be updated.

 $ ris g feature-plugin 

 $ ris g feature-plugin [19:07:27] Read project configuration [started] [19:07:27] Read project configuration [completed] [19:07:27] Scan plugins [started] [19:07:27] Scan plugins [completed] [19:07:27] Init generators [started] [19:07:28] Init generators [completed] [19:07:28] Check generator [started] [19:07:28] Check generator [completed] [19:07:28] Select plugin [started] [19:07:28] Select plugin [skipped] [19:07:28] Enter plugin name [started] ? Enter plugin name: home [19:07:38] Enter plugin name [completed] [19:07:38] Run generator [started] ? Enter package name: @project/home ? Enter plugin router path: / [19:07:58] Run generator [completed] [19:07:58] Bootstrap plugin dependencies [started] ... [19:08:26] Read project configuration [started] [19:08:26] Read project configuration [completed] [19:08:26] Clean cache [started] [19:08:26] Clean cache [completed] Done in 28.54s. [19:08:27] Bootstrap plugin dependencies [completed] 

Now we have a customized plugin, then we need to deliver the main component for the page. Again we will use the ris g container generator, in the process we will be asked to choose which plugin to generate. By the way, to see all the generators, you can simply specify ris g - a list of generators available in the project will be displayed.

 $ ris g container 

 $ ris g container [19:12:20] Read project configuration [started] [19:12:20] Read project configuration [completed] [19:12:20] Scan plugins [started] [19:12:20] Scan plugins [completed] [19:12:20] Init generators [started] [19:12:20] Init generators [completed] [19:12:20] Check generator [started] [19:12:20] Check generator [completed] [19:12:20] Select plugin [started] ? Select plugin: @project/home [19:12:25] Select plugin [completed] [19:12:25] Enter plugin name [started] [19:12:25] Enter plugin name [skipped] [19:12:25] Run generator [started] ? How should it be called? Home [19:12:29] Run generator [completed] [19:12:29] Bootstrap plugin dependencies [started] [19:12:29] Bootstrap plugin dependencies [skipped] 

Now it is necessary to associate the route with the created container, this is done in the file /packages/home/src/register.js :

 import Home from './containers/Home/Home' // import reducer, { action } from './redux/reducer' // import { match } from '@rispa/redux' const registerReducer = () => { // store.injectReducer('reducerName', reducer) } const registerWhen = () => { // when(match('/'), action) } const registerModule = context => { registerReducer(context) registerWhen(context) return Home } export default registerModule 

Now you can run the project and after completing the build, see it in the browser:

 $ ris server start-dev 

Conclusion

Now Rispa allows you to solve problems to simplify the start of the project, as well as streamline the workflow. Further development is seen precisely in the expansion of capabilities and support of new frameworks, and in addition, in the search for other possible applications for the base itself in the form of rispa-cli + rispa-core .

If you like the idea of ​​this project, then I would like to read more about, for example, features of the organization of plug-in infrastructure on top of a mono-repository, or a tutorial on plug-in development and an overview of environments or modes that cli works with the project, or the client application architecture it reacts specific, it remains fairly standard. As the project develops, and there is no possibility to test everything in various environments, problems that can be covered through issues are not excluded. We also invite those who wish to join this project in order to develop it for the benefit of the developers.

Some plans for the future

• Expansion of the list of supported frameworks, it is planned to add Angular, vue.js;
• Completion of moving the yard and plug-ins on TypeScript and typing of all interfaces;
• Adding detailed documentation for plugins;
• Implementing the plug-in infrastructure on the client.

useful links

• Project organization on github
• Project page with basic info rispa.io
• The company, under the wing of which it was all born CSSSR