CMS do it yourself. Theory of cycling
How many funny guys
And everyone makes a bike.
And one of them one morning
Comes up with gunpowder.
Viktor Tsoi.
At first, I wanted to write an article in the section “I am promoting myself” about what a young man I was and what a wonderful thing I did, but, searching a little on the net, I discovered without surprise that I was not at all the only one of its kind. Then I decided to go from the opposite: probably, almost every Web programmer at least once in his life tries to write a full-fledged CMS. At the same time, in the design process (and this process, often, comes already at the time of writing code), the developer certainly has questions. With these questions, he turns to the search engines and gets to the sites of those who have already gone through such a rake.
')
So, I began to look at what requests beginner “bike developers” get to me, and tried to highlight some things that were not obvious to me at the beginning of my work.
Wherever the conversation about the development of web applications, the fashionable abbreviation MVC (Model-View-Controller) immediately pops up. This approach says that the interface should be separated from logic, and logic from data. I will not say that I am fully imbued with these ideas, but that changes in design (or even designs) should not affect the logic of the code - I am ready to defend myself foaming at the mouth :)
This is where the rake number one rests: the appearance must be separated from the logic of the program. How to do it - everyone decides for himself. On this issue, the copies are broken quite a bit: here and various template engines, and xslt conversions, and just php + html rendered into separate files. The choice is large, but the “silver bullet”, as usual, does not exist: on one side of the scale lies flexibility, on the other - clarity. Even Smarty , with its “programming for the smallest,” seems difficult to many users. So if we focus on the user who wants to put the system "out of the box" and file it for their needs with minimal programming knowledge, then it is worth it.
In addition, designs should be replaceable and, possibly, even on the fly. That is, you need to provide for their convenient storage and editing. And one more thing that many ignored: the design should be easy to edit and update. If you had to adapt the design of some free forum, consisting of two hundred templates, in which everything is tightly “nailed” by tables and pieces of JavaScript are inserted “from logic”, then you understand what you don’t want to see.
I came up with this design: in the user design there is only what is not in the basic design. That is, in the most minimalist case, the design consists of an empty catalog with the name of the design. It is clear that in this case the design will look exactly the same as the base one, since all the missing parts will be taken from it, but as a starting point it is very convenient. If css appears in the design, the system automatically switches to it (while still html borrows from the base one). The same with JS. What we get from this: in the user design are only those files that he himself made. The user does not need to remember which file he corrected, and which one he simply copied from the basic design at the beginning of the work. Also on the site are displayed almost all the innovations of the basic design without editing the user. Such a system seemed convenient and logical to me, although to some it seems somewhat unexpected. Whether to take it into service or come up with your own is up to you.
Start writing a kernel. What should the core do? And it should do all the “dirty” work: determine the site settings, the rights and settings of user groups, the used modules, templates, caching parameters, localization, etc. That is, so that by the time plug-ins started working, they could get all the information of interest from the kernel. It sounds scary, but all this is relatively easy to write and work if you clearly understand the interaction of elements.
I decided for myself that the site would not be a bunch of pages dumped somewhere in the database, but a strict hierarchy. As a result, the site structure is tree-like and the missing parts, as in the case of designs, are inherited from the parents. The structure of user groups is also tree-like - the rights and settings are also inherited from the parents. Localization files and modules also have a simple hierarchy. A clear hierarchy made it possible to shift all kinds of unpleasant things to the engine, such as automatic generation of a site map, various menus, distribution of rights (yes, yes, to give the right to something to several groups, it is not necessary to edit each one - it’s enough to define the hierarchy), etc. Live and rejoice! And everything would be fine if it were not for the rake:
While I was engaged in designing my “megadvigka” it was somehow not up to caching ... Yes, and you think - what is so complicated about it? I placed the page in a variable, saved it to a file, and next time I showed it from there. Business ... at any time you can attach! Oh ... and we have another page for registered users ... Um ... well, think about it - save two pages to the cache! And in the header you need to output “hello, Vasya” ... well, then this fragment in the header is not to be cached. and the same fragment in the basement ... and in the middle ... Hmm ... I still need to cache different parts of the page for different periods ... Sit down and rewrite the caching engine and caching for block caching - so that each block has its own lifetime.
How?! Again caching? After all, they did everything beautifully! Well, yes ... they did ... and it even worked until the task arose to generate content for each user based on his personal settings. The cache size at the same time grows with the speed of a jet fighter, and its contents become obsolete much earlier than will be requested again. Instead of speeding up the site, we get its slowdown, and gigabytes of useless cached pages ... The main script on the site is “his majesty” cache invalidator. Hmm ... rewrite the engine again: this time we implement caching at the database query level, since this is the bottleneck in performance. Rewrote ... all - nirvana.
You look at your creation and feel like a complete idiot: instead of saving the whole page, every time you create it. But caching was conceived with precision for the reverse! How am I so burdock something?
The result was that some modules are cached by blocks, and some at the request level. This allowed for months to store in the cache such rarely changed items as, for example, the menu of the site.
I learned the lesson for myself: the system must be initially designed in such a way that at certain stages of execution you have grouped data that can be easily put into the cache or taken from there. Also, the caching system should not be rigidly tied to the engine, since today it is enough to store the cache in files, and tomorrow (or even tonight) already submit work with the memcache server.
In the process of writing the engine, it also makes sense to read a smart book about refactoring ... or read it after writing the engine. In any case, all three (you, the engine and the book) can be beneficial.
The modern system is hard to imagine as a “thing in itself” - it must have interfaces for expanding its functionality. Thus, we turn to the most delicious part of the CMS - writing modules. It is also full of questions: how should the module look, how to connect to the system, etc.
In some systems, the call of modules is rigidly registered in the kernel of the system, so if you want to write a module or install a third-party module, then climb into the source code, arm with the installation manual and enter the corresponding calls into the code. With all the idiocy of the approach, many systems function exactly like that. There is a variation of this solution: each module is a separate file in a certain directory. In this case, we get not just the need to register the module into the kernel, but also execute it as a single file. There were also variants with active templates: that is, they entered {module_name} in the template and when the parser reached this tag, the module module is called for execution, the result of which is in place of the tag. Perhaps this way is convenient, but in this case we not only did not separate the logic from the presentation, but just the opposite - we mixed them well.
After a certain number of cones, I came to a system that, perhaps, is not an example of conciseness and simplicity, but seems to me quite comfortable. Each module is a separate directory from which the kernel calls only one file (index.php). This file can both output “Hello world!” And include control files of a hyperspatial quasi-emitter - this will be convenient as a module developer. In the same directory is an xml file with a description of the parameters of the module, possible settings and the system of rights. This file is used so that the system can add modules by itself and not shift this headache to the user: clicked the “install module” button — please receive.
With the installation sorted out. A new problem arises - how not to allow the user to place on one page, say, a photo album and, for example, a forum? To common sense, hope is useless, so we need typing modules. A module of this type (for such modules I have peeped on the concept of “component” somewhere) there can be only one on the page.
Well, well, there will be only one component on the page, but there may be many others — in what order should they be connected? After all, it would be stupid if a module of a skin switch that at the end of its work transfers the user to the home page of the site connects after it has completed the module for analyzing the dynamics of the number of rabbits in the mating season - and we don’t see the results of the analysis . Consequently, the modules must be specified the order of their connection.
Some enter for this an analog nix runlevel, where for each module it is necessary to prescribe between which modules it should be connected. As a user, such a decision plunged me into prostration, but as a developer, I came up with almost the same thing: the modules are divided into three large groups. One of the groups is the “component” already mentioned, the other two differ only in that the modules of one group are connected before the component, and the other after. Moreover, I hid this separation from the user, so that only the “component” and “just a module” remained for him.
So, the module was designed, the connection was determined ... Now we need to decide how to configure it and how to allow it. And then everything is simple: since we have the kernel designed for dirty work, then let him have a headache and hurt - the module in xml issued a list of settings, and let the kernel parse it, store it and provide it on request - that's all simply.
Another problem I encountered during the development process is data transfer between modules. According to the conditions of the problem, they do not know anything about each other and are called by the kernel in the order of numbers. But I want to transfer from one module to another news! For this, I introduced a special class for global variables, where each module can save something for memory for those who come after it. This is perhaps not a very elegant solution.
You always want to have the latest version, but to make for this a minimum of gestures. Hence the desire to automate the update process. And here again there is, albeit not very extensive, but still a zoo of solutions. The most progressive ones offer to put 777 rights on all directories, and 666 on files, and then “the script will do everything itself”. The fact that this is a security hole the size of the Grand Canyon, in general, does not bother anyone.
I had ideas regarding two options: the script downloads updates to a temporary directory, and then, after asking the user for the FTP access parameters, updates itself. So he doesn’t have to give any extra rights, and everything happens automatically, and updates are chased within the server ... you just have to either ask the user for FTP access parameters every time, or store them right there on the site ... that is, all the eggs are in one basket. Therefore, I preferred another option: the user himself downloads updates (archive or via svn), uploads them to the site, and the code on the site, feeling that he has become “newer” makes the necessary corrections to the database and / or settings ... And the first option was more beautiful ... but I did not dare.
These are the most memorable milestones of my "bicycle construction". Reading this, I very much regret that at the beginning of my work I didn’t meet something like this, and I myself didn’t have enough experience and bumps to ask the "right" questions. What I got in the end, you can see here and if you have any questions, I am ready to answer them.
By the way, if you have any answers to unasked questions, I will also listen to them with pleasure. After all, experience - it is known, whose son, and it is almost impossible to make all the mistakes alone;)
And everyone makes a bike.
And one of them one morning
Comes up with gunpowder.
Viktor Tsoi.
At first, I wanted to write an article in the section “I am promoting myself” about what a young man I was and what a wonderful thing I did, but, searching a little on the net, I discovered without surprise that I was not at all the only one of its kind. Then I decided to go from the opposite: probably, almost every Web programmer at least once in his life tries to write a full-fledged CMS. At the same time, in the design process (and this process, often, comes already at the time of writing code), the developer certainly has questions. With these questions, he turns to the search engines and gets to the sites of those who have already gone through such a rake.
')
So, I began to look at what requests beginner “bike developers” get to me, and tried to highlight some things that were not obvious to me at the beginning of my work.
1. MVC - our everything!
Wherever the conversation about the development of web applications, the fashionable abbreviation MVC (Model-View-Controller) immediately pops up. This approach says that the interface should be separated from logic, and logic from data. I will not say that I am fully imbued with these ideas, but that changes in design (or even designs) should not affect the logic of the code - I am ready to defend myself foaming at the mouth :)
This is where the rake number one rests: the appearance must be separated from the logic of the program. How to do it - everyone decides for himself. On this issue, the copies are broken quite a bit: here and various template engines, and xslt conversions, and just php + html rendered into separate files. The choice is large, but the “silver bullet”, as usual, does not exist: on one side of the scale lies flexibility, on the other - clarity. Even Smarty , with its “programming for the smallest,” seems difficult to many users. So if we focus on the user who wants to put the system "out of the box" and file it for their needs with minimal programming knowledge, then it is worth it.
In addition, designs should be replaceable and, possibly, even on the fly. That is, you need to provide for their convenient storage and editing. And one more thing that many ignored: the design should be easy to edit and update. If you had to adapt the design of some free forum, consisting of two hundred templates, in which everything is tightly “nailed” by tables and pieces of JavaScript are inserted “from logic”, then you understand what you don’t want to see.
I came up with this design: in the user design there is only what is not in the basic design. That is, in the most minimalist case, the design consists of an empty catalog with the name of the design. It is clear that in this case the design will look exactly the same as the base one, since all the missing parts will be taken from it, but as a starting point it is very convenient. If css appears in the design, the system automatically switches to it (while still html borrows from the base one). The same with JS. What we get from this: in the user design are only those files that he himself made. The user does not need to remember which file he corrected, and which one he simply copied from the basic design at the beginning of the work. Also on the site are displayed almost all the innovations of the basic design without editing the user. Such a system seemed convenient and logical to me, although to some it seems somewhat unexpected. Whether to take it into service or come up with your own is up to you.
2. Site structure
Start writing a kernel. What should the core do? And it should do all the “dirty” work: determine the site settings, the rights and settings of user groups, the used modules, templates, caching parameters, localization, etc. That is, so that by the time plug-ins started working, they could get all the information of interest from the kernel. It sounds scary, but all this is relatively easy to write and work if you clearly understand the interaction of elements.
I decided for myself that the site would not be a bunch of pages dumped somewhere in the database, but a strict hierarchy. As a result, the site structure is tree-like and the missing parts, as in the case of designs, are inherited from the parents. The structure of user groups is also tree-like - the rights and settings are also inherited from the parents. Localization files and modules also have a simple hierarchy. A clear hierarchy made it possible to shift all kinds of unpleasant things to the engine, such as automatic generation of a site map, various menus, distribution of rights (yes, yes, to give the right to something to several groups, it is not necessary to edit each one - it’s enough to define the hierarchy), etc. Live and rejoice! And everything would be fine if it were not for the rake:
Rakes first. Caching
While I was engaged in designing my “megadvigka” it was somehow not up to caching ... Yes, and you think - what is so complicated about it? I placed the page in a variable, saved it to a file, and next time I showed it from there. Business ... at any time you can attach! Oh ... and we have another page for registered users ... Um ... well, think about it - save two pages to the cache! And in the header you need to output “hello, Vasya” ... well, then this fragment in the header is not to be cached. and the same fragment in the basement ... and in the middle ... Hmm ... I still need to cache different parts of the page for different periods ... Sit down and rewrite the caching engine and caching for block caching - so that each block has its own lifetime.
The second rake. Caching
How?! Again caching? After all, they did everything beautifully! Well, yes ... they did ... and it even worked until the task arose to generate content for each user based on his personal settings. The cache size at the same time grows with the speed of a jet fighter, and its contents become obsolete much earlier than will be requested again. Instead of speeding up the site, we get its slowdown, and gigabytes of useless cached pages ... The main script on the site is “his majesty” cache invalidator. Hmm ... rewrite the engine again: this time we implement caching at the database query level, since this is the bottleneck in performance. Rewrote ... all - nirvana.
The third rake. Caching
You look at your creation and feel like a complete idiot: instead of saving the whole page, every time you create it. But caching was conceived with precision for the reverse! How am I so burdock something?
The result was that some modules are cached by blocks, and some at the request level. This allowed for months to store in the cache such rarely changed items as, for example, the menu of the site.
I learned the lesson for myself: the system must be initially designed in such a way that at certain stages of execution you have grouped data that can be easily put into the cache or taken from there. Also, the caching system should not be rigidly tied to the engine, since today it is enough to store the cache in files, and tomorrow (or even tonight) already submit work with the memcache server.
In the process of writing the engine, it also makes sense to read a smart book about refactoring ... or read it after writing the engine. In any case, all three (you, the engine and the book) can be beneficial.
3. Modularity.
The modern system is hard to imagine as a “thing in itself” - it must have interfaces for expanding its functionality. Thus, we turn to the most delicious part of the CMS - writing modules. It is also full of questions: how should the module look, how to connect to the system, etc.
In some systems, the call of modules is rigidly registered in the kernel of the system, so if you want to write a module or install a third-party module, then climb into the source code, arm with the installation manual and enter the corresponding calls into the code. With all the idiocy of the approach, many systems function exactly like that. There is a variation of this solution: each module is a separate file in a certain directory. In this case, we get not just the need to register the module into the kernel, but also execute it as a single file. There were also variants with active templates: that is, they entered {module_name} in the template and when the parser reached this tag, the module module is called for execution, the result of which is in place of the tag. Perhaps this way is convenient, but in this case we not only did not separate the logic from the presentation, but just the opposite - we mixed them well.
After a certain number of cones, I came to a system that, perhaps, is not an example of conciseness and simplicity, but seems to me quite comfortable. Each module is a separate directory from which the kernel calls only one file (index.php). This file can both output “Hello world!” And include control files of a hyperspatial quasi-emitter - this will be convenient as a module developer. In the same directory is an xml file with a description of the parameters of the module, possible settings and the system of rights. This file is used so that the system can add modules by itself and not shift this headache to the user: clicked the “install module” button — please receive.
With the installation sorted out. A new problem arises - how not to allow the user to place on one page, say, a photo album and, for example, a forum? To common sense, hope is useless, so we need typing modules. A module of this type (for such modules I have peeped on the concept of “component” somewhere) there can be only one on the page.
Well, well, there will be only one component on the page, but there may be many others — in what order should they be connected? After all, it would be stupid if a module of a skin switch that at the end of its work transfers the user to the home page of the site connects after it has completed the module for analyzing the dynamics of the number of rabbits in the mating season - and we don’t see the results of the analysis . Consequently, the modules must be specified the order of their connection.
Some enter for this an analog nix runlevel, where for each module it is necessary to prescribe between which modules it should be connected. As a user, such a decision plunged me into prostration, but as a developer, I came up with almost the same thing: the modules are divided into three large groups. One of the groups is the “component” already mentioned, the other two differ only in that the modules of one group are connected before the component, and the other after. Moreover, I hid this separation from the user, so that only the “component” and “just a module” remained for him.
So, the module was designed, the connection was determined ... Now we need to decide how to configure it and how to allow it. And then everything is simple: since we have the kernel designed for dirty work, then let him have a headache and hurt - the module in xml issued a list of settings, and let the kernel parse it, store it and provide it on request - that's all simply.
Another problem I encountered during the development process is data transfer between modules. According to the conditions of the problem, they do not know anything about each other and are called by the kernel in the order of numbers. But I want to transfer from one module to another news! For this, I introduced a special class for global variables, where each module can save something for memory for those who come after it. This is perhaps not a very elegant solution.
4. Updates
You always want to have the latest version, but to make for this a minimum of gestures. Hence the desire to automate the update process. And here again there is, albeit not very extensive, but still a zoo of solutions. The most progressive ones offer to put 777 rights on all directories, and 666 on files, and then “the script will do everything itself”. The fact that this is a security hole the size of the Grand Canyon, in general, does not bother anyone.
I had ideas regarding two options: the script downloads updates to a temporary directory, and then, after asking the user for the FTP access parameters, updates itself. So he doesn’t have to give any extra rights, and everything happens automatically, and updates are chased within the server ... you just have to either ask the user for FTP access parameters every time, or store them right there on the site ... that is, all the eggs are in one basket. Therefore, I preferred another option: the user himself downloads updates (archive or via svn), uploads them to the site, and the code on the site, feeling that he has become “newer” makes the necessary corrections to the database and / or settings ... And the first option was more beautiful ... but I did not dare.
These are the most memorable milestones of my "bicycle construction". Reading this, I very much regret that at the beginning of my work I didn’t meet something like this, and I myself didn’t have enough experience and bumps to ask the "right" questions. What I got in the end, you can see here and if you have any questions, I am ready to answer them.
By the way, if you have any answers to unasked questions, I will also listen to them with pleasure. After all, experience - it is known, whose son, and it is almost impossible to make all the mistakes alone;)
Source: https://habr.com/ru/post/109310/
All Articles