CacheAccelerator for MODx Evo. A decrease in the number of database requests by caching dynamic snippets
Hello. I just recently met with the MODx CMF. I am currently mastering the Evolution version. The system as a whole is rather pleasant and very flexible, however, having familiarized myself more closely, I discovered a number of shortcomings. And some of them did not give me any peace and leave it as it is, I could not.
I will dwell on one of the most sensitive criteria for any CMS / CMF performance.
In general, with performance MODx all the rules. He himself was written quite competently, optimized. Moreover, due to its flexibility, it gives the developer the opportunity to manage the bottlenecks in the implemented project.
Nevertheless, I was just shocked by the method of processing news output using Ditto, comments using Jot, and so on. Namely, the need to disable caching for the entire page from Ditto (due to problems with working with PHx), and to call the snippet itself from Jot.
')
After all, if there are a lot of records, then they will not fit on one page, which means that, for example, a news feed should be broken up into several pages. But, if MODx has caching of this page enabled, then when switching between parts of the news feed, we will see all the same content that the first one got into the cache!
What is advised by official sources?
They advise that snippets working with multiple
pages never cached.
The snippet call page is not cached “Page Setup => Cached => Off”
The snippet call page is cached “Page Setup => Cached => On”
As we see, MODx caching is not performed in both cases. The lack of caching directly affects the speed of the system, since at each page call, all snippet data is collected from the database.
According to my tests, the Ditto call page with 10 entries on the page, displaying the title, brief annotation and publication date creates about 11 queries to the database. The use of additional filters, the search conditions on the TV fields, will give even more depressing statistics.
demo
The same thing happens on the Jot call page.
Jot call with 10 comments per page, standard information about each
comments and with a form for adding new messages, also creates an order
22 - 24 queries to the database.
demo
This is uncritical as long as the base is very small, but as the base grows, such a number of requests will increase the response time exponentially relative to the number of documents.
A little thought, I wrote CacheAccelerator, consisting of one module, one snippet and a third-party library .
Cache Accelerator, by caching the output of any arbitrary snippet (not only Ditto and Jot), increases the speed of the system and reduces the number of requests to the database.
According to the tests, a repeated request to the page above with Ditto gives the result in the form of reducing the number of calls to the database from 11 to 3 (!), And to the page with Jot, from 22 to 1 (!).
This product does not use any hack methods of integration into the MODx engine. It is a completely separate block of code. Very easy to install and use.
First, download the fileCache from:
http://neo22s.com/filecache/
either direct link:
http://lab.neo22s.com/fileCache/fileCache.zip
Create the / assets / plugins / cacheaccelerator directory
In the cacheaccelerator directory, create a cache directory (/ assets / plugins / cacheaccelerator / cache)
On both created directories install chmod 777
Next, from the downloaded archive, copy the fileCache.php file to the / assets / plugins / cacheaccelerator directory
After that, in the MODx manager, click on Elements -> Element Management -> Snippets -> New Snippet . Create a new snippet named CacheAccelerator.
There we copy the contents of the snippet.
We save the created snippet. Then go to the Plugins tab.
Click Create Plugin .
We give it the name CacheAcceleratorClear. Copy the contents of the plugin into it.
Attention! After copying the contents of the plugin, go to the System Events tab, in which we tick the OnCacheUpdate event in the Cache Service Events section!
Then click the Save button.
Everything, installation of CacheAccelerator is complete.
This plugin performs one function. When MODx clears its cache, it also clears the cache with CacheAccelerator
CacheAccelerator can be used absolutely for any snippets. Cache any issue.
For the time being, I will consider its use with examples of Ditto and Jot calls.
The Ditto call will look like this:
Instead of the Ditto call itself, the CacheAccelerator snippet is called, and the name of the cached snippet (in this case, Ditto) is specified in the snippetToCache parameter.
Next comes the cacheId parameter. It is designed to separate cached content in the case of multiple cached snippets on a page. For example, the left menu, the latest news and the news feed itself. May contain any value to which the cached content will bind.
Note that if Ditto was called in double brackets [[]], then this call should also be contained in them.
Jot call example:
The cached snippet itself is also specified in the snippetToCache parameter, and in the cacheId the identifier of the cached content on the page.
There are two more parameters, dropCacheField and noCacheGroups. We will focus on them in more detail.
The fact is that, unlike the news blocks, the addition of which comes from the MODx manager and where, after adding each of the news, the cache is cleared, in the Jot comments feed any user who visits the site can add a comment. At the same time, in order for this comment to be visible both to him and other users, it is necessary to clear the cache of the CacheAccelerator. This action may be required not only for Jot, but also for many other snippets, which I would like to cache, but also to provide for the possibility of updating the cache after any user actions.
For these purposes, the dropCacheField parameter serves.
It contains a list of conditions separated by ||. When any of these conditions are triggered, the CacheAccelerator cache is cleared.
The condition can be a field name.
If it is detected in a GET or POST request, the cache will be updated.
May be a comparison.
A semicolon lists:
If the field name itself is empty, then all existing fields are compared.
Here is an example:
Here is indicated:
In case of any coincidence, the cache will be cleared.
When posting a new message to Jot, a form is transmitted that contains the JotForm field. Thus, after posting a comment, the cache will be cleared and the information will remain relevant. After the first request for this page, the cache will be created again and subsequent requests will be given from the cache.
The noCacheGroups parameter contains a list, divided by ||, which contains groups of web users for which cache data will not be returned, and a snippet will be executed on each call. Because moderators, for example, the output form differs in the control buttons and, provided that such a request gets into the cache, other incoming users will also see a form that has elements that are not related to their request.
By default, also, cache processing does not occur for those users who are authorized in the MODx manager. And, if the user, under which you edit materials, is in the users of the admin panel, then this problem disappears by itself.
You can also clear the CacheAccelerator cache from any place by calling:
From chunk:
From snippet:
Any placeholders specified by the cached snippet are also cached and written when called from the cache.
Example:
List of parameters:
snippetToCache is the name of the snippet to cache, for example `Ditto`.
cacheId - cached snippet identifier on the page, for example `News`
dropCacheField - a list of fields and conditions for them that will cause a cache to be reset, for example, `JotForm || post; true; 2`
noCacheGroups - a list of groups for which no caching will be done, for example `admins || moderators`
clearCache - if set to 1, the cache is forcibly cleared.
logMessages - if set to 1, before the contents of the snippet, system messages about cache hits and so on will be displayed.
List of conditions:
1 ! = Not equal
2 = Equal to
3 <Less than
4 > More than
5 <= Less than or equal
6 > = Greater than or equal to
7 Contains
8 Does not contain
You can download the finished version of CacheAccelerator on this site .
In no case do I claim to complete the product and the absence of errors in its work. This is the very first alpha version, besides, I have little experience in writing such manuals.
I will be glad to any help in the development and elimination of deficiencies and errors.
Added new snippet options:
noCacheRoles - a list of manager roles for which no caching will be done
for example `Administrator || Editor`
checkURL - create a separate cache for different URLs (1 | 0). Enabled by default. Enable is useful for page-based snippets.
And also a new parameter for the plugin:
only_manual - allow only manual reset of the cache.
Thanks for the refinement Andchir !
You can still download the new version from here or from here.
I will dwell on one of the most sensitive criteria for any CMS / CMF performance.
In general, with performance MODx all the rules. He himself was written quite competently, optimized. Moreover, due to its flexibility, it gives the developer the opportunity to manage the bottlenecks in the implemented project.
Nevertheless, I was just shocked by the method of processing news output using Ditto, comments using Jot, and so on. Namely, the need to disable caching for the entire page from Ditto (due to problems with working with PHx), and to call the snippet itself from Jot.
')
After all, if there are a lot of records, then they will not fit on one page, which means that, for example, a news feed should be broken up into several pages. But, if MODx has caching of this page enabled, then when switching between parts of the news feed, we will see all the same content that the first one got into the cache!
What is advised by official sources?
They advise that snippets working with multiple
pages never cached.
Here are some examples of calling Ditto and Jot:
The snippet call page is not cached “Page Setup => Cached => Off”
[[Ditto?
&parents=`1`
&display=`10`
&paginate=`1`
&paginateAlwaysShowLinks=`1`
]]The snippet call page is cached “Page Setup => Cached => On”
[!Jot?
&customfields=`name,email`
&pagination=`10`
&badwords=`*****`
&canmoderate=`Site Admins`
&captcha=`1`!]
!]As we see, MODx caching is not performed in both cases. The lack of caching directly affects the speed of the system, since at each page call, all snippet data is collected from the database.
According to my tests, the Ditto call page with 10 entries on the page, displaying the title, brief annotation and publication date creates about 11 queries to the database. The use of additional filters, the search conditions on the TV fields, will give even more depressing statistics.
demo
The same thing happens on the Jot call page.
Jot call with 10 comments per page, standard information about each
comments and with a form for adding new messages, also creates an order
22 - 24 queries to the database.
demo
This is uncritical as long as the base is very small, but as the base grows, such a number of requests will increase the response time exponentially relative to the number of documents.
A little thought, I wrote CacheAccelerator, consisting of one module, one snippet and a third-party library .
Cache Accelerator, by caching the output of any arbitrary snippet (not only Ditto and Jot), increases the speed of the system and reduces the number of requests to the database.
According to the tests, a repeated request to the page above with Ditto gives the result in the form of reducing the number of calls to the database from 11 to 3 (!), And to the page with Jot, from 22 to 1 (!).
This product does not use any hack methods of integration into the MODx engine. It is a completely separate block of code. Very easy to install and use.
So, the installation:
First, download the fileCache from:
http://neo22s.com/filecache/
either direct link:
http://lab.neo22s.com/fileCache/fileCache.zip
Create the / assets / plugins / cacheaccelerator directory
In the cacheaccelerator directory, create a cache directory (/ assets / plugins / cacheaccelerator / cache)
On both created directories install chmod 777
Next, from the downloaded archive, copy the fileCache.php file to the / assets / plugins / cacheaccelerator directory
After that, in the MODx manager, click on Elements -> Element Management -> Snippets -> New Snippet . Create a new snippet named CacheAccelerator.
There we copy the contents of the snippet.
Snippet CacheAccelerator:
<?php
// . Ditto
if(!function_exists(cacheFieldsCompare)) {
function cacheFieldsCompare ($param1, $param2, $param3){
/*
1 or !=
2 or =
3 or <
4 or >
5 or <=
6 or >=
7
8
*/
switch($param3){
case 1:
return $param1 != $param2;
break;
case 2:
return $param1 == $param2;
break;
case 3:
return $param1 < $param2;
break;
case 4:
return $param1 > $param2;
break;
case 5:
return $param1 <= $param2;
break;
case 6:
return $param1 >= $param2;
break;
case 7:
return stristr($param1, $param2);
break;
case 8:
return !stristr($param1, $param2);
}
}
}
$nocache = isset($nocache)? $nocache : 0; //
$url = $_SERVER["REQUEST_URI"]; // ,
$path_to_cacheengine=$modx->config['base_path']."assets/plugins/cacheaccelerator/"; // CacheAccelerator
$path_to_cache=$modx->config['base_path']."assets/plugins/cacheaccelerator/cache/"; // ( )
require_once ($path_to_cacheengine."fileCache.php"); // fileCache
$cache = fileCache::GetInstance(84600*7,$path_to_cache);// fileCache
//
if((int)$clearCache){
if($logMessages) echo("Clearing cache...");
$cache->deleteCache(0);
return;
}
// , ( , )
$noCacheGroups = isset($noCacheGroups) ? $noCacheGroups : "";
$nocache = intval($modx->isMemberOfWebGroup(explode("||",$nocacheGroups)) || $modx->checkSession()) ? 2 : $nocache;
if($nocache == 2){
if($logMessages) echo("No caching for this web group.");
}
/* -, . , */
if(isset($dropCacheField)){
$fieldsArray = explode("||", $dropCacheField);
foreach ($fieldsArray as $field){
$field1 = explode(";", $field);
if($field1[1] && $field[2]){
if(empty($field1[0])){
foreach ($_POST as $key => $postField){
if(cacheFieldsCompare($postField, $field1[1], $field1[2])){
$nocache = 1;
continue;
}
}
foreach ($_GET as $key => $getField){
if(cacheFieldsCompare($getField, $field1[1], $field1[2])){
$nocache = 1;
continue;
}
}
} else {
if(!empty($_POST[$field1[0]])){
if(cacheFieldsCompare($_POST[$field1[0]], $field1[1], $field1[2])){
$nocache = 1;
continue;
}
}
if(!empty($_GET[$field1[0]])){
if(cacheFieldsCompare($_GET[$field1[0]], $field1[1], $field1[2])){
$nocache = 1;
continue;
}
}
}
} else {
if(!empty($_POST[$field1[0]]) || !empty($_GET[$field1[0]]))
$nocache = 1;
}
}
//
if($nocache == 1){
if($logMessages) echo(" Clearing cache... ");
$cache->deleteCache(0);
}
}
//
if($nocache == 0){
$cached = $cache->cache($cacheId.$url);
if(isset($cached)){
if($logMessages) echo(" Cache hit! ");
$modx->placeholders = $cached['placeholders']; //
return $cached['content']; //
}
}
$output = $modx->runSnippet($snippetToCache, $modx->event->params); //
//
if($nocache == 0){
if($logMessages) echo("Storing to cache...");
$cache->cache($cacheId.$url,array('placeholders' => $modx->placeholders, 'content' => $output));
}
// MODx
return($output);
?>We save the created snippet. Then go to the Plugins tab.
Click Create Plugin .
We give it the name CacheAcceleratorClear. Copy the contents of the plugin into it.
CacheAcceleratorClear plugin:
$path_to_cacheengine=$modx->config['base_path']."assets/plugins/cacheaccelerator/"; // CacheAccelerator
$path_to_cache=$modx->config['base_path']."assets/plugins/cacheaccelerator/cache/"; // ( , CacheAccelerator)
require_once ($path_to_cacheengine."fileCache.php");// fileCache
$cache = fileCache::GetInstance(84600*7,$path_to_cache);// fileCache
$cache->deleteCache(0);//
return;Attention! After copying the contents of the plugin, go to the System Events tab, in which we tick the OnCacheUpdate event in the Cache Service Events section!
Then click the Save button.
Everything, installation of CacheAccelerator is complete.
This plugin performs one function. When MODx clears its cache, it also clears the cache with CacheAccelerator
Using CacheAccelerator:
CacheAccelerator can be used absolutely for any snippets. Cache any issue.
For the time being, I will consider its use with examples of Ditto and Jot calls.
The Ditto call will look like this:
[[CacheAccelerator?
&snippetToCache=`Ditto`
&cacheId=`News`
&parents=`1`
&display=`10`
&paginate=`1`
&paginateAlwaysShowLinks=`1`
]]Instead of the Ditto call itself, the CacheAccelerator snippet is called, and the name of the cached snippet (in this case, Ditto) is specified in the snippetToCache parameter.
Next comes the cacheId parameter. It is designed to separate cached content in the case of multiple cached snippets on a page. For example, the left menu, the latest news and the news feed itself. May contain any value to which the cached content will bind.
Note that if Ditto was called in double brackets [[]], then this call should also be contained in them.
Jot call example:
[!CacheAccelerator?
&snippetToCache=`Jot`
&cacheId=`Comments`
&dropCacheField=`JotForm||post;true;2||;publish;2||;unpublish;2||;delete;2||;edit;2`
&noCacheGroups=`Site Admins`
&customfields=`name,email`
&pagination=`10`
&badwords=`*****`
&canmoderate=`Site Admins`
&captcha=`1`
!]The cached snippet itself is also specified in the snippetToCache parameter, and in the cacheId the identifier of the cached content on the page.
There are two more parameters, dropCacheField and noCacheGroups. We will focus on them in more detail.
The fact is that, unlike the news blocks, the addition of which comes from the MODx manager and where, after adding each of the news, the cache is cleared, in the Jot comments feed any user who visits the site can add a comment. At the same time, in order for this comment to be visible both to him and other users, it is necessary to clear the cache of the CacheAccelerator. This action may be required not only for Jot, but also for many other snippets, which I would like to cache, but also to provide for the possibility of updating the cache after any user actions.
For these purposes, the dropCacheField parameter serves.
It contains a list of conditions separated by ||. When any of these conditions are triggered, the CacheAccelerator cache is cleared.
The condition can be a field name.
If it is detected in a GET or POST request, the cache will be updated.
May be a comparison.
A semicolon lists:
;;_If the field name itself is empty, then all existing fields are compared.
Here is an example:
&dropCacheField=`JotForm||post;true;2||;delete;2`Here is indicated:
- If there is a JotForm field in the request
- If there is a post field and it is true
- If any of the fields is set to delete
In case of any coincidence, the cache will be cleared.
When posting a new message to Jot, a form is transmitted that contains the JotForm field. Thus, after posting a comment, the cache will be cleared and the information will remain relevant. After the first request for this page, the cache will be created again and subsequent requests will be given from the cache.
The noCacheGroups parameter contains a list, divided by ||, which contains groups of web users for which cache data will not be returned, and a snippet will be executed on each call. Because moderators, for example, the output form differs in the control buttons and, provided that such a request gets into the cache, other incoming users will also see a form that has elements that are not related to their request.
By default, also, cache processing does not occur for those users who are authorized in the MODx manager. And, if the user, under which you edit materials, is in the users of the admin panel, then this problem disappears by itself.
You can also clear the CacheAccelerator cache from any place by calling:
From chunk:
[!CacheAccelerator? &clearCache=`1`!]From snippet:
$modx->runSnippet("CacheAccelerator", array("clearCache" => 1))Any placeholders specified by the cached snippet are also cached and written when called from the cache.
Example:
[[CacheAccelerator?&snippetToCache=`Ditto`&cacheId=`News`]]
[+start+] - [+stop+] [+total+] <br>
[+previous+] [+pages+] [+next+]<br>List of parameters:
snippetToCache is the name of the snippet to cache, for example `Ditto`.
cacheId - cached snippet identifier on the page, for example `News`
dropCacheField - a list of fields and conditions for them that will cause a cache to be reset, for example, `JotForm || post; true; 2`
noCacheGroups - a list of groups for which no caching will be done, for example `admins || moderators`
clearCache - if set to 1, the cache is forcibly cleared.
logMessages - if set to 1, before the contents of the snippet, system messages about cache hits and so on will be displayed.
List of conditions:
1 ! = Not equal
2 = Equal to
3 <Less than
4 > More than
5 <= Less than or equal
6 > = Greater than or equal to
7 Contains
8 Does not contain
You can download the finished version of CacheAccelerator on this site .
In no case do I claim to complete the product and the absence of errors in its work. This is the very first alpha version, besides, I have little experience in writing such manuals.
I will be glad to any help in the development and elimination of deficiencies and errors.
UPD .
Added new snippet options:
noCacheRoles - a list of manager roles for which no caching will be done
for example `Administrator || Editor`
checkURL - create a separate cache for different URLs (1 | 0). Enabled by default. Enable is useful for page-based snippets.
And also a new parameter for the plugin:
only_manual - allow only manual reset of the cache.
Thanks for the refinement Andchir !
You can still download the new version from here or from here.
Source: https://habr.com/ru/post/113719/
All Articles