How to make a quick web application or 8 tips to developers on code optimization
Easy and fast - these are the two words we pray for when creating Diafan.CMS. We do not have large libraries in case of atomic war, and everything new is added by necessity. The overall logic of the system has been refined and polished over many years, so the system allows you to allow yourself to branch and refine the functionality, leaving the code easy to understand and easy to develop CMS. How is this achieved? We formed a few tips.
Our system has repeatedly won the title of fast and optimized system, the code of which shows itself most clearly under loads (the first study , the second study ). In the comments to the studies, we promised that we would write an article about how such results are achieved. And where else to publish such an article, if not on HabrĂŠ?
Despite the fact that practical examples are considered on the basis of our Diafan.CMS, the tips below will be useful to any web developers of sites, individual modules, mobile applications and network programs.
Move from easy to hard.
')
Let's say your project needed SMS-notifications. The SMS sending code is quite simple, in a few lines: this is just a phone validation and a GET request to the SMS sending server - integrate it directly.
For example, when SMS notifications to the administrator on receipt of orders were required in Diafan.CMS, we put this simple code directly in the Store module, in the function that creates the order.
Then we needed SMS notifications in the âFeedbackâ and âMailingâ module. Then we brought this functionality as a global function into a separate file includes / sms.php and calmly used it in different modules in this form:
When it was required to supplement SMS notifications with settings and our interface, we moved this functionality into a separate module.
And if you need the ability to connect different backends, for example, to work with different SMS operators, this can be done in the module, the general structure of the system allows it.
The bottom line is that you do not need to develop any complex libraries in the first stages to work with the new functionality, you do not need to make 100,500 settings that will confuse the site administrator and require writing bales of documentation. And then maybe not needed. It is better to do only what is required at this stage.
This does not lead to a snowball if the system has a clear hierarchy of âcode - pluggable functions â pluggable modulesâ. Your application will remain simple and easy to understand at every stage of development.
Refusing extra code that is not used at the current stage of development, you can kill two birds with one stone:
Moreover, this approach led us to another feature. We use very little external code, plugins and libraries. Contrary to the "bicycle" error, we believe that it is better to write one function for your tasks than to include a huge third-party library.
Analysis of links by the server is a bottleneck through which all pages of the site go. Try to make this place as easy as possible for the server.
Most often, for address navigation through the pages of the site, instead of simple server addresses like site.ru/?PAGE=34, the so-called âCNCâ (human-understandable URL) is used, when each page of the site corresponds to a beautiful string friendly address. For example, at the site.ru/o_kompanii/ address the âAbout the Companyâ page opens. Here, âo_kompaniiâ is a string CNC interpreted in site.ru/?url=o_kompanii by which the CMS searches for a page named o_kompanii in the database and displays its contents on the site. As if to request it by PAGE = 34. However, in addition to the string name of the page, some parameters can also be passed through the address bar, and there may be many of them. The analysis of such CNC can be very difficult.
In Diafan.CMS, there is only one template for link building: / stringCNC / numeric parameter 3 / another numeric parameter 25/3, 25 are the parameter values. Those. if the URL has numeric parameters, it means there are variables in front of them, in any case, this is the most frequent structure.
We immediately discard all the numeric parameters, and then, if something remains, look at the string CNC in the database and the page corresponding to it. If we donât find the page on the CNC, then we start adding one parameter to the CNC and look again in the database. Yes, such an algorithm consists of several unnecessary SQL queries, but they will be only for such specific pages or for pages 404, if artificially substituting variables there. That does not load the base for all other (and most of them) pages.
This is only easier for ugly links with $ _GET data, such as site.ru/?url=page:13&m=3&m2=25.
When launching any page of the site, first download only the most necessary functionality, which in 99% of cases will be used. All the rest run only according to needs.
For example, if in Diafan.CMS you need to connect a product to a rating, we simply refer to the variable $ this-> diafan -> _ rating and it will give the entire connected part of the Rating module. Since the first access to the pseudo-variable $ this-> diafan _ ***, the presence of the requested module will be checked and the correct connection to it will be made as an instance of the class. It is convenient to use, and cuts off any extra functionality in the current script execution. If the rating on the page is not needed, it will not run at all.
Do not connect additional functionality globally to all modules, let's optionally enable it.
For example, in any module where comments are used, in Diafan.CMS there is an option âConnect commentsâ. This eliminates situations when comments to products are needed, but not to news, and the webmaster simply hides the output of comments in the news template, but the CMS continues to execute the SQL query, look for comments to the news, generate all the necessary data. In other words, waste server resources in vain.
The same applies to any other functionality that uses the resources of the server. Don't need a visitor counter, news rating, images for menu items? Take out their connection in the settings and disable the default. This will save resources.
The same applies to hidden settings, for example, âAccess Rightsâ, when you need to flexibly configure access restrictions to different specific site materials for different users (Guest, User, Wholesaler, etc.). To ensure this feature, the easiest way is for all SQL queries of all materials. add:
... LEFT JOIN access AS a ON a.element_id = 'id_material' AND (e.access = 'yes' AND a.role = 'role_current_user') , which checks if the current user has access to the requested material.
And why do we need this load, if all the news is seen by all users, and these situations are much more? It makes sense to add this LEFT JOIN to the SQL query optionally. In Diafan.CMS, this setting is turned on / off automatically without the involvement of the administrator. That is, if for some material you need to restrict access, it is enough to note the setting and queries to the database are complicated. If not, the requests are simple.
Use prepare functions.
It happens that the functionality of one module is used in another. Imagine that we have 10 products in the shop module and each image from the image module. The most common mistake made by beginners is when additional requests are made to the image table in the product output cycle.
* here and below the code is conditional, for clarity of the algorithm and the transfer of meaning.
Thus it turns out 1 + 10 = 11 separate SELECT queries to the SQL server. However, it is better to query the products, remember all the IDs of the goods received, and then make only one SQL query: select all the images for all our products at once. It turns out only 2 SELECT queries, instead of 11.
But here the problem is that the logic of the cycles is violated and the code becomes inconvenient. We managed to find an elegant solution that is beautiful for use and optimal for the server: the prepare function. The code is:
The prepare function simply remembers all the elements for which you need to get pictures. And the feature of the get function is that the SQL query will be executed upon the first call and the result will also be remembered. All other get calls from this loop will work with the data received for the first time.
And this is done for any functional: CNC query for link building, tag query, rating query, etc. Therefore, in Diafan.CMS, calls to the SQL server are minimal.
Write your own caching algorithms, use the server.
If the online store is settled and new products are not added, why every time to contact the database to create a list of products for each customer? It is enough to contact the SQL-server once, save the results of the query in a text file in the cache folder and then take them from there. If a new product is added to the list, or a different page is requested, for example, a sorted list of products, you can reset the cache file and return to the database again. But only once, in order to update the cache.
You can cache as parts of the site, blocks, lists, cards and items, and all pages of the site. More details .
It is important to remember that static resources charge the server by an order of magnitude less than dynamic ones, so try to turn to SQL and the PHP interpreter as little as possible. By default, the file cache is used in Diafan.CMS, but you can also connect Memcache. We cache only data. All design in the cache does not fall. This allows you to save space for the cache and as a result, work with it faster.
It is better to wait for the site administrator once, than to wait for each user.
For example, in Diafan.CMS there is a flexible system of discounts, when one product can have several prices, each with an established discount, also in different currencies. We do not calculate the new prices on the site for each site visitor. We do this when installing discounts in the administrative part of the site, count all the different price options for selected products and collect them into a separate shop_price table with a price_id sign. The initial price for the product is defined as id = price_id. If id <> price_id, then this is a variation of the price either at a discount or in the main currency of the site. And this table immediately indicates for whom this price (if it is set for a particular group of users).
Yes, if there are a lot of goods, when adding or changing a discount, the administrator will wait, looking at the spinning wheel. But the site visitors will not have to wait, because when outputting on the site, as well as when searching, sorting and other capacious operations, the system simply chooses the smallest one from all the available prices from one table one by one price_id.
Spread the complex structure of information on different simple tables and use indexes on them. It happens that it is more profitable to make two simple queries than to build one complex INNER JOIN.
In Diafan.CMS, basically all adjacent tables have approximately one structure: id, element_id, element_type, module_name. Queries on them are extremely simple.
SELECT * FROM rating WHERE element_id = 3 AND module_name = 'shop' AND element_type = 'element'
And for all the specified fields indices are required.
If you look, we have not reported anything new and almost all the points are obvious. However, many people know the theory, but they are not always used in practice. We hope our tips will help you get together, stop being lazy and create beautiful, fast and optimized applications.
Our system has repeatedly won the title of fast and optimized system, the code of which shows itself most clearly under loads (the first study , the second study ). In the comments to the studies, we promised that we would write an article about how such results are achieved. And where else to publish such an article, if not on HabrĂŠ?
Despite the fact that practical examples are considered on the basis of our Diafan.CMS, the tips below will be useful to any web developers of sites, individual modules, mobile applications and network programs.
1. Scale your app to fit your needs.
Move from easy to hard.
')
Let's say your project needed SMS-notifications. The SMS sending code is quite simple, in a few lines: this is just a phone validation and a GET request to the SMS sending server - integrate it directly.
For example, when SMS notifications to the administrator on receipt of orders were required in Diafan.CMS, we put this simple code directly in the Store module, in the function that creates the order.
Then we needed SMS notifications in the âFeedbackâ and âMailingâ module. Then we brought this functionality as a global function into a separate file includes / sms.php and calmly used it in different modules in this form:
include_once('includes/sms.php'); Sms::send($message, $to); When it was required to supplement SMS notifications with settings and our interface, we moved this functionality into a separate module.
And if you need the ability to connect different backends, for example, to work with different SMS operators, this can be done in the module, the general structure of the system allows it.
The bottom line is that you do not need to develop any complex libraries in the first stages to work with the new functionality, you do not need to make 100,500 settings that will confuse the site administrator and require writing bales of documentation. And then maybe not needed. It is better to do only what is required at this stage.
This does not lead to a snowball if the system has a clear hierarchy of âcode - pluggable functions â pluggable modulesâ. Your application will remain simple and easy to understand at every stage of development.
Refusing extra code that is not used at the current stage of development, you can kill two birds with one stone:
- complete control of the code, eliminating logical errors and security problems;
- total application size will not grow.
Moreover, this approach led us to another feature. We use very little external code, plugins and libraries. Contrary to the "bicycle" error, we believe that it is better to write one function for your tasks than to include a huge third-party library.
2. Pay attention to URL handling.
Analysis of links by the server is a bottleneck through which all pages of the site go. Try to make this place as easy as possible for the server.
Most often, for address navigation through the pages of the site, instead of simple server addresses like site.ru/?PAGE=34, the so-called âCNCâ (human-understandable URL) is used, when each page of the site corresponds to a beautiful string friendly address. For example, at the site.ru/o_kompanii/ address the âAbout the Companyâ page opens. Here, âo_kompaniiâ is a string CNC interpreted in site.ru/?url=o_kompanii by which the CMS searches for a page named o_kompanii in the database and displays its contents on the site. As if to request it by PAGE = 34. However, in addition to the string name of the page, some parameters can also be passed through the address bar, and there may be many of them. The analysis of such CNC can be very difficult.
In Diafan.CMS, there is only one template for link building: / stringCNC / numeric parameter 3 / another numeric parameter 25/3, 25 are the parameter values. Those. if the URL has numeric parameters, it means there are variables in front of them, in any case, this is the most frequent structure.
We immediately discard all the numeric parameters, and then, if something remains, look at the string CNC in the database and the page corresponding to it. If we donât find the page on the CNC, then we start adding one parameter to the CNC and look again in the database. Yes, such an algorithm consists of several unnecessary SQL queries, but they will be only for such specific pages or for pages 404, if artificially substituting variables there. That does not load the base for all other (and most of them) pages.
This is only easier for ugly links with $ _GET data, such as site.ru/?url=page:13&m=3&m2=25.
3. Run the functionality only when necessary.
When launching any page of the site, first download only the most necessary functionality, which in 99% of cases will be used. All the rest run only according to needs.
For example, if in Diafan.CMS you need to connect a product to a rating, we simply refer to the variable $ this-> diafan -> _ rating and it will give the entire connected part of the Rating module. Since the first access to the pseudo-variable $ this-> diafan _ ***, the presence of the requested module will be checked and the correct connection to it will be made as an instance of the class. It is convenient to use, and cuts off any extra functionality in the current script execution. If the rating on the page is not needed, it will not run at all.
4. Functionality, creating additional load, connect optional
Do not connect additional functionality globally to all modules, let's optionally enable it.
For example, in any module where comments are used, in Diafan.CMS there is an option âConnect commentsâ. This eliminates situations when comments to products are needed, but not to news, and the webmaster simply hides the output of comments in the news template, but the CMS continues to execute the SQL query, look for comments to the news, generate all the necessary data. In other words, waste server resources in vain.
The same applies to any other functionality that uses the resources of the server. Don't need a visitor counter, news rating, images for menu items? Take out their connection in the settings and disable the default. This will save resources.
The same applies to hidden settings, for example, âAccess Rightsâ, when you need to flexibly configure access restrictions to different specific site materials for different users (Guest, User, Wholesaler, etc.). To ensure this feature, the easiest way is for all SQL queries of all materials. add:
... LEFT JOIN access AS a ON a.element_id = 'id_material' AND (e.access = 'yes' AND a.role = 'role_current_user') , which checks if the current user has access to the requested material.
And why do we need this load, if all the news is seen by all users, and these situations are much more? It makes sense to add this LEFT JOIN to the SQL query optionally. In Diafan.CMS, this setting is turned on / off automatically without the involvement of the administrator. That is, if for some material you need to restrict access, it is enough to note the setting and queries to the database are complicated. If not, the requests are simple.
5. Minimize the number of queries to the database
Use prepare functions.
It happens that the functionality of one module is used in another. Imagine that we have 10 products in the shop module and each image from the image module. The most common mistake made by beginners is when additional requests are made to the image table in the product output cycle.
$rows = SELECT * FROM shop; // foreach ($rows as $row) { echo "".$row['id']; // $img = SELECT * FROM img WHERE shopid=$row['id']; // foreach ($img) { echo " ".$img["link"]; // } } * here and below the code is conditional, for clarity of the algorithm and the transfer of meaning.
Thus it turns out 1 + 10 = 11 separate SELECT queries to the SQL server. However, it is better to query the products, remember all the IDs of the goods received, and then make only one SQL query: select all the images for all our products at once. It turns out only 2 SELECT queries, instead of 11.
$rows = SELECT * FROM shop; // foreach ($rows as $row) { $goods[] = $row['id']; // } $img = SELECT * FROM img WHERE shopid IN implode($goods[]); // foreach ($img) { $images[] = $img['link']; // } foreach ($goods[]) { echo "".$goods[]; // echo " ".$images[$goods[id]]; // } But here the problem is that the logic of the cycles is violated and the code becomes inconvenient. We managed to find an elegant solution that is beautiful for use and optimal for the server: the prepare function. The code is:
$rows = SELECT * FROM shop; // foreach($rows as $row) { $this->diafan->_images->prepare($row['id'], 'shop'); // } foreach($rows as $row) { $images[$id] = $this->diafan->_images->get($row['id'], 'shop'); // get echo "".$row['id']; // echo " ".$images['id']; // } The prepare function simply remembers all the elements for which you need to get pictures. And the feature of the get function is that the SQL query will be executed upon the first call and the result will also be remembered. All other get calls from this loop will work with the data received for the first time.
And this is done for any functional: CNC query for link building, tag query, rating query, etc. Therefore, in Diafan.CMS, calls to the SQL server are minimal.
6. Use caching
Write your own caching algorithms, use the server.
If the online store is settled and new products are not added, why every time to contact the database to create a list of products for each customer? It is enough to contact the SQL-server once, save the results of the query in a text file in the cache folder and then take them from there. If a new product is added to the list, or a different page is requested, for example, a sorted list of products, you can reset the cache file and return to the database again. But only once, in order to update the cache.
You can cache as parts of the site, blocks, lists, cards and items, and all pages of the site. More details .
It is important to remember that static resources charge the server by an order of magnitude less than dynamic ones, so try to turn to SQL and the PHP interpreter as little as possible. By default, the file cache is used in Diafan.CMS, but you can also connect Memcache. We cache only data. All design in the cache does not fall. This allows you to save space for the cache and as a result, work with it faster.
7. Minimize dynamic data acquisition.
It is better to wait for the site administrator once, than to wait for each user.
For example, in Diafan.CMS there is a flexible system of discounts, when one product can have several prices, each with an established discount, also in different currencies. We do not calculate the new prices on the site for each site visitor. We do this when installing discounts in the administrative part of the site, count all the different price options for selected products and collect them into a separate shop_price table with a price_id sign. The initial price for the product is defined as id = price_id. If id <> price_id, then this is a variation of the price either at a discount or in the main currency of the site. And this table immediately indicates for whom this price (if it is set for a particular group of users).
Yes, if there are a lot of goods, when adding or changing a discount, the administrator will wait, looking at the spinning wheel. But the site visitors will not have to wait, because when outputting on the site, as well as when searching, sorting and other capacious operations, the system simply chooses the smallest one from all the available prices from one table one by one price_id.
8. Use database normalization and indexes in tables.
Spread the complex structure of information on different simple tables and use indexes on them. It happens that it is more profitable to make two simple queries than to build one complex INNER JOIN.
In Diafan.CMS, basically all adjacent tables have approximately one structure: id, element_id, element_type, module_name. Queries on them are extremely simple.
SELECT * FROM rating WHERE element_id = 3 AND module_name = 'shop' AND element_type = 'element'
And for all the specified fields indices are required.
Conclusion
If you look, we have not reported anything new and almost all the points are obvious. However, many people know the theory, but they are not always used in practice. We hope our tips will help you get together, stop being lazy and create beautiful, fast and optimized applications.
Source: https://habr.com/ru/post/300930/
All Articles