Caching in symfony. Ideology of HTML caching. Components & partials

For 2.5 years of using symfony, I constantly have to deal with the problem of misunderstanding by symfony programmers of html caching ideas. The purpose of this post is to convey to the bright minds of symfony-developers an awareness of the paradigm of using partials & components.



So, what problem are we talking about? Any person who has come to understand the idea of ​​the framework, in particular the symphony, somehow heard about optimization. The desire for premature optimization of project developers kills all the ideas on which the symfony cache system is built.
')
For example, I will use our recently launched project Emotions have not cooled. The essence of the project: the sale of burning cheap vouchers. Very often, the updated information should be aggregated into the rss-tape, shown on the main pages of the site. The main page is a listing with filters, the main interface of the site, it should work with minimal resources and as fast as possible.

Information is updated very often: offers are relevant for several days, information on them may often change. It turns out such a public blog, which is written by specially trained people :)

We turn to a simplified implementation.

Our example: there are tours - Table A, there is information about tour operators - Tabitsa B, there is information on hotels - Table C. There is a lot more there, but for us it does not matter.

As ORM we use Propel.

Task: create product listing page. In the listing you need to display a brief information on the hotel, in which the tourist will go and on the service provider - the tour operator. The same is true when listing refrigerators or phones or anything else you can sell: information about the manufacturer, information about the category of goods, about the supplier, about the delivery ... The essence is the same: you need to select a lot of entities from the database. And not anyhow, but preferably using ORM paradigms.

The standard solution that comes to mind is to select indexed arrays of objects from tables A, B, C, using unique entity keys as indices. And select the main action to which the control flow is transferred on the listing page.

We believe that in the model we have the following classes: A, B, C, APeer, BPeer, CPeer.

Listing is done with some filters, the subtleties of which we absolutely do not care.

1. <? php
2. class indexActions extends sfActions {
4. public function executeListing (sfWebRequest $ r) {
6. $ f = new AFormFilter ();
7. $ f-> bind ($ r-> getParameter ('filter'));
8. if ($ f-> isValid ()) {
10. $ c = $ f-> buildCriteria ();
11. $ this-> array_of_a = APeer :: doSelect ($ c);
13. $ this-> array_of_b = BPeer :: retrieveBySelectedA ($ this-> array_of_a);
14. $ this-> array_of_c = CPeer :: retrieveBySelectedA ($ this-> array_of_a);
15. } else {
17. $ this-> getUser () -> setFlash ('error', 'Error while retrieving data.');
18. }
19. }
20. }

What we have done here: According to the filters obtained from the request, the Criteria object was returned from the AFormFilter object of the instanceof sfFormFilter. Then we selected the records we need from tables B and C, for example, by the id of all selected entities from table A and indexed. And we did all this in the methods created by us retrieveBySelectedA in the classes BPeer and CPeer. 3 targeted appeals to the database. Not bad.

Next, we create / actions/index/templates/listingSuccess.php, in which we list the elements.

Naturally, you can still hide the code for maximum abstraction, I will not do this, the idea is important to us.

Turn on the cache. The first thing we encounter is that if there are parameters (GET or POST request), the Action is not cached. Ok, what to do: make a request to the database in the component, passing it only an array of incoming parameters.

All right, and if we have not a listing page, but a regular ordinary index page. You can cache everything with a layout by specifying the following parameter in /actions/index/config/cache.yml:

1. list:
2. enabled: on
3. with_layout: true

This is suitable for a reinforced static page. But what if we have authorization, then the header is clearly different for each user. Cache the entire page can not be. Conclusion - to make everything in partials and kmponenta.

A symphony is remarkable in that it caches any element of a page added via PartialHelper: include_partial and include_component. In this case, it caches all items in general, those if the cache of the parent element were cleared - the cache of the child elements will be saved.

So back to the listing page. We take the sample into a separate component:

1. <? php
3. / * actions.class.php * /
4. class indexActions extends sfActions {
6. public function executeListing (sfWebRequest $ r) {
9. $ this-> filterParams = $ r-> getParameter ('filter');
10. }
11. }
12. / * end of actions.class.php * /
14. / * components.class.php * /
15. class IndexComponents extends sfComponents {
17. public function executeListingBlock () {
19. $ f = new AFormFilter ();
20. $ f-> bind ($ this-> filterParams);
22. if ($ f-> isValid ()) {
24. $ c = $ f-> buildCriteria ();
25. $ this-> array_of_a = APeer :: doSelect ($ c);
27. $ this-> array_of_b = BPeer :: retrieveBySelectedA ($ this-> array_of_a);
28. $ this-> array_of_c = CPeer :: retrieveBySelectedA ($ this-> array_of_a);
29. } else {
31. $ this-> getUser () -> setFlash ('error', 'Error while retrieving data.');
32. }
33. }
34. }
35. / * end of components.class.php * /

1. / * listingSuccess.php * /
2. <? php include_component ('index', 'listingBlock', array ('filterParams' => $ filterParams))?>
3. / * end of listingSuccess.php * /

Great, now we have a cached pattern for the same filter parameters, and thus the number of target queries is 0. For the new parameters, it still equals 3.

All is well, until a new product is added. As the right guys, we hang up the following code on the save action in the admin panel:

1. <? php
2. $ configuration = ProjectConfiguration :: getApplicationConfiguration ('frontend', 'prod', false);
3. sfContext :: createInstance ($ configuration, 'frontend');
4. $ cacheManager = sfContext :: getInstance ('frontend') -> getViewCacheManager ();
6. $ cacheManager-> remove ('@ sf_cache_partial? module = index & action = _listingBlock & sf_cache_key = *');

Cache cleared. For all requests, again 3 calls to the database.

The system works smoothly. In fact, of course, everything is more complicated. The first problem we encountered is ORM ate a lot of memory. We tried to optimize by transferring to a normal sql query with a selection of only the necessary values. There were problems with the lack of abstraction - I still for the code standards imposed by the framework, and I do not like the invention of bicycles. Everything turned out exactly as in the textbooks: the code written by one person, with large labor costs was modified by other developers. Sooooo wanted to use adequately abstracted ORM.

Naturally, we reached the caching of each sentence, while making additional requests into a separate component for sentences:

1. / * components.class.php * /
2. class IndexComponents extends sfComponents {
4. public function executeListingBlock () {
6. $ f = new AFormFilter ();
7. $ f-> bind ($ this-> filterParams);
9. if ($ f-> isValid ()) {
11. $ c = $ f-> buildCriteria ();
12. $ this-> array_of_a = APeer :: doSelect ($ c);
14. } else {
16. $ this-> getUser () -> setFlash ('error', 'Error while retrieving data.');
17. }
18. }
20. public function executeOfferItem () {
22. $ this-> b = $ this-> a-> getBRelatedByB ();
23. $ this-> c = $ this-> a-> getCRelatedByC ();
24. }
25. }
26. / * end of components.class.php * /

1. / * _listingBlock.php * /
2. <? foreach ($ array_of_a as $ a):?>
3. <? include_component ('index', 'offerItem', array ('a' => $ a))?>
4. <? endforeach?>

Yes, at first glance, I went to the crime. Carelessly I appeal to the database 2 times for the sake of each element, for which I have often been reproached by colleagues who do not want to think on the scale of the system as a whole.

Let's all super simplistic calculate:

Let me have 5 filters, each with 3 options, the average sample returns 100 results. Updating occurs every 5 minutes (adding / changing 1 record). Per day, let us have 50k hosts on this page, that for a flat account 2000 hosts per hour.

The first option is to remember, yes? - 3 requests for each option. We have 45 ~ = 50 requests for full coverage of all options. Then if we have ~ 150 hosts in 5 minutes, then we consider ~ 50 requests to the database and 100 readings from the cache (of course, I exaggerate a lot, considering that every 5 minute period covers all the options). During the day we have: 50 * 12 * 24 ~ = 15k target readings from the base.

The second option with caching a single sentence:
to cover all the options you need to do: 15 options * (1 request + 100 offers * 2 dop. request) = 3015 target readings. Every 5 minutes there is a change:
3 requests * 12 * 24 ~ = 900 requests. Total per day we have about 4k target readings from the base.

In this unobvious way, we have reduced the number of requests by 4 times. What if we have some kind of blog, where users constantly update information - we have a strong reduction in the update period, where the benefit of using more detailed caching becomes much more noticeable, already going into order.

I am afraid to talk about the time of page generation, it all depends on the settings. In our project we used XCache for ViewCache:

1. / * factories.yml * /
2. view_cache:
3. class: sfXCacheCache
4. param:
5. automaticCleaningFactor: 0
6. storeCacheInfo: true

Of course, the page generation time when using the standard sfFileCache is incomparably longer. But let's leave the system configuration issue aside.

In addition to winning the number of target calls to the database, we have a code with super obvious transparent logic and maximum abstraction, I think it’s not worth explaining the benefits.

When using ORM in a symphony by default, the requests themselves are not cached . This is not because the developers are stupid, but because a different level of cache is used here. Another idea.

The disadvantages of this approach are sometimes attributed to the very confusing logic of removing items from the cache. Suppose a user changes the data on a video uploaded by him, and you have to clear the cache for all elements with this video on the listing pages (for all variants), for a detailed page, for elements from rss-feed if there is one, and so on. All this has to be thought about and remembered at the right time, so as a recommendation, I can advise you to immediately design a system with regard to caching.

In our project, the admin panel is physically located in another container, which makes it difficult to manage the cache. So we had to make a simple XMLRPC interface for this.

Delivering costly operations to individual components is the main idea of ​​html caching. No need to reinvent the wheel, everything is already working, and it is very simple and functional. If all the guys from my team understand this, I can stop swearing at them :)