How Yandex.DZen, the caching plugin for WordPress and hosting have increased my pressure

This article, rather, from the series "hostess note." Well, a little more about personal experiences and razdolbaystve.

If you have to deal with sites on WordPress, on which the WP Super Cache caching plugin is installed, then the article may be useful for you. Otherwise, it's just a little story.

In short, the beginning of the story is this: the site lived. Thematic information and news. Made by an unknown master on CMS WordPress. And done exactly as the funds of its owner allowed at that moment. Over time, the site gained popularity, the number of visitors grew. The site itself "earned" by advertising. Those. the more visitors, the better. At some point, with peak loads, the site began to fail: pages were loading slowly and all that. But, of course, until the rooster pecked, no one really moved. The rooster appeared at the moment when the site was supposed to cover some event that attracted the attention of the public. The visitors came running, the site went to bed, and all the expected income was melting before our eyes. Due to circumstances, I had to act as a resuscitator. About this I wrote an article then. My decision was horrible (do not repeat), but the main goal was fulfilled - dreams were saved. Basically.
')
Of course, no one was interested in repeating this situation and worked on the site. First of all, we found out the reasons for the fall and high load on the server. They were commonplace: the lack of caching on the site as such and a poorly implemented mechanism for accounting and displaying the number of views of articles. The latter in itself created a decent load: with each request for the article page, he took from the database the number of views of this article, output it, and then add one and write back to the database. At the same time, the wp_postmeta table with metadata was used for storage (as it seems to be called in WP). In which the mass was kept completely unrelated to the accounting of information views and which was very large.

The problem of caching was solved simply: in a calm atmosphere, the WP Super Cache plugin was installed normally. What, many, if I remember correctly, advised me to make instead of the eerie crutch that I built in the comments to my article. Honestly, even then I was poorly oriented in the WordPress ecosystem, and therefore that crutch appeared. The caching plug-in was set up and set up by knowledgeable people already, and it well stood right out of the box. Thus, the caching problem was solved. Why this particular plugin was chosen - I do not know for sure. As far as I understand, this is one of the most popular plugins of this type for WP.

In view of the views received a little differently. It is clear that the original mechanism had to be abandoned. I didn’t want to refuse to take into account viewing itself: at least, it was shown that it was showing blocks like “the most popular articles of the week. All plug-ins to record views, even if they were “friends” with the cache plug-in, turned out to be very voracious and, most often, implemented the same mechanism with recording and extracting data from wp_postmeta. For a small site, this is quite suitable. For a site with fairly solid peak attendance rates, it is no longer there: too gluttonous for such a small function. Here it turned out that unused hosting paid for the years ahead turned up. The easiest and cheapest. The duties of storing, recording and returning data about views were piled on him. Everything is asynchronous, i.e. even if it falls, the main site will continue to calmly display articles, advertisements and everything that should show. The online storage of the browsing data was assigned to Redis, and the long-term storage was assigned to MySQL. So it turns, there is not particularly requests and eats out 1-2% of the maximum allowed load on that hosting.

And now, it takes quite a long time. Again the night and again the old story. The site is quite powerful traffic and the site begins to fall. And again, I am the only awake conditional specialist in the district. I am, of course, surprised by the repetition of events. First thought - someone accidentally turned off caching. But no, in the source code of the site page, which gives me a falling server, I see signs of a caching plug-in. Everything should be ok. But in the server control panel, I see that there is no RAM left, the number of queries to the database is incredible and everything is bad. First of all, I open a page with a description of the plug-in, quickly run through her eyes, find nothing and leave this occupation. The next step is to look at the statistics. I see that the main stream of traffic flows to the site with Yandex.DZen. And the query that leads the user to the site looks like this:

https://example.com ? utm_referrer = https:% 2F% 2Fzen.yandex.com

Those. Yandex.DZen adds its utm-tag to the address. Since the server is already completely lying and gives me only 500, then I can’t really check the theory that pages with such a “appendix” are not cached for some reason. Therefore, the next “crutch” is born (it was later replaced): a redirect is added to .htaccess, which converts the request with utm-tags into a request without them before WordPress and all its powerful plug-ins come into play. The machine restarts and, voila, everything works: the site loads quickly, the server rustles quietly at low revs. Like nothing was.

Then I relaxed and climbed calmly to see the settings for the caching plug-in. First of all - a tick "Do not cache pages with GET parameters", which is there. Everything is fine, it is not worth it. Plus, as the experiment showed, the plugin does an excellent job of caching requests with any set of arbitrary parameters. If these are not utm-tags (in fact, I cut with a redirect only requests of a certain kind, but not all where there are utm-tags).

This is where I took a closer look (using CTRL + F) to the page of the plugin. And I found there in the FAQ the glorious item “How should I use the utm_source tracking tools in Google Analytics with this plugin?”. It is clear that at the first cursory inspection I safely ignored him: it seemed to be nothing in common with the problem. At the same time, by the way, it is not that very informative and does not offer any specific solution.

The only possible useful conclusion in the article is: if you have a WP site with the WP Super Cache plugin, then check what it does (or does not do), if you request it with the parameters “? Utm_referrer = https:% 2F% 2Fzen”. yandex.com "etc. I'm not sure that when installing this plugin, everyone reads its FAQ so carefully. The concrete implementation, apparently, remains with the site owners: when I last looked at updating this plugin, I did not see any changes regarding its strange reaction to the utm-tags.

But the story would be incomplete (and I would not tell it), if it had not happened yet another confirmation of Murphy’s law. While we peacefully corresponded with the site owner and watched the site work quietly for about an hour ... the site crashed. Unexpectedly and finally. No access to the server control panel, does not work FTP, SSH and everything else. Nothing works. If before that my pressure was only slightly raised by solving the task that Ya. Zen threw and caching plug-in (after all, this is a separate pleasure - to quickly understand and repair something in light-hearted hassle), then the whole mini-panic attack happened to me. And only a vague feeling that with a couple of lines in .htaccess it is impossible to kill everything in an hour after inserting these lines, did not allow the “mini” to grow into something more complete. In general, after a couple of minutes of exchange of surprised messages, we got into the personal account of the hosting provider with whom the server lives. And there in the tickets we found a warning about the "technical work from X to Y to MSK". The most interesting thing is that in the mail, no matter how we looked for, we did not find any messages from the host about these works. The ticket was at least a day old. Prior to this, such messages came to the post office, and no one usually doesn’t look at all at all for helpdesk tickets (and the host’s office itself) without special need. Therefore, what happened was a big surprise. After that we scolded the hoster’s eyes, laughed, waited until everything worked and went to sleep.

Here is a story.