An instructive story about what can happen with a site on a shared-hosting
The working day was slowly but surely coming to an end. Sunlight streamed through the blinds and flooded the office with golden crimson. Somewhere in the corner, a coffee machine was buzzing, squeezing out the remaining coffee from the capsule. Our project discussed something lively with the designer, and I ruled shoals kindly left to me by a junior programmer.
And everything seems to be nothing if it were not for the message: "What about your T site?"
One of my good friends noticed that one of our sites is displayed incorrectly and let me know.
I dropped all the cases and loaded the page with the site. No problems were observed on it. Well, unless a small redesign was required ...
“Everything is all right with the site,” I wrote to my friend and again plunged into the fascinating world of code and bugs.
After a while, my brain still took out the disturbing signal of my friend from memory and I involuntarily useful to re-check the site. For some reason confident in the performance of the site, I experienced a light self-confidence.
The main page of the site, sadly greeted me with a stray coding and the complete absence of css. Black abracadabra symbols on a pristine white background brought me back to reality. Self-confidence instantly disappeared and I began frantically pressing CTRL + F5 into the keyboard.
"This is just a cache ... Yes ... Just a cache ..." - I repeated myself over and over again.
When I realized that my tenth attempt to reset the browser cache does not give any results, and the main page of the site continues to change its appearance with each press on the cherished keys, the first thing that flashed through my head was “Hacking ?!”. My fears began to grow stronger when, after the next page reload, I saw a red and white inscription stating that such and such a table was not found.
Hands themselves began to change everything related to the site, access: admin panel, database, SSH.
After, I began to carefully study the logs. Although the logs - it says loudly. At my disposal were only reports on the web server. MySQL crash logs and failed authorization attempts via SSH are not issued on shared hosting.
Nothing strange was revealed in the logs and I went straight to the SSH console in order to connect to MySQL directly, because I clearly remember that the site cursed the lack of tables in the database.
Very strange, but all the tables were in place (at least those on which the site cursed were exactly in place).
As a CMS, on site T, 1C-Bitrix is ​​used. We love this system very much and admire it in every possible way (with rare exceptions).
What was my surprise when, going into the site settings, I saw the data from a completely foreign site R. The path to the root, the domain name and some other settings were changed. I, stunned, looked at the monitor with round eyes in surprise. At the same time, somewhere behind me, our project has already poured Valerian tincture. For some reason I pressed F5. What happened after that plunged me into the deepest shock and gloom. Site settings again took normal values.
At this moment, for a moment I doubted the adequacy of the entire IT industry as a whole and began to believe in miracles. My wife's phone call brought me out of my stupor. I picked up the phone. Until now, I can’t remember what she was telling me, since the sensation of something mysterious did not leave me and my brain frantically tried to find at least some explanation of what had happened.
I grunted something into the phone and hung up. Well now, at least I'm sure of something. I am sure that a lot of interesting information about myself awaits me at home.
But this later, but now we need to understand what is happening.
After the next reload of the page with the settings, I again saw other people's values.
Judging by the absolute path to the root folder of the web server, we had prescribed the settings of the site, which was located on the same hosting as we.
I instantly felt my handset and rang the tech support for the Hosting.
From the conversation it became clear that they are not able to solve our problems right now. The technical support officer politely let me know that I should write an appeal by email and my appeal will be reviewed within 24 hours, in the queue. After I, just as politely, told them everything I thought about them, I hung up and wrote a letter to them, filled with epithets. Just in case, we also wrote a letter to tech support for 1C-Bitrix.
The dead silence of the office was broken only by the ticking of a large wall clock and the barely audible noise of the cooling system of the system unit. A round-shaped white clock performed several functions in the office. First, they were an element of the interior. There is not much furniture in our office, nothing superfluous. Although, I absolutely do not mind buying a large and soft sofa, so that in the breaks between tasks, you can lie down with a cup of coffee in your hands and, stretching your legs, immerse yourself in dreams that some day we will move away from developing websites and develop interesting ones. and of course innovative, in terms of gameplay, games.
Secondly, this snow-white round clock still fulfilled the function embodied in them - they showed time. If someone needed to know the time, then he certainly turned to this watch. They have an inexplicable and almost hypnotic attraction. And nobody cared that there was a clock in the smartphone at hand, or on the monitor screen.
')
So at this moment our pedantic warder inexorably counted time with every step of the second hand.
And we all expected that just now, now, our mail server will receive the long-awaited response from any of the technical support.
But there was still no letter. About any work and speech could not be. My brain, chasing a gray matter, tried to unravel this strange metamorphosis with the site T.
Understanding that if the site is seen by its visitors in such a state, they will obviously not be happy, I’d come to close it from public access. Moreover, if there is a hacking, it is quite possible that the site is already full of injections, in order to steal cookies, and therefore, a visit to this site is absolutely contraindicated.
After pressing the cherished button in the settings of the main module, the site plunged into hibernation.
Out of curiosity, I went to the site R, whose settings were registered in the settings of our site.
Site Under Construction was on the main page of R.
The brain tried to establish a connection between the two latest events and I again opened access to the site T.
Returning to the site R, I saw that he recovered his performance.
Coincidence or dependence?
Having repeated the manipulations in the settings of the main module, I made sure that the dependence of site R on our site T and vice versa takes place here.
But how can this be?
All, than are connected sites T and R - this is hosting.
“And let's call the phone number listed on the R website and try to explain the situation to them,” our project unexpectedly suggested to everyone.
We did not have time to feel the whole genius of this idea, when suddenly a telephone call rang in the office.
At the end of the tube were the guys who developed the site R and at the moment, too, are at a loss from what is happening.
I was handed the phone.
After a short conversation, it turned out that they, like us, have no idea about the causes of what is happening.
"Apparently on the hosting, in some strange way, there is a symbolic link between our two bases," I suggested.
Well, what other explanation could you find here? Access to both databases is different, but, in some strange way, changes in their database affect ours and vice versa.
“Have you tried creating another database?” I continued.
“Yes, this is the third” - they answered me at the other end with a sadness in my voice.
Thus, the version with the symbolic link did not withstand any criticism.
Those guys also wrote an appeal to technical support for hosting. We exchanged names, skype and phones, having agreed to keep each other up to date.
News from tech support Hosting all was not. I dialed their number again and started explaining to the tech support employee that the situation is a stalemate and the same problem is already observed on two Hosting accounts.
I did not hear anything intelligible in response. At the end of a meaningless and fruitless conversation, I was finally convinced that hosting support would not help us.
Well, at least it was clear that this is not hacking.
From this moment on, I stopped hoping for technical support for the Hosting and took everything into my own hands.
Something told me that I would get answers to my questions by going to the list of tables through the 1C-Bitrix admin panel. Well, in fact, there were no other options from where to start the search.
I was surprised at least the same when I found out that the contents of the b_lang table (where the site settings are stored) and the contents of the admin page of the site settings differ.
What the hell?! How can this be ?!
“There are two of them!” Flashed through my mind.
“There are two connections to the database” - I became more and more entrenched in this thought.
How else to explain that the admin panel shows one thing, and the contents of the tables - another?
A little more developing this idea, I remembered the constant connection to the database .
Although, judging by the description of the technology of persistent connections, they are separated by at least a username and password when connecting to the database, but these data are different for the R and T sites.
And yet, can the caching system somehow remember the connection and give it away with two completely different connections?
By this time I was ready to believe in anything.
I turned off the function of persistent connections in the 1C-Bitrix settings and, at the same time, left the caching type empty (before that there was the value - “APC”).
I asked the same to do and my colleagues in misfortune.
When everything was ready, I pressed F5. These were the longest and most exciting seconds of my life. Well, it can not be so simple.
Finally the page was loaded and ... the site earned! Site R also had everything in order. There was only one way to check if everything was normal. I entered the main module settings and again sent the site to the “Under Construction” state. Site T, obediently followed this instruction, and site R remained in working condition. This fact told us that the problem was successfully solved and the bases were no longer connected to each other.
But what is the problem? In persistent connections or in caching?
It was possible to find out that the problem was that the sites that are not connected at all with each other were mixed with the APC cache.
1C-Bitrix, in the dbconn.php file, forcibly caches the following tables:
The list may vary.
Among other tables, the very b_lang is clearly visible, where the settings of sites are stored. Consequently, the T and R sites alternately recorded data in this table and cached it in APC. When site R cached a table, site T took a cached copy and vice versa.
But here's the most important question - how is it possible on a hosting, with thousands of working accounts, mixing the cache?
After all, it turns out that any site that uses APC can interfere with the performance of any (almost) other site (also using APC), within the limits of this hosting (more precisely, the server).
As a result - losses due to idle sites and, possibly, data theft, because you can cache anything.
Is this the logic of the hosting operation is normal?
After a brief discussion with the developers of the site R, we came to the conclusion that you can simply set a different BX_CACHE_SID for our sites.
Obviously, there is a certain problem, since site T has worked on this hosting for about a year and a half and did not cause any complaints of this nature. And then suddenly such an incident.
Why did our cache get mixed up with site R?
Why there are no mechanisms for delimiting the cache of different accounts on such a large hosting?
With these questions I went home. They did not go out of my head until the evening.
At the door I was greeted warmly, already having time to miss me, my wife, and on the table stood a hot and deliciously smelling supper.
“This misunderstanding is over,” I thought with relief.
Yes, someday we will definitely play games ...
UPD 1: having considered the comments, I came to the conclusion that it is necessary to summarize a brief summary of the above.
Always set a unique cache ID and back up regularly
UPD 2: As a result of communication with the technical support of the Hosting, it turned out that they do not provide the account sharing service at all.
They gave contacts to the director of technical support, to which you can send your wishes on the work of hosting.
UPD 3: The head of the technical support service of the Hosting service shared with me wonderful information about the development of a new hosting environment in which separation of account caches will be provided.
And everything seems to be nothing if it were not for the message: "What about your T site?"
One of my good friends noticed that one of our sites is displayed incorrectly and let me know.
I dropped all the cases and loaded the page with the site. No problems were observed on it. Well, unless a small redesign was required ...
“Everything is all right with the site,” I wrote to my friend and again plunged into the fascinating world of code and bugs.
After a while, my brain still took out the disturbing signal of my friend from memory and I involuntarily useful to re-check the site. For some reason confident in the performance of the site, I experienced a light self-confidence.
The main page of the site, sadly greeted me with a stray coding and the complete absence of css. Black abracadabra symbols on a pristine white background brought me back to reality. Self-confidence instantly disappeared and I began frantically pressing CTRL + F5 into the keyboard.
"This is just a cache ... Yes ... Just a cache ..." - I repeated myself over and over again.
When I realized that my tenth attempt to reset the browser cache does not give any results, and the main page of the site continues to change its appearance with each press on the cherished keys, the first thing that flashed through my head was “Hacking ?!”. My fears began to grow stronger when, after the next page reload, I saw a red and white inscription stating that such and such a table was not found.
Hands themselves began to change everything related to the site, access: admin panel, database, SSH.
After, I began to carefully study the logs. Although the logs - it says loudly. At my disposal were only reports on the web server. MySQL crash logs and failed authorization attempts via SSH are not issued on shared hosting.
Nothing strange was revealed in the logs and I went straight to the SSH console in order to connect to MySQL directly, because I clearly remember that the site cursed the lack of tables in the database.
Very strange, but all the tables were in place (at least those on which the site cursed were exactly in place).
As a CMS, on site T, 1C-Bitrix is ​​used. We love this system very much and admire it in every possible way (with rare exceptions).
What was my surprise when, going into the site settings, I saw the data from a completely foreign site R. The path to the root, the domain name and some other settings were changed. I, stunned, looked at the monitor with round eyes in surprise. At the same time, somewhere behind me, our project has already poured Valerian tincture. For some reason I pressed F5. What happened after that plunged me into the deepest shock and gloom. Site settings again took normal values.
At this moment, for a moment I doubted the adequacy of the entire IT industry as a whole and began to believe in miracles. My wife's phone call brought me out of my stupor. I picked up the phone. Until now, I can’t remember what she was telling me, since the sensation of something mysterious did not leave me and my brain frantically tried to find at least some explanation of what had happened.
I grunted something into the phone and hung up. Well now, at least I'm sure of something. I am sure that a lot of interesting information about myself awaits me at home.
But this later, but now we need to understand what is happening.
After the next reload of the page with the settings, I again saw other people's values.
Judging by the absolute path to the root folder of the web server, we had prescribed the settings of the site, which was located on the same hosting as we.
I instantly felt my handset and rang the tech support for the Hosting.
From the conversation it became clear that they are not able to solve our problems right now. The technical support officer politely let me know that I should write an appeal by email and my appeal will be reviewed within 24 hours, in the queue. After I, just as politely, told them everything I thought about them, I hung up and wrote a letter to them, filled with epithets. Just in case, we also wrote a letter to tech support for 1C-Bitrix.
The dead silence of the office was broken only by the ticking of a large wall clock and the barely audible noise of the cooling system of the system unit. A round-shaped white clock performed several functions in the office. First, they were an element of the interior. There is not much furniture in our office, nothing superfluous. Although, I absolutely do not mind buying a large and soft sofa, so that in the breaks between tasks, you can lie down with a cup of coffee in your hands and, stretching your legs, immerse yourself in dreams that some day we will move away from developing websites and develop interesting ones. and of course innovative, in terms of gameplay, games.
Secondly, this snow-white round clock still fulfilled the function embodied in them - they showed time. If someone needed to know the time, then he certainly turned to this watch. They have an inexplicable and almost hypnotic attraction. And nobody cared that there was a clock in the smartphone at hand, or on the monitor screen.
')
So at this moment our pedantic warder inexorably counted time with every step of the second hand.
And we all expected that just now, now, our mail server will receive the long-awaited response from any of the technical support.
But there was still no letter. About any work and speech could not be. My brain, chasing a gray matter, tried to unravel this strange metamorphosis with the site T.
Understanding that if the site is seen by its visitors in such a state, they will obviously not be happy, I’d come to close it from public access. Moreover, if there is a hacking, it is quite possible that the site is already full of injections, in order to steal cookies, and therefore, a visit to this site is absolutely contraindicated.
After pressing the cherished button in the settings of the main module, the site plunged into hibernation.
Out of curiosity, I went to the site R, whose settings were registered in the settings of our site.
Site Under Construction was on the main page of R.
The brain tried to establish a connection between the two latest events and I again opened access to the site T.
Returning to the site R, I saw that he recovered his performance.
Coincidence or dependence?
Having repeated the manipulations in the settings of the main module, I made sure that the dependence of site R on our site T and vice versa takes place here.
But how can this be?
All, than are connected sites T and R - this is hosting.
“And let's call the phone number listed on the R website and try to explain the situation to them,” our project unexpectedly suggested to everyone.
We did not have time to feel the whole genius of this idea, when suddenly a telephone call rang in the office.
At the end of the tube were the guys who developed the site R and at the moment, too, are at a loss from what is happening.
I was handed the phone.
After a short conversation, it turned out that they, like us, have no idea about the causes of what is happening.
"Apparently on the hosting, in some strange way, there is a symbolic link between our two bases," I suggested.
Well, what other explanation could you find here? Access to both databases is different, but, in some strange way, changes in their database affect ours and vice versa.
“Have you tried creating another database?” I continued.
“Yes, this is the third” - they answered me at the other end with a sadness in my voice.
Thus, the version with the symbolic link did not withstand any criticism.
Those guys also wrote an appeal to technical support for hosting. We exchanged names, skype and phones, having agreed to keep each other up to date.
News from tech support Hosting all was not. I dialed their number again and started explaining to the tech support employee that the situation is a stalemate and the same problem is already observed on two Hosting accounts.
I did not hear anything intelligible in response. At the end of a meaningless and fruitless conversation, I was finally convinced that hosting support would not help us.
Well, at least it was clear that this is not hacking.
From this moment on, I stopped hoping for technical support for the Hosting and took everything into my own hands.
Something told me that I would get answers to my questions by going to the list of tables through the 1C-Bitrix admin panel. Well, in fact, there were no other options from where to start the search.
I was surprised at least the same when I found out that the contents of the b_lang table (where the site settings are stored) and the contents of the admin page of the site settings differ.
What the hell?! How can this be ?!
“There are two of them!” Flashed through my mind.
“There are two connections to the database” - I became more and more entrenched in this thought.
How else to explain that the admin panel shows one thing, and the contents of the tables - another?
A little more developing this idea, I remembered the constant connection to the database .
Although, judging by the description of the technology of persistent connections, they are separated by at least a username and password when connecting to the database, but these data are different for the R and T sites.
And yet, can the caching system somehow remember the connection and give it away with two completely different connections?
By this time I was ready to believe in anything.
I turned off the function of persistent connections in the 1C-Bitrix settings and, at the same time, left the caching type empty (before that there was the value - “APC”).
I asked the same to do and my colleagues in misfortune.
When everything was ready, I pressed F5. These were the longest and most exciting seconds of my life. Well, it can not be so simple.
Finally the page was loaded and ... the site earned! Site R also had everything in order. There was only one way to check if everything was normal. I entered the main module settings and again sent the site to the “Under Construction” state. Site T, obediently followed this instruction, and site R remained in working condition. This fact told us that the problem was successfully solved and the bases were no longer connected to each other.
But what is the problem? In persistent connections or in caching?
It was possible to find out that the problem was that the sites that are not connected at all with each other were mixed with the APC cache.
1C-Bitrix, in the dbconn.php file, forcibly caches the following tables:
- b_file
- b_file_bucket_size
- b_lang
- b_option
- b_lang_domain
- b_site_template
- b_event
- b_agent
The list may vary.
Among other tables, the very b_lang is clearly visible, where the settings of sites are stored. Consequently, the T and R sites alternately recorded data in this table and cached it in APC. When site R cached a table, site T took a cached copy and vice versa.
But here's the most important question - how is it possible on a hosting, with thousands of working accounts, mixing the cache?
After all, it turns out that any site that uses APC can interfere with the performance of any (almost) other site (also using APC), within the limits of this hosting (more precisely, the server).
As a result - losses due to idle sites and, possibly, data theft, because you can cache anything.
Is this the logic of the hosting operation is normal?
After a brief discussion with the developers of the site R, we came to the conclusion that you can simply set a different BX_CACHE_SID for our sites.
Obviously, there is a certain problem, since site T has worked on this hosting for about a year and a half and did not cause any complaints of this nature. And then suddenly such an incident.
Why did our cache get mixed up with site R?
Why there are no mechanisms for delimiting the cache of different accounts on such a large hosting?
With these questions I went home. They did not go out of my head until the evening.
At the door I was greeted warmly, already having time to miss me, my wife, and on the table stood a hot and deliciously smelling supper.
“This misunderstanding is over,” I thought with relief.
Yes, someday we will definitely play games ...
UPD 1: having considered the comments, I came to the conclusion that it is necessary to summarize a brief summary of the above.
Always set a unique cache ID and back up regularly
UPD 2: As a result of communication with the technical support of the Hosting, it turned out that they do not provide the account sharing service at all.
They gave contacts to the director of technical support, to which you can send your wishes on the work of hosting.
UPD 3: The head of the technical support service of the Hosting service shared with me wonderful information about the development of a new hosting environment in which separation of account caches will be provided.
Source: https://habr.com/ru/post/234071/
All Articles