Energy efficient synergy
Today our guest is Dmitry Bornyakov, head of the information technology department at the Baltika Novosibirsk plant. Under the leadership of Dmitry, a project was implemented for the implementation of a cold water conditioning system in the server room of the Baltika-Novosibirsk plant without the use of a cooling machine (chiller). High-performance precision cooling of IT equipment is provided by industrial ethylene glycol used in the brewing process.
One of the key risks in the comprehensive assessment of the IT infrastructure of the Novosibirsk branch was the risk of stopping the server room due to the possible failure of the air conditioning system. This was facilitated by the fact that climate systems de jure are not within IT’s area of responsibility, and, accordingly, IT can only indirectly affect the continuity of their work.
Evaluating possible options for leveling such risks, it was noted that the climatic conditions of western Siberia make it possible to use outside air to remove heat. Moreover, circuits with the organization of cooling the coolant with outside air are already used for the technological needs of the Novosibirsk site.
To calculate the payback of possible solutions, the operational costs of operating and maintaining the existing classic freon data center system were calculated.
In general, everything was like ... And it was boring. Everything worked. There was some kind of rustling around the server room (maybe I would call it a data center), and all sorts of attempts to increase entropy and other activities, giving importance and value to this place, closed themselves on themselves.
Folk began to relax, more and more often play kicker (table football) in the warehouse and discuss the prospects of the Kazakh space program, supplying them with drawings of a three-stage horse.
')
At the same time, with the scheduled maintenance of the air conditioning system, the next brigade revealed a bug that could turn into a decisive jamb for us! Namely - a very large inrush current of the freon compressor could put uninterrupted power supply. Not daring to force the UPS more than 20kW bursts, I ordered to dismantle the unit and carry out a thorough diagnosis, thinking to present warranty claims. No matter how wrong! Everything indicated that the previous service was performed by people, to put it mildly, unprofessional. A badly pumped-out track resulted in the copper-plating of the compressor, which will unambiguously lead to the seizure of the bearings. The previous office from the energy service, which performed the service, disowned our claim by our own acts. A strange way to do long-term business. Successes them.
So, as soon as I climbed, a lot of questions fell on my head - how to live with such an approach? The situation with the near-contractors contractors did not suit me, and I began to pull the blanket over myself, i.e. legitimize the ownership of air conditioning in the server. Usually, in such cases, no one resists.
After counting budgets and understanding the necessary amount of money for quarterly maintenance of three systems (one cools the UPS in a separate room), I felt a little uneasy. Of course, it is possible to replace the “quarterly” with “once in half a year”, but what was removed and washed out of the external blocks, directly said, “once in half a year” I will once recall.
Representatives of the APC, who visited them, urged that colleagues in Slavutych (Kiev) based on InRow 100, built a very interesting concept to actively study the issue. His key feature was water cooling not with a chiller, but with a heat exchanger with technological glycol. It has become enviable, all the more so since cold glycol is like a fool of a shag!
The integrator said that the designers are ready to work on parole, but they needed a constructive team with the bosses, and I did it! When reporting the financial benefits of abandoning the secondary water circuit, it was allowed to initiate the design. A preliminary reading on the night of the InRow datasheet revealed his ability to work on ethylene / propylene glycol with concentrations up to 50%! What made it possible to make bold and trump claims that “there will be no water!” In the director’s office.
And then ... Having ignited the importance, I informed the designers of the ideological wisdom, while listening to the proposed innovations. The result was the rejection of metal pipelines in order to reduce condensate, the use of heat insulating paint on nodes with complex geometry, and most importantly, the rejection of the heat exchanger and the virtualization of the second circuit using a three-way valve! He also insisted on bringing in and discharging all media from below the blocks, through the overlap. Still, the liquid ...
The knot of mixing turned out impressive, but to the touch flimsy. And since it was necessary to locate it in the warehouse behind the server, the risk of demolishing it with an overly quick loader was evident. Had to order fencing. InRow heavy blocks !!! Cheerfully drag them to the second floor of the stairs did not work. I bow to the strong guys from the warehouse of finished products. Those barely managed the six of them. Alas, there were traces on the linoleum in the hall and the anger of our curling team.
During the installation of InRow, an extra server cabinet was formed on the seats. Seriously, it was planned for transfer to a backup server. But this required a serious stop of the infrastructure, so all the action took place at night, and the darkest over the past 465 years. Coincided so. And further. When you realize that all the cabinets easily roll on their wheels, a person who has been rolling wardrobe on felt since childhood becomes a little awkward. But now in the department there are two pairs of new boots!
The launch was postponed three times. Officially, due to the fact that it was difficult to gather all three participants in the process in one place and time. But, IMHO, everyone stuffed his own price, revenge and sneered at me. However, to the credit of all participants, the launch was made from the first time with absolutely no comments. It would not be enough for someone to spoil my birthday.
Testing began the very next day, and with vicious addiction. The system was displayed on supercritical modes, readings and alarms of emergency sensors were recorded. Standing in jets of cold wind in the outer sheepskin coat, I took a parade of technologies. The cap moved down to the back of the head, but did not fall - a halo interfered with it.
Who are interested in boring technical details, can read further.
The next stage was the internal coordination of the idea, which took more than 4 months, after which a certain extract from the best world practices and experience of colleagues in the Slavutych company was obtained. The main task that had to be solved was to ensure the circulation of the coolant at a rate of at least 0.2 l / s and an inRow inlet temperature not lower than +5.6 ⁰. Those. in fact, there was a difference between the existing temperature and the minimum allowable temperature for InRow, equal to 10 ° C, which does not allow connecting them directly to the technological circuit. Experts of various services and levels, including the plant director, took part in the evaluation of the proposed options. As a result, the reference scheme was adopted, excluding the presence of water and intermediate heat exchangers, designed to level the above-mentioned temperature difference. A concept in the form of a design specification was issued to the integrator in October 2010, and a month later I had a ready-made project on my desk, the possibility of implementation of which was confirmed by the manufacturer of the equipment and the service of the chief engineer of the plant.
R-Style Siberia Integration acted as the general contractor. The design and installation of the hydraulic part of the solution was carried out by the company "Engineering Technologies". I didn’t doubt the possibilities of the first company because of the great experience of working with them. Good for the work "Engineering Technologies" was given for the outstanding expert skills of their leader, whose subordinates were surprised by the highest discipline and general work culture during the audit and diagnostics of the existing air conditioning system. Under the terms of the APC company, in order to preserve the guarantee, InRow had to be started and configured by a specialist, who was granted the right to press the system launch button. In total, the project involved 4 involved specialists, including installation, and 2 internal experts.
Expressing the similarity of the solution, I would not want to take away the bread from the designers and experts who performed the calculation of the hydraulic resistance of the system, pipe diameters and characteristics of other elements. That is why I will not give a calculation model here. However, this does not prevent me from voicing the initial data with which we operated when calculating this model. As already mentioned, the solution to the problem is to obtain the required circulation rate of the coolant at a given temperature. In addition to the previously mentioned characteristics, the pressures in the supply and discharge pipelines of the coolant, type, concentration, and glycol temperature in the system were taken into account. In countable form, these data are shown in the table.
Due to the need to supply the InRow modules with a coolant with a sufficiently low temperature and eliminate the risk of leakage and leakage of condensed moisture, it was decided to enter the media into the modules from below through the overlap. Also during the installation, a low-condensing polypropylene pipe and a special heat-insulating paint were used. This has significantly reduced the cost of piping.
After the launch, a series of tests of the system was carried out on the subject of sensors triggering and checking the logic of the emergency automation operation. An overview of the values of the results of these tests is given in the table.
I want to note that a number of the obtained values are beyond the scope of the InRow documentation, however, within these modes, the equipment behaves stably, without frosting, overheating, alarm display, etc.
InRow modules have quite adequate means of local and remote monitoring of the state and environments, and can not only send status messages, but also be covered by MSSCOM. The pumping cabinet and InRow cabinets receive power from the Galaxy 5000 server UPS. The electrical power consumption of the solution is 906 watts. At the time of this writing, the possibility of operating the server room from batteries was 97 minutes. By placing the InRow modules in this room, the formation of a rather isolated hot corridor with a volume of about 9 m3 has been achieved. In view of the fairly close distance between the modules, they were assembled into a logical group. Appearance of the equipment presented in the photo.
The mixing unit is located outside the server room, however, to reduce the hydraulic resistance, it is as close as possible to it. For the convenience of starting tests, electronic measuring instruments on it are duplicated by mechanical ones. When replicating a project, the scheme can be simplified.
The maintenance of the solution during the warranty period was carried out by the specialists of the plant, all the more so that in the first approximation the devices look quite simple. From maintenance operations, periodic cleaning (washing) of air and glycol filters is seen without stopping the solution. Operations that require switching to the freon system include global work on the glycol circuit of the plant. For example, drain parts of the system. During the three-year operation of the plant, a similar procedure was carried out only once - at the start of the blending department. That is why part of the Freon system is left as a safety node and is supported by the automation cabinet in the StandBy mode.
For an understanding of the scheme and logic of the organization of the existing hydraulic component of the solution, the scheme of the mixing unit is shown in the figure.
During the operation, a number of irrelevant drawbacks of the design solution were revealed, which were leveled by the next version, which expands the application of the system to the main UPS. The first and most important thing that has been done is the layout of the mixing unit has been completely revised. Not that I didn’t seem to like him and looked like a flimsy design ... it was bulky and reminded one of the “patented inventions” of the guys from our department. The role of the main pumps was revised and instead of supporting the three-way valve, they turned into pumping pumps. Thus, it was possible to solve all previously identified shortcomings and implement new plans.
Namely:
• ensure a uniform circulation rate of the glycol mixture.
• to ensure the independence of the system from the operation of industrial pumps of the main glycol circuit.
• raise the differential pressure at the inlet and outlet to the level required to connect additional cooling modules in other IT rooms.
The logic of the automatics operation was improved, in order to be able to select the manual control mode. For example, forcibly include backup freon air conditioners. Written work instructions and solutions for sample cases.
To place an uninterruptible power supply located at +0, i.e. at 3.5m below the server, a slightly different scheme was proposed, allowing to realize the logical redundancy of the room cooling system with a shortage of floor space. Namely, a fan coil was added to the air line of the freon heat exchanger, which is able to accept the glycolic mixture as a coolant. Understanding the idea of the contractor helped a lot with three-dimensional drawings, which he periodically sent for approval. Such a scheme is a scheme, and a visual understanding of what you get is worth a lot.
The appearance of the updated solution is presented in the photo. Mixing node is freed from unnecessary balancers and measuring devices has become much more compact and easier.
In conclusion, I would like to say that the assembled mixing unit is much simpler, cheaper and less than a water chiller. And the fact that we are fixing a steady decrease in the daily energy consumption of IT systems from 740 to 530 kW / h indicates a very high investment and operating efficiency of the solution.
Thanks:
• Vladimir Zakharov, Lead Engineer - Mechanics of the Baltika-Novosibirsk branch, for comprehensive technical advice.
• Director of the Baltika-Novosibirsk branch Yuriy Centyrev for professional opposition.
• Yashkan Vitaliy, Director of Engineering Technologies for providing a combination of innovations, smart heads and skillful hands.
• Alexander Samokhin, Head of SchneiderElectric Company's Siberian Federal Distribution Authority for working with regional customers, for customer-oriented consulting in preparing and launching the solution.
• Chief Project Engineer of the R-Style Siberia Integration Company, Alexander Bakhlykovous, that he was able to bring all these people together.
One of the key risks in the comprehensive assessment of the IT infrastructure of the Novosibirsk branch was the risk of stopping the server room due to the possible failure of the air conditioning system. This was facilitated by the fact that climate systems de jure are not within IT’s area of responsibility, and, accordingly, IT can only indirectly affect the continuity of their work.
Evaluating possible options for leveling such risks, it was noted that the climatic conditions of western Siberia make it possible to use outside air to remove heat. Moreover, circuits with the organization of cooling the coolant with outside air are already used for the technological needs of the Novosibirsk site.
To calculate the payback of possible solutions, the operational costs of operating and maintaining the existing classic freon data center system were calculated.
In general, everything was like ... And it was boring. Everything worked. There was some kind of rustling around the server room (maybe I would call it a data center), and all sorts of attempts to increase entropy and other activities, giving importance and value to this place, closed themselves on themselves.
Folk began to relax, more and more often play kicker (table football) in the warehouse and discuss the prospects of the Kazakh space program, supplying them with drawings of a three-stage horse.
')
At the same time, with the scheduled maintenance of the air conditioning system, the next brigade revealed a bug that could turn into a decisive jamb for us! Namely - a very large inrush current of the freon compressor could put uninterrupted power supply. Not daring to force the UPS more than 20kW bursts, I ordered to dismantle the unit and carry out a thorough diagnosis, thinking to present warranty claims. No matter how wrong! Everything indicated that the previous service was performed by people, to put it mildly, unprofessional. A badly pumped-out track resulted in the copper-plating of the compressor, which will unambiguously lead to the seizure of the bearings. The previous office from the energy service, which performed the service, disowned our claim by our own acts. A strange way to do long-term business. Successes them.
So, as soon as I climbed, a lot of questions fell on my head - how to live with such an approach? The situation with the near-contractors contractors did not suit me, and I began to pull the blanket over myself, i.e. legitimize the ownership of air conditioning in the server. Usually, in such cases, no one resists.
After counting budgets and understanding the necessary amount of money for quarterly maintenance of three systems (one cools the UPS in a separate room), I felt a little uneasy. Of course, it is possible to replace the “quarterly” with “once in half a year”, but what was removed and washed out of the external blocks, directly said, “once in half a year” I will once recall.
Representatives of the APC, who visited them, urged that colleagues in Slavutych (Kiev) based on InRow 100, built a very interesting concept to actively study the issue. His key feature was water cooling not with a chiller, but with a heat exchanger with technological glycol. It has become enviable, all the more so since cold glycol is like a fool of a shag!
The integrator said that the designers are ready to work on parole, but they needed a constructive team with the bosses, and I did it! When reporting the financial benefits of abandoning the secondary water circuit, it was allowed to initiate the design. A preliminary reading on the night of the InRow datasheet revealed his ability to work on ethylene / propylene glycol with concentrations up to 50%! What made it possible to make bold and trump claims that “there will be no water!” In the director’s office.
And then ... Having ignited the importance, I informed the designers of the ideological wisdom, while listening to the proposed innovations. The result was the rejection of metal pipelines in order to reduce condensate, the use of heat insulating paint on nodes with complex geometry, and most importantly, the rejection of the heat exchanger and the virtualization of the second circuit using a three-way valve! He also insisted on bringing in and discharging all media from below the blocks, through the overlap. Still, the liquid ...
The knot of mixing turned out impressive, but to the touch flimsy. And since it was necessary to locate it in the warehouse behind the server, the risk of demolishing it with an overly quick loader was evident. Had to order fencing. InRow heavy blocks !!! Cheerfully drag them to the second floor of the stairs did not work. I bow to the strong guys from the warehouse of finished products. Those barely managed the six of them. Alas, there were traces on the linoleum in the hall and the anger of our curling team.
During the installation of InRow, an extra server cabinet was formed on the seats. Seriously, it was planned for transfer to a backup server. But this required a serious stop of the infrastructure, so all the action took place at night, and the darkest over the past 465 years. Coincided so. And further. When you realize that all the cabinets easily roll on their wheels, a person who has been rolling wardrobe on felt since childhood becomes a little awkward. But now in the department there are two pairs of new boots!
The launch was postponed three times. Officially, due to the fact that it was difficult to gather all three participants in the process in one place and time. But, IMHO, everyone stuffed his own price, revenge and sneered at me. However, to the credit of all participants, the launch was made from the first time with absolutely no comments. It would not be enough for someone to spoil my birthday.
Testing began the very next day, and with vicious addiction. The system was displayed on supercritical modes, readings and alarms of emergency sensors were recorded. Standing in jets of cold wind in the outer sheepskin coat, I took a parade of technologies. The cap moved down to the back of the head, but did not fall - a halo interfered with it.
Who are interested in boring technical details, can read further.
The next stage was the internal coordination of the idea, which took more than 4 months, after which a certain extract from the best world practices and experience of colleagues in the Slavutych company was obtained. The main task that had to be solved was to ensure the circulation of the coolant at a rate of at least 0.2 l / s and an inRow inlet temperature not lower than +5.6 ⁰. Those. in fact, there was a difference between the existing temperature and the minimum allowable temperature for InRow, equal to 10 ° C, which does not allow connecting them directly to the technological circuit. Experts of various services and levels, including the plant director, took part in the evaluation of the proposed options. As a result, the reference scheme was adopted, excluding the presence of water and intermediate heat exchangers, designed to level the above-mentioned temperature difference. A concept in the form of a design specification was issued to the integrator in October 2010, and a month later I had a ready-made project on my desk, the possibility of implementation of which was confirmed by the manufacturer of the equipment and the service of the chief engineer of the plant.
R-Style Siberia Integration acted as the general contractor. The design and installation of the hydraulic part of the solution was carried out by the company "Engineering Technologies". I didn’t doubt the possibilities of the first company because of the great experience of working with them. Good for the work "Engineering Technologies" was given for the outstanding expert skills of their leader, whose subordinates were surprised by the highest discipline and general work culture during the audit and diagnostics of the existing air conditioning system. Under the terms of the APC company, in order to preserve the guarantee, InRow had to be started and configured by a specialist, who was granted the right to press the system launch button. In total, the project involved 4 involved specialists, including installation, and 2 internal experts.
Expressing the similarity of the solution, I would not want to take away the bread from the designers and experts who performed the calculation of the hydraulic resistance of the system, pipe diameters and characteristics of other elements. That is why I will not give a calculation model here. However, this does not prevent me from voicing the initial data with which we operated when calculating this model. As already mentioned, the solution to the problem is to obtain the required circulation rate of the coolant at a given temperature. In addition to the previously mentioned characteristics, the pressures in the supply and discharge pipelines of the coolant, type, concentration, and glycol temperature in the system were taken into account. In countable form, these data are shown in the table.
Due to the need to supply the InRow modules with a coolant with a sufficiently low temperature and eliminate the risk of leakage and leakage of condensed moisture, it was decided to enter the media into the modules from below through the overlap. Also during the installation, a low-condensing polypropylene pipe and a special heat-insulating paint were used. This has significantly reduced the cost of piping.
After the launch, a series of tests of the system was carried out on the subject of sensors triggering and checking the logic of the emergency automation operation. An overview of the values of the results of these tests is given in the table.
I want to note that a number of the obtained values are beyond the scope of the InRow documentation, however, within these modes, the equipment behaves stably, without frosting, overheating, alarm display, etc.
InRow modules have quite adequate means of local and remote monitoring of the state and environments, and can not only send status messages, but also be covered by MSSCOM. The pumping cabinet and InRow cabinets receive power from the Galaxy 5000 server UPS. The electrical power consumption of the solution is 906 watts. At the time of this writing, the possibility of operating the server room from batteries was 97 minutes. By placing the InRow modules in this room, the formation of a rather isolated hot corridor with a volume of about 9 m3 has been achieved. In view of the fairly close distance between the modules, they were assembled into a logical group. Appearance of the equipment presented in the photo.
The mixing unit is located outside the server room, however, to reduce the hydraulic resistance, it is as close as possible to it. For the convenience of starting tests, electronic measuring instruments on it are duplicated by mechanical ones. When replicating a project, the scheme can be simplified.
The maintenance of the solution during the warranty period was carried out by the specialists of the plant, all the more so that in the first approximation the devices look quite simple. From maintenance operations, periodic cleaning (washing) of air and glycol filters is seen without stopping the solution. Operations that require switching to the freon system include global work on the glycol circuit of the plant. For example, drain parts of the system. During the three-year operation of the plant, a similar procedure was carried out only once - at the start of the blending department. That is why part of the Freon system is left as a safety node and is supported by the automation cabinet in the StandBy mode.
For an understanding of the scheme and logic of the organization of the existing hydraulic component of the solution, the scheme of the mixing unit is shown in the figure.
During the operation, a number of irrelevant drawbacks of the design solution were revealed, which were leveled by the next version, which expands the application of the system to the main UPS. The first and most important thing that has been done is the layout of the mixing unit has been completely revised. Not that I didn’t seem to like him and looked like a flimsy design ... it was bulky and reminded one of the “patented inventions” of the guys from our department. The role of the main pumps was revised and instead of supporting the three-way valve, they turned into pumping pumps. Thus, it was possible to solve all previously identified shortcomings and implement new plans.
Namely:
• ensure a uniform circulation rate of the glycol mixture.
• to ensure the independence of the system from the operation of industrial pumps of the main glycol circuit.
• raise the differential pressure at the inlet and outlet to the level required to connect additional cooling modules in other IT rooms.
The logic of the automatics operation was improved, in order to be able to select the manual control mode. For example, forcibly include backup freon air conditioners. Written work instructions and solutions for sample cases.
To place an uninterruptible power supply located at +0, i.e. at 3.5m below the server, a slightly different scheme was proposed, allowing to realize the logical redundancy of the room cooling system with a shortage of floor space. Namely, a fan coil was added to the air line of the freon heat exchanger, which is able to accept the glycolic mixture as a coolant. Understanding the idea of the contractor helped a lot with three-dimensional drawings, which he periodically sent for approval. Such a scheme is a scheme, and a visual understanding of what you get is worth a lot.
The appearance of the updated solution is presented in the photo. Mixing node is freed from unnecessary balancers and measuring devices has become much more compact and easier.
In conclusion, I would like to say that the assembled mixing unit is much simpler, cheaper and less than a water chiller. And the fact that we are fixing a steady decrease in the daily energy consumption of IT systems from 740 to 530 kW / h indicates a very high investment and operating efficiency of the solution.
Thanks:
• Vladimir Zakharov, Lead Engineer - Mechanics of the Baltika-Novosibirsk branch, for comprehensive technical advice.
• Director of the Baltika-Novosibirsk branch Yuriy Centyrev for professional opposition.
• Yashkan Vitaliy, Director of Engineering Technologies for providing a combination of innovations, smart heads and skillful hands.
• Alexander Samokhin, Head of SchneiderElectric Company's Siberian Federal Distribution Authority for working with regional customers, for customer-oriented consulting in preparing and launching the solution.
• Chief Project Engineer of the R-Style Siberia Integration Company, Alexander Bakhlykovous, that he was able to bring all these people together.
Source: https://habr.com/ru/post/128778/
All Articles