Cooling server. Divide and rule
Good day to all.
It so happened that the articles begin to form a cycle. The first part of the cycle is here habrahabr.ru/post/235593
In this part, I will try to simplifyly explain why you need to manage the air flow in the server, how it is implemented, the pros and cons of different approaches, as well as the economic justification for large strokes.
Our main goals are:
1. Supply cold air to the equipment air intakes.
2. Take away the hot air from the exhaust and supply it to the air intake of the air conditioner.
3. Providing the necessary volume and pressure of air.
4. Separation of hot and cold air streams.
What is it for?
')
As already mentioned in the previous article, the main problem with the cold in the server is the mixing of air. Stirring creates 2 problems:
1. The air temperature at the server air intake is higher than required.
2. The air temperature at the air intake of the air conditioner is lower than planned by the designers. This reduces the efficiency of the air conditioner, creates problems with regulation, in some systems leads to errors that can simply turn off the air conditioner.
How are these problems solved?
Splitting flows. There are 3 main ways to separate threads:
1. Rack Air Containment Insulation (RACS).
2. Cold Air Containment System (CACS) insulation.
3. Hot aisle isolation (Hot Air Containment System - HACS).
Insulation at rack level is as follows
The picture shows a row air conditioner and a rack, retrofitted with RACS elements.
RACS is used in 3 cases:
1. When it is necessary to increase the cooling capacity of the in-line conditioner that cools this rack directly. Row precision air conditioners are designed to work with high return air temperatures. The higher the temperature, the greater the cooling capacity of the air conditioner. Again, the less cold air disperses around the room, the more it gets to the rack air intake.
2. When you need to cool the rack in a large room, but the room itself is not efficiently cooled.
3. When in the room there are, for example, standard racks with a heat release of 5-8 kW and a pair of racks with a heat dissipation of 20-70 kW. In this case, you build one system for all racks with the use of expensive specialized solutions. Or build a simple system based on perimeter air conditioners for low consumption racks, and build a separate container for high load racks.
Isolation of the cold corridor looks like this:
For CACS, it is required to arrange the cabinets in one row opposite the wall or in 2 rows opposite each other. The cold air supply area is overlapped by the roof and doors / curtains, creating an insulated volume.
CACS is a fairly traditional version. It has been used for a long time, by virtue of the established traditions of design, when the racks are placed facing each other. Therefore, it is the most suitable option for upgraded data centers.
Insulation hot corridor looks like this:
In the same way as for CACS, HACS requires row placement of cabinets, with the only difference that the cabinets should be located "back" to each other. This isolates the hot air zone.
In either case, it is possible to use both in-line and perimeter air conditioners. In the case of CACS, a raised floor can be used to supply air; hot air is drawn from the volume of the room. In the case of HACS, the flow is made to the volume of the room. In perimeter cooling, a plenum of a false ceiling or air ducts is used for air intake. At the same time, the roof of the corridor changes to an air duct (there is another term chimney - literally “chimney”), as in the picture above.
As an option for using HACS, so-called “exhaust doors” can be used. This is a unit with fans, which is hung instead of the back door of the cabinet and provides air intake and supply it to the plenum of a false ceiling. A specific thing can be used when it is necessary to isolate several racks in the room, or when upgrading old data centers, where the cabinets were placed chaotically, and there is no possibility to move them to form a corridor.
A very important point when using insulation systems. No need to think that if you closed the roof and the ends of the corridor, your work is done perfectly. There are a bunch of holes that need to be blocked:
1. Empty units in the cabinets. We use special plugs, cardboard, polycarbonate, boards.
2. The gap between the cabinets. We buy special plugs from the cabinet manufacturer or take polyurethane self-adhesive tape.
3. The gap between the raised floor tiles, the gap in the junction of the raised floor to the walls and structural elements. We use seals on the stringers of the floor (you have in fact the floor in the data center is assembled with stringers). In places of junction to the walls - again, the tape from p.2.
4. Cable entries through which air can flow from one zone to another. Here you need to use brush entries.
Features of the use of a particular system.
RACS. Everything is cool here except for problems with fire fighting. You must either create a single volume with the room by implementing automatic door opening, or start a fire extinguishing system in the RACS.
CACS and HACS. The main difference in what kind of air in our room is cold or hot. When using CACS, we will not be able to place unstable equipment in the room, since the whole room will be a hot air zone. The temperature there can reach 35 degrees and above. There are no such problems with HACS. Again, according to sanitary standards, it is possible to work at high temperatures for a limited amount of time. This eliminates the need for lengthy indoor work when using CACS. And the third - the supply of cold air in case of a crash of air conditioners is different, at HACS it is more. A very important point when using plenums for air supply is turbulence. If you use the raised floor, you must provide a sufficient section of it. We can only operate with height, since the area we have is determined by the room. Reducing the height leads to an increase in the speed of air flow (the air volume is constant here), high speeds in conditions of a large number of obstacles (in the form of raised floor racks, trays, etc.) leads to the formation of turbulence (turbulence), turbulence leads to a decrease in pressure hence the volume of air that comes out directly from the perforated floor tiles is reduced. We must always strive for a laminar flow. To calculate the height, you can use CFD software (correctly and with the highest confidence), or make an approximate calculation. Powerful article on this topic www.ccc.ru/magazine/depot/07_12/read.html?Wcd7c2146c471e.htm .
Economy.
Some good guys conducted a mathematical simulation. For the calculation was taken the same data center. On row cooling, CAPEX and OPEX were calculated with and without HACS. On perimeter cooling, CAPEX and OPEX were considered comparing HACS and CACS. What did the study show?
1. In-line cooling solution with isolated corridors on average 5-7% cheaper in CAPEX. By increasing the capture index (the ability of the air conditioner to “capture” hot air), the number of air conditioners used has decreased. Operating costs are also reduced by reducing the number of air conditioners (reducing power consumption) and by increasing efficiency, because we were able to raise the temperature in the hot aisle.
2. In perimeter cooling all the more confusing. The lowest CAPEX solution is without isolation. But these solutions can only be applied up to a capacity of 5-7 kW per rack. Then only isolation. Up to a power of 12–15 kW, a CACS rack is 2–6% cheaper than a HACS, since HACS requires a hot air exhaust system (ducts, plenum of a false ceiling). However, after this threshold, CACS requires the use of air closers from under the raised floor, which significantly increases the cost of the solution (by 7-10%). Operational costs for isolated solutions are 12% lower than for non-insulated ones. At the same time, up to a threshold of 12–15 kW per rack, the costs of CACS and HACS are the same and begin to diverge after this threshold. CACS becomes an average of 4% more expensive, because the consumption of air closers is added to the balance.
How to choose what suits you?
If the ceilings are low (hence it is not possible to install a raised floor), build an in-line solution with HACS at any densities.
If you have a low density (up to 6 kW per rack) and high ceilings, build a non-insulated perimeter solution with air distribution under the raised floor.
If you have high densities (15 kW and above) build HACS, in-line or perimeter - depends on the geometry of the room.
If the density is average (6-12 kW), build a CACS.
Border cases require calculations and modeling, it all depends on the redundancy, the number of racks and the geometry of the room. The scale can lean in any direction.
That's all, if you have questions - I will answer in the comments.
It so happened that the articles begin to form a cycle. The first part of the cycle is here habrahabr.ru/post/235593
In this part, I will try to simplifyly explain why you need to manage the air flow in the server, how it is implemented, the pros and cons of different approaches, as well as the economic justification for large strokes.
Our main goals are:
1. Supply cold air to the equipment air intakes.
2. Take away the hot air from the exhaust and supply it to the air intake of the air conditioner.
3. Providing the necessary volume and pressure of air.
4. Separation of hot and cold air streams.
What is it for?
')
As already mentioned in the previous article, the main problem with the cold in the server is the mixing of air. Stirring creates 2 problems:
1. The air temperature at the server air intake is higher than required.
2. The air temperature at the air intake of the air conditioner is lower than planned by the designers. This reduces the efficiency of the air conditioner, creates problems with regulation, in some systems leads to errors that can simply turn off the air conditioner.
How are these problems solved?
Splitting flows. There are 3 main ways to separate threads:
1. Rack Air Containment Insulation (RACS).
2. Cold Air Containment System (CACS) insulation.
3. Hot aisle isolation (Hot Air Containment System - HACS).
Insulation at rack level is as follows
The picture shows a row air conditioner and a rack, retrofitted with RACS elements.
RACS is used in 3 cases:
1. When it is necessary to increase the cooling capacity of the in-line conditioner that cools this rack directly. Row precision air conditioners are designed to work with high return air temperatures. The higher the temperature, the greater the cooling capacity of the air conditioner. Again, the less cold air disperses around the room, the more it gets to the rack air intake.
2. When you need to cool the rack in a large room, but the room itself is not efficiently cooled.
3. When in the room there are, for example, standard racks with a heat release of 5-8 kW and a pair of racks with a heat dissipation of 20-70 kW. In this case, you build one system for all racks with the use of expensive specialized solutions. Or build a simple system based on perimeter air conditioners for low consumption racks, and build a separate container for high load racks.
Isolation of the cold corridor looks like this:
For CACS, it is required to arrange the cabinets in one row opposite the wall or in 2 rows opposite each other. The cold air supply area is overlapped by the roof and doors / curtains, creating an insulated volume.
CACS is a fairly traditional version. It has been used for a long time, by virtue of the established traditions of design, when the racks are placed facing each other. Therefore, it is the most suitable option for upgraded data centers.
Insulation hot corridor looks like this:
In the same way as for CACS, HACS requires row placement of cabinets, with the only difference that the cabinets should be located "back" to each other. This isolates the hot air zone.
In either case, it is possible to use both in-line and perimeter air conditioners. In the case of CACS, a raised floor can be used to supply air; hot air is drawn from the volume of the room. In the case of HACS, the flow is made to the volume of the room. In perimeter cooling, a plenum of a false ceiling or air ducts is used for air intake. At the same time, the roof of the corridor changes to an air duct (there is another term chimney - literally “chimney”), as in the picture above.
As an option for using HACS, so-called “exhaust doors” can be used. This is a unit with fans, which is hung instead of the back door of the cabinet and provides air intake and supply it to the plenum of a false ceiling. A specific thing can be used when it is necessary to isolate several racks in the room, or when upgrading old data centers, where the cabinets were placed chaotically, and there is no possibility to move them to form a corridor.
A very important point when using insulation systems. No need to think that if you closed the roof and the ends of the corridor, your work is done perfectly. There are a bunch of holes that need to be blocked:
1. Empty units in the cabinets. We use special plugs, cardboard, polycarbonate, boards.
2. The gap between the cabinets. We buy special plugs from the cabinet manufacturer or take polyurethane self-adhesive tape.
3. The gap between the raised floor tiles, the gap in the junction of the raised floor to the walls and structural elements. We use seals on the stringers of the floor (you have in fact the floor in the data center is assembled with stringers). In places of junction to the walls - again, the tape from p.2.
4. Cable entries through which air can flow from one zone to another. Here you need to use brush entries.
Features of the use of a particular system.
RACS. Everything is cool here except for problems with fire fighting. You must either create a single volume with the room by implementing automatic door opening, or start a fire extinguishing system in the RACS.
CACS and HACS. The main difference in what kind of air in our room is cold or hot. When using CACS, we will not be able to place unstable equipment in the room, since the whole room will be a hot air zone. The temperature there can reach 35 degrees and above. There are no such problems with HACS. Again, according to sanitary standards, it is possible to work at high temperatures for a limited amount of time. This eliminates the need for lengthy indoor work when using CACS. And the third - the supply of cold air in case of a crash of air conditioners is different, at HACS it is more. A very important point when using plenums for air supply is turbulence. If you use the raised floor, you must provide a sufficient section of it. We can only operate with height, since the area we have is determined by the room. Reducing the height leads to an increase in the speed of air flow (the air volume is constant here), high speeds in conditions of a large number of obstacles (in the form of raised floor racks, trays, etc.) leads to the formation of turbulence (turbulence), turbulence leads to a decrease in pressure hence the volume of air that comes out directly from the perforated floor tiles is reduced. We must always strive for a laminar flow. To calculate the height, you can use CFD software (correctly and with the highest confidence), or make an approximate calculation. Powerful article on this topic www.ccc.ru/magazine/depot/07_12/read.html?Wcd7c2146c471e.htm .
Economy.
Some good guys conducted a mathematical simulation. For the calculation was taken the same data center. On row cooling, CAPEX and OPEX were calculated with and without HACS. On perimeter cooling, CAPEX and OPEX were considered comparing HACS and CACS. What did the study show?
1. In-line cooling solution with isolated corridors on average 5-7% cheaper in CAPEX. By increasing the capture index (the ability of the air conditioner to “capture” hot air), the number of air conditioners used has decreased. Operating costs are also reduced by reducing the number of air conditioners (reducing power consumption) and by increasing efficiency, because we were able to raise the temperature in the hot aisle.
2. In perimeter cooling all the more confusing. The lowest CAPEX solution is without isolation. But these solutions can only be applied up to a capacity of 5-7 kW per rack. Then only isolation. Up to a power of 12–15 kW, a CACS rack is 2–6% cheaper than a HACS, since HACS requires a hot air exhaust system (ducts, plenum of a false ceiling). However, after this threshold, CACS requires the use of air closers from under the raised floor, which significantly increases the cost of the solution (by 7-10%). Operational costs for isolated solutions are 12% lower than for non-insulated ones. At the same time, up to a threshold of 12–15 kW per rack, the costs of CACS and HACS are the same and begin to diverge after this threshold. CACS becomes an average of 4% more expensive, because the consumption of air closers is added to the balance.
How to choose what suits you?
If the ceilings are low (hence it is not possible to install a raised floor), build an in-line solution with HACS at any densities.
If you have a low density (up to 6 kW per rack) and high ceilings, build a non-insulated perimeter solution with air distribution under the raised floor.
If you have high densities (15 kW and above) build HACS, in-line or perimeter - depends on the geometry of the room.
If the density is average (6-12 kW), build a CACS.
Border cases require calculations and modeling, it all depends on the redundancy, the number of racks and the geometry of the room. The scale can lean in any direction.
That's all, if you have questions - I will answer in the comments.
Source: https://habr.com/ru/post/243725/
All Articles