Immersion cooling - immersion cooling. In the last article on immersion cooling , the advantages of placing servers in mineral oil were essentially considered. I was deeply mistaken when it seemed to me that this is the most effective solution in global terms. After reviewing some other developments, I realized that the basic principle itself is effective - immersion cooling, but everything else, as they say, “depends on” ...

Today I want to tell you about the relatively new two-phase cooling system Immersion-2, which uses all the advantages of 3M Novec “dry water” to achieve truly magical results, turning disadvantages into advantages!

What is 3M Novec will not be repeated, this has already been written in RuNet many times, and even on Habré there is an excellent article , and how it can be used to cool servers in practice and why “dry water” turns out to be more efficient than mineral oil, liquid cooling systems electronics in closed circuits, consider now.
')


The cooling system of electronics in liquids in closed circuits has existed and has been used for several decades, for example, in the military and aviation fields. It is problematic and expensive in production and operation, since pressure can build up over time in closed systems. Among other things, these systems are directed to smaller pieces of hardware with a lower power density, which leads to duplication of components in use.

For example, 20 motherboards will require 20 closed cases, 20 heat exchangers, 40 connectors for fluid circulation (20 incoming and 20 outgoing), 20 power connectors, 20 control devices, etc. All these components will be very difficult to maintain. In the open approach, thanks to 3M Novec, use one immersion cooling half-open bath with 2-phase cooling, where all the equipment can be placed in one body as a whole.



Due to the fact that 3M Novec begins to boil at atmospheric pressure even at 34 ° C, 49 ° C or 56 ° C (depending on the type used, the boiling point differs) its phase transition from liquid to gaseous occurs. This property can be used to cool equipment without the use of hydraulic pumps, as in the case of cooling with mineral oil, as there is no need to pump the liquid inside the tank, 3M Novec boils and mixes, removes excess heat due to physical processes of evaporation and condensation, two phase transitions what the design of the system is greatly simplified.

For a better understanding, I will give the stages of 2-phase immersion cooling:







Like immersion cooling, “sealed servers” use a single-phase cooling process. This is much less efficient compared to a 2-phase system (heat transfer coefficient is much higher) and requires more components. High density, for example, in passive 2-phase systems, is simply not possible, because the mixing process goes well without the use of pumps. Technically sealed servers use dielectric fluid, such as mineral oil or fluid that is similar to the fluid used in passive 2-phase immersion cooling. But the process of heat removal does not occur due to evaporation and condensation, the main process is mixing with the help of pumps, which in any case only increases energy consumption and reduces the cooling efficiency.



In general, it is worth noting that the idea of ​​Immersion-2 is in fact not new. Back in the 1970s, relevant studies were carried out using C6F14 fluid, which boiled on the surface of the silicon chip. However, the chip itself was far from perfect and the fluid had a negative effect on it. In short, computer and chip technology has matured only in the last few decades. What we could not do 20 years ago is now possible thanks to a modern chip package that includes heat distributors with high-performance interfaces, ssd disks that lack rotating components, and sealed HDD disks that can be used in a liquid medium. Another factor that influenced the fact that the technology was not previously studied was the lack of high-density computing systems at the time and only in the last few years this problem has become topical.

Allied Control took seriously the development of this idea, put it into practice, and this is what came of it:



As a test project in Hong Kong, in a small office building, a mini data center was built, which became the most energy-efficient data center not only in Asia, but perhaps in the whole world, as the PUE energy efficiency ratio of 1.01 was achieved!



According to Allied Control, the maximum power per rack when using their cooling system can be up to 225 kW. For testing purposes, a cooling system with a capacity of up to 500 kW was installed for 24 racks. In this office mini data center is quieter than in the library!



The funny thing is that the world's first energy-efficient data center in history was used not for scientific work, but for mining BitCoin cryptocurrency, the realities of the modern world, but it is necessary to pay back the innovations! For the growth of the mining farm even reserved additional space:



And this is what the “blades” look like, which can be up to 92 in each unit:



The liquid cooling system itself is reserved:



Of course, as well as communication channels and power supply.

Benefits

- 4000 times more efficient than air cooling (electronic components are immersed in a dielectric fluid, which has a much higher thermal conductivity and heat capacity than air);
- energy efficiency 1.01 (the possibility of removal from 18 to 225 kW of heat from the rack);



- “cool hundreds of kilowatts” using only 1,500 watts (savings on electricity for the current site of 24 cabinets filled with equipment amount to more than $ 60,000 / month);
- low boiling point promotes mixing and passive cooling due to evaporation, eliminates the need for pumps (fluid with a boiling point of 34 ° C (93 ° F) called 3M Novec 7000 is used);
- higher density (space occupied by the equipment is reduced many times, thereby allowing you to save on all other engineering systems, data center);



- improved performance and reliability (there are no fluctuations in temperature and hot spots, there are no unnecessary parts, since the cooling system is passive, it is possible to get rid of duplication of hardware, the transfer of heat occurs only once);
- incredible cleanliness (two-phase immersion cooling system has beneficial side effects - massive air flow, dust and noise, are excluded from the object, something immersed in the fluid not only does not need to be cleaned and dried when removed, but even becomes even cleaner due to evaporation fluid from the surface);



- extremely high degree of environmental friendliness and fire safety (does not contain substances that destroy ozone, moreover, has the property of displacing oxygen, thereby preventing ignition processes, which is actively used in gas fire extinguishing systems based on 3M Novec);
- long lifespan (over 30 years) and various boiling points of the cooling substance (34 ° C, 49 °, 56 ° C, etc., are mainly used by Novec 7000 and Novec 649 with temperatures of 34 and 49 degrees, respectively);



- safety (during operation due to low boiling points it is impossible to get a burn, and due to the absence of toxicity - poisoning);
- not patented (the possibility of universal design for a specific project);
- ready-made modular solution for the customer with a quick commissioning period (up to 6 months).

On the last paragraph we will dwell in more detail The serial solution received the name DataTank and provides the ability to divert up to 1.4 MW of heat from equipment located in one container:



DataTank modules are third-generation systems and are able to work effectively even in the hot and humid climates of Asia, while providing PUE 1.01 or even less. Multiple containers can be combined to deploy a large-scale container farm.



In general, we can conclude that the future is for immersion cooling, although of course we would like such innovations to be primarily used not only for mining BitCoin, but also for scientific and industrial purposes!