⬆️ ⬇️

Electricity in the data center





The work of any data center rests on the "three pillars" - power supply, cooling and security systems. And the key infrastructure among them, no doubt, is the power supply system.



Smooth base



The Selectel “Flower 2” data center is designed for a power consumption of 2500 kVA and corresponds to the Tier III level, according to which the power supply and cooling channels are duplicated according to the 2N redundancy scheme. This gives a significant advantage in terms of the reliability of power supply both to customers who place their equipment and to the maintenance service of the data center itself - maintenance work or replacement of failed components can be performed without turning off the power supply to the data center (including customer equipment).





')

Electricity enters the data center through two independent inputs from the city power grid and is supplied to two transformer substations of the TP 10 (6) / 0.4. The substations are owned by Selectel and are located in a complex of buildings also owned by the company. This is another important moment for customers regarding the reliability of the infrastructure - in Russia, a fairly large number of data centers are located on rented premises, and therefore depend on the rental market conditions and even on the mood of the owners.







In transformer substations, a high “city” voltage of 6 kV is reduced to 380 V (400 V), and then supplied to the main switchboards (MSB) with a protective automation system.







This power supply can have various kinds of interference (bursts and dips, high-frequency noise, etc.), and therefore is not suitable for interference-sensitive servers and communication equipment, as well as air-conditioning systems and automation of the data center itself.







Therefore, along the bus lines, the power supply of 380 V (400 V) is then supplied to 2 groups of industrial UPSs (uninterruptible power supply units), each of which is a 6x400 kVA module with a double conversion function. Here, the alternating voltage is rectified into a direct current voltage, and then again converted into alternating 380 V (400 V), but with an ideal sinusoid shape.







In addition to the function of cleaning from interference, UPS perform another important task - provide power to the data center from the battery (rechargeable batteries) in case of complete disconnection of the external "urban" power supply during the transition to the emergency power source - DGU (diesel generator sets). For this purpose, the UPSs are connected to mutually redundant matrices of 130 rechargeable 12 V batteries. The charge of these batteries is enough for the period of data center operation under full load up to 10 minutes. When the external power supply fails, the automatic equipment of the ATS (automatic input reserve) launches three DGUs with a capacity of 1,800 kVA each in a separate room, which can provide the data center with electricity for an unlimited time.







DGUs have N + 1 redundancy, that is, two DGUs are enough for normal operation of the Tsvetochnaya 2 data center, plus one DGU - in reserve. On the territory of the data center, there is a reserve of fuel for 48 hours of DGU operation, and there is also a service agreement with two suppliers for emergency supply of fuel in case of prolonged urban blackout.







On average, it takes up to 3 minutes to start the DGU and their release to the operating load mode - the UPS battery charge is enough for this operation. In order to ensure steady start of diesel engines even in winter time, there is a constant heating of the coolant in a diesel generator set, and this parameter is controlled both during regular rounds and remotely through the monitoring system.







The inputs of the external "urban" power supply and the DGI are connected in parallel to each other via the ATS and their outputs supply the input-switchgear (2x ASG). From the output of two ASUs, power is supplied respectively to 2 groups of UPSs, then from each group of UPSs, via separate busbars, is supplied to shields of uninterrupted power supply (SHGP), and then directly to the machine halls to distribution cabinets of power racks. In these cabinets, 380 V (400 V) is divided into 230 V buses for powering computer and network equipment. UPSs provide European 400/230 V power standard.



4 separate busbars from two independent sources come to each machine hall, and they are laid in different ways in order to avoid simultaneous mechanical damage. This is how 2N power redundancy occurs.







Accurate consumption metering



Each cabinet is equipped with an intelligent device that records electricity consumption in each rack and records readings to a log file. Although for customers who host their servers or racks in the computer room, power consumption does not affect the rental price, this information is extremely important for data center personnel. Unusually high, or vice versa, low consumption indicates a possible malfunction and requires the attention of staff.











From distribution cabinets, power is supplied to the racks also via 2 separate cables (as ordered by customers, including with the AVR device in the rack). Servers with 2 power supply units (PSUs) are connected to sockets from different inlets in the rack, and equipment with one power supply unit is connected to AVR sockets (optional).







Maintenance and monitoring - a guarantee of reliability of power supply



Whatever degree of reliability is initially incorporated into the power supply system of the data center, the equipment requires constant monitoring and regular maintenance. All information on the state of power supply systems, including DGS, is collected on large screens in the Monitoring Center. In addition, in case of deviations from the norm, messages come to the plates of engineers on duty.







According to the regulations, on-duty engineers and electricians make constant rounds of the engine rooms and, accordingly, of the rooms where the power electrical equipment is located. Each employee has a tablet with an inspection plan. On the tablet, in addition to the bypass plan, there is also a description of the inspection task at each bypass point.







After inspecting the premises or site with the equipment, the employee makes a mark on the tablet, and at the end of the tour, a tour protocol is automatically generated, which is stored in the archive of the data center. Outwardly, this is similar to how pilots necessarily read out the instructions (check list) for checking aircraft systems before takeoff, even if they know this instruction by heart.



In addition to monitoring, electrical equipment with a certain frequency of routine maintenance. Initially, it is inspected in working condition with a thermal imager for heated contacts, then it is turned off and all contacts are pulled. At the same time, the cabinets and other units are cleaned from dust. By the way, the entrance to the premises with the equipment is allowed only in shoe covers, and in some of them - also in dust-proof overalls. Here is a complex electrical economy provides power to the servers and network equipment of Selectel customers.

Source: https://habr.com/ru/post/343866/



All Articles