The path of electricity in the data center
The power supply system is one of the most critical components of the engineering infrastructure of the data center. Without electricity, a data center in a split second can turn into a useless building with a pile of idle IT equipment. To prevent this from happening, the data center has on-board diesel generators and uninterruptible power supplies.
Today we will talk in detail about the power supply system of the data center: find out which way electricity travels to the rack and what happens when urban electricity is lost in the data center. A subject we will sort on an example of one of our data-centers.
Before we go through the electricity routes in the data center, let us examine the basic terms and definitions that we need today.
')
The data center is connected to the city power network through two inputs. These inputs are independent: they come from different substations and along separate routes. We follow the path of electricity through one of these inputs from the transformer to the server in the rack.
Through the inputs of urban electricity falls on the step-down transformers data center. Here 10 kV is converted to 400 V.
These 400 V come to the main switchboard (MSB). From here, power diverges across all engineering systems and IT equipment of the data center.
Two rows of gray cabinets - this is the main switchboard. Each row (section) is powered by its input and duplicates the one that stands opposite.
MSB includes:
In the room with the main switchboard, there is also a shield in which the outlets from all the DGS (usually from 3 to 5 pieces) are collected. When the city power supply disappears, the power from this shield is fed to the MSB and diverges further throughout the data center.
All the electricity generated by the diesel generator set goes to the copper distribution bus and from there goes to the induction automat from the diesel generator set.
So, getting into the main switchboard, 400V come to the introductory machine from the city. Next, electricity falls on the distribution bus and tap-off machines. AVR in case of interruptions to the urban power grid, switches the data center to power from the diesel generator set. From MSB power supply is already guaranteed, but still:
a) “dirty”: city stress is not too stable, there are jumps, temporary rises and falls.
b) not uninterrupted: the DGU will take up to 30 seconds to start and go to workload.
Both of these problems are solved by an uninterruptible power supply (UPS). It is there that electricity comes after MSB. The UPS levels the voltage to the set value, and when power is lost from the city, the UPS automatically switches its load to the batteries. On batteries, the data center can operate autonomously for about 15 minutes. This is more than enough for the DGU to start.
UPS ordinary.
Racks with batteries.
From the UPS, the UPS distribution panel (SCHIBP) is powered. From diversion automat SCHIBP, electricity flows to the switchboards of the machine rooms (SCHR), air conditioners (SCHK), emergency lighting (SCHS) and other panels.
In ShchR, for each rack, a separate automatic switch of the desired nominal value and phase is set. For example, to power a 6.5 kW rack, we install a single-phase automatic 32A.
SchR in the engine room.
In the NORD 4 machine halls, instead of the SCH, busbars are used under the raised floor.
On the busbar are installed tap-off boxes with automatic machines leading to the racks.
It remains to power the equipment in the rack. To do this, two PDUs are installed in the rack, which are connected to different power beams. Equipment with two power supplies is connected to different PDUs. If the equipment has one power supply, it is connected to the PDU via ATS. When power is lost on one of the PDUs, the ATS transfers the load to the second.
It is fundamentally important to properly connect equipment to the ATS and PDU . If you mess up, then no backup for power in the data center will not help your server to survive interruptions to the city's power supply.
Diagram of the proper connection of servers with one and two power supplies in a rack.
This is the way electricity passes through one beam from the transformer to the server in the rack. In a similar route, electricity reaches the rack and along the second beam. In the general scheme, all of the above looks like this:
Now let's look at the situation when urban electricity supply is lost in the data center. In this scenario, the following happens:
- The UPS switches to battery power;
- automatic input of the reserve in the main switchboard starts the diesel generator set and switches the data center to the power supply from them.
Here’s how it happens:
/>
When the power of the city network is resumed, the same AVR automatically returns the data center to the power supply from the city network:
One of the DGU groups at the OST site.
In this post we have discussed in detail how the power supply system looks in dynamics. If you want to know the details of the capacities, brands and backup schemes of each element of the system in the TIER III data center, then you can run through our last tour of NORD 4 . Ask questions in the comments, if something interesting for you is left overs.
Today we will talk in detail about the power supply system of the data center: find out which way electricity travels to the rack and what happens when urban electricity is lost in the data center. A subject we will sort on an example of one of our data-centers.
Let's talk on the concepts
Before we go through the electricity routes in the data center, let us examine the basic terms and definitions that we need today.
')
Entry - a place to connect the data center to the city power grid.
Transformer - a device for converting electrical voltage, in our case from 10 kV to 400 V.
A beam is an electrical route from a transformer to a server power supply.
Switchboard - a device for receiving, distributing electricity and protecting against overloads and short circuits.
Diesel Generator Set (DGS) is an autonomous power plant that produces electricity by burning diesel fuel. In the data center with the help of DGU is guaranteed food. Usually DGU stand off, but warm. It takes about 30 seconds to launch.
Uninterruptible Power Supply (UPS) - a device that powers the data center during a brief disconnection of the main source of power supply. The loss of electricity for at least a second, will lead to a shutdown of IT equipment. The task of the UPS is to prevent this interruption.
Automatic input reserve (ATS) - a device used in cases where the power supply system has at least two supply inputs. In case of problems with one of the inputs, the load automatically switches to the second. In data centers, ATS is used to switch the load on a data center to a diesel generator set and to back up power supply to servers with a single power supply.
Power distribution unit, PDU (power distribution unit) - a device for the distribution of electricity. In the English tradition, PDUs are referred to as electric panels and socket outlets. In our case, these are only socket outlets in the rack.
Let there be light
The data center is connected to the city power network through two inputs. These inputs are independent: they come from different substations and along separate routes. We follow the path of electricity through one of these inputs from the transformer to the server in the rack.
Through the inputs of urban electricity falls on the step-down transformers data center. Here 10 kV is converted to 400 V.
These 400 V come to the main switchboard (MSB). From here, power diverges across all engineering systems and IT equipment of the data center.
Two rows of gray cabinets - this is the main switchboard. Each row (section) is powered by its input and duplicates the one that stands opposite.
MSB includes:
- input circuit breaker (input automats) . If on the way from the main switchboard to the load a short circuit or excess of the permissible current occurs, the input automaton will switch off. The principle of operation of these machines is the same as that of traffic jams in the electrical panel in your home. In each section of the main switchboard there are two input automata: one works when the data center is powered from the city power network, the other when the diesel generator set is turned on.
- distribution bus . All electricity flows to it and is transmitted further to UPS or other switchboards.
In the upper part are the cables through which electricity comes from the transformer. In the middle part there is an introductory automat, below is a copper distribution bus.
- automatic reserve transfer (ATS) . He controls the input automata from the city and the DGU, gives commands to start and stop the DGU, switches the power of the data center from transformers to the DGU and back.
The display shows which source the data center is powered from. At the moment, food comes from the city network (LN1). LN 2 is a backup power line from DGU.
- diverting machines . They protect against the overload and short circuit of each route, which departs to other shields and end users: racks, UPS, air conditioners, lighting, etc.
- section switch. We have two main counters because 2 inputs have been connected to the data center. It allows you to power both sections of the MSB from one input when you need to repair one of the transformers.
In the room with the main switchboard, there is also a shield in which the outlets from all the DGS (usually from 3 to 5 pieces) are collected. When the city power supply disappears, the power from this shield is fed to the MSB and diverges further throughout the data center.
All the electricity generated by the diesel generator set goes to the copper distribution bus and from there goes to the induction automat from the diesel generator set.
So, getting into the main switchboard, 400V come to the introductory machine from the city. Next, electricity falls on the distribution bus and tap-off machines. AVR in case of interruptions to the urban power grid, switches the data center to power from the diesel generator set. From MSB power supply is already guaranteed, but still:
a) “dirty”: city stress is not too stable, there are jumps, temporary rises and falls.
b) not uninterrupted: the DGU will take up to 30 seconds to start and go to workload.
Both of these problems are solved by an uninterruptible power supply (UPS). It is there that electricity comes after MSB. The UPS levels the voltage to the set value, and when power is lost from the city, the UPS automatically switches its load to the batteries. On batteries, the data center can operate autonomously for about 15 minutes. This is more than enough for the DGU to start.
UPS ordinary.
Racks with batteries.
From the UPS, the UPS distribution panel (SCHIBP) is powered. From diversion automat SCHIBP, electricity flows to the switchboards of the machine rooms (SCHR), air conditioners (SCHK), emergency lighting (SCHS) and other panels.
In ShchR, for each rack, a separate automatic switch of the desired nominal value and phase is set. For example, to power a 6.5 kW rack, we install a single-phase automatic 32A.
SchR in the engine room.
In the NORD 4 machine halls, instead of the SCH, busbars are used under the raised floor.
On the busbar are installed tap-off boxes with automatic machines leading to the racks.
It remains to power the equipment in the rack. To do this, two PDUs are installed in the rack, which are connected to different power beams. Equipment with two power supplies is connected to different PDUs. If the equipment has one power supply, it is connected to the PDU via ATS. When power is lost on one of the PDUs, the ATS transfers the load to the second.
It is fundamentally important to properly connect equipment to the ATS and PDU . If you mess up, then no backup for power in the data center will not help your server to survive interruptions to the city's power supply.
Diagram of the proper connection of servers with one and two power supplies in a rack.
This is the way electricity passes through one beam from the transformer to the server in the rack. In a similar route, electricity reaches the rack and along the second beam. In the general scheme, all of the above looks like this:
Extinguish the light
Now let's look at the situation when urban electricity supply is lost in the data center. In this scenario, the following happens:
- The UPS switches to battery power;
- automatic input of the reserve in the main switchboard starts the diesel generator set and switches the data center to the power supply from them.
Here’s how it happens:
- The UPS records the loss of power at the input and automatically switches to autonomous power supply from the batteries.
- At the same time, the ATS in the main switchboard also fixes a breakdown with urban nutrition. AVR Waits 5 seconds. If the power is not restored, then the ATS sends a signal to start the diesel generator set.
- DGS start and synchronize with each other in 15-30 seconds.
- When power from the diesel generator set appears on the backup line, the ATS opens the circuit breaker of the main line and closes the switch of the backup line with an exposure time of 10 seconds. Since then, the data center is powered by the DGU.
- The UPS sees power recovery at the input and returns to standard mode.
/>
When the power of the city network is resumed, the same AVR automatically returns the data center to the power supply from the city network:
- AVR fixes the return of city power, waits 5 seconds to disconnect from the diesel generator set.
- At the same time, the UPS switches to battery power.
- AVR opens the switch on the backup line (power supply from the diesel generator set), and after 5 seconds closes the switch on the main power line from the city. The UPS returns to standard mode.
- AVR waits 60 seconds, after which gives the command to stop the diesel generator set.
One of the DGU groups at the OST site.
In this post we have discussed in detail how the power supply system looks in dynamics. If you want to know the details of the capacities, brands and backup schemes of each element of the system in the TIER III data center, then you can run through our last tour of NORD 4 . Ask questions in the comments, if something interesting for you is left overs.
Source: https://habr.com/ru/post/310356/
All Articles