The history of a single data center
Recently I visited the opening of one promising data center - Oversan .
Under the cut - photos DC and Gordon Freeman :)
About the company Oversan-Mercury, many probably do not know, but I was minimally aware of, because From time to time I look at Lebedev’s site (mainly in business lynch) and have seen the logo creation process there. It is very unusual to admit.
')
It all started with a small conference, which was held in the building of the prefecture of the South-Eastern Administrative Okrug - this place is painfully familiar to me, because just five minutes away is the institute, where I spent the best years of my life :)
Three people spoke at the presentation - Ioshpa Igor (Development Director), Bakanov Pavel (Director of the Network Control Center) and Dembovsky Alexander (Project Administrator).
Each of them told something different that it would be easier for me to retell with extracts from official sources, rather than recall - I still would not remember the abundance of nuances. By the way, about memorizing - during the conference, the laptop from which the presentation was on the screen went into sleep mode, and upon waking up, demanded a password.
Looking at “Mac” and at the blocked Vista screensaver, the recent incident about Steve Ballmer was involuntarily remembered - “ Steve, please sign on my laptop, there is Windows, honestly! ":)
So let's get down to business. The data center Oversan is a commercial project and is located in a large building near the metro station. Aviamotornaya - those who study at MEI and MTUCI certainly know the cool shop of the “Forum” equipment - just in those parts. Immediately after the presentation, we went by bus to the DC building.
Next to him there are no strategic facilities that could carry a potential threat - factories or gas stations, as well as residential buildings - so as not to interfere with the sleep of peaceful-minded citizens.
According to the creators, the data center is one of the most modern in Russia and meets all international standards - first of all it is intended for the implementation of the most resource-intensive and technological projects.
The computing heart operates at the expense of two transformer substations, from which a total of 4.5 MW of electricity was supplied.
Power stability is provided by a variety of uninterruptible power supplies and diesel generator sets.
If you believe the speakers, then in case of power failure "from outside", the generators will be able to ensure the full operation of the entire DC for a dozen hours (without bringing additional fuel). And yes, the availability of power supply is 99.982%.
The data network is built on the equipment of the world leader in the telecommunications industry - Cisco Systems (USA). The failover core of the network consists of Catalyst 6509NEB switches and GSR 12406 series routers. The security systems are based on Cisco ASA5580 high-performance firewall with 20 Gbps throughput and IPS 4270 attack prevention system that detects all known types of attacks)
Independent optical backbones leading to the main Russian communications centers of OJSC MMTS No. 9 and OJSC MMTS No. 10 have a total bandwidth of 70 Gbit / s and have the capacity to build up to 400 Gbit / s, which ensures reliable connectivity with the Internet.
The width of the channel to the rack can reach 10 Gb / s. The data transmission system is managed by employees with Cisco Systems certificates. The percentage of successful packet delivery is 99%; the delay in packet transmission from the client equipment port to the carrier port does not exceed 30 ms.
Having its own points of presence at the M9 and M10 communication centers, the data center is able to provide a channel rental service that is not represented in the proper amount on the Russian market. Optical lines allow you to organize up to 96 channels of communication with Internet providers and companies interested in direct peering with a data center (which allows you to gradually increase bandwidth without the extra cost of expensive DWDM equipment).
The entire room of the DC is divided into two large containment areas, in which there can be about 200 racks in total.
And under 50 of them liquid cooling is brought. Air-cooled cooling zones are cooled using a precision air-conditioning system with cold air outflow under the raised floor (it is at a height of 70 cm and can withstand a load of 1200 kg / m2).
Under it passes the main part of the wires.
The air temperature in the rack is maintained in the range of 18 to 21 degrees with humidity in the range of 49-53%.
The standard rack load is approximately 7kW and, if the client so desires, it is possible to install additional equipment, such as KVM switches or a managed power distribution system (PDU), which allows monitoring the power parameters in real time.
Water-cooled racks (Schaefer Unoqle) are capable of diverting up to 30 kW of heat generation - they are connected to four huge chillers (GeoClima), which are located outdoors.
As for security, the building has 24-hour video surveillance, a gateway system is used between rooms, and they, in turn, are equipped with modern security and access control systems (TAC and I / NET SEVEN) with authorization via proximity-cards.
And if we talk about the structure of the network itself, then in addition to hardware firewalls, traffic is divided into virtual networks (VLANs on Cisco switches), centralized anti-virus scanning, authorized SSL certificates and something else. Already, all password procedures are strictly regulated, the software is updated with an enviable consistency, and the plans include periodic security system tests with simulations of various types of impacts.
A separate item deserves the attention of the gas fire extinguishing system.
Thanks to a multitude of sensors, the system can operate in automatic mode - in one of the corridors there are special installations with two independent squibs, which in the shortest possible time allow the gas to extinguish even the recurrence of a fire (which is very unlikely).
All gas extinguishing systems are reserved and connected to uninterruptible power supplies.
Soooo ... here we have a plus ...
... then somewhere there should be a minus ...
It was really interesting to wander around the building - you see, you see it not every day. I constantly had the feeling that you were at a military base or in a building of foreign supercomputers. All neat and clean - class!
The corridor of the building is decorated with busts of famous people. There was a place for the busts of the Strugatsky brothers - after all, the name of the data center was born in one of their stories (Oversan - jump through the sun).
Here and the fairytale is the end ... and who listened is in vain, because it is the text)
Successes!
Under the cut - photos DC and Gordon Freeman :)
About the company Oversan-Mercury, many probably do not know, but I was minimally aware of, because From time to time I look at Lebedev’s site (mainly in business lynch) and have seen the logo creation process there. It is very unusual to admit.
')
It all started with a small conference, which was held in the building of the prefecture of the South-Eastern Administrative Okrug - this place is painfully familiar to me, because just five minutes away is the institute, where I spent the best years of my life :)
Three people spoke at the presentation - Ioshpa Igor (Development Director), Bakanov Pavel (Director of the Network Control Center) and Dembovsky Alexander (Project Administrator).
Each of them told something different that it would be easier for me to retell with extracts from official sources, rather than recall - I still would not remember the abundance of nuances. By the way, about memorizing - during the conference, the laptop from which the presentation was on the screen went into sleep mode, and upon waking up, demanded a password.
Looking at “Mac” and at the blocked Vista screensaver, the recent incident about Steve Ballmer was involuntarily remembered - “ Steve, please sign on my laptop, there is Windows, honestly! ":)
So let's get down to business. The data center Oversan is a commercial project and is located in a large building near the metro station. Aviamotornaya - those who study at MEI and MTUCI certainly know the cool shop of the “Forum” equipment - just in those parts. Immediately after the presentation, we went by bus to the DC building.
Next to him there are no strategic facilities that could carry a potential threat - factories or gas stations, as well as residential buildings - so as not to interfere with the sleep of peaceful-minded citizens.
According to the creators, the data center is one of the most modern in Russia and meets all international standards - first of all it is intended for the implementation of the most resource-intensive and technological projects.
The computing heart operates at the expense of two transformer substations, from which a total of 4.5 MW of electricity was supplied.
Power stability is provided by a variety of uninterruptible power supplies and diesel generator sets.
If you believe the speakers, then in case of power failure "from outside", the generators will be able to ensure the full operation of the entire DC for a dozen hours (without bringing additional fuel). And yes, the availability of power supply is 99.982%.
The data network is built on the equipment of the world leader in the telecommunications industry - Cisco Systems (USA). The failover core of the network consists of Catalyst 6509NEB switches and GSR 12406 series routers. The security systems are based on Cisco ASA5580 high-performance firewall with 20 Gbps throughput and IPS 4270 attack prevention system that detects all known types of attacks)
Independent optical backbones leading to the main Russian communications centers of OJSC MMTS No. 9 and OJSC MMTS No. 10 have a total bandwidth of 70 Gbit / s and have the capacity to build up to 400 Gbit / s, which ensures reliable connectivity with the Internet.
The width of the channel to the rack can reach 10 Gb / s. The data transmission system is managed by employees with Cisco Systems certificates. The percentage of successful packet delivery is 99%; the delay in packet transmission from the client equipment port to the carrier port does not exceed 30 ms.
Having its own points of presence at the M9 and M10 communication centers, the data center is able to provide a channel rental service that is not represented in the proper amount on the Russian market. Optical lines allow you to organize up to 96 channels of communication with Internet providers and companies interested in direct peering with a data center (which allows you to gradually increase bandwidth without the extra cost of expensive DWDM equipment).
The entire room of the DC is divided into two large containment areas, in which there can be about 200 racks in total.
And under 50 of them liquid cooling is brought. Air-cooled cooling zones are cooled using a precision air-conditioning system with cold air outflow under the raised floor (it is at a height of 70 cm and can withstand a load of 1200 kg / m2).
Under it passes the main part of the wires.
The air temperature in the rack is maintained in the range of 18 to 21 degrees with humidity in the range of 49-53%.
The standard rack load is approximately 7kW and, if the client so desires, it is possible to install additional equipment, such as KVM switches or a managed power distribution system (PDU), which allows monitoring the power parameters in real time.
Water-cooled racks (Schaefer Unoqle) are capable of diverting up to 30 kW of heat generation - they are connected to four huge chillers (GeoClima), which are located outdoors.
As for security, the building has 24-hour video surveillance, a gateway system is used between rooms, and they, in turn, are equipped with modern security and access control systems (TAC and I / NET SEVEN) with authorization via proximity-cards.
And if we talk about the structure of the network itself, then in addition to hardware firewalls, traffic is divided into virtual networks (VLANs on Cisco switches), centralized anti-virus scanning, authorized SSL certificates and something else. Already, all password procedures are strictly regulated, the software is updated with an enviable consistency, and the plans include periodic security system tests with simulations of various types of impacts.
A separate item deserves the attention of the gas fire extinguishing system.
Thanks to a multitude of sensors, the system can operate in automatic mode - in one of the corridors there are special installations with two independent squibs, which in the shortest possible time allow the gas to extinguish even the recurrence of a fire (which is very unlikely).
All gas extinguishing systems are reserved and connected to uninterruptible power supplies.
Soooo ... here we have a plus ...
... then somewhere there should be a minus ...
It was really interesting to wander around the building - you see, you see it not every day. I constantly had the feeling that you were at a military base or in a building of foreign supercomputers. All neat and clean - class!
The corridor of the building is decorated with busts of famous people. There was a place for the busts of the Strugatsky brothers - after all, the name of the data center was born in one of their stories (Oversan - jump through the sun).
Here and the fairytale is the end ... and who listened is in vain, because it is the text)
Successes!Source: https://habr.com/ru/post/83155/
All Articles