Where to store data: a tour to the IXcellerate data center

We often hear that storing data and putting its systems to IT outsourcing is dangerous and unreliable. Allegedly, the data may be unavailable due to technical problems with the provider, the data center can become the object of a hacker attack, and simply calmer when all the IT equipment and all of the company's data are near, “close by”, and not stored and maintained “on side. "

I propose to see how the IXcellerate data center is arranged, in which we store our data and customer data. We are a service provider of Onlanta . Our cloud OnCloud.ru is based on several geographically distributed data centers. IXcellerate data center is one of them.


IXcellerate's data processing center is located in the former industrial zone and reinforced concrete plant in the Altufyevo area on a site of more than 15 thousand square meters. m - this is one of the largest data centers in Moscow.
')

The choice of the territory of a concrete product plant for creating a data center was not accidental - such enterprises usually have powerful lines for the input of electricity, which is exactly what is needed for the data center. There are two such lines on the territory of the data center - from two different distribution substations. The distance between their entry into the building is at least 150 m. This greatly increases the reliability and availability of data center services, since the probability of failures due to accidents on the power grid is greatly reduced.

IXcellerate data center began to be built in 2012 and a year later the first machine hall was put into operation. The founder of IXcellerate is a well-known international investor and British businessman Guy Wilner .

Guy Willner (right)

Co-founder Guy Wilner was another international businessman - Cliff Gauntlett, who began working in the Soviet Union in the late 1980s. He turned out to be one of those foreigners who came here during perestroika, fell in love with the country, and, despite all her problems, remained here forever (Cliff died in 2013). Probably many people remember phone cards ROL. They were released by the Russian provider Golden Telecom, where Cliff worked as a senior vice president. In addition, Cliff was also an avid "rocker" - headed by the famous ROL Band, and his songs are even in the repertoire of the Scorpions group. Hence the motto of the IXcellerate data center - Rockin'Russia.

Korportivny festival Rockin'Russia: Cliff Gauntlett with a guitar, behind the drums - Alexander Vershbow, the US Ambassador to Russia. (photo www.passportmagazine.ru )

The first machine room of the IXcellerate data center was named in honor of Cliff Gauntlett.

Cliff's guitar still hangs in the data center, with Paul McCartney’s original autograph.

The design capacity of the data center is 13.7 megawatts, and the capacity is 1835 racks, and this number is growing. The building area is 6000 square meters. m. In addition, the data center has more than 600 square meters. m warehouse for customers, where they can store the necessary equipment for the work.

Two access roads lead to the data center building. The territory is removed from airports, railway tracks, highways, various kinds of shopping and entertainment centers, as well as residential buildings.

Data center building with installed cooling devices (chillers)

In addition to the main power supply from the two electrical substations in the Moscow region, the data center also has backup diesel generators that generate electricity in emergency situations, for example, if both substations fail during a disaster.

For diesel generators supplied special fuel - summer and winter. Special protective concrete blocks were placed near the diesel generators so that they would not be accidentally damaged by moving equipment.

Outdoor container with diesel generator behind protective concrete blocks

The performance of diesel engines is monitored around the clock online, and even the duty officer performs a detour every four hours. During our visit, diesel generators were just serviced, and there were vehicles with test equipment on the territory.

In the event of a power failure, the batteries create the required time interval until the backup diesel generators, after starting, reach the required power.

Diesel Generator Maintenance

The data center strictly complies with the requirements of both physical and information security, as well as the security of power grids. The data center building is faced with so-called. sandwich panels both outside and inside. They provide thermal insulation and physical protection. The data center was built according to the “box-in-box” principle, that is, trees, a concrete fence, a lawn, and multi-layered sandwich panels separate the interior from the street.

The roof of the building is located at a height of 14 m, and the roof of the engine room is 9 m. Between them there is a protective layer against leakage.

Multi-layering allows you to maintain certain parameters of temperature and humidity that are optimal for the operation of server equipment. Microclimate is also created by air coolers (chillers) and precision air conditioners.

Creating a data center

The IXcellerate data center was built modularly, in three phases. The engine room of the first phase during construction in 2012 was the largest in Moscow. The halls of the second and third phase round the hall of the first phase around the perimeter of the letter G.

Each machine room has its own loading and unloading area. A separate discharge zone is also provided for the fire station.

IXcellerate Data Center Layout

IXcellerate was created as a “neutral” data center, independent of telecommunication and cloud service providers, and focused exclusively on co-location services, i.e. placement of customer equipment, its protection and provision of the required engineering infrastructure with an established level of service under a standard contract.

It happens that customers ask about cloud services. For such cases, the data center has an ecosystem of partners, on the equipment of which cloud solutions are provided. We are doing just that.

Let's see what's inside

The data center requires employees and authorized clients to undergo dual identification, including a standard ID-card and biometric control (hand palm scanner).

Employee Access Control

Access to cells (“cells”) where client equipment is installed is under round-the-clock control, that is, only data center employees authorized by the client can get there. These cells are separated from each other by a metal grid.

The cells also have their own physical protection, for example, combination locks and video cameras. In total, there are five lines of physical security with a constantly working building monitoring and control system (BMS) Computrols.

In the data center, it has become a good tradition to call each machine room in honor of the company or the person who made the greatest contribution to its creation.

The first phase hall, named after Cliff Gauntlett, is designed for 220 seats with customer equipment.

During the construction of the hall of the second phase, the investments of the Japanese company Sumitomo Corp. were used, therefore it is called The Summit Hall. The third-phase hall is named “Height” in honor of the technical director of the data center, Sergey Vladimirovich Vyshemirsky, who used to work at the Chkalovsky airfield.

Plate with the name of the third engine room

Power and Cooling

High-quality electricity is used to power the equipment of the customers. What does the word “quality” mean for electricity? It is no secret that industrial electric current sometimes has peak bursts that can damage customer equipment. Therefore, there is a special “electric room” in the data center, where these emissions are eliminated. The result is an electric current with a frequency of 50 Hertz, which has the form of a "pure" sinusoid, without emissions and peaks. Electrical distribution equipment constantly monitors nine parameters of current quality.

Data center power supply redundancy is organized at the 2N level: as already mentioned, two independent power supply beams approach the data center. Damage to one route of laying will not entail stopping the data center or its part, as there is always a backup route.

This is how incoming lines from the power substations on one of the laying routes look like.

Power supply input from a transformer substation

Here's another little touch on the topic of electrical and fire safety. The wiring from the switches is laid in metal pipes so that when a short circuit is possible, no sparks fly out.

Switches for lighting lamps with wiring in metal pipes

All power supply cables pass under the raised floor 1 m high. Under the raised floor, cold air with a predetermined humidity is supplied to the equipment racks in full compliance with the international ASHRAE standard.

Temperature and humidity sensors are installed on the cells with client equipment, which not only visually display these parameters, but also regularly transmit them to the centralized control system of the engineering infrastructure of the data center.

Temperature and humidity sensor

To power each rack under the raised floor there are two sockets - each from a separate input line from different electrical substations. For even greater reliability, they are laid in separate gutters.

Two rack power sockets (through the raised floor grille)

Some clients also install their own video cameras in their cells with equipment for increased security.

The data center has “telecom zones”, where the equipment of 44 telecom operators is installed, including Orange, Sprint, Beeline, MegaFon, MTS, Telia, other operators and Internet providers, as well as the MSK-IX node and the Eurasia Peering traffic exchange platform IX, uniting 38 resident operators.


In the data center for the installation of client equipment accepted any size and functionality, even tape drives.

One of our clients - a television channel with a multimillion audience - makes reservations here for broadcasting in HD.

Fire safety

The ability to quickly and effectively extinguish the fire, and it is better not to allow it at all - one of the most important characteristics of any data center.

IXcellerate uses a three-tier fire detection and extinguishing system. The first level is the Vesda ultra-early smoke detection system, which reacts to a change in air density and the amount of impurities and dust of just 0.01%. A self-learning system with elements of artificial intelligence keeps in mind a certain matrix of air parameters. Such a system can recognize smoke at a very early stage, as well as determine the sector where the change in parameters occurred.

Apollo - an integrated fire alarm and warning system - the next step in the fire safety data center.

But IXcellerate's pride is the HI-FOG automatic fire extinguishing system, which is capable of localizing fires on an area of ​​4 x 4 square meters. m (with the possibility of finding personnel directly in the workshop and without impact and consequences for other sites). For this purpose, water sprayers (sprinklers) are used under the HI-FOG system, which are under constant pressure and are placed above each local perimeter of 4 x 4 m.

It is important to note that the HI-FOG uses a dry-tube system, i.e. the fire extinguishing agent used in the normal mode - distilled water - is stored in a tank that is placed in a fire-pumping station.

One of the sprinklers

In the sprinkler there is a ball with an essential substance, which bursts at a temperature of 57 ^o C, after which the fire extinguishing system automatically starts.

The HI-FOG system creates a fine mist that closes the fire zone, displaces oxygen from it and lowers the temperature. That is, the system allows to eliminate two causes of their three possible fires. The third theoretically possible cause of a fire is the presence of combustible material. However, the already mentioned sandwich panels, which revetted both external and internal walls, as well as raised floor plates in the hall, are made of non-combustible material. These panels can provide up to 90 minutes of fire protection. In addition, only low smoke cables are used in the data center.

The HI-FOG system is, perhaps, optimal for use in the data center, since it provides local extinguishing rather than filling the entire hall with gas foam, causing damage to equipment not covered by fire. In addition, its use does not require mandatory evacuation of people.

The FM Approved certificate, issued by an international leader in independent testing, allows the use of HI-FOG technology also in the energy industry, in the processing and processing industry, as well as in submarines and cruise liners.

Fire station

A tank with distilled water (blue, visible from the back in the photo) is designed to extinguish the entire area of ​​the data center. The same system works in the Moscow metro and the Bolshoi Theater.

Remote control and monitoring

The 24 x 7 control and monitoring control panel contains duty officers who monitor the operation of engineering systems and receive applications from clients. The video surveillance system allows you to quickly move from displaying the general scheme of the data center to images of individual rooms, cells, corridors and equipment racks, components of the fire extinguishing system (including sprinklers), power supplies, and air conditioning.

The standard SLA agreement provides that the first response to a request for technical support must be given within 30 minutes. However, in the data center, the actual time of customer feedback does not exceed six minutes, according to statistics from the Zendesk portal, which is used to register requests.

Support is provided in Russian, English and Chinese.

Control and monitoring system console

Since the launch of the data center since its launch in 2013, there has not been a single downtime (downtime) or a single violation of the SLA.

The data center and its well-coordinated team leaves the impression of a living organism.

Inside the data center there is even a place for music: when you cross the corridor from one hall to another, you can not miss the equipment for the rock band in which Cliff Gauntlett once played and who performs at corporate events and the mandatory annual open air festival IXcellerate - Rockin ' Russia.

Corridor between the data center rooms

IXellerate plans to build new sites, in addition to Moscow One there will be data centers Moscow Two, Moscow Three, etc.

Our company " Onlanta " began cooperation with IXcellerate with two racks for the implementation of an ambitious at that time project to create a geo-distributed cluster. Then no one on the Russian market had such a solution. Channels of dark optics cost us cosmic money compared to the profit from the services we provided. Nobody bought a separate service for a disaster-resistant cloud here, so we just began to develop our cloud at two sites at once. Then a third one appeared, as it became obvious that it was impossible to provide full fault tolerance and load balancing without a separate “witness” who would not be tied to one of the two sites. But objectively, one can say that, of all three sites, IXcellerate is the best - both in terms of service organization, and in general, in terms of management adequacy.

Share and you what you value most in your data center and, conversely, what annoys you most?