The data center of our dreams in Yaroslavl: photos of construction and launch

Bird's eye view

We have long wanted to build a data center, the capacity of which will be enough for the next 10 years. From December 2011 we talked about its design and our dreams. Now is the time to show the first results. Yaroslavl Technical Center has a TIER III reliability level according to the Uptime Institute classification. Today there are less than a dozen data centers of this level in Russia.

Now tests of the water cooling system are being conducted, work is being completed on the premises of the backbone network. A full launch is expected in early December after the completion of this work and the approval of government agencies for commissioning the office part. Caution traffic .
')

Chronology

In 2011, decisions were made on the construction of the Yaroslavl Data Processing Center of VimpelCom. From October 2011 to August 2013, the project was made and certified, we wrote about it (links at the end of the post). In June 2012, the zero cycle of building construction began (site preparation, foundation, etc.), in December - capital construction. In August 2013 we began to mount technical equipment, and in October 2014 we launched the first data center module in operation.



Our object looks like a complex of buildings, which include:
- main building;
- power modules for placing uninterruptible power systems on diesel-dynamic uninterruptible power supplies;
- distribution substations;
- checkpoint;
- boiler room;
- fuel storage;
- sewage treatment plants;
- various ancillary buildings.


Center layout

In general, everything was going according to plan, despite the previously agreed deadlines. Construction work began as the project was released for each object and the project documentation was approved by government agencies.

The active construction phase began in December 2012. At this point, the contractors had a project for all auxiliary buildings, on-site networks; started with power modules and gearbox. In February 2013, work began on the construction of the main building, and in September the installation of the technological equipment of the data center began.

Photographs about the construction process
Main building. Foundation construction:



Main building. Installation of metal structures of the central part of the building and staircases:



Main building. Installation of metal structures - construction of a hangar for data center modules. The span is 40 meters:



Main building. Installation of metal structures - the supporting structure of the data center module was erected inside the building frame:



Sweet November. Last year, the weather presented a gift, thanks to which we began to build the machine hall, practically under the open sky:



Rigging works on the installation of a 23-ton 3000-horsepower diesel engine with a 100-ton crane:



Diesel engine in a regular place:



Installation of water cooling piping:



Installation of one of the three chillers, we affectionately call them spiders:



Construction of the machine room:



Installation of a structured cable network in the data center module:



Yaroslavl Technical Center in numbers:

Engineering difficulties

In construction, as in any other project, something unexpected may happen (supply failure, illness of a whole team of workers, bad weather conditions, an accident and much more), which will stop the work of one subcontractor first and then potentially, and the entire construction. These risks “multiplied” by the number of contractors determine the probabilistic nature of the construction process. The probabilistic nature takes on an extreme form when the project is innovative, and its participants must solve problems that they have never faced before. Just our case.

The surprises are "regular", which fit into the planning horizons, and there are those that require changes in the project in the course of execution. One of the most difficult and demanding tasks was the abandonment of the traditional concrete roof, on which the original was intended to install the metal structure of the colors of our striped logo. The choice was made in favor of a soft roof with a black and yellow membrane after the start of the active phase of construction. This solution allowed to significantly reduce the load on the building frame and foundation, which allowed to save the budget on materials and optimize the timing of work.

One of the surprises for designers was the realization of what is a diesel generator silencer with a capacity of three thousand horsepower. At the factory in Germany during the acceptance of the equipment of the dynamic uninterruptible power supply system, we all took pictures against the background of some kind of “submarine”, and then an employee came up and said: “And these are your guys, your silencer”. Length - 5.7 meters, weight - 1.5 tons. While he was taken to Yaroslavl, we had to urgently supplement the section on rigging work.

One of the features of our data center was to be a gradual launch: we planned to rebuild and equip one section of the main building, and then continue to work. So, it turned out that the installation of process equipment in the context of ongoing construction and installation work is not a trivial task. The number of questions that had to be resolved every day began to peak, the situation was heating up, and the argument during negotiations with foreign suppliers reached “I don't give a shit”. In general, everything turned out to be possible, but unusual for all participants in the process.

Some of the contractors could not cope and had to leave the project: we changed the general contractor, the technical customer (capital construction management experts). In order not to violate the deadlines for individual works, it was necessary to attract trusted Beeline partners.

It is necessary to admit that we didn’t have the experience of capital construction of such scale at that time, which created additional difficulties. For example, at first the deadline for completion of construction was set for May 2013. Unreal date, from the point of view, technology works, or at least because only in August the design phase was completed. We compared the situation with our capabilities in time and adjusted the deadlines.

They quickly learned, attracted new people, looked for specialists within the company who came in contact from construction. Over time, we have a team of experienced experts in capital construction, power systems, IT infrastructure, and the transportation network. And as a result, they were able to solve problems directly at the construction site.

Before construction, our colleague had a conversation with the head of the newly built pharmaceutical plant, who said: “The main thing is purchases.” We realized this truth. The slightest mistake when choosing a contractor, and a lot of problems will fall on you: the project budget will increase, and deadlines will fly with a whistle.

One such contractor quickly raises the level of chaos at a construction site, his problems become the problems of all. He quickly forgets everything that he promised you at the tender stage, and suddenly it turns out that promises depend on many new factors beyond the control of a poor contractor, who need help without fail (most often with money and agreeing on an increase in time). If we consider the construction site as an ecosystem, we can say that this contagion is spreading rapidly, and soon other contractors who start to behave in a similar way find themselves in the affected area. And if they are, why should we try? In general, everything, as in life, only at stake are millions of rubles.

What do we know now

1. Project team . Need a team of external or internal staff, which would have expertise in all areas of the project. Ideally, everyone should have experience building data centers. This is difficult or costly and difficult to justify to management (be sure to voice the risks before you accept it). Key employees (project manager, his assistants, chief project engineer, etc.) should have this experience. We now have such a team.
2. Project management . We need an extremely clear organizational structure, good communication between the participants, clear areas of responsibility, decision-making processes and the processing of requests from contractors.
3. Planning. Schedule for capital construction may consist of several thousand points with a complex relationship. We need a separate person who would update this schedule weekly. So you will always have a clear picture for making the right decisions.
4. Procurement (selection of suppliers and contractors):
- in any procurement expert group must be sufficiently competent. We need professionals at the stage of choosing a concept and drafting technical tasks;
- the work of the purchasing department should be optimized from the point of view of the specifics of the construction;
- It is better to reduce the number of contractors: one general designer (even if your company already has a valid contract, for example, on security systems), one general contractor;
- the procedure should provide for one or two backup suppliers for all types of work that could be included in the project if difficulties arise;
- Excellent practice - insurance of large contracts by bank guarantee;
- in the scope of work of the general contractor include the commissioning of the building;
- thick contracts in which everything is spelled out in detail is good.
5. Expertise, consulting .
We need to invite experts to the project in all areas involved in the project: data center design, capital construction, installation of process equipment, etc. Prior to the start of commissioning works, it is worth hiring an independent organization with experience in the data center to help develop a test program and methodology. Under its control will be commissioning, as well as comprehensive testing of all data center systems. These people should be held liable for the result of their work.
6. Service operation .
Before starting the design of the object, find the head of the maintenance department Decisions on the design and construction of the building should be made in terms of operation (convenience, cost effectiveness, minimal risks, etc.) Reliability and availability of the data center will depend on the correctness of the chosen solutions.

How to fly into space

Founder of Beeline Dmitry Zimin puts his signature on the NFC situational plan

Toward the end of construction, Dmitry Zimin, the founder of VimpelCom, came to us. In general, he came to meet with local journalists, but we called him to our construction site. Dmitry Borisovich turned out to be incredibly friendly and sincere. And we tried to imagine what was going on in his heart: once his business started from one base station within the walls of the Radio-Technical Institute in Moscow, and now he is in another city and watches how a 10-megawatt data center of his company is built ( and this is how it all began ).

Now

A single modular data center has now been commissioned to accommodate 236 standard telecommunications racks. In total, it is planned to launch 6 such halls, that is, in total, 1416 racks can be placed in the NFC.

Initial calculations showed that the number of racks in one hall would be no more than 200, but during the design we were able to increase this figure to 236.

We have summed up the backbone network (but have not yet tested everything), we are driving cooling tests. How to finish - there will be a big cleaning, coordination and acceptance. After that, in December, the first “combat” launch is planned.

How we designed all this, you can read here: general , about power , about cooling . This is the 2011-2012 year.