Walks on Peter's data centers and telecoms
The first days of summer are great for educational trips to St. Petersburg.
We will visit Miran, linxdatacenter, RETN and Metrotek.
5 am, Moscow railway station, KFC, Moika Embankment, Plate, pigeons from the roof, Isaaca, the Field of Mars, Captures yandexodrayovy, and here it is - Miran.
Our lab with Eva, the broadcast server, the virtual Mikrotikovsky Routeros, hosting linkmeup with IPv6 accessibility and now hosting podcasts have been living in Miran for several years now.
')
At the entrance to Miran we were met at first by the impressive size of the DSU.
Most diesel gensets are on wheels for two reasons:
DGU Mirana in winter and summer starts in a few seconds. During this time, the entire data center is powered by batteries.
And iron HP in the stand of Miran and the tour became possible thanks to Fyodor Rusakov.
The whole tour took place in a cozy homely atmosphere. Coffee with cookies, talk for telecoms, for the origins of the company, about the transport of two hundred servers vkshechki from Moscow to St. Petersburg under cover of night. Without haste, we walked through two data center buildings. Both - BrownField - converted from the factory premises. The first one, owned by Miran, from which we started and where on the second floor was charged after a four-hour walk through Peter, was once a factory of diesel installations, truly gigantic diesel installations. Actually the second floor was not - there were huge gates to round-trip. Everything was dismantled, the second floor was done, active cooling, redundant power supply, gas fire extinguishing system.
Classic hot corridor, from where the air is supplied to the air conditioners and under the raised floor to the closed cold corridor.
Also under the raised floor power cables are hidden.
Miran was not born as a platform for one anchor client, but rather systematically developed the offer in a market where there is demand.
At one time there was a large bank with all the grids and access control systems, but it had already moved out.
Demand continued to grow, Miran, too, a second building appeared - already rented - also BrownField - two hundred meters from the first. The road lies along the embankment of Bolshaya Nevka past the Nobel house.
The new building is built on a modular basis with the ability to transport the entire module in case of problems with the landlord. The first two boxes were bought from the manufacturer, the third one was already collected with regard to the old mistakes.
The modular architecture turned out to be cons - they are relatively narrow - a hot corridor - 60 cm - two people do not disperse. And most importantly - the cost of the move, as it turned out at the first calculation, is comparable to the construction of a new one.
But they look both outside and inside fresh and fashionable.
NOC guys on the Orthodox Linux, stable debian.
The company, by the way, received its name from the board Meridian Integrated Recorded Announcement (MIRAN) System in the telephone switchboard Meridian. And after that the data center was called that.
They decided to build a data center in a difficult 2008 year.
Since then, the guys provide Dedic and Bell. In special rooms installed equipment operators.
And it imposes certain restrictions and features on work. For example, in racks there is the most varied equipment, both in terms of machines and in terms of networking. In Miran I didn’t see Arista and Mellanox. All equipment imposes different requirements on the temperature in the cold corridor, so you have to keep it very low - about 20 ° C, that is, you don’t have to say about low PUE - it is 1.5 for Miran.
Another nuance - with snake weddings in racks. As soon as you let external comrades into the room with their commutations, write “lost beauty”. No one has exceptions.
Alas, to introduce draconian measures here may be to his detriment.
Recent photos.
And then merch from our table to yours and from yours to ours. Loaded the whole crowd in the taxi - and in the links.
These are serious guys, on the temporary pass of which even the Security Matters is written. This will be the leitmotif of the whole tour.
After the gingerbread Miran in linxdatacenter you realize that the luxury of the DC has come. On their day off, 4 people came together directly, not to the city center, to tell about themselves to other five people who are unlikely to become their clients :)
We were met by Maria Svistunova, held at a large negotiating table. A bit of idle talk for Guatemalan coffee and go.
Excursions are put on stream. Maxim and Oleg know their business: they tell it by the thumb: clearly and with passion. Where necessary, an emphasis is made, and where not, it is bypassed. With a sense of humor, too, a full order, so more than two hours passed quickly.
The pride of the data center is uptime at 100% since its launch in 2011.
The confidence is so high that for the tenths there is not even a place on the scoreboard.
Linx is an example of a DC in GreenField format. The building was specially built for the placement of a DC in it. Therefore, the height of the ceilings, the strength of the overlappings of the power supply inputs, its reservation, SCS - was incorporated into the project and taken into account during the construction phase.
Thus, by default, the floor withstands a rack of 800 kg, but the mass can be further increased after additional reinforcement. They say racks cost in Falcon9 require a special relationship.
The DC itself is powered by a gas-piston installation, and urban electricity is only a reserve. According to Maxim, it was a very correct decision, because during the well-known Petersburg blackout, the office and the nearby business center suffered, and the data center continued its work.
The diesel generator is no longer a trailer on wheels, but a completely stationary room with chimneys billowing high above the landscape.
Startup time is no more than two minutes, and on batteries the entire data center can work for about an hour. Diesel can work 3 days without additional refueling.
Here in everything reigns order and thoughtfulness.
This, for example, is a key, where data center customers themselves can take keys from their racks by fingerprint.
And this is a room where customers can carry out any engineering work they need, for which a wire tool hangs on the wall.
These are already shelves for local operation with all vital in daily work:
It is always charged and ready to go.
The baseball cap of the beginner hiphopera in the photo above saved more than one person’s head. It is a solid natural helmet like a natural helmet - the very thing is to climb under a half meter raised floor, where no, no, yes, some sharp piece of iron sticks out.
Actually employees from operation:
This unremarkable device is a differential pressure meter.
It is designed to solve a simple problem - to maintain the pressure difference between the corridor and the engine room. For example, in order that it was inside a little higher and when you open the door, the dust did not fly inside, but instead flew away from the door.
The fire extinguishing system is also gas - in case of trouble, the gas displaces oxygen and stops burning.
Only serious and very serious guys who have reasons for paranoia in size to Ararat get up in the link.
Grid and separate from data center access control - this is just the beginning.
That's when two people come to solve a problem or switch to new commutations - one with a material key, the other with an electronic one - this is more interesting. And when an auditor in glasses comes from a client, descends under a raised floor, pushes two sheets of grids and says that a child’s head will crawl through here, you understand that a person has seen some interesting things in his life.
There were funny situations when, when designing a cell, they took into account everything that was possible, all security systems, all sizes that allow you to deliver a rack and install equipment in it, but somehow did not think about the dimensions of the engineers who would work with it. And they neither in the front nor in the profile do not fit between the cell and the stand.
If I were a FSB, I would put my iron in linx :)
After the tour we sat for quite a long time, mentally chatting under the pizza.
From this realm of security and order, we moved to RETN.
We are familiar with these guys for a long time and close. Therefore, it is not surprising that finally we gathered on their roofs to fry kebabs and poison bikes.
We have already been with them too, although, it seems, they didn’t tell. Last year, plans to seize the roofs of St. Petersburg failed because of Fifa.
RETN is actually the largest trunk operator with fiber all over the planet. Any spider will envy their web.
A DC on the bypass is a home project for the soul that has found its niche in a growing market. These are those who need a thick connection with the world and access to other local operators.
For example, in RETN is worth CDN Netflix'a.
I was surprised to learn that each of their nodes was individually signed by hand with a phrase.
And such holiday boxes are threshed in different cities on POPs RETN (and, obviously, not only him), delivering to you as soon as possible the “Black Mirror” and “Sharp Visors”.
There is also a Google CDN and a bunch of operators, both local and federal.
At RETN, two tiny (compared to Miranovsky) diesel generators and a car with cans stand next to the building to go to the gas station. It sounds almost like a family business.
Well, the evening talks on the roof brought us to the point that RETN would give us a virtual box for a backup site for hosting podcasts and pictures.
World guys, super-lamp data center, amazing network.
For some, this day ended with a duke nukem tournament, for someone a welcome sleep in a hotel.
The next day we didn’t have many plans: Metrotek and the podcast live - linkmeup IRL .
Just a month ago, Misha Sokolov and Denis Gabidullin from STC Metrotek Slash Plumspace came to visit a telecom podcast, and we analyzed what a Smart SFP is. And now we were able to touch them alive.
Denis, who conducted a tour of Metrotek St. Petersburg office (which by the way has not changed since our last trip to them), actually gave one of the samples from the threshold.
At first I was surprised how clumsy it was made - as if with a jigsaw and file it was turned from a 32 key.
It turned out that the case of the prototype was printed on a 3D printer made from metal powder, which can be seen on the rough surface. Others - those that are ready for the series - are in perfect order.
But in general, looking at them, you are amazed how they still managed to hide such a number of brains in an almost standard building.
I advise you to listen to the podcast about Rosa, Toza and their sister Boza . It turned out for my taste interesting.
Office design is the most common. Relatively small rectangular cabinets, elongated toward the window, are rather dark, which pressed a little. But the atmosphere, despite the weekend, reigned working.
Registration in two years has not changed. We were pleased with some posters and boards.
We walked through several rooms: R & D, systems engineers, FPGAs, held different types of SFPs in our hands, programmers, saw the first model in which Smart was rendered from SFP on the wire - one of the first experiments - rather, the Proof of Concept, that it generally works .
They say it is even convenient for some customers to have such a tail sticking out of their iron.
The very first goal two years ago was to assemble a conventional optical SFP that would at least transmit a signal.
Then, the Smart part was separately assembled with a foreign SFP (just on a string) to make sure that they were able to assemble the chips.
Metrotek turned huge work. Just think that for 2 years we have assembled a fully working optical module that can encapsulate, telemetry, an exact time server, an interface converter.
And at the same time debug to the ideal (although this has yet to be verified by time) the basic functionality of signal transmission. There should be no attenuation, there should not even be rare drops caused by SFP. And it should not warm up, despite the pack of ASICs, FPGA and CPU inside.
Bravo!
And then I remembered that, in general, no more than two years ago, I worked in a single international wide company where conferences were banned and knowledge exchange was absent as a class. And it was I who was one of the evangelists of weekly meetings to share experiences that seemed to save lonely engineers in remote regions.
In general, Metroteca regularly conducts internal trainings and lectures, why they are great.
And then we were taken to an amazing installation shop. How many times I have been in them, it seems that my childhood delight does not fade.
Metrotek has an equipment for printing on boards right in the office.
A stencil for applying solder pads, a SMD component printer (otherwise I just cannot name it), a small stove for baking boards and nearby workstations for manual installation.
To my surprise, I learned that microchips with hundreds of legs are also soldered by hand. Denis says that their guys just do it with amazing precision and speed, soldering the smallest details and sometimes it makes no sense to run Shaitan-car - a person will make it faster.
We have called together the Metrotek Eltex-Verilite production line, on the wave of a relatively recent tour to them.
In addition, the office has a 3d printer and a milling machine for domestic use, which, however, has already engraved medals for last year’s Chaos Construction .
From there we went to the stunningly ideological Krang Pizza, where the linkmeup was already swallowed by the mutant and pizza-burger in size and moved to the IQ option office, where they recorded the first release of IRL linkmeup .
The next day, we looked at the Mammoth Laser Center, where we had a technocratic ecstasy and a bit of inflammation of greed, but this will be a different story.
In conclusion, I would like to say that all the excursions and meetings turned out to be very different, both in scale of action and in focus.
Miran who hospitably sheltered our hosting, fast growing, but still friendly simple.
Lynx is a truly cool data center, with security at the forefront and experienced marketers.
RETN- young jolly guys “out of the garages,” who make a giant net and still a little TsODik and are not averse to making a fire on the roof.
Metrotek is a relatively small team of Russian developers of hardware and software, which makes almost a bunch of “measuring tools” more than competitive equipment, and is now entering a new market with an ambitious task.
And the Laser Center, which goes beyond the interests of the linkmeup, but is the object of our personal interests. He left a very pleasant impression and engraving of the microphone on the recorder.
We will visit Miran, linxdatacenter, RETN and Metrotek.
5 am, Moscow railway station, KFC, Moika Embankment, Plate, pigeons from the roof, Isaaca, the Field of Mars, Captures yandexodrayovy, and here it is - Miran.
Miran
Our lab with Eva, the broadcast server, the virtual Mikrotikovsky Routeros, hosting linkmeup with IPv6 accessibility and now hosting podcasts have been living in Miran for several years now.
')
At the entrance to Miran we were met at first by the impressive size of the DSU.
Most diesel gensets are on wheels for two reasons:
- Mobility - there are several more Miran modules in a couple of hundred meters. They have their own generators - but you never know - you have to roll.
- Mobility - they do not need to be registered as stationary power plants and, accordingly, project documentation is not needed.
DGU Mirana in winter and summer starts in a few seconds. During this time, the entire data center is powered by batteries.
And iron HP in the stand of Miran and the tour became possible thanks to Fyodor Rusakov.
The whole tour took place in a cozy homely atmosphere. Coffee with cookies, talk for telecoms, for the origins of the company, about the transport of two hundred servers vkshechki from Moscow to St. Petersburg under cover of night. Without haste, we walked through two data center buildings. Both - BrownField - converted from the factory premises. The first one, owned by Miran, from which we started and where on the second floor was charged after a four-hour walk through Peter, was once a factory of diesel installations, truly gigantic diesel installations. Actually the second floor was not - there were huge gates to round-trip. Everything was dismantled, the second floor was done, active cooling, redundant power supply, gas fire extinguishing system.
Classic hot corridor, from where the air is supplied to the air conditioners and under the raised floor to the closed cold corridor.
Also under the raised floor power cables are hidden.
Miran was not born as a platform for one anchor client, but rather systematically developed the offer in a market where there is demand.
At one time there was a large bank with all the grids and access control systems, but it had already moved out.
Demand continued to grow, Miran, too, a second building appeared - already rented - also BrownField - two hundred meters from the first. The road lies along the embankment of Bolshaya Nevka past the Nobel house.
The new building is built on a modular basis with the ability to transport the entire module in case of problems with the landlord. The first two boxes were bought from the manufacturer, the third one was already collected with regard to the old mistakes.
The modular architecture turned out to be cons - they are relatively narrow - a hot corridor - 60 cm - two people do not disperse. And most importantly - the cost of the move, as it turned out at the first calculation, is comparable to the construction of a new one.
But they look both outside and inside fresh and fashionable.
NOC guys on the Orthodox Linux, stable debian.
The company, by the way, received its name from the board Meridian Integrated Recorded Announcement (MIRAN) System in the telephone switchboard Meridian. And after that the data center was called that.
They decided to build a data center in a difficult 2008 year.
Since then, the guys provide Dedic and Bell. In special rooms installed equipment operators.
And it imposes certain restrictions and features on work. For example, in racks there is the most varied equipment, both in terms of machines and in terms of networking. In Miran I didn’t see Arista and Mellanox. All equipment imposes different requirements on the temperature in the cold corridor, so you have to keep it very low - about 20 ° C, that is, you don’t have to say about low PUE - it is 1.5 for Miran.
Another nuance - with snake weddings in racks. As soon as you let external comrades into the room with their commutations, write “lost beauty”. No one has exceptions.
Alas, to introduce draconian measures here may be to his detriment.
Recent photos.
And then merch from our table to yours and from yours to ours. Loaded the whole crowd in the taxi - and in the links.
linxdatacenter
These are serious guys, on the temporary pass of which even the Security Matters is written. This will be the leitmotif of the whole tour.
After the gingerbread Miran in linxdatacenter you realize that the luxury of the DC has come. On their day off, 4 people came together directly, not to the city center, to tell about themselves to other five people who are unlikely to become their clients :)
We were met by Maria Svistunova, held at a large negotiating table. A bit of idle talk for Guatemalan coffee and go.
Excursions are put on stream. Maxim and Oleg know their business: they tell it by the thumb: clearly and with passion. Where necessary, an emphasis is made, and where not, it is bypassed. With a sense of humor, too, a full order, so more than two hours passed quickly.
The pride of the data center is uptime at 100% since its launch in 2011.
The confidence is so high that for the tenths there is not even a place on the scoreboard.
Linx is an example of a DC in GreenField format. The building was specially built for the placement of a DC in it. Therefore, the height of the ceilings, the strength of the overlappings of the power supply inputs, its reservation, SCS - was incorporated into the project and taken into account during the construction phase.
Thus, by default, the floor withstands a rack of 800 kg, but the mass can be further increased after additional reinforcement. They say racks cost in Falcon9 require a special relationship.
The DC itself is powered by a gas-piston installation, and urban electricity is only a reserve. According to Maxim, it was a very correct decision, because during the well-known Petersburg blackout, the office and the nearby business center suffered, and the data center continued its work.
The diesel generator is no longer a trailer on wheels, but a completely stationary room with chimneys billowing high above the landscape.
Startup time is no more than two minutes, and on batteries the entire data center can work for about an hour. Diesel can work 3 days without additional refueling.
Here in everything reigns order and thoughtfulness.
This, for example, is a key, where data center customers themselves can take keys from their racks by fingerprint.
And this is a room where customers can carry out any engineering work they need, for which a wire tool hangs on the wall.
These are already shelves for local operation with all vital in daily work:
It is always charged and ready to go.
The baseball cap of the beginner hiphopera in the photo above saved more than one person’s head. It is a solid natural helmet like a natural helmet - the very thing is to climb under a half meter raised floor, where no, no, yes, some sharp piece of iron sticks out.
Actually employees from operation:
This unremarkable device is a differential pressure meter.
It is designed to solve a simple problem - to maintain the pressure difference between the corridor and the engine room. For example, in order that it was inside a little higher and when you open the door, the dust did not fly inside, but instead flew away from the door.
The fire extinguishing system is also gas - in case of trouble, the gas displaces oxygen and stops burning.
Only serious and very serious guys who have reasons for paranoia in size to Ararat get up in the link.
Grid and separate from data center access control - this is just the beginning.
That's when two people come to solve a problem or switch to new commutations - one with a material key, the other with an electronic one - this is more interesting. And when an auditor in glasses comes from a client, descends under a raised floor, pushes two sheets of grids and says that a child’s head will crawl through here, you understand that a person has seen some interesting things in his life.
There were funny situations when, when designing a cell, they took into account everything that was possible, all security systems, all sizes that allow you to deliver a rack and install equipment in it, but somehow did not think about the dimensions of the engineers who would work with it. And they neither in the front nor in the profile do not fit between the cell and the stand.
If I were a FSB, I would put my iron in linx :)
After the tour we sat for quite a long time, mentally chatting under the pizza.
From this realm of security and order, we moved to RETN.
RETN
We are familiar with these guys for a long time and close. Therefore, it is not surprising that finally we gathered on their roofs to fry kebabs and poison bikes.
We have already been with them too, although, it seems, they didn’t tell. Last year, plans to seize the roofs of St. Petersburg failed because of Fifa.
RETN is actually the largest trunk operator with fiber all over the planet. Any spider will envy their web.
A DC on the bypass is a home project for the soul that has found its niche in a growing market. These are those who need a thick connection with the world and access to other local operators.
For example, in RETN is worth CDN Netflix'a.
I was surprised to learn that each of their nodes was individually signed by hand with a phrase.
And such holiday boxes are threshed in different cities on POPs RETN (and, obviously, not only him), delivering to you as soon as possible the “Black Mirror” and “Sharp Visors”.
There is also a Google CDN and a bunch of operators, both local and federal.
At RETN, two tiny (compared to Miranovsky) diesel generators and a car with cans stand next to the building to go to the gas station. It sounds almost like a family business.
Well, the evening talks on the roof brought us to the point that RETN would give us a virtual box for a backup site for hosting podcasts and pictures.
World guys, super-lamp data center, amazing network.
For some, this day ended with a duke nukem tournament, for someone a welcome sleep in a hotel.
The next day we didn’t have many plans: Metrotek and the podcast live - linkmeup IRL .
Metrotek
Just a month ago, Misha Sokolov and Denis Gabidullin from STC Metrotek Slash Plumspace came to visit a telecom podcast, and we analyzed what a Smart SFP is. And now we were able to touch them alive.
Denis, who conducted a tour of Metrotek St. Petersburg office (which by the way has not changed since our last trip to them), actually gave one of the samples from the threshold.
At first I was surprised how clumsy it was made - as if with a jigsaw and file it was turned from a 32 key.
It turned out that the case of the prototype was printed on a 3D printer made from metal powder, which can be seen on the rough surface. Others - those that are ready for the series - are in perfect order.
But in general, looking at them, you are amazed how they still managed to hide such a number of brains in an almost standard building.
I advise you to listen to the podcast about Rosa, Toza and their sister Boza . It turned out for my taste interesting.
Office design is the most common. Relatively small rectangular cabinets, elongated toward the window, are rather dark, which pressed a little. But the atmosphere, despite the weekend, reigned working.
Registration in two years has not changed. We were pleased with some posters and boards.
We walked through several rooms: R & D, systems engineers, FPGAs, held different types of SFPs in our hands, programmers, saw the first model in which Smart was rendered from SFP on the wire - one of the first experiments - rather, the Proof of Concept, that it generally works .
They say it is even convenient for some customers to have such a tail sticking out of their iron.
The very first goal two years ago was to assemble a conventional optical SFP that would at least transmit a signal.
Then, the Smart part was separately assembled with a foreign SFP (just on a string) to make sure that they were able to assemble the chips.
Metrotek turned huge work. Just think that for 2 years we have assembled a fully working optical module that can encapsulate, telemetry, an exact time server, an interface converter.
And at the same time debug to the ideal (although this has yet to be verified by time) the basic functionality of signal transmission. There should be no attenuation, there should not even be rare drops caused by SFP. And it should not warm up, despite the pack of ASICs, FPGA and CPU inside.
Bravo!
Full computing system:Denis noted that the company practices and encourages the exchange of knowledge and experience. And then I thought: “Is it different? 0_o. - Yes, often people come to them, who in the previous place of work cooked in their own juice, did not communicate, did double and triple work and generally avoided communication attempts.
- CPU: NXP i.MX6 ARM up to 900 MHz
- FPGA: Lattice ECP5 up to 85K LUTs
- RAM: DDR3 up to 1 GB
And then I remembered that, in general, no more than two years ago, I worked in a single international wide company where conferences were banned and knowledge exchange was absent as a class. And it was I who was one of the evangelists of weekly meetings to share experiences that seemed to save lonely engineers in remote regions.
In general, Metroteca regularly conducts internal trainings and lectures, why they are great.
And then we were taken to an amazing installation shop. How many times I have been in them, it seems that my childhood delight does not fade.
Metrotek has an equipment for printing on boards right in the office.
A stencil for applying solder pads, a SMD component printer (otherwise I just cannot name it), a small stove for baking boards and nearby workstations for manual installation.
To my surprise, I learned that microchips with hundreds of legs are also soldered by hand. Denis says that their guys just do it with amazing precision and speed, soldering the smallest details and sometimes it makes no sense to run Shaitan-car - a person will make it faster.
We have called together the Metrotek Eltex-Verilite production line, on the wave of a relatively recent tour to them.
In addition, the office has a 3d printer and a milling machine for domestic use, which, however, has already engraved medals for last year’s Chaos Construction .
From there we went to the stunningly ideological Krang Pizza, where the linkmeup was already swallowed by the mutant and pizza-burger in size and moved to the IQ option office, where they recorded the first release of IRL linkmeup .
The next day, we looked at the Mammoth Laser Center, where we had a technocratic ecstasy and a bit of inflammation of greed, but this will be a different story.
In conclusion, I would like to say that all the excursions and meetings turned out to be very different, both in scale of action and in focus.
Miran who hospitably sheltered our hosting, fast growing, but still friendly simple.
Lynx is a truly cool data center, with security at the forefront and experienced marketers.
RETN- young jolly guys “out of the garages,” who make a giant net and still a little TsODik and are not averse to making a fire on the roof.
Metrotek is a relatively small team of Russian developers of hardware and software, which makes almost a bunch of “measuring tools” more than competitive equipment, and is now entering a new market with an ambitious task.
And the Laser Center, which goes beyond the interests of the linkmeup, but is the object of our personal interests. He left a very pleasant impression and engraving of the microphone on the recorder.
Source: https://habr.com/ru/post/455788/
All Articles