WI-FI in the subway: network architecture and underground stones

In just a couple of years, the Muscovite’s trip in the metro has ceased to be a daily routine. Earlier, the only entertainment in the subway was reading books, the press and an MP3 player, now they have been added to online shopping, watching TV shows, business correspondence, even acquaintances in Tinder and quests. And all thanks to the appearance of a free Wi-Fi network in the metro. About 80% of Muscovites regularly connect to the MT_FREE network in the metro, without thinking about how it works and with what forces it is done. There is an opinion that Wi-Fi in the metro was “held” by the metro itself, but this is not quite true. A wireless network is a MaximTelecom project. For the company, this was the first experience of building a high-speed Wi-Fi network with engineering and technical solutions that are unique in world practice. In this post, we will describe how a Wi-Fi network is organized in the Moscow metro.

We actually have two networks ...

Radio network inside the cars

You enter the car, you see a sticker with the name of the network, or, out of habit, you turn on Wi-Fi on your phone. At the same time, the device connects to the network with the SSID MT_FREE. It is organized by high-density access points that are located in each car, operate in two 2.4 GHz and 5 GHz bands and support the 802.11a / b / g / n standards. They are controlled by the controller in the head car. As part of these cars, two, and therefore the controller, too, two. All equipment in the rolling stock, including between cars, connect cables - twisted pair.
')

Radio network train tunnel

After passing through the internal train network, data is transmitted to the fixed network infrastructure using a train-tunnel radio channel. It is installed between the base station located in each head car and the base stations located along the route of the rolling stock in the tunnel, as well as in open sections of the track. The location of base stations along the tracks is such that the train moves in a solid radio field. Due to this, communication breaks are minimal. Base stations on the train are located in the same way as the access point controllers on each head car, while only one of the stations is active during the movement of the train. The radio channel works in the same frequency range as Wi-Fi - 5 GHz, but uses a proprietary data transfer protocol. Unlike the equipment inside the train, the equipment of the radio channel train-tunnel can be seen outside the rolling stock and in the tunnels / on the open sections of the track.

Network architecture

Each base station on the train’s route is connected to the switching nodes located in the metro office premises using a dedicated fiber-optic network. Uninterrupted power supply of base stations is also organized with the help of equipment installed in these switching nodes.

The architecture of the fixed data network does not differ from the typical architecture of telecom operators. This is a “double star” with geographic redundancy of communication channels and key equipment. The network has several communication channels with backbone operators, with a total bandwidth of more than 60 Gbit / s.

Houston, we have problems ...

Technological complexity

To work in the underground you need equipment:

• can withstand harsh operating conditions in the tunnel (suspended metal dust and engine oil) and on rolling stock (extreme temperatures and vibrations);
• met the requirements of the metro (use non-combustible materials, comply with the requirements for electromagnetic compatibility, work from non-standard power sources);
• having the necessary network functionality.

In the metro in the rolling stock used direct current with a rated voltage of 80V. However, depending on the state of the batteries and the number of breaks in the contact rail, the actual voltage “jumps” from 30V to 150V .

We did not manage to find an acceptable power supply unit with such parameters, and the cost of suitable options made the project impractical.

Here, Sibcontact, a company from Novosibirsk, really helped us out. Under our requirements, colleagues made a power supply, which we successfully tested and later used in all compositions. The devices turned out to be very reliable, they were inexpensive, and the supplier managed to produce the required amount in a few weeks, not months, as it usually happens.

We also encountered a non-standard power supply in the tunnel - a two-phase network with a voltage of 127V. It is impossible to power the equipment operating from a single-phase 220V network from it, and we pulled new cables from our own power sources installed in the technical premises of the stations. This increased the reliability of the network, as we applied uninterruptible power supplies and automatic input reserve.



The variety of types of trains also caused great difficulties. This influenced the work on the design of equipment placement of local networks of trains - it was enormous. Secondly, during the construction it turned out that almost all the compositions, even of one series and year of manufacture, are different. This is due to the fact that they are constantly upgraded and installed additional equipment. Such work was carried out separately for each train, and each time we equipped the train in a unique way.

Serious questions have arisen in the radio network planning. They were associated with both the variety of materials from which the tunnels were made, and the lack of basic information on their design, geometry, as well as branches and obstacles.

We ourselves completely surveyed all the tunnels - in the Moscow metro there are more than 330 km of them, and in double-track terms more than 660 km. We applied certain metrics and rules for the placement of base stations, and after installing and launching the equipment during operation, we measured radio coverage and refined the optimal equipment placement points. Some base stations we had to move after installation.

These difficulties forced us, together with colleagues from the Nizhny Novgorod company Radio Gigabit, to conduct research and develop a unique radio planning method in tunnels, which is based on simulation (mathematical modeling) of the channel and system level of the radio network in tunnels and in open areas. In new projects, we are no longer guessing, but we know exactly how to arrange the equipment for obtaining the specified channel characteristics.

Architectural complexity

The main equipment forming the radio channel between the train and the tunnel is located in the “head” (remember that there are two of them), while the cars are constantly changing when arriving at the depot in rolling stock. The network operates in constant motion, as a result of which the network ports and physical devices, through which the traffic of the same sessions from the same composition, changes all the time. In this regard, we solved a number of architectural problems:

• fully automatic configuration of the train network when changing or changing the order of cars
• distribution of intra-car access points between two W-Fi controllers in the train
• correct receipt by users and equipment as part of IP addresses
• “Exit” of user traffic through the correct base station currently active
• skipping MAC addresses from one port of a fixed switch to another when a train is moving (this does not happen or happens very rarely on a fixed network), requiring constant “retraining” of network ports at the MAC level

A separate problem was the monitoring of our network. Conventional network equipment monitoring systems are unable to distinguish the situation in which the composition leaves the radio coverage area from the equipment failure of the composition. This leads to a large number of false alarms of the fault warning system. In addition, the unit of monitoring and maintenance is the train (since you always need to understand where a particular piece of equipment is located at the moment and how it is connected to other network elements), but in practice metro trains are dynamic, the composition of cars in which can be changed daily, or even twice a day. We have created our own monitoring tools that automatically detect the appearance of a train in the coverage area, “bypass” it, determining the composition and order of the cars and equipment installed in them and provide the network control center operators with data already in terms of the actual trains on the line.

These are the key technical tasks that MaksimaTelecom solved when planning and creating a network. Moreover, this process continues to this day, as the load on the network grows and new metro stations appear. Many of the lessons learned during the Moscow project, we applied in the construction of a Wi-Fi network in the St. Petersburg metro, thanks to which we managed to make it much more productive and fast. But we will tell about it in the following posts.