How to deploy HD Wi-Fi for 45 thousand fans in 10 days? Stadium "Spartacus" in anticipation of the 2018 World Cup
You can not just take and invite foreign football teams to play at the stadium as part of the World Cup. Is the stadium recently built and in compliance with international standards? This is not enough. You need to get certified FIFA. The International Football Federation, among other things, controls the free access of fans to the Internet. So, we had to deploy the network in the stands of the home stadium of Spartak Otkrytie Arena football club. In order to catch the matches of the Confederations Cup, which actually became a rehearsal for the World Cup, we had a little more than a month. And directly on the installation of access points and debugging the system left 10 days.
I will say in advance that in Russia then there had not yet been such large-scale projects of equipping Wi-Fi stadiums.
The stadium was built with private money and first of all had to meet the requirements of the Spartak football club: it was planned to hold matches of the Russian Championship and the Cup of Russia, the UEFA and the Champions League, as well as concert and entertainment events. I recall that we already wrote in detail on Habré how they built the infrastructure of this stadium.
')
For the Confederations Cup, and later the World Cup 2018, the Spartak stadium needed to be brought into compliance with all FIFA requirements. In particular, to organize separate premises for FIFA employees, a zone for media work (including changing the configuration of the media tribune - aka press box - for the required number of chairs), build temporary buildings and facilities for technical staff / broadcasters, and meet many other requirements.
As for the IT infrastructure, one of the requirements of FIFA is to provide Internet access for fans in all stands via Wi-Fi.
A source
In 2013, during the construction of the stadium's IT infrastructure, access to Wi-Fi was presented in the zones: VIP (Sky-boxes), press and restaurants. This Wi-Fi coverage was sufficient for the tasks existing at that time.
In May 2017, the stadium owner contacted us to organize Wi-Fi in all the stands. The first match at the Otkrytie Arena stadium was scheduled for June 18, and such a short time to complete the project might seem unrealistic. However, we had a serious theoretical background, as previously LANIT-Integration worked out a project for another stadium in detail.
According to the FIFA requirements, a Wi-Fi network in the stands must provide access to at least 15% of the total number of all viewers in both bands (2.4 and 5 GHz) and at least 2 Mb / s Internet access speed. Looking ahead to say that we focused on higher requirements. In addition to the requirements of FIFA, we had to comply with the requirements of our legislation as well: the identification of subscribers when connecting to a public Wi-Fi network.
Of the currently existing subscriber identification methods that comply with the requirements of the legislation of the Russian Federation, we have chosen the most popular and convenient - by cell phone number. It works as follows: the subscriber, after connecting to the Wi-Fi network, opens any http page, redirects it to the Web portal, in which he goes through the identification and authorization procedure: the user enters his phone number and confirms it with the code that is sent to mobile phone in sms.
Web portal with SMS authorization
During the construction of the stadium, the IT infrastructure was built on the equipment of Huawei.
This vendor was chosen for the construction of HD Wi-Fi on the eve of the 2018 World Cup. Thus, we were able to use the Internet access points established in 2013, and saved the customer’s investment.
For Huawei, our project has become the first case in Russia. In principle, the creation of a network of high-density Wi-Fi can be considered the most difficult task for equipment manufacturers.
We had to choose the model of the access point, the place of their installation, the type of antennas and the ways of fixing all this stuff. The choice fell on the AP8030DN, since this model is, firstly, high-performance, supports the IEEE 802.11ac Wave1 standard and works in both frequency bands (2.4 and 5 GHz). Secondly, it has a built-in directional sector antenna, which is important when covering areas with high subscriber density. Thirdly, the model can work in an extended temperature range and maintains a high IP67 protection class, which is mandatory when mounted on a stadium bowl.
To increase the number of serviced Wi-Fi subscribers, increase productivity and throughput, and to support the further scalability of the Wi-Fi network, we decided to replace the Wi-Fi controller used at the stadium with a fault-tolerant solution from two ACU2 controllers. This model is designed to deploy high-density Wi-Fi and supports up to 2048 access points and 32,000 Wi-Fi users.
These controllers are a line card that is installed directly into existing stadium core switches. To the cluster of two new controllers, we planned to switch all existing customer access points installed in the VIP-bed, under the tribunes, including food courts and halls.
By the way, it is worth noting the efficiency of the partner - Huawei: they took into account our circumstances, produced and delivered all the equipment not for the standard 6-8 weeks, but for 3 weeks.
While the equipment was manufactured at the factory and flew to Moscow, we needed to prepare working documentation as quickly as possible and give the contractor a task to start work on laying the SCS to the places where Wi-Fi access points were installed, to prepare and install fasteners for them.
During the pre-project survey, we decided on the main concept - how we will build everything: we agreed with the customer on the installation sites for Wi-Fi access points, where, how and from where we will “light up” the stands, we conducted radio modeling and test radio surveys, which resulted in the required number of access points .
The basic concept assumed four levels (contours) of fastening of access points. The distance between adjacent access points of the same level is about 8 meters.
The fourth level (upper stands): mounting on supporting columns with an angle of inclination towards the fans, covering the upper part of the upper stands.
The third level (upper stands): installation on technological gangways from them, we cover the lower part of the upper stands.
The second level (lower stands): installation under the visor from the upper stands with an angle of inclination towards the fans, cover the upper part of the lower stands.
The first level (lower stands): installation on advertising banners located along the football field, we cover the lower part of the lower stands.
Some installation locations may seem unusual. For example, I do not know of such stadiums, where access points would be attached to advertising banners. But this, by the way, is a worthy decision, considering how difficult it is to find an optimal place in a stadium that has been built and used for a long time.
As soon as we placed an order with the vendor and started preparing the working documentation, at the stage of its coordination with the customer it became clear that a part of our concept needs to be changed.
First it turned out that advertising banners during the Confederations Cup will be removed and replaced by others that are definitely not suitable for us. Then we did not manage to coordinate the placement of all-weather outdoor cabinets on technological gangways, the switches in which were necessary for points of the third circuit as “air”, since the lengths of SCS cables to the nearest possible cross-overs were much higher than 100 m. So we “lost” the supposed first and third circuits .
I had to look for new installation sites for access points that would provide the same optimal coverage and anti-vandal protection, which is important in the stadium, where, after the derby, the access point can be carried away as a “trophy”.
Taking into account all the remarks, the concept was refined: the points of the first and third contours moved to the facades of the stands, i.e. access points of the first and second, third and fourth circuits should “shine” on each other. According to calculations, one access point was supposed to cover a cell / cell for about 170 places and, if we take into account the fact that about 15% of fans should connect to the access point, then about 30 subscribers for an access point of this class were not the limit.
Four contours of installation of access points (visual presentation of Huawei)
In total, it turned out about 250 Wi-Fi access points. Taking into account the already existing access points, the total number was 350.
It was necessary to conduct power to the access points and connect them to the switches. There were no difficulties with the three lower contours - we used the cross rooms of the stadium. For the upper contour it was necessary to come up with some other solution. And it was found. We decided to install six weatherproof outdoor cabinets in which the temperature required for the operation of the equipment and the level of humidity are maintained.
Finally, after all our approvals, the contractor was able to proceed to laying the SCS and preparing installation sites for the delivery of access points. Four teams worked around the clock in three shifts. Before the arrival and arrival of the equipment, there was not much time left, and we needed to prepare for its meeting: to make a radio frequency plan, settings templates, to deploy and test the SMS authorization system with the controller existing at the customer.
Additional bracket for access points on the facades
After the equipment flew to Moscow and passed all the necessary registration procedures, we decided to immediately take it to the stadium so as not to waste precious time, because we had only 10 days to install the access points, start the system and test it.
When the stadium was being prepared for the Confederations Cup, a huge number of contractors for various systems were present on it every day, as if the stadium was being built anew.
Access to the stadium was carried out under strict control, as if it were the “clean zone” of the airport. All contractors were given their territory for the placement / storage of their working equipment and equipment outside the stadium. We were “lucky”: our playground was right next to the stadium, but it was possible to get to the entrance groups only by passing all the barriers across the field. And this is three kilometers!
Only the entrance control and preparation of the brought equipment for installation took us two days, including re-transportation from the wagons to the bowl itself. There were always problems with equipment being carried to the stadium: every thing was checked through the introscope, then it would be forbidden to enter the stadium at all.
Preparing installation access points
Finally, the equipment got into the arena, and we set about installing and configuring it: some engineers took up the launch of switches, and the other - “fine-tuning” access points. A team of installers was allocated to each level (contour) of mounting the access points, and as soon as the access point was mounted and connected to the network, the engineer selected the optimal angle of inclination.
Access points ready for installation
One of the unpleasant features of the deployment of high-density Wi-Fi networks is a lot of work with each access point: you need to tune in the radio broadcasting — manually adjust the channels and power at each access point, set the required tilt angle and constantly conduct radio surveys.
The problem is that you can see the real picture of signal propagation only in a filled stadium, since the radio broadcast of an empty and full stadium is dramatically different. Each smartphone, regardless of the operating system installed, polls Wi-Fi networks stored in it approximately every 30 seconds.
Now imagine: 45,000 people in one place, and the telephone of each of the spectators broadcasts something every 30 seconds. This creates an incredible level of noise in the radio environment, which entails completely different requirements for the signal level in such networks. During a football match, did you try to talk to your neighbor, sitting 2 places from you? So that he at least heard something, you have to shout. I think the analogy with the Wi-Fi signal is clear.
Another problem - the signal fades when passing through the audience. Therefore, ideally, the antenna should be located in front of the viewer so that there are no obstacles between the end user and the antenna. Unfortunately, this is not always possible due to the design features of the stadium, and you have to put points where you can fix them and where you can reach the necessary SCS. Such an arrangement is often not very efficient. For example, the points at the bottom of the stands, “looking” at the spectators from the bottom up, during the full house only 8-10 rows of spectators break through. And then the signal becomes too weak.
Separately, there was a problem of the frequency range: since in the 2.4 GHz band the signal propagates further than 5 GHz and the band has only 3 non-overlapping channels, in order to avoid interference, we had to turn off the 2.4 GHz band at access points through one. Of course, we had to tinker with power for a long time, but after a few matches we selected the optimal EIRP for each sector.
Mounted access points
Since this was the Confederations Cup under the auspices of FIFA, we had to coordinate and share radio frequency channels with various new Wi-Fi segments. And some of the access points had to be disconnected because of the temporary Wi-Fi segment from FIFA itself, which was deployed all over the western stand, where they provided access to the network not only by their employees, but also the entire press box.
Construction of HD Wi-Fi is in full swing: access points of the second circuit assembly
The subscriber's identification and authorization system was set up by our partner Netams, who deployed a self-developed boxed solution - WNAM. Before the arrival of new equipment at the stadium, their team managed to deploy their system and successfully test it with the Wi-Fi controller and access points existing at the time.
Colleagues, like us, had to face an interesting challenge: two days before the first match, the Organizing Committee gave a new introduction: to make a Web-portal page where the user goes through SMS authentication in the style of the Confederations Cup, and not just change a couple of pictures and text, but redo it completely (HTML-layout), including the logic of work. It was necessary to implement multilanguage support, new design of buttons and frames, auto-adjust to the size of the mobile device, work with all cellular operators in the world. But the partner successfully coped with this task.
Stadium waiting for the first match
The first match at the stadium was between the national teams of Chile and Cameroon. The stands were scored by 70%. Advertising on the work of Wi-Fi at the stadium the Organizing Committee did not give, which allowed us to test the new HD Wi-Fi without a large load.
In the first match there were about 1500 unique users. In subsequent matches, Wi-Fi was already advertised, and we were only engaged in selecting the optimal power of access points. The largest influx of users was in the last match for the third place: almost at the full stadium there were about 4000 unique users.
In conclusion, I would like to add that this was the first stadium in Russia already built and operating on such a scale that they launched HD Wi-Fi in 10 days. Now thousands of fans go online and share photos or videos on social networks right during the match.
PS We wrote this article together with my colleague ISkrypnik , who in this project helped me in developing the concept and in the technical press, so we will respond to your comments together.
I will say in advance that in Russia then there had not yet been such large-scale projects of equipping Wi-Fi stadiums.
Why did I have to retrofit the stadium for the World Cup
The stadium was built with private money and first of all had to meet the requirements of the Spartak football club: it was planned to hold matches of the Russian Championship and the Cup of Russia, the UEFA and the Champions League, as well as concert and entertainment events. I recall that we already wrote in detail on Habré how they built the infrastructure of this stadium.
')
For the Confederations Cup, and later the World Cup 2018, the Spartak stadium needed to be brought into compliance with all FIFA requirements. In particular, to organize separate premises for FIFA employees, a zone for media work (including changing the configuration of the media tribune - aka press box - for the required number of chairs), build temporary buildings and facilities for technical staff / broadcasters, and meet many other requirements.
As for the IT infrastructure, one of the requirements of FIFA is to provide Internet access for fans in all stands via Wi-Fi.
In 2013, during the construction of the stadium's IT infrastructure, access to Wi-Fi was presented in the zones: VIP (Sky-boxes), press and restaurants. This Wi-Fi coverage was sufficient for the tasks existing at that time.
In May 2017, the stadium owner contacted us to organize Wi-Fi in all the stands. The first match at the Otkrytie Arena stadium was scheduled for June 18, and such a short time to complete the project might seem unrealistic. However, we had a serious theoretical background, as previously LANIT-Integration worked out a project for another stadium in detail.
Design
According to the FIFA requirements, a Wi-Fi network in the stands must provide access to at least 15% of the total number of all viewers in both bands (2.4 and 5 GHz) and at least 2 Mb / s Internet access speed. Looking ahead to say that we focused on higher requirements. In addition to the requirements of FIFA, we had to comply with the requirements of our legislation as well: the identification of subscribers when connecting to a public Wi-Fi network.
Of the currently existing subscriber identification methods that comply with the requirements of the legislation of the Russian Federation, we have chosen the most popular and convenient - by cell phone number. It works as follows: the subscriber, after connecting to the Wi-Fi network, opens any http page, redirects it to the Web portal, in which he goes through the identification and authorization procedure: the user enters his phone number and confirms it with the code that is sent to mobile phone in sms.
During the construction of the stadium, the IT infrastructure was built on the equipment of Huawei.
This vendor was chosen for the construction of HD Wi-Fi on the eve of the 2018 World Cup. Thus, we were able to use the Internet access points established in 2013, and saved the customer’s investment.
For Huawei, our project has become the first case in Russia. In principle, the creation of a network of high-density Wi-Fi can be considered the most difficult task for equipment manufacturers.
We had to choose the model of the access point, the place of their installation, the type of antennas and the ways of fixing all this stuff. The choice fell on the AP8030DN, since this model is, firstly, high-performance, supports the IEEE 802.11ac Wave1 standard and works in both frequency bands (2.4 and 5 GHz). Secondly, it has a built-in directional sector antenna, which is important when covering areas with high subscriber density. Thirdly, the model can work in an extended temperature range and maintains a high IP67 protection class, which is mandatory when mounted on a stadium bowl.
To increase the number of serviced Wi-Fi subscribers, increase productivity and throughput, and to support the further scalability of the Wi-Fi network, we decided to replace the Wi-Fi controller used at the stadium with a fault-tolerant solution from two ACU2 controllers. This model is designed to deploy high-density Wi-Fi and supports up to 2048 access points and 32,000 Wi-Fi users.
These controllers are a line card that is installed directly into existing stadium core switches. To the cluster of two new controllers, we planned to switch all existing customer access points installed in the VIP-bed, under the tribunes, including food courts and halls.
By the way, it is worth noting the efficiency of the partner - Huawei: they took into account our circumstances, produced and delivered all the equipment not for the standard 6-8 weeks, but for 3 weeks.
While the equipment was manufactured at the factory and flew to Moscow, we needed to prepare working documentation as quickly as possible and give the contractor a task to start work on laying the SCS to the places where Wi-Fi access points were installed, to prepare and install fasteners for them.
During the pre-project survey, we decided on the main concept - how we will build everything: we agreed with the customer on the installation sites for Wi-Fi access points, where, how and from where we will “light up” the stands, we conducted radio modeling and test radio surveys, which resulted in the required number of access points .
The basic concept assumed four levels (contours) of fastening of access points. The distance between adjacent access points of the same level is about 8 meters.
The fourth level (upper stands): mounting on supporting columns with an angle of inclination towards the fans, covering the upper part of the upper stands.
The third level (upper stands): installation on technological gangways from them, we cover the lower part of the upper stands.
The second level (lower stands): installation under the visor from the upper stands with an angle of inclination towards the fans, cover the upper part of the lower stands.
The first level (lower stands): installation on advertising banners located along the football field, we cover the lower part of the lower stands.
Some installation locations may seem unusual. For example, I do not know of such stadiums, where access points would be attached to advertising banners. But this, by the way, is a worthy decision, considering how difficult it is to find an optimal place in a stadium that has been built and used for a long time.
First problems
As soon as we placed an order with the vendor and started preparing the working documentation, at the stage of its coordination with the customer it became clear that a part of our concept needs to be changed.
First it turned out that advertising banners during the Confederations Cup will be removed and replaced by others that are definitely not suitable for us. Then we did not manage to coordinate the placement of all-weather outdoor cabinets on technological gangways, the switches in which were necessary for points of the third circuit as “air”, since the lengths of SCS cables to the nearest possible cross-overs were much higher than 100 m. So we “lost” the supposed first and third circuits .
I had to look for new installation sites for access points that would provide the same optimal coverage and anti-vandal protection, which is important in the stadium, where, after the derby, the access point can be carried away as a “trophy”.
Taking into account all the remarks, the concept was refined: the points of the first and third contours moved to the facades of the stands, i.e. access points of the first and second, third and fourth circuits should “shine” on each other. According to calculations, one access point was supposed to cover a cell / cell for about 170 places and, if we take into account the fact that about 15% of fans should connect to the access point, then about 30 subscribers for an access point of this class were not the limit.
In total, it turned out about 250 Wi-Fi access points. Taking into account the already existing access points, the total number was 350.
It was necessary to conduct power to the access points and connect them to the switches. There were no difficulties with the three lower contours - we used the cross rooms of the stadium. For the upper contour it was necessary to come up with some other solution. And it was found. We decided to install six weatherproof outdoor cabinets in which the temperature required for the operation of the equipment and the level of humidity are maintained.
Finally, after all our approvals, the contractor was able to proceed to laying the SCS and preparing installation sites for the delivery of access points. Four teams worked around the clock in three shifts. Before the arrival and arrival of the equipment, there was not much time left, and we needed to prepare for its meeting: to make a radio frequency plan, settings templates, to deploy and test the SMS authorization system with the controller existing at the customer.
Ready Steady Go
After the equipment flew to Moscow and passed all the necessary registration procedures, we decided to immediately take it to the stadium so as not to waste precious time, because we had only 10 days to install the access points, start the system and test it.
When the stadium was being prepared for the Confederations Cup, a huge number of contractors for various systems were present on it every day, as if the stadium was being built anew.
Access to the stadium was carried out under strict control, as if it were the “clean zone” of the airport. All contractors were given their territory for the placement / storage of their working equipment and equipment outside the stadium. We were “lucky”: our playground was right next to the stadium, but it was possible to get to the entrance groups only by passing all the barriers across the field. And this is three kilometers!
Only the entrance control and preparation of the brought equipment for installation took us two days, including re-transportation from the wagons to the bowl itself. There were always problems with equipment being carried to the stadium: every thing was checked through the introscope, then it would be forbidden to enter the stadium at all.
Finally, the equipment got into the arena, and we set about installing and configuring it: some engineers took up the launch of switches, and the other - “fine-tuning” access points. A team of installers was allocated to each level (contour) of mounting the access points, and as soon as the access point was mounted and connected to the network, the engineer selected the optimal angle of inclination.
One of the unpleasant features of the deployment of high-density Wi-Fi networks is a lot of work with each access point: you need to tune in the radio broadcasting — manually adjust the channels and power at each access point, set the required tilt angle and constantly conduct radio surveys.
The problem is that you can see the real picture of signal propagation only in a filled stadium, since the radio broadcast of an empty and full stadium is dramatically different. Each smartphone, regardless of the operating system installed, polls Wi-Fi networks stored in it approximately every 30 seconds.
Now imagine: 45,000 people in one place, and the telephone of each of the spectators broadcasts something every 30 seconds. This creates an incredible level of noise in the radio environment, which entails completely different requirements for the signal level in such networks. During a football match, did you try to talk to your neighbor, sitting 2 places from you? So that he at least heard something, you have to shout. I think the analogy with the Wi-Fi signal is clear.
Another problem - the signal fades when passing through the audience. Therefore, ideally, the antenna should be located in front of the viewer so that there are no obstacles between the end user and the antenna. Unfortunately, this is not always possible due to the design features of the stadium, and you have to put points where you can fix them and where you can reach the necessary SCS. Such an arrangement is often not very efficient. For example, the points at the bottom of the stands, “looking” at the spectators from the bottom up, during the full house only 8-10 rows of spectators break through. And then the signal becomes too weak.
Separately, there was a problem of the frequency range: since in the 2.4 GHz band the signal propagates further than 5 GHz and the band has only 3 non-overlapping channels, in order to avoid interference, we had to turn off the 2.4 GHz band at access points through one. Of course, we had to tinker with power for a long time, but after a few matches we selected the optimal EIRP for each sector.
Since this was the Confederations Cup under the auspices of FIFA, we had to coordinate and share radio frequency channels with various new Wi-Fi segments. And some of the access points had to be disconnected because of the temporary Wi-Fi segment from FIFA itself, which was deployed all over the western stand, where they provided access to the network not only by their employees, but also the entire press box.
The subscriber's identification and authorization system was set up by our partner Netams, who deployed a self-developed boxed solution - WNAM. Before the arrival of new equipment at the stadium, their team managed to deploy their system and successfully test it with the Wi-Fi controller and access points existing at the time.
Colleagues, like us, had to face an interesting challenge: two days before the first match, the Organizing Committee gave a new introduction: to make a Web-portal page where the user goes through SMS authentication in the style of the Confederations Cup, and not just change a couple of pictures and text, but redo it completely (HTML-layout), including the logic of work. It was necessary to implement multilanguage support, new design of buttons and frames, auto-adjust to the size of the mobile device, work with all cellular operators in the world. But the partner successfully coped with this task.
Works?
The first match at the stadium was between the national teams of Chile and Cameroon. The stands were scored by 70%. Advertising on the work of Wi-Fi at the stadium the Organizing Committee did not give, which allowed us to test the new HD Wi-Fi without a large load.
In the first match there were about 1500 unique users. In subsequent matches, Wi-Fi was already advertised, and we were only engaged in selecting the optimal power of access points. The largest influx of users was in the last match for the third place: almost at the full stadium there were about 4000 unique users.
In conclusion, I would like to add that this was the first stadium in Russia already built and operating on such a scale that they launched HD Wi-Fi in 10 days. Now thousands of fans go online and share photos or videos on social networks right during the match.
And here is the summary: our project in numbers
- The entire project of deploying high-density Wi-Fi at the Otkrytie Arena stadium took 5 weeks.
- It took 10 days to develop a radio coverage plan.
- 3.5 weeks after the start of the project, the equipment was delivered by plane to Moscow.
- In the stadium bowl there are 250 Wi-Fi access points.
- In total, the stadium has about 350 Wi-Fi access points.
- The high-density Wi-Fi system at the stadium is able to provide Internet access to 8 thousand visitors at a time.
PS We wrote this article together with my colleague ISkrypnik , who in this project helped me in developing the concept and in the technical press, so we will respond to your comments together.
By the way, we have open vacancies.
Source: https://habr.com/ru/post/348916/
All Articles