As we did HD Wi-Fi in the stadium in Krasnodar on 34 thousand people
It may seem that Wi-Fi at the stadium is not needed at all. I came to be sick - so watch the game, but do not post a selfie and read the mail. But no, Wi-Fi is needed for two important reasons. First, with such a crowd of people, the cellular network channels will be extremely quickly dismantled, and most of the fans will remain for a time without a telephone. Secondly, in fact, the stadium owners really wanted to give a special mobile application for all fans.
Inside - layouts of players on the field, statistics on the match and players, plus streaming video from cameras, so that you can see close-ups on faces on your tablet or phone what is happening. Well, and immediately repeats important moments from different angles.
On the radio survey
')
The result of the examination of the membrane
For all this, you need fast Wi-Fi high density and with minimal interference phenomena. Ordinary hotspots here do not stick - you need a good examination of radio surveys, plus special equipment. The result is 150 tricky access points with beamforming after rather nontrivial work.
High density in relation to Wi-Fi is a rather vague and conventional concept. Very simplifying, this is a network condition where there are as many active users as they are unable to digest a wireless network, made according to the classic design (access points with omnidirectional antennas, comfortably working with a maximum of 20-30 clients on the radio interface and a coverage radius of 15 meters typical office). Accordingly, a special coverage is required (so that subscribers and points do not interfere with each other), and a large number of customers impose bandwidth requirements on all network elements: the controller, switches, routers, firewalls and, in fact, Internet access channels.
When creating a wireless network for facilities with a high subscriber density, which is a stadium, you need to consider the following factors:
Wireless network, built on the traditional design, can not cope with the tasks. As a result, there would be large packet loss, a large number of retransmitts, and, as a result, an extremely low data transfer rate, unstable service operation.
Reducing the coverage of access points is achieved by:
This is not all. More required:
At the Krasnodar FC stadium, we used the Cisco Wi-Fi solution, which includes the new Cisco narrow-beam antennas, high-performance dual-band Cisco access points in conjunction with software algorithms to automatically tune the wireless network depending on the constantly changing radio frequency environment, and CleanAir technology, which includes Access points with built-in intelligent hardware features that can detect, classify, and reduce the effects of radio frequency interference. To provide monitoring and network management, the solution includes Cisco WLC wireless multifunction controllers and Cisco Prime Infrastructure infrastructure management software.
Cisco Antenna:
Narrow-directional sector antenna with MIMO support. It has 6 elements, 3 each for operation in 2.4 and 5 GHz. The beam width of the DND in the 2.4 GHz band is 36 ° in the horizontal plane and 36 ° in the vertical plane, and in the 5 GHz range it is 55 ° and 48 °, respectively. The case of the antenna provides protection from external influence of dust and moisture according to the class IP65. Designed specifically for use with a Cisco access point in a high-density environment. It is used to cover the stands of the stadium.
Cisco Antenna:
Sector antenna with MIMO support. It has 4 elements simultaneously operating in 2.4 and 5 GHz. The width of the beam in the 2.4 GHz band is 65 ° in the horizontal plane and 105 ° in the vertical plane, and in the 5 GHz range it is 55 ° and 110 °, respectively. The case of the antenna provides protection from external influence of dust and moisture in class IP54. Used in conjunction with a Cisco access point. It is used to cover stadium stands, VIP boxes, press zones, zones close to the entrance to the field and rooms with high subscriber density.
Cisco access point:
"Stadium" dual-band access point, specially designed for use in areas of high concentration of users. The access point is designed and certified to work with narrowly directed antennas that have a gain of up to 13 dB (2.4 GHz) and 7 dB (5 GHz), such as the Cisco antenna described above, for example. It has advanced settings for the radio transmitter, which is indispensable for such installations. Equipped with a built-in spectrum analyzer (CleanAir technology), which allows you to detect and avoid radio frequency interference of any origin. Supports IEEE 802.11n standard and 2x3 MIMO configuration with two spatial streams.
Cisco access point:
Dual-band access point with IEEE 802.11ac standard support and 4x4 MIMO configuration with three spatial streams. Equipped with a built-in spectrum analyzer (CleanAir technology), which allows you to detect and avoid radio frequency interference of any origin. It is used together with the sector antenna for the first rows of the stands of the stadium, VIP lodges, press zones, zones close to the entrance to the field and rooms with high subscriber density, as well as in zones with low subscriber density together with omnidirectional dipole antennas.
Cisco Wireless Controller:
Centralized control of access points is provided by the high-performance Cisco WLC. This controller model supports simultaneous control of up to 6000 access points. Provides bandwidth of 10 Gbit / s and supports simultaneous operation of up to 64,000 users, which is indispensable for such installations. System fault tolerance is achieved by combining two controllers into a cluster. Switching time from active to standby controller is less than a second without interrupting client sessions.
The wireless network controller is responsible for such functions as ensuring secure seamless roaming, automated radio control (controlling the carrier frequency and power of radio transmitters of access points), eliminating coverage “holes” in the access points failure, switching users from loaded to less loaded ones, detecting and avoiding the mutual influence of wireless devices and interference, detecting intrusions into a wireless network, ensuring security policies and QoS policies.
We were lucky, and the customer turned at the moment when the concrete was not yet poured, and the stadium was only designed. This meant that we could pre-lay places for equipment, cable routes and other infrastructure. This is important, because besides the Wi-Fi we did IP TV and Digital Signage - in other words, huge screens with what is happening on the field.
In total, the stadium needed to combine about 50 subsystems. For my department it all started with the design of our infrastructure. The problem is that the exact location of the radio equipment can be identified only after understanding what form and what kind of material the stadium will be. It is like a sound - you need to measure it in order to get an accurate picture of the wave propagation. Accordingly, in the project we thought that we could have fixed (in particular, it was especially good that we managed to lay the tracks and these special slots for equipment in the photo below). At this stage, there is no understanding, for example, who will do roofing at all, so there is little accurate data.
When designing, we coordinated the installation sites for a long time: for example, in the aisles where people walk, you cannot install equipment that will bulge. These are the requirements of firefighters. We offered options with shtrobleny and holes in reinforced concrete structures, but then the operation decided that the object will begin to collapse ahead of time from such holes. Also on this site the architect had aesthetic requirements - and the project laid more strength. The combination of aesthetics and increased strength allowed us to convince the operation, and we designed the grooves for flush mounting of antennas and access points in thermo boxes.
The next stage is the stage of working documentation. When it became clear who made what kind of system, and it was approximately clear what solutions would be applied (first the concrete and structures were worked through, after which the working documentation on engineering and automated systems was prepared). The peculiarity of the stage is that there are changes at the construction site and sometimes it is necessary to make revisions to the working documentation. The main task is to provide everything you need. That there was no such "and here we are demolishing the partition to hell." It is important to think in advance also because the project must be within budget. Here the most important part for us is to conduct a radio survey “live” when the concrete is only laid. That is, here in this state of the stadium:
On the radio survey
The next stage of SMR is the funniest part, where you have to argue a lot and defend decisions with contractors on adjacent systems. You also have to cooperate with contractors, who, say, make the roof, on which it is necessary to provide fasteners, which should allow easy operation of the system. We repeatedly tested this stage at large sites, especially we received a lot of experience from the stadiums - we also made the IT infrastructure and sound of the CSKA and Spartak stadiums (30 and 44 thousand places, respectively).
Here, as usual, there are a lot of contractors, and no one wants to wait for the others to work. It also touched us - one contractor laid the stove without our thermoboxes, and we had to look for new thermoboxes for installation. We have added a new rule - to conduct purchases after creating a test node, additional components may be needed. It doesn't matter how well the site was designed by all organizations.
The main task is to evenly cover the bowl, taking into account that users will sit very tightly. To increase the density of coverage, antennas with a small coverage area are used, due to which more access points are installed without negative interference.
For this you need:
I remind you that we did it when only part of the structures were ready:
That's right, on the scotch test
It was not possible to inspect the upper platform because of the absence of a winch and a crane. More precisely, they were, but could not get under the roof. I had to conduct an examination on the lower stand and make a projection on the upper stand.
It was very difficult to determine the distance from the antenna on the crane to the stands in the vertical. We measured the distance with a laser. The radio operator did not have a walkie-talkie, I had to shout and use gestures. The process was a long one. We moved the cradle of the crane during the day to examine a quarter, after we projected the results onto the upper tribune.
Measuring equipment is mounted on the cradle, and then ...
She rises ...
And depicts a piece of roofing.
When the roof was partially ready, they summoned the spiers to check the result. For them, we conducted a briefing, gave them radios. In general, they were quickly taught to do a procedure that they could not do themselves without tolerances. Thus, we checked the results of the radio survey and the transfer of the projection. As we understood that if the slope of the tribune is the same as the top and bottom, the measurement results will coincide - and this theory was confirmed.
Such schemes turned out here:
Tribunes
Membrane
Sectional view
Cisco Wi-Fi is used not only in Krasnodar. These systems are supplied by American stadiums. Specifically, on the technical side accumulated experience is. It was much more interesting for the customer to look at the financial and practical results of the implementation of such an infrastructure. Probably one of the best examples is the implementation of the Connected Stadium system at the CONSOL Energy Center (for the Pittsburgh Penguins NHL football team). There was also HD Wi-Fi, a lot of screens like this:
And polycars visible from the stands. So, the number of advertisers has doubled, and sponsorship revenue has tripled. Stadium brand has become more recognizable and memorable. 80% of viewers saved the content that they watched at the stadium, relaying it to social networks.
Stadium completed. According to our subsystem - now we are implementing a system for broadcasting video traffic via Wi-Fi for a special mobile application. The peculiarity is that the data is transmitted to the devices using multicast. The use of streaming video broadcasting via multicast mailing significantly reduces the load on communication channels, radio environment resources and network equipment resources. This effect is achieved due to the fact that the data from the source is not transmitted individually to each recipient (unicast), which breeds copies of uniform traffic, but that the source sends a single copy of the data along a common route to those recipients who have subscribed to the newsletter. The advantage of this approach: the addition of new users does not entail the need to increase network bandwidth along a common route to service consumers.Accordingly, the load on the intermediate equipment is also reduced.
I will answer questions in the comments or by mail 2710@croc.ru
Inside - layouts of players on the field, statistics on the match and players, plus streaming video from cameras, so that you can see close-ups on faces on your tablet or phone what is happening. Well, and immediately repeats important moments from different angles.
On the radio survey
')
The result of the examination of the membrane
For all this, you need fast Wi-Fi high density and with minimal interference phenomena. Ordinary hotspots here do not stick - you need a good examination of radio surveys, plus special equipment. The result is 150 tricky access points with beamforming after rather nontrivial work.
Why HD, and what is the difficulty?
High density in relation to Wi-Fi is a rather vague and conventional concept. Very simplifying, this is a network condition where there are as many active users as they are unable to digest a wireless network, made according to the classic design (access points with omnidirectional antennas, comfortably working with a maximum of 20-30 clients on the radio interface and a coverage radius of 15 meters typical office). Accordingly, a special coverage is required (so that subscribers and points do not interfere with each other), and a large number of customers impose bandwidth requirements on all network elements: the controller, switches, routers, firewalls and, in fact, Internet access channels.
When creating a wireless network for facilities with a high subscriber density, which is a stadium, you need to consider the following factors:
- The total number of subscribers - the stadium, when fully loaded, holds 34,000 potential users. Accordingly, our network at the peak can be loaded with about 30 thousand connections to streaming video.
- The density of subscriber devices - the fans sitting in the stands are placed very tightly, on 10 square meters on average 60 people are accommodated. The key indicator by which a sufficiently suitable network architecture is tested is the required bandwidth per subscriber. Each access point has a certain band passing in the radio channel, evenly dividing between clients, and the radius of the coverage. The higher the density of clients, the more clients will connect to one access point, and the less bandwidth per client. Logical output is to put a greater number of points, but we are limited in free radio resources in the 2.4 GHz band (there are only 3 non-overlapping channels - 1, 6 and 11), in the 5 GHz band it is simpler - there are 11 channels, but not all mobile devices support 5 GHz. Therefore, accordingly, we cannot refuse 2.4. Accordingly, it is necessary to reduce the radius of the points or more precisely to form radiation patterns in order to reduce the negative impact on neighboring points operating at the same frequencies and optimally allocate radio resources.
- Characteristics of the implantation object that affect signal propagation — the stadium bowl is a large open space in which inevitable interference between access points;
Wireless network, built on the traditional design, can not cope with the tasks. As a result, there would be large packet loss, a large number of retransmitts, and, as a result, an extremely low data transfer rate, unstable service operation.
Reducing the coverage of access points is achieved by:
- the use of specialized narrow antenna (more on this below);
- shutdown of low channel speeds (up to 12 Mbit / s);
- shutdown of processing packets of subscribers with a low signal level (RX-SOP);
- lower transmitter power;
This is not all. More required:
- Protect the HD coverage area from a spurious Wi-Fi signal. Negative phenomena to cover the spectator area of the stadium bowl can provide my own points located in VIP-zones, lobbies near the exits to the pitch. The recipe is the same - control power and radio coverage.
- Reduce radio channel utilization: do not use more than four SSIDs (ideally, use one) in visual zones, since Each SSID requires a separate Beacon packet to be sent, and each broadcast SSID responds to a null probe request; to deal with customer-provoked interference, inviting viewers to connect to a Wi-Fi network, because the connected device starts to send the probe request many times less; remove extraneous access points and Ad Hoc devices as sources of interference and beacon generators, probe request, probe response packets and eliminate the use of "adjacent" Wi-Fi networks and use a single wireless network.
What equipment was used
At the Krasnodar FC stadium, we used the Cisco Wi-Fi solution, which includes the new Cisco narrow-beam antennas, high-performance dual-band Cisco access points in conjunction with software algorithms to automatically tune the wireless network depending on the constantly changing radio frequency environment, and CleanAir technology, which includes Access points with built-in intelligent hardware features that can detect, classify, and reduce the effects of radio frequency interference. To provide monitoring and network management, the solution includes Cisco WLC wireless multifunction controllers and Cisco Prime Infrastructure infrastructure management software.
Cisco Antenna:
Narrow-directional sector antenna with MIMO support. It has 6 elements, 3 each for operation in 2.4 and 5 GHz. The beam width of the DND in the 2.4 GHz band is 36 ° in the horizontal plane and 36 ° in the vertical plane, and in the 5 GHz range it is 55 ° and 48 °, respectively. The case of the antenna provides protection from external influence of dust and moisture according to the class IP65. Designed specifically for use with a Cisco access point in a high-density environment. It is used to cover the stands of the stadium.
Cisco Antenna:
Sector antenna with MIMO support. It has 4 elements simultaneously operating in 2.4 and 5 GHz. The width of the beam in the 2.4 GHz band is 65 ° in the horizontal plane and 105 ° in the vertical plane, and in the 5 GHz range it is 55 ° and 110 °, respectively. The case of the antenna provides protection from external influence of dust and moisture in class IP54. Used in conjunction with a Cisco access point. It is used to cover stadium stands, VIP boxes, press zones, zones close to the entrance to the field and rooms with high subscriber density.
Cisco access point:
"Stadium" dual-band access point, specially designed for use in areas of high concentration of users. The access point is designed and certified to work with narrowly directed antennas that have a gain of up to 13 dB (2.4 GHz) and 7 dB (5 GHz), such as the Cisco antenna described above, for example. It has advanced settings for the radio transmitter, which is indispensable for such installations. Equipped with a built-in spectrum analyzer (CleanAir technology), which allows you to detect and avoid radio frequency interference of any origin. Supports IEEE 802.11n standard and 2x3 MIMO configuration with two spatial streams.
Cisco access point:
Dual-band access point with IEEE 802.11ac standard support and 4x4 MIMO configuration with three spatial streams. Equipped with a built-in spectrum analyzer (CleanAir technology), which allows you to detect and avoid radio frequency interference of any origin. It is used together with the sector antenna for the first rows of the stands of the stadium, VIP lodges, press zones, zones close to the entrance to the field and rooms with high subscriber density, as well as in zones with low subscriber density together with omnidirectional dipole antennas.
Cisco Wireless Controller:
Centralized control of access points is provided by the high-performance Cisco WLC. This controller model supports simultaneous control of up to 6000 access points. Provides bandwidth of 10 Gbit / s and supports simultaneous operation of up to 64,000 users, which is indispensable for such installations. System fault tolerance is achieved by combining two controllers into a cluster. Switching time from active to standby controller is less than a second without interrupting client sessions.
The wireless network controller is responsible for such functions as ensuring secure seamless roaming, automated radio control (controlling the carrier frequency and power of radio transmitters of access points), eliminating coverage “holes” in the access points failure, switching users from loaded to less loaded ones, detecting and avoiding the mutual influence of wireless devices and interference, detecting intrusions into a wireless network, ensuring security policies and QoS policies.
Progress
We were lucky, and the customer turned at the moment when the concrete was not yet poured, and the stadium was only designed. This meant that we could pre-lay places for equipment, cable routes and other infrastructure. This is important, because besides the Wi-Fi we did IP TV and Digital Signage - in other words, huge screens with what is happening on the field.
In total, the stadium needed to combine about 50 subsystems. For my department it all started with the design of our infrastructure. The problem is that the exact location of the radio equipment can be identified only after understanding what form and what kind of material the stadium will be. It is like a sound - you need to measure it in order to get an accurate picture of the wave propagation. Accordingly, in the project we thought that we could have fixed (in particular, it was especially good that we managed to lay the tracks and these special slots for equipment in the photo below). At this stage, there is no understanding, for example, who will do roofing at all, so there is little accurate data.
When designing, we coordinated the installation sites for a long time: for example, in the aisles where people walk, you cannot install equipment that will bulge. These are the requirements of firefighters. We offered options with shtrobleny and holes in reinforced concrete structures, but then the operation decided that the object will begin to collapse ahead of time from such holes. Also on this site the architect had aesthetic requirements - and the project laid more strength. The combination of aesthetics and increased strength allowed us to convince the operation, and we designed the grooves for flush mounting of antennas and access points in thermo boxes.
The next stage is the stage of working documentation. When it became clear who made what kind of system, and it was approximately clear what solutions would be applied (first the concrete and structures were worked through, after which the working documentation on engineering and automated systems was prepared). The peculiarity of the stage is that there are changes at the construction site and sometimes it is necessary to make revisions to the working documentation. The main task is to provide everything you need. That there was no such "and here we are demolishing the partition to hell." It is important to think in advance also because the project must be within budget. Here the most important part for us is to conduct a radio survey “live” when the concrete is only laid. That is, here in this state of the stadium:
On the radio survey
The next stage of SMR is the funniest part, where you have to argue a lot and defend decisions with contractors on adjacent systems. You also have to cooperate with contractors, who, say, make the roof, on which it is necessary to provide fasteners, which should allow easy operation of the system. We repeatedly tested this stage at large sites, especially we received a lot of experience from the stadiums - we also made the IT infrastructure and sound of the CSKA and Spartak stadiums (30 and 44 thousand places, respectively).
Here, as usual, there are a lot of contractors, and no one wants to wait for the others to work. It also touched us - one contractor laid the stove without our thermoboxes, and we had to look for new thermoboxes for installation. We have added a new rule - to conduct purchases after creating a test node, additional components may be needed. It doesn't matter how well the site was designed by all organizations.
What is the complexity of the problem?
The main task is to evenly cover the bowl, taking into account that users will sit very tightly. To increase the density of coverage, antennas with a small coverage area are used, due to which more access points are installed without negative interference.
For this you need:
- Antennas with high gain at frequencies of both 2.4 and 5 GHz.
- Required - with multi-channel reception-transmission (MIMO) 4 * 4 to increase the capacity in the radio channel (but it happens that the receivers do not have MIMO or 1 * 1). The data transfer rate depends on the type of MIMO on the client side.
- Further, we believe that on this project the receiver (subscriber terminal, for example, a telephone) and the transmitter (our antenna) should be on the line of sight. It is necessary to create all the conditions for this to be done in most cases.
- In the future, it is good to use the 802.11ac standard, but not all user devices will support it. As well as not all user devices operate at 5 GHz. For the future, this is a very good solution - the 802.11ac standard will support MU-MIMO. Multi user MIMO, in contrast to the usual Single user MIMO, sends a frame to a set of recipients, not one. That is, an access point with 4 antennas can send one stream to 3 user devices at a time.
Radio survey
I remind you that we did it when only part of the structures were ready:
That's right, on the scotch test
It was not possible to inspect the upper platform because of the absence of a winch and a crane. More precisely, they were, but could not get under the roof. I had to conduct an examination on the lower stand and make a projection on the upper stand.
It was very difficult to determine the distance from the antenna on the crane to the stands in the vertical. We measured the distance with a laser. The radio operator did not have a walkie-talkie, I had to shout and use gestures. The process was a long one. We moved the cradle of the crane during the day to examine a quarter, after we projected the results onto the upper tribune.
Measuring equipment is mounted on the cradle, and then ...
She rises ...
And depicts a piece of roofing.
When the roof was partially ready, they summoned the spiers to check the result. For them, we conducted a briefing, gave them radios. In general, they were quickly taught to do a procedure that they could not do themselves without tolerances. Thus, we checked the results of the radio survey and the transfer of the projection. As we understood that if the slope of the tribune is the same as the top and bottom, the measurement results will coincide - and this theory was confirmed.
Such schemes turned out here:
Tribunes
Membrane
Sectional view
Foreign experience
Cisco Wi-Fi is used not only in Krasnodar. These systems are supplied by American stadiums. Specifically, on the technical side accumulated experience is. It was much more interesting for the customer to look at the financial and practical results of the implementation of such an infrastructure. Probably one of the best examples is the implementation of the Connected Stadium system at the CONSOL Energy Center (for the Pittsburgh Penguins NHL football team). There was also HD Wi-Fi, a lot of screens like this:
And polycars visible from the stands. So, the number of advertisers has doubled, and sponsorship revenue has tripled. Stadium brand has become more recognizable and memorable. 80% of viewers saved the content that they watched at the stadium, relaying it to social networks.
Next steps
Stadium completed. According to our subsystem - now we are implementing a system for broadcasting video traffic via Wi-Fi for a special mobile application. The peculiarity is that the data is transmitted to the devices using multicast. The use of streaming video broadcasting via multicast mailing significantly reduces the load on communication channels, radio environment resources and network equipment resources. This effect is achieved due to the fact that the data from the source is not transmitted individually to each recipient (unicast), which breeds copies of uniform traffic, but that the source sends a single copy of the data along a common route to those recipients who have subscribed to the newsletter. The advantage of this approach: the addition of new users does not entail the need to increase network bandwidth along a common route to service consumers.Accordingly, the load on the intermediate equipment is also reduced.
I will answer questions in the comments or by mail 2710@croc.ru
Source: https://habr.com/ru/post/269585/
All Articles