Study of the sustainability of national segments of the Internet for 2018

This study explains how the failure of one autonomous system (AS) affects the global connectivity of a particular region, especially when it comes to the country's largest Internet provider (ISP). The connectivity of the Internet at the network level is due to the interaction between autonomous systems. As the number of alternative routes increases between ASs, resistance to failures arises and the stability of the Internet in a given country increases. However, some paths become more important than others and the availability of as many alternative routes as possible is the only viable way to ensure the reliability of the system (in the sense of AS).

The global connectivity of any AS, regardless of whether it represents a secondary Internet provider or an international giant with millions of service consumers, depends on the quantity and quality of its paths to Tier-1 providers. As a rule, Tier-1 implies an international company offering a global IP transit service and connecting to other Tier-1 operators. However, within this elite club there is no obligation to maintain such a connection. Only the market can motivate such companies to unconditionally connect with each other, providing high quality service. Is this a sufficient incentive? We will answer this question below - in the section on IPv6 connectivity.
')
If an Internet provider loses connection with at least one of its own Tier-1 connections, it will most likely be unavailable in some parts of the Earth.

Short facts TL; DR :
Romania and Luxembourg came out of the top 20 from 11th and 20th place, respectively, at the end of 2017;
Singapore jumped 18 places to 5th position;
Hong Kong fell 13 places to 15 position;
The Netherlands entered the top 20 in the 17th position;
18 of the 20 countries remained in the top 20 compared with last year.

Measuring Internet reliability

Imagine that the AS is experiencing significant network degradation. We are looking for the answer to the following question: “What percentage of ASs in this region can lose touch with Tier-1 operators, thereby losing global accessibility?”

Why simulate a similar situation? Strictly speaking, when BGP and the world of interdomain routing were at the design stage, the creators assumed that each non-transit AS would have at least two upstream vendors (upstream) to ensure fault tolerance in case one of them falls. However, in reality, everything is completely different - more than 45% of ISPs have only one connection to the upstream transit. A set of unconventional relationships between transit ISPs further reduces overall reliability. So do transit ISPs fall? The answer is yes, and this happens quite often. The correct question in this case is: “When will a particular ISP experience a degradation of connectivity?”. If such problems seem remote to someone, it is worth remembering Murphy's law: “Everything that can go wrong will go wrong.”

To simulate a similar scenario, we use the same model for the third year in a row. In the same year, we didn’t just repeat the previous calculations - we significantly expanded the field of study. The following steps were completed to assess the reliability of the AS:

For each AS in the world, we obtain all alternative paths to Tier-1 operators using the AS relationship model, which serves as the core of the Qrator.Radar product;
Using the IPIP geodatabase, we compared countries with the provided address of each AS;
For each AS, we calculated the share of its address space corresponding to the selected region. This helped filter out situations where an Internet provider may be present at the exchange point in a particular country, but does not have a presence in the region as a whole. An illustrative example is Hong Kong, where hundreds of participants of the largest Asian Internet exchange HKIX exchange traffic, having a zero presence in the Internet segment of Hong Kong;
Having obtained clear results for AS in the region, we assess the impact of the possible failure of this AS on other AS and the countries in which they are represented;
In the end, for each country we found a specific AS, affecting the largest percentage of other AS in the region. Foreign AS are not considered.

IPv4 Reliability

Below you can see the top 20 countries in terms of reliability, in terms of resiliency in case of a single AS failure. In practice, this means that the country has good Internet connectivity, and the percentage reflects the share of AS, which will lose global connectivity in the event of failure of the largest AS.


While individual countries can move up and down by rating, the overall picture has not changed significantly since 2017. Last year, the average failure rate in the world was 41%, in 2018 it decreased by 3% to 38% with a small one. The number of countries that reduced their dependence on one AS to less than 10% (which is a sign of high resiliency) increased by one to 30.

Another noticeable change was a significant increase in Internet reliability in small countries in South Asia and Africa. These regions are still developing, but a significant improvement in diversity in the IP transit market is a sign of accelerated progress.

IPv6 game

It is believed that if the technology works well in IPv4, it can be easily ported to IPv6. This erroneous assumption may be a structural problem for the entire IPv6 development process.

Measuring global reliability between two protocol versions is not as easy as it may seem. To maintain global connectivity in IPv4, any and only path to the Tier-1 provider will be sufficient. But in IPv6 this may not be the case. Because of the ongoing peering wars between several top-level IPv6 providers, they are not all related to each other. At least two pairs of providers decided to break the peer-to-peer relationship (“de-peer”) in IPv6: Cogent (AS174) and Hurricane Electric (AS6939), Deutsche Telekom (AS3320) and Verizon US (AS701). These telecommunications companies may have a variety of reasons for their conflicts, but if the network is connected to only one side of them, it will not have full-fledged connectivity in IPv6. It also affects the reliability of an Internet provider with multiple upstream services - failure of one can lead to connectivity problems.

Wanting to fix these problems, we adjusted the measurement process to check if full IPv6 connectivity is maintained during a crash. In other words, to ensure full connectivity and the highest reliability, the path to Tier-1 operators must be constantly present. We also calculated the percentage of ASs in a country that have only partial connectivity in IPv6 due to peer-to-peer wars. Here are the results:



A general comparison of IPv4 and IPv6 in the case of a single failure shows that 86% of the national IPv4 segments have a much higher connectivity. An important discovery in the IPv6 world is that many Internet service providers do not have proper connectivity even under normal conditions - without any failures. For example, in the US, this refers to about 10% of all ASs that support IPv6, and in China, the situation is even worse, since China Telecom (AS4134) receives global connectivity from only one provider - Hurricane Electric.

As stated earlier, no one can force Tier-1 providers to connect with each other other than their customers. Recent data clearly shows that consumer demand is not a sufficient incentive to connect them to each other and achieve 100% network visibility. The only way to improve this situation seems to be appeals to the proper level of IPv6 service. The Qrator.Radar team is considering various options for how to make this information more apparent to every ISP in the world, thereby increasing community awareness on this issue.

As for partial connectivity, several countries “do not see” more than 20% of the IPv6 address space. It:



The finding of the two former island colonies, Saint-Martin and Saint-Barthélemy, explainable by looking at the overall IPv4 rating; microstates, in most cases, depend on a single Internet service provider. The remaining 9 states with values ​​above 20%, and in particular the United Arab Emirates with 25% of partially connected (respectively, without a full-fledged global IPv6 peering) networks, are surprising.

Broadband Internet and PTR Records

Repeating the question we asked ourselves since last year: “Is it true that the leading provider in the country always affects regional reliability more than everyone else or some other?”, We developed an additional metric for further study.

Perhaps the most significant (in terms of the client base) Internet provider in this area will not necessarily be the autonomous system that will become most important in ensuring global connectivity. An example can be quickly found in the topmost position in the IPv4 rating - Germany. Last year it was clear that German transit was at least 2.29% dependent on the provider Versatel. This year, Versatel was replaced by DTAG (Deutsche Telekom), which improved connectivity in the country by 0.03%.

But knowing about the position of DTAG did not let us sleep for a long time and we tried to develop a metric that would sometimes include invisible connections (ASN can change affiliation or geography, the number can be released) to show what really happens where we know for sure about the existence of a dominant local player. Russia with Rostelecom, the United States with omnipotent Comcast, who own the lion's share of the broadband user base in their own countries. So what happens in the region if such a provider suddenly fails?

After numerous experiments, we determined that the most accurate indicator of the actual value of the provider can be based on the analysis of PTR records. Typically, they are used for reverse DNS lookup: using the IP address, you can identify the associated hostname or domain name.

This means that the PTR can allow the measurement of specific equipment in the address space of an individual operator. Since we already know the fattest AS for every country in the world, we could count the PTR records in the networks of these suppliers, determining their share among all the PTR records of the region. You should immediately make a disclaimer: we counted ONLY PTR records and did not calculate the ratio of IP addresses without PTR records to IP addresses with PTR records.

So, then we talk exclusively about IP addresses with PTR records present. Creating them is not a general rule, so some providers enter PTR, while others do not.

We showed how many of these IP addresses with the specified PTR records will be disconnected in case of disconnection from / together with the largest (by PTR) autonomous system in the specified country. The figure reflects the percentage of all IP addresses with PTR support in the region.

Let's compare the 20 most stable countries in IPv4 with a PTR rating:



Obviously, the approach of examining PTR records gives completely different results. In most cases, not only the central AS in the region changes, but the percentage of instability for the specified AS is completely different. In all reliable, from the point of view of global availability, the number of IP-addresses with PTR support that will turn off due to the fall of AS is ten times higher.

This may mean that the leading national Internet provider always owns the end users. Thus, we must assume that this percentage is part of the user and customer base of the ISP that will be disabled (in case you cannot switch to an alternative provider) in the event of a failure. From this point of view, countries no longer seem as reliable as they appear in terms of transit. We leave the reader to possible conclusions from comparing the top 20 IPv44 with PTR rating values.

Detail changes in the regions

Last year, we clearly pointed out the significant influence of the AS174 operator Cogent. This year, given the static nature of the 90% rating year-on-year, when Cogent served regions such as France, the United Kingdom, the United States and Ireland, we see all these countries in all positions with the same values ​​depending on one upstream. However, changes occurred in the USA, where AS209, Centurylink, dislodged Cogent. In response, Cogent added Spain and Belgium to the list of countries where it is the dominant supplier of the compound.

Although this means that disabling Cogent represents a risk for several regions at once, it is also necessary to recognize that the market position of this particular provider is primarily the result of its good IPv4 connectivity. Despite the fact that Cogent has added two more countries to its strategic portfolio, the denial of service will not lead to complete inaccessibility in these highly diversified national segments of the Internet.

But the big news is precisely what happened in the United States. For two years in a row - 2016 and 2017 - we have defined AS174, Cogent, as decisive in this market. This is no longer the case - in 2018, AS 209 CenturyLink replaced it, sending the United States up three places to 7th place in the IPv4 rating.

Turning to the region of the former USSR, we see small changes compared to last year. Rostelecom remains the main Internet provider in Russia (AS 12389), whose market share is quite significant (reflected in PTR values). However, in 2018, the shutdown of Rostelecom will lead to a lack of global connectivity in only 5.27% of autonomous systems in the country, which puts Russia in 13th place in terms of reliability.

As in the past year, the transit market in Russia is represented mainly by medium Tier-2 networks, which leads to high availability values. The same cannot be said about Uzbekistan, where for the third year in a row we observe 99.9% dependence on one supplier (AS 28910). In Turkmenistan, there is only one upstream supplier: Rostelecom. The Tajikistan index is “much better” with a value of 78.14% of instability, which puts the country in 202nd place in the overall rating. The fourth country in the region with a high instability is Azerbaijan, although its 47% dependence on AS 29049 is not so bad compared to the rest.

The downgrade of Ukraine’s rating by four positions can be explained by ongoing legislative processes inside the country, where ways are being sought to determine an acceptable level of applicable Internet regulation. Such intervention inevitably leads to some loss of reliability, although the position of Ukraine in the top 20 is quite stable for three years in a row.

There are “only” 83 countries with more than 40% dependence on one supplier, 65 of them have values ​​higher than 50%. There are 39, for the most part, small countries with instability indexes of 90% or more. There are exceptions: North Korea’s index is 100% for AS 131729, which needs no explanation; the same applies to Eritrea, Greenland and New Caledonia, having their own reasons. 99% of Ethiopia’s only AS 24757 (EthioNet), in a country of 100 million people, is an unexpected fact in 2018. Syria is 99.5% dependent on AS 29256, which represents a significant loss compared to last year’s 88.75%, caused by the displacement of last year’s AS owned by German telecom. Cuba has a 97% dependence on AS 11960. The Jamaica index is 91.3% for AS 34520 owned by Columbus Networks, a company that we often encounter as the only provider of connectivity in remote locations off the coast of North and South America.

Compared with Luxembourg, which is on the 30th place of the classic rating with a value of 9.8% depending on AS6661, the 66% rate of Monaco for AS6758 is not peculiar to the country of such a standard of living and geography.



Over the past two years, significant changes have taken place in the Asian market of Internet providers.

Singapore’s increased connectivity is due to its position among Asian tigers, the region’s fastest growing economies. This competition is fierce at the top of the sustainability rating, where the positions are fairly tight as a percentage. Hong Kong and Singapore swapped on the basis of a 2% change in the index.

According to independent observers interviewed by us, Singtel positions in Singapore are weakening. Last year, the key AS was owned by StarHub, a Singapore broadband Internet access provider. In 2018, the main AS in Singapore is already becoming SingNet AS3758 (SingTel's broadband division). We were told that lately, SingTel has been experiencing a lack of capital, since their main business in Singapore has stopped active growth (which can be further confirmed by looking at the dynamics of the company's shares over the past two years). SingTel’s difficulties are exacerbated by a mutual integration process with the Philippine ISP Globe - this is probably another reason why SingNet has lost consumers, thereby losing market share and real connections. These events together dramatically and rather sharply decentralized the ISP market of Singapore, which led to a jump of 18 positions to the current 5th line in the sustainability rating.

Speaking of Globe ISP, owning AS4775, last year was leading for the Philippines, in 2018 it ceased to be - AS9299 surpassed it. AS9299 is owned by PLDT, which is reportedly becoming increasingly aggressive in the Philippine market. In addition, the position of the region is restored. The Philippines fell from 20th place in the reliability rating to 31st in 2017, and rose to 27th now. This fact alone clearly shows that competition is good, especially when it comes to the reliability of the Internet in a particular region. However, we should not forget that Globe is SingTel, and PLDT is, in fact, an NTT affiliate in the Philippines.

Hong Kong's loss of reliability rating may be related to PCCW activities in the region. Local observers tell us that in 2018, PCCW chose an active customer acquisition strategy, gaining market share and active customers in the economic zone itself.

Last year, the main AS of Hong Kong belonged to Level3 (AS3356). After the merger with CenturyLink at the end of 2017, the position of Level3 could not fail to change to a larger, but at the same time less stable from a regional point of view.

---

With the increasing risks of cyber security and, in fact, the constant flow of news about attacks on the Internet infrastructure, it is time for all governments, private and public companies, but first of all for ordinary users to carefully assess their positions. The risks associated with the connectivity of the regions must be studied carefully and honestly, analyzing the true levels of reliability. Even low values ​​in the rating of instability can cause real problems with accessibility in the event of a massive attack on a large, nationwide, provider of a critical service, say, DNS. Do not forget also that the outside world will disconnect from services and data located within the region, in case of complete loss of connectivity.

Our research clearly shows that the markets of Internet providers and telecom operators, based on competition, ultimately develop more dynamically in order to become much more stable and fault tolerant with respect to risks inside and even outside of a specific region. Without a competitive market, the failure of one AS can and will result in the loss of a network connection from a significant proportion of users from a country or a wider region.