How Google Cloud breathed life into Pokémon GO
During my entire engineering career, I had a hand in launching a multitude of products that won millions of users. People accept the product, usually gradually, over several months, when the introduction of new features and architectural changes is planned for a long enough period. But never before have I seen anything even close to the growth that Niantic , a Google Cloud client, experienced with Pokémon GO.
As a seed, I'll start with a picture worth a thousand words:
Our colleagues in the technical community asked which infrastructure helped Pokémon GO to survive with millions of users. Niantic and the Google Cloud team together wrote this post to highlight some of the key components that make one of the most popular mobile games in history work.
')
At our Horizon event, we presented Google Customer Reliability Engineering (CRE), a new work model in which Google’s technicians integrate with the customer’s team and share responsibility for the reliability and success of critical cloud applications. The first Google CRE client was Niantic, and the first job was the launch of the Pokémon GO - the best test of all!
Within 15 minutes of its launch in Australia and New Zealand, traffic far exceeded studio expectations. This was the first indication that the Niantic team was able to create something truly special. Niantic called Google CRE for help in anticipation of the American launch, which was to take place the next day. Niantic and Google Cloud - including CRE, SRE, developers, products, support and executives - prepared for the invasion of new Pokemon trainers, since, according to all predictions, the Pokémon GO had to beat all previous predictions on the number of players.
Pokémon GO is a mobile application that uses many Google Cloud services, but Cloud Datastore became the direct “culprit” of the game’s popularity , considering its role as the main database of the game world in which Pokémon are caught. The schedule at the beginning of this post tells the whole story: the team counted on 1x players, and the worst scenario involved 5x. The rapid growth of the popularity of Pokémon GO has led to the fact that traffic is over the top 50x from the initial assessment, 10x more than the critical level. In response, Google CRE, on behalf of Niantic, seamlessly prepared additional resources to stay ahead of their record growth.
Not everything went smoothly at startup. When there were problems with the stability of the game, the engineers of Niantic and Google consistently solved each task, quickly developing and deploying solutions. The Google CRE team worked hand in hand with Niantic, redefining their entire architecture with the help of Google Cloud's key engineers and product managers — all amid connecting millions of new players to the game.
Besides the fact that the Pokémon GO has become a global phenomenon, it is also one of the best examples of container development. The game logic works in the Google Container Engine (GKE) , based on the open source project Kubernetes . Niantic chose GKE for its ability to manage a container cluster on a planetary scale, unleashing the team’s strength to make "live" changes to the game. Thus, Niantic used Google Cloud to turn Pokémon GO into a service for millions of players, constantly adapting and improving it.
One of the most ambitious technical solutions used by Niantic and the Google CRE team was the transition to the new version of GKE, which could add over a thousand additional nodes to the container cluster in anticipation of the long-awaited launch of the game in Japan. This is almost like replacing the engine on a flying plane - it was necessary to take measures in order not to harm existing players, the transition to the new version took place at the same time, when millions of new players joined the world of Pok © mon. In addition to this upgrade, Niantic and Google engineers worked together to replace the network load balancer , deploying a new and more complex HTTP / S Load Balancer in its place. HTTP / S Load Balancer is a global system that works with HTTPS traffic and gives much more control to administrators, faster connections to users and greater firewall throughput, which is more suitable for the type and amount of traffic Pokémon GO works with.
The lessons learned from the American launch - allocating resources with a margin, replacing the architecture with the latest version of the Container Engine and improving to the HTTP / S Load Balancer - came in handy when the game started in Japan. In this country, the number of users was three times more than in the US two weeks earlier, and yet the launch went smoothly.
The Google Cloud GKE / Kubernetes team that supports many of our clients, like Niantic
Niantic's Pokémon GO was a launch in which everyone went all out; he demanded quick and informed decisions from half a dozen teams. The sheer scale and ambitions of the game require Niantic to use the architectural and operational practices of those engineering teams that create the relevant products on which it is based. On behalf of the Google CRE team, I can say that it was a real pleasure to be part of such a memorable launch that gave pleasure to play to such a huge number of people all over the world.
As a seed, I'll start with a picture worth a thousand words:
Our colleagues in the technical community asked which infrastructure helped Pokémon GO to survive with millions of users. Niantic and the Google Cloud team together wrote this post to highlight some of the key components that make one of the most popular mobile games in history work.
')
Common fate
At our Horizon event, we presented Google Customer Reliability Engineering (CRE), a new work model in which Google’s technicians integrate with the customer’s team and share responsibility for the reliability and success of critical cloud applications. The first Google CRE client was Niantic, and the first job was the launch of the Pokémon GO - the best test of all!
Within 15 minutes of its launch in Australia and New Zealand, traffic far exceeded studio expectations. This was the first indication that the Niantic team was able to create something truly special. Niantic called Google CRE for help in anticipation of the American launch, which was to take place the next day. Niantic and Google Cloud - including CRE, SRE, developers, products, support and executives - prepared for the invasion of new Pokemon trainers, since, according to all predictions, the Pokémon GO had to beat all previous predictions on the number of players.
Create a pokemon game world
Pokémon GO is a mobile application that uses many Google Cloud services, but Cloud Datastore became the direct “culprit” of the game’s popularity , considering its role as the main database of the game world in which Pokémon are caught. The schedule at the beginning of this post tells the whole story: the team counted on 1x players, and the worst scenario involved 5x. The rapid growth of the popularity of Pokémon GO has led to the fact that traffic is over the top 50x from the initial assessment, 10x more than the critical level. In response, Google CRE, on behalf of Niantic, seamlessly prepared additional resources to stay ahead of their record growth.
Not everything went smoothly at startup. When there were problems with the stability of the game, the engineers of Niantic and Google consistently solved each task, quickly developing and deploying solutions. The Google CRE team worked hand in hand with Niantic, redefining their entire architecture with the help of Google Cloud's key engineers and product managers — all amid connecting millions of new players to the game.
Pokemon in containers
Besides the fact that the Pokémon GO has become a global phenomenon, it is also one of the best examples of container development. The game logic works in the Google Container Engine (GKE) , based on the open source project Kubernetes . Niantic chose GKE for its ability to manage a container cluster on a planetary scale, unleashing the team’s strength to make "live" changes to the game. Thus, Niantic used Google Cloud to turn Pokémon GO into a service for millions of players, constantly adapting and improving it.
One of the most ambitious technical solutions used by Niantic and the Google CRE team was the transition to the new version of GKE, which could add over a thousand additional nodes to the container cluster in anticipation of the long-awaited launch of the game in Japan. This is almost like replacing the engine on a flying plane - it was necessary to take measures in order not to harm existing players, the transition to the new version took place at the same time, when millions of new players joined the world of Pok © mon. In addition to this upgrade, Niantic and Google engineers worked together to replace the network load balancer , deploying a new and more complex HTTP / S Load Balancer in its place. HTTP / S Load Balancer is a global system that works with HTTPS traffic and gives much more control to administrators, faster connections to users and greater firewall throughput, which is more suitable for the type and amount of traffic Pokémon GO works with.
The lessons learned from the American launch - allocating resources with a margin, replacing the architecture with the latest version of the Container Engine and improving to the HTTP / S Load Balancer - came in handy when the game started in Japan. In this country, the number of users was three times more than in the US two weeks earlier, and yet the launch went smoothly.
The Google Cloud GKE / Kubernetes team that supports many of our clients, like Niantic
More interesting facts:
- The game world Pokémon GO uses more than ten Google Cloud services.
- Pokémon GO has become the largest Kubernetes deployment to Google Container Engine in history. Due to the size of the cluster and related traffic, many errors were identified, they were fixed and the code was added to the open source project .
- To support the huge Pokémon GO user base, Google provided tens of thousands of cores for the Niantic cluster in the Container Engine.
- The global network of Google has allowed to reduce delays in the work of Pokemon coaches who inhabit the common gaming world. Gaming traffic was mostly transmitted on Google’s private optical network (and it’s even under water !), Providing a reliable connection with low latency for players around the world.
Niantic's Pokémon GO was a launch in which everyone went all out; he demanded quick and informed decisions from half a dozen teams. The sheer scale and ambitions of the game require Niantic to use the architectural and operational practices of those engineering teams that create the relevant products on which it is based. On behalf of the Google CRE team, I can say that it was a real pleasure to be part of such a memorable launch that gave pleasure to play to such a huge number of people all over the world.
Source: https://habr.com/ru/post/311996/
All Articles