What famous companies use Docker in production and for what?
It is 2017 in the yard, and in various articles and, especially, comments to them now and then we have to meet with the opinion that Docker is the lot of inexperienced enthusiasts, exotic one-day companies and any experiments that are as far away as possible from real production. And while colleagues are preparing new technical materials, I decided to show on publicly available examples and facts that this is more a myth and a stereotype than the world in which IT lives today.
The starting point for this article is the statistics from the Datadog monitoring service , which first appeared in 2015 and was last updated just a few days ago, in April 2017. Datadog is not some traditional analytic company, but a monitoring system known for its friendliness to cloud environments (its agent code is written in Python and published as Open Source) .
Having at its disposal a huge database of infrastructure devices in companies from all over the world, Datadog collected Docker's Adaptation Statistics, which she herself described as “no hype, just facts.” The collected statistics represent 10 thousand companies and 185 million containers used for any purpose (of course, this is not necessarily production, but at least - part of the infrastructure that it makes sense to monitor) . Here are the facts:
')
1. If at the beginning of March 2016, about 13.6% of Datadog's customers used Docker, then after a year (ie, by March 2017), this figure grew by ~ 38%, reaching 18.8% .
2. Docker is running on 15% of all hosts represented in monitoring. For comparison: two years ago such hosts were only 3%, and this is much more significant growth.
3. Leaders of adaptation Docker are large companies . The more hosts the company has in monitoring, the more likely Docker will be. About 60% of organizations with 500+ hosts are named by Docker users. The last update of the study also showed a growing interest in Docker from medium-sized organizations (from 100 to 500 hosts): the number of Docker users among them was already close to the figure among large companies.
4. About 40% of Docker users use container tools such as Kubernetes, Mesos, Amazon ECS, and Google Container Engine. A year ago there were half those.
5. Docker users approximately 5 times increase the number of running containers in the period from the first to the tenth month of its use.
6. Most often, Docker launches the following services: nginx, Redis, Elasticsearch, Registry (18% of companies using Docker), PostgreSQL, MySQL, etcd, Fluentd, MongoDB, RabbitMQ.
7. On average , 7 Docker-containers are launched on one host. 9 months ago, the figure was lower - 5 containers.
8. The average lifespan of a Docker container is 2.5 days; for comparison, the virtual machine - 23 days. A significant dependence was recorded on the availability of orchestration instruments: in companies without them, containers on average live for 5.5 days.
Similar statistics on Docker were collected in Sysdig ( published April 12, 2017), a service specializing in monitoring Docker and featuring good support for related technologies. True, they analyzed much less: only 45,000 containers. Of the interesting data in terms of their data, we can note the great popularity of Kubernetes (43% of all users, and this category includes products like OpenShift and Tectonic), Quay (22% of users against 15% of Docker Registry).
(The most popular services launched in Docker, according to Sysdig)
Finally, we turned to our Russian colleagues from okmeter.io , whose monitoring we use in our projects. According to the statistics, the users of this service also have a stable and significant increase in Docker popularity. If in January 2016 only 9% of companies used Docker, and it was installed on 3% of hosts, then in April 2017 the percentage of companies almost doubled, reaching 17% , and hosts increased more than 7 times, rising to 23% . Thus, the indicator of the number of users is very close to the largest sample from Datadog, and the prevalence of Docker on the hosts of okmeter.io clients was one and a half times higher.
From the general - to the particulars. On the Docker Inc site itself, of course, there is a section with customers and their success stories. In it you can find really famous brands. I will not deal with the full listing and instead give examples of some of them with the years of implementations / updates.
BBC News publishes 80,000 news stories every day in English only, which are serviced by a system running on 150 servers with JVM and PHP applications. The infrastructure had ten different environments for Continuous Integration, running a huge number of tasks used by 500+ developers. Tasks were performed for a very long time (half an hour to be scheduled and another half-hour to be performed) and consistently. Turning to containers in 2015, we managed to get rid of half an hour of planning, because the launch of tasks became parallel, and the execution time of tasks was reduced. In addition, the new environment has become more open, scalable and reliable, which allowed it to become the best practice adapted in other BBC projects. (Details about the project are available in a 2-page PDF file .)
One of the largest online stores faced with the limitations of virtual machines that were used to optimize application development processes. Turning to Docker for continuous integration, the companies organized a process that provides rollout of containerized applications to developers' laptops, test environments and in production. In 2014, the eBay PaaS Team in its technology blog published details on the CI model used with Mesos. In 2015, eBay representatives talked about using Docker containers with Kubernetes (and OpenStack).
Expedia Travel Company (more than 18 thousand employees in 60 countries) decided to switch to cloud infrastructure in 2012, thanks to which its engineers began their journey to micro-service architecture. To create and deploy such applications, we needed suitable and reliable tools. The result was that in 2016, over 200 containerized applications were launched in production using Docker (hosted by AWS). In addition, the developers have their own data center, in which the developers have also been deployed. Among the tools used are the Docker Engine, the Docker Trusted Registry (more than 1000 repositories) and the Docker Compose. Expedia has a speech on YouTube with details on their use of Docker in production. By the way, this year Expedia London subdivision posted on GitHub its Docker Developer Education course , consisting of 8 lessons.
Groupon's e-commerce service was one of Docker's early users: back in 2014, engineers at this company introduced containers for continuous integration. In the same year, at the DockerCon conference, they presented their DotCi Open Source project — a plugin for Jenkins that simplifies task management (with built-in Docker support). It was already then that he became a crucial tool for managing the assembly processes and releases, as the Technical Director of Groupon spoke about to the press.
The well-known electronic payment service PayPal (processes 200+ payments every second) uses the private OpenStack cloud, which runs more than 100,000 virtual machines (they run all web applications and services). The task was to modernize the infrastructure in order to make it more responsive, safe and cost-effective. As a tool, the way of refactoring existing applications (in Java and C ++) was chosen with the aim of their containerization and convenient deployment both to its cloud and to the public (which may turn out to be more profitable than supporting its own infrastructure). The result was the introduction in 2014 of the Docker technology stack in production, which includes the Docker Commercially Supported Engine (CS Engine), the Docker Trusted Registry and the Docker Compose. Containers coexist with virtual machines and allowed to significantly reduce the total number of VMs, which led to the optimization of infrastructure. In addition, the rollout speed of new applications has been significantly reduced due to the lightness of the containers (compared to virtualization). In 2015, PayPal engineers went further, speaking at the DockerCon SF conference with a report ( video , presentation ) about their infrastructure in several data centers, CI with Jenkins and working with Docker images. And in 2016, LinuxCon NA revealed that PayPal's CI system, using Git, Docker, Mesos, Jenkins, Aurora, and Travis CI API, led to a 10-fold saving.
(Orchestration Docker to PayPal)
The popular news edition of The Washington Post had difficulties with a growing number of applications written in different languages ​​(Java, Scala, Node, Python, Go). Deciding to build a continuous deployment system for their AWS-based infrastructure, the engineers chose Docker Engine and Docker Compose. The current environment has more than 2500 Docker containers, which fully serve the entire production, including the website for online readers (news, mobile app, mobile version of the site), their publishing platform, Arc Publishing, and a news analytics system.
Among other major Docker users listed on the developer’s website are General Electric Appliances, New Relic, New York Times, Spotify, Uber, Yelp and Yandex (talking about PaaS Cocaine ). By the way, about Russia . Using Docker in Russian companies / Russian engineers is also a reality. Here are some examples:
By the way, if you also use Docker in production - please tell about it in the comments . I am sure that it will be very interesting to many!
(View the Docker containers using the baDocker utility created in Badoo)
The popularity and prevalence of software products, especially those forming an infrastructure, is determined not only by their customers, but also by how they are supported by the giants of the industry. And it’s about Docker that’s just lazy. In particular (examples, of course, are much more, therefore only about the main thing) :
(Microsoft Active Friendship with Docker)
Particular attention is paid to the growing ecosystem of projects created by Google: Kubernetes from Google, Tectonic and Quay from CoreOS, RancherOS, Apache Mesos and DC / OS, Contiv from Cisco ... Finally, I mention extensive cooperation with non-profit organizations and projects such as Cloud Foundry , Open Containers Initiative , Cloud Native Computing Foundation .
Docker really “shot” and, not being some kind of supernova and super-complex technology, came at the right time to take its place in the world of DevOps and CI / CD. We ourselves also use it in production for more than one year, talk about our experience and will continue to do so. Subscribe to our hub if you are interested :)
Scales and docking trends
The starting point for this article is the statistics from the Datadog monitoring service , which first appeared in 2015 and was last updated just a few days ago, in April 2017. Datadog is not some traditional analytic company, but a monitoring system known for its friendliness to cloud environments (its agent code is written in Python and published as Open Source) .
Having at its disposal a huge database of infrastructure devices in companies from all over the world, Datadog collected Docker's Adaptation Statistics, which she herself described as “no hype, just facts.” The collected statistics represent 10 thousand companies and 185 million containers used for any purpose (of course, this is not necessarily production, but at least - part of the infrastructure that it makes sense to monitor) . Here are the facts:
')
1. If at the beginning of March 2016, about 13.6% of Datadog's customers used Docker, then after a year (ie, by March 2017), this figure grew by ~ 38%, reaching 18.8% .
2. Docker is running on 15% of all hosts represented in monitoring. For comparison: two years ago such hosts were only 3%, and this is much more significant growth.
3. Leaders of adaptation Docker are large companies . The more hosts the company has in monitoring, the more likely Docker will be. About 60% of organizations with 500+ hosts are named by Docker users. The last update of the study also showed a growing interest in Docker from medium-sized organizations (from 100 to 500 hosts): the number of Docker users among them was already close to the figure among large companies.
4. About 40% of Docker users use container tools such as Kubernetes, Mesos, Amazon ECS, and Google Container Engine. A year ago there were half those.
5. Docker users approximately 5 times increase the number of running containers in the period from the first to the tenth month of its use.
6. Most often, Docker launches the following services: nginx, Redis, Elasticsearch, Registry (18% of companies using Docker), PostgreSQL, MySQL, etcd, Fluentd, MongoDB, RabbitMQ.
7. On average , 7 Docker-containers are launched on one host. 9 months ago, the figure was lower - 5 containers.
8. The average lifespan of a Docker container is 2.5 days; for comparison, the virtual machine - 23 days. A significant dependence was recorded on the availability of orchestration instruments: in companies without them, containers on average live for 5.5 days.
Sysdig statistics
Similar statistics on Docker were collected in Sysdig ( published April 12, 2017), a service specializing in monitoring Docker and featuring good support for related technologies. True, they analyzed much less: only 45,000 containers. Of the interesting data in terms of their data, we can note the great popularity of Kubernetes (43% of all users, and this category includes products like OpenShift and Tectonic), Quay (22% of users against 15% of Docker Registry).
(The most popular services launched in Docker, according to Sysdig)
Statistics from Russia
Finally, we turned to our Russian colleagues from okmeter.io , whose monitoring we use in our projects. According to the statistics, the users of this service also have a stable and significant increase in Docker popularity. If in January 2016 only 9% of companies used Docker, and it was installed on 3% of hosts, then in April 2017 the percentage of companies almost doubled, reaching 17% , and hosts increased more than 7 times, rising to 23% . Thus, the indicator of the number of users is very close to the largest sample from Datadog, and the prevalence of Docker on the hosts of okmeter.io clients was one and a half times higher.
Docker Inc Success Stories
From the general - to the particulars. On the Docker Inc site itself, of course, there is a section with customers and their success stories. In it you can find really famous brands. I will not deal with the full listing and instead give examples of some of them with the years of implementations / updates.
BBC News
BBC News publishes 80,000 news stories every day in English only, which are serviced by a system running on 150 servers with JVM and PHP applications. The infrastructure had ten different environments for Continuous Integration, running a huge number of tasks used by 500+ developers. Tasks were performed for a very long time (half an hour to be scheduled and another half-hour to be performed) and consistently. Turning to containers in 2015, we managed to get rid of half an hour of planning, because the launch of tasks became parallel, and the execution time of tasks was reduced. In addition, the new environment has become more open, scalable and reliable, which allowed it to become the best practice adapted in other BBC projects. (Details about the project are available in a 2-page PDF file .)
ebay
One of the largest online stores faced with the limitations of virtual machines that were used to optimize application development processes. Turning to Docker for continuous integration, the companies organized a process that provides rollout of containerized applications to developers' laptops, test environments and in production. In 2014, the eBay PaaS Team in its technology blog published details on the CI model used with Mesos. In 2015, eBay representatives talked about using Docker containers with Kubernetes (and OpenStack).
Expedia
Expedia Travel Company (more than 18 thousand employees in 60 countries) decided to switch to cloud infrastructure in 2012, thanks to which its engineers began their journey to micro-service architecture. To create and deploy such applications, we needed suitable and reliable tools. The result was that in 2016, over 200 containerized applications were launched in production using Docker (hosted by AWS). In addition, the developers have their own data center, in which the developers have also been deployed. Among the tools used are the Docker Engine, the Docker Trusted Registry (more than 1000 repositories) and the Docker Compose. Expedia has a speech on YouTube with details on their use of Docker in production. By the way, this year Expedia London subdivision posted on GitHub its Docker Developer Education course , consisting of 8 lessons.
Groupon
Groupon's e-commerce service was one of Docker's early users: back in 2014, engineers at this company introduced containers for continuous integration. In the same year, at the DockerCon conference, they presented their DotCi Open Source project — a plugin for Jenkins that simplifies task management (with built-in Docker support). It was already then that he became a crucial tool for managing the assembly processes and releases, as the Technical Director of Groupon spoke about to the press.
Paypal
The well-known electronic payment service PayPal (processes 200+ payments every second) uses the private OpenStack cloud, which runs more than 100,000 virtual machines (they run all web applications and services). The task was to modernize the infrastructure in order to make it more responsive, safe and cost-effective. As a tool, the way of refactoring existing applications (in Java and C ++) was chosen with the aim of their containerization and convenient deployment both to its cloud and to the public (which may turn out to be more profitable than supporting its own infrastructure). The result was the introduction in 2014 of the Docker technology stack in production, which includes the Docker Commercially Supported Engine (CS Engine), the Docker Trusted Registry and the Docker Compose. Containers coexist with virtual machines and allowed to significantly reduce the total number of VMs, which led to the optimization of infrastructure. In addition, the rollout speed of new applications has been significantly reduced due to the lightness of the containers (compared to virtualization). In 2015, PayPal engineers went further, speaking at the DockerCon SF conference with a report ( video , presentation ) about their infrastructure in several data centers, CI with Jenkins and working with Docker images. And in 2016, LinuxCon NA revealed that PayPal's CI system, using Git, Docker, Mesos, Jenkins, Aurora, and Travis CI API, led to a 10-fold saving.
(Orchestration Docker to PayPal)
The washington post
The popular news edition of The Washington Post had difficulties with a growing number of applications written in different languages ​​(Java, Scala, Node, Python, Go). Deciding to build a continuous deployment system for their AWS-based infrastructure, the engineers chose Docker Engine and Docker Compose. The current environment has more than 2500 Docker containers, which fully serve the entire production, including the website for online readers (news, mobile app, mobile version of the site), their publishing platform, Arc Publishing, and a news analytics system.
Other examples
Among other major Docker users listed on the developer’s website are General Electric Appliances, New Relic, New York Times, Spotify, Uber, Yelp and Yandex (talking about PaaS Cocaine ). By the way, about Russia . Using Docker in Russian companies / Russian engineers is also a reality. Here are some examples:
- Badoo's look at the year of use of Docker in production;
- a report about the development and deployment of services using Docker and Kubernetes in Avito ;
- a report from Aviasales.ru about the migration of the search engine to Docker;
- and for a change - impressions of using Docker in a bank (from the architect of Alpha Laboratories, the special purpose department of Alfa Bank ).
By the way, if you also use Docker in production - please tell about it in the comments . I am sure that it will be very interesting to many!
(View the Docker containers using the baDocker utility created in Badoo)
Adoption in IT
The popularity and prevalence of software products, especially those forming an infrastructure, is determined not only by their customers, but also by how they are supported by the giants of the industry. And it’s about Docker that’s just lazy. In particular (examples, of course, are much more, therefore only about the main thing) :
- Docker integrates / is supported / maintained in Amazon Web Services, Google Cloud Platform, IBM Bluemix, Jelastic, Microsoft Azure, OpenStack, Oracle Container Cloud Service, VMware vSphere Integrated Containers ...
- Supported in configuration management systems: Ansible, CFEngine, Chef, Puppet, Salt.
- Red Hat does not just "support" Docker, but develops its OpenShift platform (based on Kubernetes), Canonical officially collaborates , and a special subscription is offered for SUSE.
- Microsoft is not limited to Azure, but recently bought Deis at all.
(Microsoft Active Friendship with Docker)
Particular attention is paid to the growing ecosystem of projects created by Google: Kubernetes from Google, Tectonic and Quay from CoreOS, RancherOS, Apache Mesos and DC / OS, Contiv from Cisco ... Finally, I mention extensive cooperation with non-profit organizations and projects such as Cloud Foundry , Open Containers Initiative , Cloud Native Computing Foundation .
Total
Docker really “shot” and, not being some kind of supernova and super-complex technology, came at the right time to take its place in the world of DevOps and CI / CD. We ourselves also use it in production for more than one year, talk about our experience and will continue to do so. Subscribe to our hub if you are interested :)
Source: https://habr.com/ru/post/326784/
All Articles