How and why $ 10 billion in public cloud costs are unwarranted losses
Public cloud service providers at the end of 2017 announced their Q3 revenue. Income indicators impress with their size and growth rate. The total income of three large companies (Amazon Web Services, Microsoft Azure and the Google Cloud Platform) amounted to $ 7.5 billion for the third quarter or $ 30 billion in annual terms.
$ 10 billion in cloud spending is a waste
RightScale analysts wondered how much revenue cloud providers accounted for tenants who do not understand how they could prevent the waste in total costs of cloud computing. Earlier in the “RightScale 2017 State of the Cloud Report” report, it was noted that cloud computing users believe that excess costs reach 30 percent of the cost of the cloud. In fact, it is likely that this figure is even higher.
')
In fact, 60 preliminary estimates of cloud computing cost optimization were considered, which were performed for RightScale customers, before the optimization activities were started. It was found that the average loss in cloud costs is 35 percent. This corresponds to losses of up to 10 billion dollars in costs for AWS, Azure and Google annually. It is likely that the number will grow next year, as the pace of cloud technology is accelerating. In such circumstances, IT professionals need to identify unnecessary costs and take the necessary measures in order to save and optimize costs.
First of all, we need to recognize that excessive IT costs are not a new problem. For decades, we have been buying infrastructure for data centers a few months or even years before it really is needed on such a scale. And with the introduction of virtualization, companies continue to deploy virtual machines, usually with more resources than they really need, and often leave them turned on even when they are not used for a long time.
However, the incentive to improve the efficiency of virtualized environments is somewhat higher, because the advantage of optimization seems less noticeable when the equipment is already purchased and fully paid.
The difference in the case of the public cloud is that the optimization of resources begins to bring in money immediately. At that moment, when you turn off the resource or reduce the amount of resources consumed, you will no longer pay for the excess and at the end of the month the bill for the cloud will be lower. So, now there is a motivation to save money and there is an opportunity to get almost instant satisfaction when you do it.
There are three factors that in some cases lead to the formation of losses in the composition of cloud costs:
The first reason that losses of this magnitude can occur in a public cloud is due to the incredible complexity of pricing and billing in the clouds. You need to “get a degree” in cloud pricing to understand all the features and track what money is spent on. To illustrate the scope of the problem, here are some examples:
In addition to the complexity of pricing described above, your spending on cloud services is constantly changing. Changes can occur not only in the volume and structure of the resources you use, but also on the part of cloud providers that change prices, develop discount models and add new services with completely new prices. It is also important to note that providers sometimes change how spending items are presented on the invoice, disrupting the automated processes that customers could implement to distribute expenses and analyze the bill.
The third cause of losses arises due to the upward embedding of the cloud. The cloud allows companies to be more flexible. Individual development teams or business units can instantly access the infrastructure they need by pressing a button or calling a simple API. You no longer have to go through the lengthy approval or allocation procedures that were required in traditional data centers.
However, experts who are engaged in the provision of resources, often do not have sufficient grounds for the effective use of cloud resources. Prices for individual virtual machines or resources seem small, but only until the use of the cloud has grown, and the cost of the cloud has not become significant. Upon reaching certain values, the business and employees are aware of how the losses accumulate. Even when they are aware of the problem, the lack of experience in cloud pricing and the lack of visibility in the structure of cloud costs make it difficult to solve the problem.
Most enterprises have centralized cloud teams that work to reduce the cost of cloud computing, making the use of cloud technologies more efficient. But in order for the changes to take effect, such specialists must work closely with business users to determine what specific actions to take.
The three broadest areas for finding sources of cloud computing savings are instances, discounts, and data storage.
The largest share in cloud costs and, therefore, the greatest opportunity for losses is instances or virtual machines.
Another important opportunity to save money is to use discounts from cloud computing providers. Each provider has different rules and approaches to the provision of discounts, but the basic principles are the same. RightScale offers a step-by-step plan and act when looking for an option to get a discount.
The size of the discount and the scheme of receipt depend on the cloud provider. The following table summarizes some of the AWS, Azure and Google discount terms.
The cost of storage is typically 10–25 percent of the cloud bill, the store can quickly grow and inflate costs. RightScale highlights the following most significant opportunities for optimizing storage costs:
Organizations implementing cloud technologies quickly understand that they need to learn how to manage the costs of cloud services. Businesses that already make extensive use of cloud technologies are seeking to optimize costs and use automated policies to keep them under control. Cloud tenants need reports and controls to help prevent losses before it happens. One of the optimization tools is RightScale Optima, however, it is only suitable when choosing a foreign cloud provider.
The virtualization platform supports not only the creation of the necessary number of virtual machines with the necessary characteristics and operating systems, but also the virtualization of networks and storage systems. Due to this, cloud tenants can combine cloud servers on the network with any topology and achieve exact compliance of IT resources with their actual business needs. The hardware part of the provider’s platform is located in several Russian TIER III data centers. Optical communication channels between data centers clouds are duplicated, forming a ring of high availability.
In the Cloud4Y cloud, physical computing and storage resources are converted to virtual pools. This common pool represents a level of abstraction from which resources can be obtained for consumption by end users as separate computing units. Since management takes place through the self-service vCloud Director 9.0 panel, the tenant himself can at any time make changes to his “prefabricated” cloud data center. For this reason, when using the Cloud4Y cloud, customers do not have a problem with the choice of instance type from the predefined provider and the problem with making changes to their cloud configuration.
Customers can view their VM statistics themselves through the Tenant Portal using the 'Monitoring Chart'. vCloud Director collects and stores various VM performance metrics. These metrics include data on CPU / memory usage / storage of the virtual machine, average latency of disk operations and so on. VM monitoring and VM metrics can be used by you to make fact-based cost optimization decisions in order to maximize benefits within the resources used.
We are not Google, this has its advantages. Cloud4Y provides a slightly different payment model, different from the instances - the “Pay as you go” model for each type of resource separately. This more approach makes it possible to pay for resources upon consumption. It is worth noting that in all cases, the CPU and RAM fees are taken only for the “on” time, at the time when you are not using the virtual machine — no fee is charged, but you will have to pay for the storage regardless of whether the machine is turned on or off - the fee is taken until the moment there is data on the disk; if the disk is blank, no fee is charged.
On the site you can find a calculator of the cost of services, but it is worth noting that it works based on the assumption of using the VM in 24x7x30 mode. Using an instance in 12x5x30 mode, the cost of the service for you may be lower by more than 60%.
A 100% return of unused funds from the customer’s personal account and post-payment on the fact of consumption at the end of the month for legal entities are provided. For customers from Russia, payment is made in rubles, and Russian-language technical support is 24x7x365.
There is a flexible system of discounts, mainly depending on the amount of resources used. If your company is developing and wants to get a virtual cloud server (IaaS) service with a 40% discount, then fill out a short form and enter the promotional code: "developer40". The offer can be used by any company registered in the Russian Federation, the main activity of which falls under the category “Software development and consulting in this area”.
$ 10 billion in cloud spending is a waste
RightScale analysts wondered how much revenue cloud providers accounted for tenants who do not understand how they could prevent the waste in total costs of cloud computing. Earlier in the “RightScale 2017 State of the Cloud Report” report, it was noted that cloud computing users believe that excess costs reach 30 percent of the cost of the cloud. In fact, it is likely that this figure is even higher.
')
In fact, 60 preliminary estimates of cloud computing cost optimization were considered, which were performed for RightScale customers, before the optimization activities were started. It was found that the average loss in cloud costs is 35 percent. This corresponds to losses of up to 10 billion dollars in costs for AWS, Azure and Google annually. It is likely that the number will grow next year, as the pace of cloud technology is accelerating. In such circumstances, IT professionals need to identify unnecessary costs and take the necessary measures in order to save and optimize costs.
Cloud Optimization Provides Instant Savings
First of all, we need to recognize that excessive IT costs are not a new problem. For decades, we have been buying infrastructure for data centers a few months or even years before it really is needed on such a scale. And with the introduction of virtualization, companies continue to deploy virtual machines, usually with more resources than they really need, and often leave them turned on even when they are not used for a long time.
However, the incentive to improve the efficiency of virtualized environments is somewhat higher, because the advantage of optimization seems less noticeable when the equipment is already purchased and fully paid.
The difference in the case of the public cloud is that the optimization of resources begins to bring in money immediately. At that moment, when you turn off the resource or reduce the amount of resources consumed, you will no longer pay for the excess and at the end of the month the bill for the cloud will be lower. So, now there is a motivation to save money and there is an opportunity to get almost instant satisfaction when you do it.
3 factors that "inflate" cloud costs
There are three factors that in some cases lead to the formation of losses in the composition of cloud costs:
- the complexity of cloud computing pricing and billing,
- constant change in prices and product portfolio of cloud providers and
- decentralization when using the cloud.
1. The complexity of cloud pricing and invoices for payment
The first reason that losses of this magnitude can occur in a public cloud is due to the incredible complexity of pricing and billing in the clouds. You need to “get a degree” in cloud pricing to understand all the features and track what money is spent on. To illustrate the scope of the problem, here are some examples:
- Cloud service bills can often have millions of line items. Do not even expect that when opening your favorite tabular editor, the account will always be of a reasonable size.
- Cloud providers offer hundreds of thousands of price points. One AWS has over 70,000 price points for instances only.
- Cloud tenants do not have a clear idea of ​​which of the options is the most affordable. When providing resources in the cloud, users often do not know which regions or types of instances that meet their needs will be offered at the lowest price.
- There are many options for discounts. AWS offers reserved instances, and more than 90 percent of the price offers will have different discount options. Azure also added the ability to purchase reserved instances of virtual machines and pricing works differently from AWS. Google has its own automated approach to providing discounts of two types: Sustained Use Discounts (increasing discount for permanent use of an instance more than 25% of the time per month) and Committed Use Discounts (discount for instances reserved for 1 or 3 years).
2. Constant changes in services by cloud providers
In addition to the complexity of pricing described above, your spending on cloud services is constantly changing. Changes can occur not only in the volume and structure of the resources you use, but also on the part of cloud providers that change prices, develop discount models and add new services with completely new prices. It is also important to note that providers sometimes change how spending items are presented on the invoice, disrupting the automated processes that customers could implement to distribute expenses and analyze the bill.
3. Decentralized use of the cloud
The third cause of losses arises due to the upward embedding of the cloud. The cloud allows companies to be more flexible. Individual development teams or business units can instantly access the infrastructure they need by pressing a button or calling a simple API. You no longer have to go through the lengthy approval or allocation procedures that were required in traditional data centers.
However, experts who are engaged in the provision of resources, often do not have sufficient grounds for the effective use of cloud resources. Prices for individual virtual machines or resources seem small, but only until the use of the cloud has grown, and the cost of the cloud has not become significant. Upon reaching certain values, the business and employees are aware of how the losses accumulate. Even when they are aware of the problem, the lack of experience in cloud pricing and the lack of visibility in the structure of cloud costs make it difficult to solve the problem.
Most enterprises have centralized cloud teams that work to reduce the cost of cloud computing, making the use of cloud technologies more efficient. But in order for the changes to take effect, such specialists must work closely with business users to determine what specific actions to take.
Where are the biggest losses in cloud costs?
The three broadest areas for finding sources of cloud computing savings are instances, discounts, and data storage.
Instances
The largest share in cloud costs and, therefore, the greatest opportunity for losses is instances or virtual machines.
- Excessive instances: on average, according to RightScale Optima, 56% of expenses are for virtual machines. However, typically 40 percent of instances are two to three times the size of current workloads. This means 50-75 percent of the overhead for each non-optimized instance. Typical savings are 20-30 percent of the cost of virtual machines, or 11-16 percent of all cloud costs.
- Easy to use virtual machines: many cloud users run their cloud servers in 24x7 mode, even if they are used only in certain hours and days. A typical example is the cloud infrastructure, which is needed only on weekdays when developers are working. Using scripts to switch these instances from a 24x7 schedule to a 12x5 schedule saves 64 percent of this type of cost. Likewise, the virtual machines needed for development, QA, training, demonstrations, and other temporary applications are often not disabled after the project is completed. Their immediate shutdown during periods of inactivity can be the cause of significant savings.
- Choosing the type of instances or region (in fact, the data center) with a higher price: many cloud computing users do not understand that choosing neighboring regions can result in significant savings depending on the type of instance chosen. For example, from AWS regions in the USA, Northern California is more expensive than the state of Oregon. Azure has several regions on the east coast of the United States, and prices vary for many virtual machine families. In addition, since cloud service providers offer new types of instances, they often have better specifications and cost less than the instances they replace. Turning off obsolete virtual machine families can save a lot of money. Choosing the best option on average saves 3-5 percent of the cost of the cloud.
Instance discounts
Another important opportunity to save money is to use discounts from cloud computing providers. Each provider has different rules and approaches to the provision of discounts, but the basic principles are the same. RightScale offers a step-by-step plan and act when looking for an option to get a discount.
The size of the discount and the scheme of receipt depend on the cloud provider. The following table summarizes some of the AWS, Azure and Google discount terms.
AWS | Azure | ||
Type of | Reserved Instance | Reserved Instance | Sustained Use Discounts (for duration of use), Committed Use Discounts |
Period | 1 or 3 years | 1 or 3 years | 1 or 3 years |
Amount of discount | 20-75% | 37-55% | |
Ability to make changes | Standard : Change AZ (zone) type network. Without the ability to change the region or family of virtual machines. Convertible : Any changes | Any changes | Change the family or type of instance. Without the ability to change the region. |
Opportunity to return money | Self selling to another user (it can be difficult) | Refund for 12% payment | No reimbursement options |
Data storage
The cost of storage is typically 10–25 percent of the cloud bill, the store can quickly grow and inflate costs. RightScale highlights the following most significant opportunities for optimizing storage costs:
- Non-attached volumes: The volumes (disks) attached to an instance are often not deleted along with the complete removal of the virtual machine. This data is accumulated. As a result, volumes unattached to existing virtual machines can occupy a significant amount of storage. The administrator needs to identify and delete unbound volumes or provide automatic actions based on the storage policy.
- Older snapshots: the snapshots created (snapshots of the state of the data in the storage system, fixed at a certain point in time) are also often not cleared. Each organization has a policy that describes how long to keep snapshots. Often, an effective solution is to automate these actions.
- Choosing expensive storage options. Often, cloud computing users choose more expensive drives (SSD vs HDD), backup options and frequency.
Manage and Optimize Cloud Computing Costs
Organizations implementing cloud technologies quickly understand that they need to learn how to manage the costs of cloud services. Businesses that already make extensive use of cloud technologies are seeking to optimize costs and use automated policies to keep them under control. Cloud tenants need reports and controls to help prevent losses before it happens. One of the optimization tools is RightScale Optima, however, it is only suitable when choosing a foreign cloud provider.
Comment Cloud4Y
Disclaimer: Translation of this article is made by the cloud provider Cloud4Y offering a competing product.Cloud Cloud 4Y is built on the basis of VMware virtualization technology stack with support for all cluster options ( HA, DRS, vMotion ) and complies with the concept of software-defined data center (SDDC). Tenants can create their own virtual data center (vDC) on their own and change its configuration in the shortest possible time using a single and convenient vCloud Director 9.0 control panel.
The virtualization platform supports not only the creation of the necessary number of virtual machines with the necessary characteristics and operating systems, but also the virtualization of networks and storage systems. Due to this, cloud tenants can combine cloud servers on the network with any topology and achieve exact compliance of IT resources with their actual business needs. The hardware part of the provider’s platform is located in several Russian TIER III data centers. Optical communication channels between data centers clouds are duplicated, forming a ring of high availability.
In the Cloud4Y cloud, physical computing and storage resources are converted to virtual pools. This common pool represents a level of abstraction from which resources can be obtained for consumption by end users as separate computing units. Since management takes place through the self-service vCloud Director 9.0 panel, the tenant himself can at any time make changes to his “prefabricated” cloud data center. For this reason, when using the Cloud4Y cloud, customers do not have a problem with the choice of instance type from the predefined provider and the problem with making changes to their cloud configuration.
Customers can view their VM statistics themselves through the Tenant Portal using the 'Monitoring Chart'. vCloud Director collects and stores various VM performance metrics. These metrics include data on CPU / memory usage / storage of the virtual machine, average latency of disk operations and so on. VM monitoring and VM metrics can be used by you to make fact-based cost optimization decisions in order to maximize benefits within the resources used.
We are not Google, this has its advantages. Cloud4Y provides a slightly different payment model, different from the instances - the “Pay as you go” model for each type of resource separately. This more approach makes it possible to pay for resources upon consumption. It is worth noting that in all cases, the CPU and RAM fees are taken only for the “on” time, at the time when you are not using the virtual machine — no fee is charged, but you will have to pay for the storage regardless of whether the machine is turned on or off - the fee is taken until the moment there is data on the disk; if the disk is blank, no fee is charged.
On the site you can find a calculator of the cost of services, but it is worth noting that it works based on the assumption of using the VM in 24x7x30 mode. Using an instance in 12x5x30 mode, the cost of the service for you may be lower by more than 60%.
A 100% return of unused funds from the customer’s personal account and post-payment on the fact of consumption at the end of the month for legal entities are provided. For customers from Russia, payment is made in rubles, and Russian-language technical support is 24x7x365.
There is a flexible system of discounts, mainly depending on the amount of resources used. If your company is developing and wants to get a virtual cloud server (IaaS) service with a 40% discount, then fill out a short form and enter the promotional code: "developer40". The offer can be used by any company registered in the Russian Federation, the main activity of which falls under the category “Software development and consulting in this area”.
Source: https://habr.com/ru/post/359338/
All Articles