Black Friday 2017 through the eyes of IT and developers. As we stood black Friday with a 10 times increase in traffic
Every year Black Friday becomes a new challenge for e-commerce solutions. Customers get the opportunity to buy products at attractive prices, and IT specialists receive a flurry of requests to servers, exorbitant loads on all resources and unavailability of external services (such as payment systems, ERP, etc.).
As we already wrote earlier , we work with large online stores in different countries, so we often face high loads. In this article, we will describe how Black Friday 2017 went on one of the client projects implemented on the VirtoCommerce platform and deployed in the Azure Cloud and how we were able to withstand a 10 times increase in traffic.
')
The initial configuration was built according to customer requirements without a CDN. On the one hand, it was necessary to keep within a strict budget, on the other hand, to lay down the possibility of scaling and changing the configuration in real time.
Products looked like this:
On Black Friday 2017, marketers predicted an increase in traffic by 10 times. Plus, during the day, it was planned to do email- or SMS-mailing, which should have led to a sharp, peak increase in requests at one point in time.
We analyzed the current system and made the following decisions that we were able to implement using Azure services.
In order to prepare for Black Friday, we have identified 4 points:
Thus, the scheme should look like this:
By default, all resources (JavaScript and CSS files) are loaded directly from the front end of the application.
Azure Content Delivery Network (CDN) allows you to cache static web content at strategically located points. This allows for maximum throughput for delivering content to users. CDN also reduces the load on the web application server.
In order to connect a CDN for caching static resources, you must:
After that, all the resources that are addressed in the themes / assets / static / folder path will be downloaded from the CDN.
Previously, we performed a series of benchmark tests for different regions, which showed the effectiveness of the CDN.
For example, for a static vendor.js file with a size of 714 KB, the following results were obtained: 216ms Vs 705ms . A good increase in speed.
Before 705 ms
After 216 ms
In client solutions, we recommend using the Azure Blob service as primary storage for binary files or product images.
Out of the box, a CDN can be connected to Azure Blob Storage for content caching. The partner services with which Microsoft collaborates within Azure CDN offer developers a global solution for delivering large volumes of content. This solution allows caching of blobs and static content of computational instances on physical nodes in the USA, Europe, Asia, Australia and South America.
In order to create a new CDN for product images, you need to:
We compared the work before and after. For example, a jpeg file with a size of 82 KB size, we received in 126 ms, instead of 259 ms.
Before 259 ms
After 126 ms
By default, we use the Azure S2: Medium computational resources for App Services, but if we need to use large CPU or memory resources, we were ready to increase the plan to S3: Large.
The tests conducted in Azure showed that the change in the amount of computing resources in the direction of increasing occurs on average within 2-5 minutes and without down-time.
Scale Out allows you to configure the number of environments in which the application runs. In the basic setting, we have set up rules for automatically increasing / decreasing the number of roles from 2 to 5, depending on the metrics.
With the usual work of the application, this can significantly save IT administrator time and budget.
For ourselves, we have chosen the following settings:
Unfortunately, it turned out that with peak loads, the automatic rules are too slow - they are late and in the end we had to switch to manual control. In normal times, the automatic Scale Out works perfectly.
Morning has come. In the browser, developers and system administrators open Live Stream metrics. Activated promotions and passed the first email-list. In Live Stream, we see an increase in the number of requests. More resources are required, but automatic rules do not have time to react to these changes.
We see an increase in Response Time. A new instance rises and the situation stabilizes.
A wave of distribution passes and resources are automatically reduced. The next wave - the picture repeats. We decide to switch to manual control of the amount of the instance in order to leave resources with a margin.
Customers note that the site works much faster, despite the load.
Here is the distribution of the number of requests from the duration:
Black Friday morning from 7:40 to 9:40. Step 2 minutes.
And this is the usual Friday:
An increase in the number of orders led to an increase in the load on the database to critical parameters. We usually assume that the operating parameter is 40% -50% DTU percentage.
On Black Friday, the load increased to 70-80%. It was decided to raise the resources to 100 DTU. Changes to the plan on the fly also passed within 2-3 minutes and without down-time, which could not but rejoice. This resolved potential stock issues. Further more requests, more traffic. More orders. And so it is until one in the morning on Saturdays. Then they all went to sleep.
We stood black Friday!
The site pleased customers with discounts, speed and 100% availability. The customers of the store, in turn, pleased the owners of the service with a number of orders.
Yes, the cost of IT resources for this day increased by 3 times, but the revenue for that day grew 100 times compared with the usual day. The main costs were associated with an increase in the number of application instances and a raising of the database plan, after the end of black Friday, we returned to the previous plans.
As the calculations showed, the CDN connection increased the cost of IT resources by only $ 2 / day, so the client decided to leave this configuration on an ongoing basis.
According to our estimates, the transfer of static resources to CDN has improved the following indicators:
Well, we will prepare for the new black Friday 2018!
As we already wrote earlier , we work with large online stores in different countries, so we often face high loads. In this article, we will describe how Black Friday 2017 went on one of the client projects implemented on the VirtoCommerce platform and deployed in the Azure Cloud and how we were able to withstand a 10 times increase in traffic.
')
The initial configuration was built according to customer requirements without a CDN. On the one hand, it was necessary to keep within a strict budget, on the other hand, to lay down the possibility of scaling and changing the configuration in real time.
Products looked like this:
- App Service for the front end of the application - working on the VirtoCommerce Storefront for ASP.NET Core 2.0 , it has three independent stores for the Scandinavian countries: each under its own domain, with its own theme, in its own language and its own currency. The application interacts with the back end system through RESTful API calls.
- The App Service for back end - working on the VirtoCommerce platform , it manages a single catalog, prices, store and orders, and also implements a RESTful API for accessing the functions and data of the platform.
- Logic Apps is designed to integrate with external systems.
- To monitor the performance of the solution, Application Insights are used.
On Black Friday 2017, marketers predicted an increase in traffic by 10 times. Plus, during the day, it was planned to do email- or SMS-mailing, which should have led to a sharp, peak increase in requests at one point in time.
Part 1. Preparation
We analyzed the current system and made the following decisions that we were able to implement using Azure services.
In order to prepare for Black Friday, we have identified 4 points:
- Activate CDN for static resources (JavaScript and CSS).
- Activate CDN for product images.
- Increase computing resources at the time of the action.
- Configure rules for automatic Scale Out in Azure.
Thus, the scheme should look like this:
Activate CDN for static resources (JavaScript and CSS).
By default, all resources (JavaScript and CSS files) are loaded directly from the front end of the application.
Azure Content Delivery Network (CDN) allows you to cache static web content at strategically located points. This allows for maximum throughput for delivering content to users. CDN also reduces the load on the web application server.
In order to connect a CDN for caching static resources, you must:
- Add code to form CDN links in the StaticAssetUrl method.
- Create a CDN for App Service docs.microsoft.com/en-us/azure/app-service/app-service-web-tutorial-content-delivery-network .
- Configure CDN Url in application config.
After that, all the resources that are addressed in the themes / assets / static / folder path will be downloaded from the CDN.
Previously, we performed a series of benchmark tests for different regions, which showed the effectiveness of the CDN.
For example, for a static vendor.js file with a size of 714 KB, the following results were obtained: 216ms Vs 705ms . A good increase in speed.
Before 705 ms
After 216 ms
Activate CDN for product images.
In client solutions, we recommend using the Azure Blob service as primary storage for binary files or product images.
Out of the box, a CDN can be connected to Azure Blob Storage for content caching. The partner services with which Microsoft collaborates within Azure CDN offer developers a global solution for delivering large volumes of content. This solution allows caching of blobs and static content of computational instances on physical nodes in the USA, Europe, Asia, Australia and South America.
In order to create a new CDN for product images, you need to:
- Install the platform Virto Commerce version v2.13.18 and higher. Plus use AzureBlobProvider.
- Create an Azure CDN for Blob storage docs.microsoft.com/en-us/azure/cdn/cdn-create-a-storage-account-with-cdn .
- Set cdnUrl attribute in AssetsConnectionString. For example: provider = AzureBlobStorage; rootPath = assets; DefaultEndpointsProtocol = https; AccountName = {AccountName}; AccountKey = {AccountKey}; cdnUrl = abc.azureedge.net
We compared the work before and after. For example, a jpeg file with a size of 82 KB size, we received in 126 ms, instead of 259 ms.
Before 259 ms
After 126 ms
Make Scale Up for the duration of the promotion.
By default, we use the Azure S2: Medium computational resources for App Services, but if we need to use large CPU or memory resources, we were ready to increase the plan to S3: Large.
The tests conducted in Azure showed that the change in the amount of computing resources in the direction of increasing occurs on average within 2-5 minutes and without down-time.
Configure rules for automatic Scale Out.
Scale Out allows you to configure the number of environments in which the application runs. In the basic setting, we have set up rules for automatically increasing / decreasing the number of roles from 2 to 5, depending on the metrics.
With the usual work of the application, this can significantly save IT administrator time and budget.
For ourselves, we have chosen the following settings:
- Increase instance count by 1 if CPUPercantage> 80 or MemoryPercentage> 90 over last 10 minutes.
- Decrease instance count by 1 if CPUPercantage <70 and MemoryPercentage <70 over last 10 minutes.
Unfortunately, it turned out that with peak loads, the automatic rules are too slow - they are late and in the end we had to switch to manual control. In normal times, the automatic Scale Out works perfectly.
Part 2. Black Friday
Morning has come. In the browser, developers and system administrators open Live Stream metrics. Activated promotions and passed the first email-list. In Live Stream, we see an increase in the number of requests. More resources are required, but automatic rules do not have time to react to these changes.
We see an increase in Response Time. A new instance rises and the situation stabilizes.
A wave of distribution passes and resources are automatically reduced. The next wave - the picture repeats. We decide to switch to manual control of the amount of the instance in order to leave resources with a margin.
Customers note that the site works much faster, despite the load.
Here is the distribution of the number of requests from the duration:
Black Friday morning from 7:40 to 9:40. Step 2 minutes.
And this is the usual Friday:
An increase in the number of orders led to an increase in the load on the database to critical parameters. We usually assume that the operating parameter is 40% -50% DTU percentage.
On Black Friday, the load increased to 70-80%. It was decided to raise the resources to 100 DTU. Changes to the plan on the fly also passed within 2-3 minutes and without down-time, which could not but rejoice. This resolved potential stock issues. Further more requests, more traffic. More orders. And so it is until one in the morning on Saturdays. Then they all went to sleep.
Part 3. Retrospective
We stood black Friday!
The site pleased customers with discounts, speed and 100% availability. The customers of the store, in turn, pleased the owners of the service with a number of orders.
Yes, the cost of IT resources for this day increased by 3 times, but the revenue for that day grew 100 times compared with the usual day. The main costs were associated with an increase in the number of application instances and a raising of the database plan, after the end of black Friday, we returned to the previous plans.
As the calculations showed, the CDN connection increased the cost of IT resources by only $ 2 / day, so the client decided to leave this configuration on an ongoing basis.
According to our estimates, the transfer of static resources to CDN has improved the following indicators:
- Reduced average response time by 400ms.
- Reduced Page View Load time by 1.5 seconds.
What's next?
- We will hide all our sites behind the CDN in order to remove the re-rendering of the home page, category pages and product cards. In our project we can do this, because the current prices, discounts for the client, shopping cart, etc. are loaded through javascript.
- We will activate API Management between the front end and back end systems.
- Add caching to improve performance in Azure API Management docs.microsoft.com/en-us/azure/api-management/api-management-howto-cache .
- Let's place App Services on several geolocations.
Well, we will prepare for the new black Friday 2018!
Source: https://habr.com/ru/post/348588/
All Articles