Online store Wildberries: what happens on the other side of the monitor
Today we decided to acquaint you with a behind the scenes of a completely different direction of human activity and consider the background of such a complex system, the servers of which are tested by thousands of transactions per second and the price of software or hardware failure may be too high. So, today we are visiting one of the largest Russian online clothing stores Wildberries
How does an online store work? To make it clearer, we propose to break up the process into main stages, examine them and create an overall picture of what is happening.
In the photo: warehouse online store Wildberries
')
So, first you need to decide what product will replenish the assortment of the store, then ask the supplier for the necessary information on the selected product (composition, equipment, price, etc.). Next is the initial monitoring of prices, and all the necessary information about the product is poured into the system. Having coordinated the order with the supplier and having received all the necessary documents from him, the manager gives the go-ahead for the shipment of the goods.
After the supplier delivers the goods to the warehouse, the second stage begins. Now the warehouse workers are checking the compliance of the new batch with the items that were entered in the system, checking the goods for factory defects, repacking the goods and pasting internal bar codes. In the event of a marriage, non-compliance of the received goods information in the system or requirements for supply, the goods are not accepted to the warehouse. In this way, the store tries to protect the customer from buying low-quality goods and reduce the number of so-called “wrong investments”. After this, the new warehouse unit is measured (do not forget that we are talking about a clothing store), the photographer takes pictures on the models, and the diligent copywriter writes the accompanying text to the site.
We will not go deep into mechanisms according to which the client gets on the page with the goods. It is clear that in one degree or another he comes to the online store prepared: perhaps he is looking for a specific thing or just became interested in an advertising link. It is much more interesting to see how the system responds to customer actions, in a word, what happens when a user has already reached the site. We know that one of the most important factors that determines how much a client lingers on a site is the speed of processing its search poll. If, according to the specified filters (blue dress, size S, knee-length), the result is given 10-15 seconds, the bored client closes the tab and the client will automatically stop. By the way, on average in Russia, the site response time to user actions is 4 seconds or more, but Wildberries managed to reduce this time to two seconds. To achieve this, the system uses proxy with nginx: behind one external IP address there are several nginx servers on the VMWare platform, which is installed on the Dell PowerEdge R620 server. Behind them, in turn, are already located directly web server. In addition, the most frequently requested data is stored in the cache memory of servers and is issued almost instantly. Fail safety of web servers is provided through clustering: at least two servers completely duplicate each other and operate in master mode. They are connected to a shared storage Dell EqualLogic PS, operating in hybrid mode with SSD and SAS. This approach made it possible to achieve high response results, despite the enormous load on the servers: the number of processed requests that went through the “outer perimeter” - outcash - during one day varies from 750 thousand to 1.5 million. Yes, this is such an uneasy hardware substrate under a simple and clear interface that helps in a few seconds to find a particular product.
The next stage begins when the customer is determined with the choice and confirms his order. In certain cases, confirmation occurs via SMS, but most often the operator calls the customer back. By the way, an important point: the system provides for automatic reservation of goods at the very moment when the client has formed an order, but has not yet received confirmation. And all because with large sales volumes, goods can be led away from under the nose while the customer is dialing the customer.
For employees of the store, this stage is not so quick and easy, because the work on processing and ordering is not as simple as it may seem. Initially, customer requests go to the warehouse, on the basis of which a so-called assembly sheet is formed, according to which the warehouse worker collects the necessary goods. The warehouse where Wildberries' goods are stored is divided into zones: in one place, for example, shoes are located, in the other - things hanging on hangers, in the third - linen, etc. The assembly sheet is not tied to a specific order, but is formed in such a way as to minimize the movement of an employee through the rather large warehouse premises and save his time. This algorithm is generated by the system automatically. By the way, it takes into account not only the optimal route, but also the number of things that a warehouse worker can carry away at one time (you know, 20 pairs of socks and 20 pairs of shoes are not the same thing).
Collected items are brought to the place of formation of orders, on which there are several dozen shelves with numbered cells (they are also jokingly called breakwaters). The employee scans the barcodes of the goods, and the system gives him the number of the rack and the cell in which this or that order is collected. They are taken away after the entire rack has been completely filled: the completed goods are sorted into vehicles that deliver them to different cities. Then the order takes the courier and takes it to the buyer at a predetermined time.
Thus, a single IT-system online store combines the frontend, implemented on Linux, and the backend - on Windows. To support and develop the system, there is a large staff of developers and system administrators, and the project’s computer park is very diverse. In places that do not require intensive workload (for example, to perform certain logistical tasks or in a call center), Wyse terminals are installed. There, where goods are accepted and orders are collected, and these are actions that require speed and responsiveness of the interface, personal computers are installed. The staff processing photos of products for the site, are PC with discrete graphics and a large amount of RAM.
The entire information base of the project - a warehouse, a call center, a store and an ERP system (and this is about 20 TB of constantly changing data), is controlled using M620 blade servers. They are installed in the chassis M1000e, which, to date, already involved four.
The use of virtualization allows you to avoid the risk of data loss in case of failure or failure of one or even several blade servers: the current load is evenly distributed among the other servers before replacing the failed element. All critical nodes are duplicated, and during the design, a certain amount of power is initially laid. By the way, Wildberries rents racks at once in two data centers, this allows the site to remain accessible even in the event of an accident in one of the data centers, an accident at an electrical substation or another man-made disaster. A direct network connection with the largest Internet service providers and traffic exchange points (MSK-IX, etc.) ensures that most user requests pass through the minimum number of network nodes before it reaches the company's servers.
The choice in favor of Dell was not accidental. The equipment of many vendors was tested by Wildberries employees with the help of synthetic tests and on non-critical services, and the decisions from the finalists - in the “combat” mode right on the store floor, and in terms of technical characteristics, as well as the level of technical support and maintenance, the choice was made in our direction . It should be added that the equipment performance indicators are monitored in real time using the advanced Zabbix functionality. All parameters of the critical nodes are displayed on the monitors (and, by the way, there are six of them!) Of the duty specialist. The current operating parameters are recorded absolutely from all devices of the enterprise: servers, disk racks, uninterruptible power supplies, network devices and communication channels, and not only in the central office, but also in the regions. In addition, all the company's specialists have their own areas of responsibility and, if some indicators deviate from the norm, they are automatically sent notifications via email, SMS and Skype.
However, Wildberries employees secretly admitted that the main and most valuable resource is not a server, but a solid team of professionals who care about their work. Lovely to hear!
PS If you want to learn more about the capabilities of Dell equipment for organizations working in the field of e-commerce, feel free to write to Marat Rakayev . Our experts will help solve almost any problem.
How does an online store work? To make it clearer, we propose to break up the process into main stages, examine them and create an overall picture of what is happening.
In the photo: warehouse online store Wildberries
')
So, first you need to decide what product will replenish the assortment of the store, then ask the supplier for the necessary information on the selected product (composition, equipment, price, etc.). Next is the initial monitoring of prices, and all the necessary information about the product is poured into the system. Having coordinated the order with the supplier and having received all the necessary documents from him, the manager gives the go-ahead for the shipment of the goods.
After the supplier delivers the goods to the warehouse, the second stage begins. Now the warehouse workers are checking the compliance of the new batch with the items that were entered in the system, checking the goods for factory defects, repacking the goods and pasting internal bar codes. In the event of a marriage, non-compliance of the received goods information in the system or requirements for supply, the goods are not accepted to the warehouse. In this way, the store tries to protect the customer from buying low-quality goods and reduce the number of so-called “wrong investments”. After this, the new warehouse unit is measured (do not forget that we are talking about a clothing store), the photographer takes pictures on the models, and the diligent copywriter writes the accompanying text to the site.
We will not go deep into mechanisms according to which the client gets on the page with the goods. It is clear that in one degree or another he comes to the online store prepared: perhaps he is looking for a specific thing or just became interested in an advertising link. It is much more interesting to see how the system responds to customer actions, in a word, what happens when a user has already reached the site. We know that one of the most important factors that determines how much a client lingers on a site is the speed of processing its search poll. If, according to the specified filters (blue dress, size S, knee-length), the result is given 10-15 seconds, the bored client closes the tab and the client will automatically stop. By the way, on average in Russia, the site response time to user actions is 4 seconds or more, but Wildberries managed to reduce this time to two seconds. To achieve this, the system uses proxy with nginx: behind one external IP address there are several nginx servers on the VMWare platform, which is installed on the Dell PowerEdge R620 server. Behind them, in turn, are already located directly web server. In addition, the most frequently requested data is stored in the cache memory of servers and is issued almost instantly. Fail safety of web servers is provided through clustering: at least two servers completely duplicate each other and operate in master mode. They are connected to a shared storage Dell EqualLogic PS, operating in hybrid mode with SSD and SAS. This approach made it possible to achieve high response results, despite the enormous load on the servers: the number of processed requests that went through the “outer perimeter” - outcash - during one day varies from 750 thousand to 1.5 million. Yes, this is such an uneasy hardware substrate under a simple and clear interface that helps in a few seconds to find a particular product.
The next stage begins when the customer is determined with the choice and confirms his order. In certain cases, confirmation occurs via SMS, but most often the operator calls the customer back. By the way, an important point: the system provides for automatic reservation of goods at the very moment when the client has formed an order, but has not yet received confirmation. And all because with large sales volumes, goods can be led away from under the nose while the customer is dialing the customer.
For employees of the store, this stage is not so quick and easy, because the work on processing and ordering is not as simple as it may seem. Initially, customer requests go to the warehouse, on the basis of which a so-called assembly sheet is formed, according to which the warehouse worker collects the necessary goods. The warehouse where Wildberries' goods are stored is divided into zones: in one place, for example, shoes are located, in the other - things hanging on hangers, in the third - linen, etc. The assembly sheet is not tied to a specific order, but is formed in such a way as to minimize the movement of an employee through the rather large warehouse premises and save his time. This algorithm is generated by the system automatically. By the way, it takes into account not only the optimal route, but also the number of things that a warehouse worker can carry away at one time (you know, 20 pairs of socks and 20 pairs of shoes are not the same thing).
Collected items are brought to the place of formation of orders, on which there are several dozen shelves with numbered cells (they are also jokingly called breakwaters). The employee scans the barcodes of the goods, and the system gives him the number of the rack and the cell in which this or that order is collected. They are taken away after the entire rack has been completely filled: the completed goods are sorted into vehicles that deliver them to different cities. Then the order takes the courier and takes it to the buyer at a predetermined time.
Thus, a single IT-system online store combines the frontend, implemented on Linux, and the backend - on Windows. To support and develop the system, there is a large staff of developers and system administrators, and the project’s computer park is very diverse. In places that do not require intensive workload (for example, to perform certain logistical tasks or in a call center), Wyse terminals are installed. There, where goods are accepted and orders are collected, and these are actions that require speed and responsiveness of the interface, personal computers are installed. The staff processing photos of products for the site, are PC with discrete graphics and a large amount of RAM.
The entire information base of the project - a warehouse, a call center, a store and an ERP system (and this is about 20 TB of constantly changing data), is controlled using M620 blade servers. They are installed in the chassis M1000e, which, to date, already involved four.
The use of virtualization allows you to avoid the risk of data loss in case of failure or failure of one or even several blade servers: the current load is evenly distributed among the other servers before replacing the failed element. All critical nodes are duplicated, and during the design, a certain amount of power is initially laid. By the way, Wildberries rents racks at once in two data centers, this allows the site to remain accessible even in the event of an accident in one of the data centers, an accident at an electrical substation or another man-made disaster. A direct network connection with the largest Internet service providers and traffic exchange points (MSK-IX, etc.) ensures that most user requests pass through the minimum number of network nodes before it reaches the company's servers.
The choice in favor of Dell was not accidental. The equipment of many vendors was tested by Wildberries employees with the help of synthetic tests and on non-critical services, and the decisions from the finalists - in the “combat” mode right on the store floor, and in terms of technical characteristics, as well as the level of technical support and maintenance, the choice was made in our direction . It should be added that the equipment performance indicators are monitored in real time using the advanced Zabbix functionality. All parameters of the critical nodes are displayed on the monitors (and, by the way, there are six of them!) Of the duty specialist. The current operating parameters are recorded absolutely from all devices of the enterprise: servers, disk racks, uninterruptible power supplies, network devices and communication channels, and not only in the central office, but also in the regions. In addition, all the company's specialists have their own areas of responsibility and, if some indicators deviate from the norm, they are automatically sent notifications via email, SMS and Skype.
However, Wildberries employees secretly admitted that the main and most valuable resource is not a server, but a solid team of professionals who care about their work. Lovely to hear!
PS If you want to learn more about the capabilities of Dell equipment for organizations working in the field of e-commerce, feel free to write to Marat Rakayev . Our experts will help solve almost any problem.
Source: https://habr.com/ru/post/207568/
All Articles