About disk counting in the cloud
We continue the cycle of articles devoted to the accounting of Selektel cloud resources.
Processor time and memory were discussed last year, now the turn of the disks has come.
There are three resources associated with the disk, each of which is counted separately:
')
Processor time (i.e., processor) and RAM, which we talked about earlier, are the essential resources of a virtual machine. If they are not there, then there is no virtual machine itself. But it is possible (albeit difficult) to imagine a virtual machine without disks and / or without network interfaces. In addition, the same disks can be connected to different virtual machines. Thus the question arises: how to take into account the disks that were first in one machine, then in another, and now they are not connected at all?
Our early accounting model implied that all these resources are attributed to the virtual machine to which they are connected. But this caused a lot of ambiguity, and we abandoned this model, returning to the model used in the Xen Cloud Platform. The picture is a simplified version of this model. Blue shows what belongs to the user, green shows the names of the objects that are being counted.
In the new model, the accounting is completely separate, for each component consumption is considered separately. For convenience of output to the user, these values ​​are summed up and shown as if they belong to the same machine, but in fact, the virtual machine has only two of its own accounting resources - this is the processor and memory. The remaining components - the disk and the network form three separate entities (I did not make a reservation - three). These are VDI, VBD and VIF. (Virtual Disk Image, Virtual Block Device, Virtual Network Interface). If with VIF everything is clear, then with VDI / VBD not quite. Why are there two of them?
The fact is that VDI is only “disk space”. It is not connected with any machine, it does not know anything about read / write operations. The object of accounting for VDI is only one value - “storage”. For the example in the figure, one disk (VDI) is not connected to the machine — it is only stored.
But VBD is just a driver that performs disk operations. It is he who counts the number of operations and the amount of read / written data. VBD interconnects VDI and VM (virtual machine).
Every time a disk connects to a virtual machine, a new VBD is created.
Simply? Yes. Difficulties begin further when we try to understand, “and when exactly will disk operations occur?” The Linux kernel developers spent a lot of time debugging and improving the caching algorithm - and it works. As the statistics show, the more loaded the machine is, the more it has write requests compared to read requests. Conversely, the less loaded the machine is, the more uncached read requests (in percent, of course) as compared to write requests due to the fact that almost every file is read “for the first time” (since the download).
Actually, here are the real numbers: across the cloud, we have a ratio of read to write operations of 3/4. On the first available car with a consumption of less than 5 p / day, the write-to-read ratio is 2: 1, to be exact, 2.2 million to 1.1 million. And on a very seriously loaded machine (about 450 rubles / day) the read / write ratio is 0.3 million to 4 million (most of the expenses of the machine are outgoing traffic, more than 300 GB / day). It is quite obvious that it’s impossible to give 300GB from nowhere, so this is the place where disk caching works well. (By the way, anticipating the question of why we have such a large amount of memory in MOD - just for such caching).
Thus, it is impossible to predict whether a particular file operation will create any actions with a block device. Although it is possible to predict that the first reading of the file will cause a read operation, and repeated reading with some probability, no.
Immediately I anticipate questions that appear periodically in tickets: any operations with virtual file systems (/ proc, / sys), mounted nfs-balls, iscsi-devices, diskless drbd-balls, etc. are not the object of accounting - only calls to / dev / xvd * devices are counted. Obviously, any operations with fifo, unix sockets, etc. also do not cause disk operations. But the banal sync command, on the contrary, will cause a surge of activity.
Processor time and memory were discussed last year, now the turn of the disks has come.
There are three resources associated with the disk, each of which is counted separately:
- disk storage
- read / write capacity
- and the number of disk operations
')
Virtual machine device
Processor time (i.e., processor) and RAM, which we talked about earlier, are the essential resources of a virtual machine. If they are not there, then there is no virtual machine itself. But it is possible (albeit difficult) to imagine a virtual machine without disks and / or without network interfaces. In addition, the same disks can be connected to different virtual machines. Thus the question arises: how to take into account the disks that were first in one machine, then in another, and now they are not connected at all?Our early accounting model implied that all these resources are attributed to the virtual machine to which they are connected. But this caused a lot of ambiguity, and we abandoned this model, returning to the model used in the Xen Cloud Platform. The picture is a simplified version of this model. Blue shows what belongs to the user, green shows the names of the objects that are being counted.
In the new model, the accounting is completely separate, for each component consumption is considered separately. For convenience of output to the user, these values ​​are summed up and shown as if they belong to the same machine, but in fact, the virtual machine has only two of its own accounting resources - this is the processor and memory. The remaining components - the disk and the network form three separate entities (I did not make a reservation - three). These are VDI, VBD and VIF. (Virtual Disk Image, Virtual Block Device, Virtual Network Interface). If with VIF everything is clear, then with VDI / VBD not quite. Why are there two of them?
The fact is that VDI is only “disk space”. It is not connected with any machine, it does not know anything about read / write operations. The object of accounting for VDI is only one value - “storage”. For the example in the figure, one disk (VDI) is not connected to the machine — it is only stored.
But VBD is just a driver that performs disk operations. It is he who counts the number of operations and the amount of read / written data. VBD interconnects VDI and VM (virtual machine).
Every time a disk connects to a virtual machine, a new VBD is created.
What is a “disk operation”?
From the point of view of Linux, VBD is the same block device as many others, with its own driver providing a standard interface. / dev / xvda is no different from / dev / sda or / dev / hda. When the operating system wants to read or write something, it calls the block device driver with the appropriate command.Simply? Yes. Difficulties begin further when we try to understand, “and when exactly will disk operations occur?” The Linux kernel developers spent a lot of time debugging and improving the caching algorithm - and it works. As the statistics show, the more loaded the machine is, the more it has write requests compared to read requests. Conversely, the less loaded the machine is, the more uncached read requests (in percent, of course) as compared to write requests due to the fact that almost every file is read “for the first time” (since the download).
Actually, here are the real numbers: across the cloud, we have a ratio of read to write operations of 3/4. On the first available car with a consumption of less than 5 p / day, the write-to-read ratio is 2: 1, to be exact, 2.2 million to 1.1 million. And on a very seriously loaded machine (about 450 rubles / day) the read / write ratio is 0.3 million to 4 million (most of the expenses of the machine are outgoing traffic, more than 300 GB / day). It is quite obvious that it’s impossible to give 300GB from nowhere, so this is the place where disk caching works well. (By the way, anticipating the question of why we have such a large amount of memory in MOD - just for such caching).
Thus, it is impossible to predict whether a particular file operation will create any actions with a block device. Although it is possible to predict that the first reading of the file will cause a read operation, and repeated reading with some probability, no.
Immediately I anticipate questions that appear periodically in tickets: any operations with virtual file systems (/ proc, / sys), mounted nfs-balls, iscsi-devices, diskless drbd-balls, etc. are not the object of accounting - only calls to / dev / xvd * devices are counted. Obviously, any operations with fifo, unix sockets, etc. also do not cause disk operations. But the banal sync command, on the contrary, will cause a surge of activity.
Source: https://habr.com/ru/post/111226/
All Articles