How to evaluate the performance of storage on Linux: benchmarking using open tools
Last time, we talked about open source tools for evaluating the performance of processors and memory. Today we are talking about benchmarks for file systems and data storage systems on Linux - Interbench, Fio, Hdparm, S and Bonnie.
Photo - Daniele Levis Pelusi - Unsplash
Fio (stands for Flexible I / O Tester) creates disk I / O streams to evaluate the performance of the Linux file system. The utility can also be run on Windows — you need to install the Cygwin command line interface. A configuration guide is in the fio repository on github .
')
Fio by Jens Axboe, responsible for the Linux IO subsystem and the developer of the blktrace utility for tracing I / O operations. He created fio because he was tired of writing programs to test a specific load manually.
The utility will calculate the IOPS and system bandwidth, as well as allow you to estimate the depth of the queue I / O operations. The utility works with special files (extension .fio), in which the settings and conditions of the test are written. There are several test options, for example, there is an arbitrary write, read and rewrite. Here is an example of the file contents for the first case:
Today, fio is used by large companies - they work with the utility at SUSE , Nutanix and IBM .
The utility was written by Canadian developer Mark Lord (Mark Lord) back in 2005. It is still supported by the author and is part of many popular distributions. The main purpose of hdparm is to configure drive parameters. But the tool can be used to conduct simple benchmarks, for example, measuring the speed of reading. To do this, write a command in the console:
The system will generate a similar response:
As for the drive settings, hdparm allows you to change the amount of cache memory, modify the sleep and power settings, and also safely erase data on the SSD. But, as experts from ArchLinux warn , a careless change of system parameters can make the data on the disk inaccessible and even damage the drive. Before working with hdparm it is better to familiarize yourself with the tutorial - just write the command man hdparm in the console.
This is a set of benchmarks for evaluating I / O performance. The authors of the utility were the development team from the AlgoDev group, which includes employees of the Italian University of Modena and Reggio Emilia .
All benchmarks are bash-scripts that evaluate the performance of the storage system - bandwidth, latency, work schedulers. For example, the throughput-sync.sh benchmark “bombards” the storage system with read or write requests (in this case, the already mentioned fio utility is used). Here is the code for this script .
Another script, comm_startup_lat.sh, measures the latency of reading data from a disk during a “cold cache” (when it does not have the necessary data). The code can also be found in the repository .
Photos - AgĂŞ Barros - Unsplash
A utility for evaluating file system performance, developed in 1989. It was authored by engineer Tim Bray. With the help of Bonnie, he planned to optimize the performance of the computing systems involved in the New Oxford English Dictionary project at the University of Waterloo.
Bonnie randomly reads and writes data to disk. After the utility displays parameters such as the number of bytes processed per CPU-second , as well as the level of CPU usage in percent. The source code of the benchmark can be found on Google Code .
On the basis of Bonnie built another set of tools for testing the hard disk - Bonnie ++ (written in C ++, instead of C). It provides additional benchmark tools. For example, zcav to evaluate the performance of various HDD zones. Bonnie ++ is also suitable for testing mail servers and database servers.
The tool was developed by Con Kolivas , an Australian anaesthesiologist, who is known for his contributions to the development of the Linux kernel and the work on a “ fair processor scheduler ”. Interbench helps you configure the I / O scheduler and file system settings.
Interbench emulates the behavior of the CPU scheduler when performing interactive tasks. These interactive tasks can be working with sound and video, launching computer games, or simply dragging a dialog box in the operating system.
Source code, examples and recommendations for setting up the tool can be found in the official repository on GitHub .
Photo - Daniele Levis Pelusi - Unsplash
Fio
Fio (stands for Flexible I / O Tester) creates disk I / O streams to evaluate the performance of the Linux file system. The utility can also be run on Windows — you need to install the Cygwin command line interface. A configuration guide is in the fio repository on github .
')
Fio by Jens Axboe, responsible for the Linux IO subsystem and the developer of the blktrace utility for tracing I / O operations. He created fio because he was tired of writing programs to test a specific load manually.
The utility will calculate the IOPS and system bandwidth, as well as allow you to estimate the depth of the queue I / O operations. The utility works with special files (extension .fio), in which the settings and conditions of the test are written. There are several test options, for example, there is an arbitrary write, read and rewrite. Here is an example of the file contents for the first case:
[global] name=fio-rand-read filename=fio-rand-read rw=randread bs=4K direct=0 numjobs=1 time_based=1 runtime=900 Today, fio is used by large companies - they work with the utility at SUSE , Nutanix and IBM .
Hdparm
The utility was written by Canadian developer Mark Lord (Mark Lord) back in 2005. It is still supported by the author and is part of many popular distributions. The main purpose of hdparm is to configure drive parameters. But the tool can be used to conduct simple benchmarks, for example, measuring the speed of reading. To do this, write a command in the console:
$ sudo hdparm -t /dev/sdb The system will generate a similar response:
Timing buffered disk reads: 242 MB in 3.01 seconds = 80.30 MB/sec As for the drive settings, hdparm allows you to change the amount of cache memory, modify the sleep and power settings, and also safely erase data on the SSD. But, as experts from ArchLinux warn , a careless change of system parameters can make the data on the disk inaccessible and even damage the drive. Before working with hdparm it is better to familiarize yourself with the tutorial - just write the command man hdparm in the console.
S
This is a set of benchmarks for evaluating I / O performance. The authors of the utility were the development team from the AlgoDev group, which includes employees of the Italian University of Modena and Reggio Emilia .
All benchmarks are bash-scripts that evaluate the performance of the storage system - bandwidth, latency, work schedulers. For example, the throughput-sync.sh benchmark “bombards” the storage system with read or write requests (in this case, the already mentioned fio utility is used). Here is the code for this script .
Another script, comm_startup_lat.sh, measures the latency of reading data from a disk during a “cold cache” (when it does not have the necessary data). The code can also be found in the repository .
Photos - AgĂŞ Barros - Unsplash
Bonnie
A utility for evaluating file system performance, developed in 1989. It was authored by engineer Tim Bray. With the help of Bonnie, he planned to optimize the performance of the computing systems involved in the New Oxford English Dictionary project at the University of Waterloo.
Bonnie randomly reads and writes data to disk. After the utility displays parameters such as the number of bytes processed per CPU-second , as well as the level of CPU usage in percent. The source code of the benchmark can be found on Google Code .
On the basis of Bonnie built another set of tools for testing the hard disk - Bonnie ++ (written in C ++, instead of C). It provides additional benchmark tools. For example, zcav to evaluate the performance of various HDD zones. Bonnie ++ is also suitable for testing mail servers and database servers.
Interbench
The tool was developed by Con Kolivas , an Australian anaesthesiologist, who is known for his contributions to the development of the Linux kernel and the work on a “ fair processor scheduler ”. Interbench helps you configure the I / O scheduler and file system settings.
Interbench emulates the behavior of the CPU scheduler when performing interactive tasks. These interactive tasks can be working with sound and video, launching computer games, or simply dragging a dialog box in the operating system.
Source code, examples and recommendations for setting up the tool can be found in the official repository on GitHub .
What we write about in our blogs:Benchmarks for Linux servers: 5 open tools
File backup: how to hedge against data loss
Inspection of gadgets at the border: how to act in order not to lose confidential data?
Source: https://habr.com/ru/post/458204/
All Articles