Testing with IOmeter

The program IOmeter is a popular tool for testing the performance of the disk subsystem and the local network. The test is "100% synthetics."

Unfortunately, some non-obviousness of the testing process in it, an outdated appearance, the absence of a full-fledged online help, and documentation, as well as a Russian-language description, often cause difficulties when trying to use it. Also on the Internet there is practically no detailed description of the method of working with it, and a description of the terms used and features.
')
A special thanks to the author wants to express to the owner of the blog aboutnetapp , whose post about practical testing of storage systems with the help of IOmeter, kicked the author to write this article.

In the world of testing software, there are currently accepted two main models of testing. The first is an imitation of some practical work. Typically, such a test includes a number of real-world applications that, with the help of scripts, perform some kind of imitation of practical work during testing — they open Vord documents, make changes to them, create and save spreadsheets. A classic example of such testing applications is the Winbench test suite.

With the indisputable advantages of such “real” tests, they also have a serious drawback, for example, it is difficult or practically impossible to create a test that goes beyond some typical actions, for example, office work.
Secondly, it is often difficult to interpret the results, since the measurement technique is rather dark and confused within the applications used in simulation testing.

For this reason, “synthetic” tests are often used for load testing of server systems, especially their disk subsystems, which are some kind of fairly clear algorithm with which you can try to simulate some kind of load that is close to real. Due to obvious algorithmic clarity, such tests are very convenient for comparative testing, as well as analyzing performance problems.

The popular IOmeter test falls into the second category, "synthetic" tests.

This program was originally developed at Intel, and subsequently, when its development at Intel stopped, was transferred to open source, the community of which continues its gradual development.

Despite the fact that the program has its own website , I recommend downloading it from the Sourceforge project page .
The fact is that the Download link from offsite is given to the last Stable, which froze for a long time (2006.07.27), whereas the last one on Sourceforge is Devel 2008.06.22 rc2, which made quite a lot of significant changes and improvements, so I recommend using it.

The following binaries are compiled for this version on Sourceforge: X86_64, IA64 (native Itanium), Win32 and Linux x86_64. From sorts, you can (try) to assemble Dynamo under the desired platform, so, after a small leveling and shamanism, he gathered under the Solaris 10 / SPARC I needed.

The fun will start from the beginning. After installation, you will find two executable files in the program folder: iometer.exe and Dynamo.exe . The fact is that IOmeter is built according to a “two-component” scheme, which allows not only to separate the system under test from the control system, but also to manage at once the set of tested computers. Iometer.exe is a control program with GUI, and Dynamo.exe is a command line utility for the corresponding platform.

This separation is very convenient, for example, for me, when my test system is in a data center in the next building, where, frankly, it is not too comfortable to be (noisy, cold, blowing, and even trivial to sit on nothing). Using such a two-component testing system, I can run Load Generator (Dynamo.exe) on the test servers in the data center, go to the office, and from there connect to them over the local network and manage all of them remotely, getting the results on my laptop.

However, if you do not have the task to do “distributed” testing, then you can not bother with all this, and just run iometer.exe. At the same time you will see the GUI-manager screen, and Dynamo will start locally afterwards.

Both Dynamo and IOmeter can be run with the /? In doing so, it is expected that you will receive a list of some useful startup keys.

IOmeter:
Iometer config_file [result_file [timeout_value]]
Iometer [/ c config_file] [/ r result_file] [/ t timeout_vaue] [/ p port] [/ m 1]

Config_file - config file. There must be a valid icf.
Result_file - the result file (in CSV format), supplemented, if it does not exist, it is created.
Timeout_value - the number of seconds that we wait for the participants to login.
Port_number - the port that we are listening for login.
/ m - show a large gauge when testing.

If both config_file and results_file are set , then Iometer tries to start in batch mode, without human intervention, start and stop tests, if the launch is defined in the Test Setup tab of the loaded config_file , record the results in result_file , and when finished, close iometer.

The ability to run a batch is very convenient and useful, since tests can take a total of many hours.

Dynamo:
dynamo [/ i iometer_computer_name / m manager_computer_name] [/ n manager_name] [/ c cpu_affinity] [/ p login_port_number]

iometer_computer_name is the name or IP address of the computer running Iometer, to which we will log in to Dynamo. Without this parameter, Dynamo will look for it on localhost.
manager_name is the name of the computer with Dynamo, in terms of IOmeter it is called “manager”. The default is the host name. It is important if you use it in the IOmeter config, for example, assign it a specific task to it in the previously recorded config.
manager_computer_name is the name or IP of the Dynamo computer with which the GUI will communicate with this Dynamo. The default is the IP address of the first network card.
login_port_number is the port on which the login will take place between Dynamo and the GUI. The default is 1066. Do not forget to open it, in the case of firewalls.
cpu_affinity is the number of a specific CPU on which Dynamo will run on a multiprocessor system. If not defined, then on the first CPU.

Thus, the line to run can be:
track @ unixbox> ./dynamo / i 192.168.1.100 / m 192.168.1.10 / n LG1

At the same time, we will launch the Dynamo instance on the machine with IP 192.168.1.10, which will appear in the GUI located at 192.168.1.100 and will be called LG1 in it

Another key not specified in /? this is the / force_raw switch . This key says to ignore the file systems found on the disk, and test it as a raw device. BE CAREFUL with its use, since with it the disk formatted by the file system will be corrupted during testing.

When launched over the network, the login process from remote Dynamo takes a few seconds; in the case of a local launch, everything happens instantly.



One of the key concepts in IOmeter is “worker”.
Worker is an object that performs the test task. That he is assigned to an action.
By default, Dynamo (in terms of IOmeter - “manager”, which is a bit unexpected, it is more logical to call this manager GUI, IOmeter.exe) for this pair creates Workers in an amount equal to the processor cores detected on the system. Keep in mind that if Hyperthreading is present, they will be recognized as two cores, which may not be quite good for running an intensive load application on each of them.
You can use any number of workers from the existing ones, you can, of course, assign a test to only one.

So, launched IOmeter.



On the left in the Topology column, our Dynamo will log in with us, on local or remote machines.
On my Asus 901Go laptop, on the Atom N270 processor, as you can see, there were two cores and two workers were created.

The first tab is called Disk Targets .
When you first start, you will see in the Targets column for the selected manager and his worker all those disks that he sees (if you run Dynamo on a remote machine, these will be the disks that he sees there, on his remote car). The disks with the recognized file system on themselves will be painted yellow, the raw partitions, the unformatted files or the filesystem that are not recognized by this OS will be shown in blue.

At first, all discs will be crossed with a red line. This means that the test file, which is necessary for testing on the file system, was not found on them (in the case of a raw partition, this is not necessary, and the “blue” disks are immediately ready for testing).
You can select a disk for testing by putting a cross in the checkbox.
The test file iobw.tst is created in the root of the corresponding disk when it is first started, and, ATTENTION, by default it takes up all the free disk space! This can be a long time, and ... uh ... unexpectedly;) for applications or OS, if you experiment, for example, on a working computer.
In order to limit its size, you can set its size in the Maximum disk size field, the size is indicated in SECTORS, 512 bytes in size!
For example, 65535 sectors are 32MB.
Starting disk sector can specify the starting sector in the case of a raw partition.
# of Outstanding IOs is an important parameter worth a detailed description.

Each task, each program executed on a computer usually performs input / output to disk. As a rule, they do this not with one stream, but simultaneously write and read several parallel processes from disk.
It is considered that 4-8 I / O streams generate quite simple applications, such as notepad.exe and calc.exe. Large MS Word programs — 32–64 are parallel to the stream, the maximum can be considered 256, corresponding to a large enterprise database like Oracle, under a good load.
The number of simultaneous I / O streams is "# of Outstanding IOs".
In order to test the system on a load close to combat, this parameter should be actively used.
But now, here, in this place, we will not touch this parameter, we will assign work with him separately later.

Go to the tab Network Targets .



With the help of IOmeter, you can also test the performance of the local network, but I have never done this, so here I can only say that you can add the Network Worker by clicking on the button on the toolbar, highlighting the desired “manager” on the left.

The next tab, Access Specification, is important. It is on it that the “test pattern” is created and set, the profile of the test load that will load our system.


Selecting the Worker on the left, select one of the load profiles we have, and add it with the << Add button to the left panel. Thus, we assign a testing task to an existing worker. If we plan to launch several workers in parallel (or sequentially), we also add a load pattern to them.
If on the left we choose not one specific worker, but an element that includes them, then the assigned pattern will be assigned to all workers below the hierarchy.
You can add and not one pattern on the worker.

You can create patterns yourself, or you can use an already specified set. Once, when IOmeter was still produced by Intel, several test patterns developed by Intel were bundled with it.
Here: iometer2.icf you can download the configuration file for the IOmeter, with a set of load test patterns visible in the screenshot, with more or less success imitating certain typical loads.
You can also correct, or even draw your own. Click New or select a pattern and click Edit (or Edit Copy), and a window for setting parameters will open.



By drawing a pattern, we set the blocks that the test will access to the storage system (or the network if we test the network interface).
Block size is the upper left panel: Transfer Request Size .
The relative number of such blocks: Percent of Access Specification . If you specify a complex pattern, then the total number of all should be equal to 100%.
The extreme left - Percent Read / Write Distribution - determines what will be the ratio of records and reads on this pattern. So, for example, for File Server 80% read and 20% write are set, and for the web server, from which, usually, only read, 100% read is set.
The fourth important parameter Percent Random / Sequental Distribution - the nature of access, is selected between Random and Sequental, random and sequential access to the test file.
The remaining parameters are rarely used.

The penultimate tab is Result Display .
On it we will see how our testing goes, its results on the selected parameters.



Two important controls on this tab are the Result Since switch and the Update Frequency engine.
The first determines what results will be displayed on the “thermometers” below - the total from the beginning of the test ( Start of Test ) or instantaneous, the current ( Last Update ).
The regulator determines the frequency of changing the readings on the screen.
Please note that all this affects only the display in the program; the maximum value achieved during the tests will be recorded in the result file.

The default display is 6 indicators. They can be changed by clicking on the “button” with the parameter and selecting the option to display in the menu. It also affects only the display in the program, they will be written into the resulting file.



And finally, let's proceed to setting up the testing process, tab Test Setup



For the selected worker, I remind you again that all actions are performed and assigned to a specific worker object, we write down the Test Description , so that we can figure out what the results are in the general CSV file of results.
Set the test run time: Run Time
And set the “acceleration time”, a preliminary period for “warming up”: Ramp Up Time .

This could be limited to if we wanted to conduct one test. But, as a rule, it is interesting to do several at once in reception, rather than poking buttons, waiting for the end of each. For this IOmeter is equipped with rich features.
In the Cycling Options drop-down list, you can set conditions for sequential execution of tests according to certain conditions and parameter changes.



For example, as I said above, I suggest not setting the # of Outstanding IOs parameter , but running the tests sequentially, with a gradual increase in it, in order to see, as a result, the dynamics of the storage system's response to the I / O load.

But there are 8 variants of Cycling Options, try to figure out the model.



By selecting the option indicated in the screenshot, you can set the fields from how many (1) to how many (256) to increase the number of threads. As a character of growth, I recommend Exponential Sepping and Power: 2. In this case, the increase will be double: 1 - 2 - 4 - 8 - 16 ... Such a growth step is sufficient to assess the load capacity of the I / O subsystem.

So, we got to the start.
Click on the toolbar button with a green flag.
We are invited to save the results to a CSV (Comma Separated Value) file with the default name of results.CSV. If this file already exists, it will be RECORDED, at the beginning of the new data the test name specified by us in the Test Setup will be indicated.
In the future, it will be possible to import this CSV into Excel and configure beautiful graphs based on its data.

During the first run for a long time, nothing visible happens as for the first run a test file is created. This is indicated by the message below: Preparing Drives . When the file is created, the red strikethrough line disappears in Disk Targets . In subsequent launches, the file is not recreated, and the test starts immediately.



In the lower left corner displays the name of the test going.
In the right, the message Ramp remaining says that the process of “warming up” is underway before the test, and Run 1 of 6 , that the first of 6 tests assigned to Cycling options is running.



To stop the current test (and continue to the next), you can click the Stop button on the toolbar, and to interrupt the entire sequence of tests, Stop All .

Now go to the Test Results tab.



“Thermometers” show the current value of the tests, and the number in the middle of it shows the current value of the selected parameter.
Run remainings: the remaining time of the current test.
The total time will be (with the previously specified parameters): (30 seconds ramp + 10 minutes of the test) * 6 tests.

During the test, its progress is displayed in the Dynamo utility window.



At the end of the tests, the CSV file with the name you chose (by default results.CSV) collects the results that can be imported into Excel and set up the necessary analytics in it.