goader - console benchmark with a bias for writing and reading files

goader is a console benchmark with a simple configuration and support for various backends for testing. The name comes from go and loader, and also has its meaning in English, "customized spear, stick"

At the moment, you can test (argument -requests-engine ):

• http (GET requests or GET + PUT)
• disk
• s3 (With authorization by ACCESS / SECRET keys, endpoint is required, but it gives the opportunity to check private s3, signature ver4 is currently not supported, but I plan)
• null and sleep to test the benchmark itself

The bias is made to write and read files, not pages.

Usage example

goader -rps=300 -wps=150 -min-body-size=1 -max-body-size=128k --max-requests=1000 -requests-engine=disk -url=tmp/NN/RRRRR 

The points appear in real time in accordance with each request; in due time, this allowed me to visually identify problems, in the case that the numbers would give little. In case of errors in their place will be E

There are many utilities for load testing, but I personally have a number of complaints about them, which inspired me to write my own ...

Problem number 1, what are we testing?

The emphasis is on “the number of parallel threads”, the search for the maximum load, the maximum number of threads. Or a fixed number of threads. This is of course great, but personally in my case, in real life, when replacing any component of the system, it was quite rare that this question arose how much it would pull. When replacing components, there are often two more questions and they are different:

a) Response at a known load. Like for example 50 PUT / s, 300 GET / s. Check the response time on the old and new system. From the replacement of components, the number of users will not grow, the goal is usually to improve the responsiveness of the system.

 goader -rps=300 -wps=50 -min-body-size=1 -max-body-size=128k -requests-engine=upload -url=http://localhost/files/user_NNNNN/file_RRRR 

b) We still want to give more than it was, we want to know the maximum of the system. Again, what is it? Number of clients that the server can handle? Number of parallel customers who receive the N-th percentage of errors? Very often there are no errors, but just get a huge response time.

Therefore, for myself, the maximum of the system I defined as “the number of parallel clients, at which the response time does not exceed the specified value."

 goader -rpw 2 -max-latency 5ms -requests-engine=disk -url=tmp/RRRRRRR -body-size=4k -max-requests=30000 

The load is aligned with WRITEs (PUT / WriteFile), the number of READs is set relative to WRITEs, that is, reads-per-write, readings per write - in Example 2, it can be a fractional number.

The number of threads will adapt to the delays and the output will be the number of simultaneous customers that we can withstand this load. Optionally, the maximum can be limited-increased by the argument -max-channels, by default 32.

The traditional “number of simultaneous clients” also exists, -wt = 5 -rt = 10 (5 write threads, 10 per read).

These are 2 modes which were not enough for me. But there was a problem number 2 for benchmarks I tested.

Configuration complexity

An additional requirement for myself was the lack of UI and configuration files. This is of course very cool when the million settings and possibilities are endless, but this requires another powerful machine, UI, half a day to write the configuration and then drag these files along. Of course, sometimes this is still a plus, it was important for me to be able to connect to any of the servers and be able to run a system test manually from memory. Or send a short message to the employees in a short message with the team, and then they will do everything themselves. In the extreme case, peep into the goader --help and again all will finish it themselves.

So far, it has been possible to do without configuration files, and I hope to keep it that way.

For example, often the list of paths is provided by a separate file. Alternative: -url=http://127.0.0.1/user/NNNN/images/RRRRRRR.jpg
NNNN - will be replaced by random numbers
RRRRRRR - will be replaced by random letters
XXXXX - will be replaced by successively increasing numbers

Visibility

Numbers in numbers, and sometimes with eyes you can see anomalies that are difficult to see in numbers, not knowing what to look for. Each request is displayed in the console with a dot (green - read, blue - record), error E, change of the number of threads with the arrow up or down.
This feature allowed us to find problems in the system, as we were able to visually see how the performance slipped at a certain moment.

In addition, the end result also tends to be concise and informative. It is possible to replace it with json, -output=json/human
Sometimes points get in the way, you can turn them off. -show-progress=false

With a large number of requests, or with remote execution, it is difficult to see progress with your eyes, for this there is an experimental function of creating an html file with visual display ( -timeline-file=timeline.html ) when the request came out and how long it took. There are plans to improve, to change the display of the lifetime of a query to a vertical one and then it will be much more obvious how many queries existed. But for now this is not a priority.

Support simple http requests

By simple I mean only GET requests to pages, without PUTs using all the above. This was not a priority, but you can use it.

Requests per second:

 goader -rps=300 --max-requests=1000 -url=http://localhost/user/NN/product/RRRR 

Number of simultaneous clients:

 goader -rt=32 --max-requests=1000 -url=http://localhost/user/NN/product/RRRR 

Search for the maximum number of simultaneous clients with a specified maximum delay:

 goader -wt=0 -max-latency=300ms --max-requests=1000 -url=http://localhost/user/NN/product/RRRR 

It is better to do more requests in this mode, for more accurate results.

By the way, it was my first experience with go, I wanted to test in practice rave reviews, I can not say that I do not support them. Yes, there are drawbacks, but in general, quick compilation, quick error checking, a single ecosystem, a simple type system - all this makes it possible, as it is fashionable to say, to concentrate on the code, and not on the features of the ecosystem.

And the fact that, without any extra magic, the output of the binaries for all systems is charm.

I only upload linux / 386 linux / amd64 windows / amd64 darwin / 386 darwin / amd64 to githabs, but if someone needs to expand this list, then no problem. Within the framework supported by the golang itself. Comments about the code itself are also welcome.

You can download it from Github or build it yourself, MIT license. For ease of use it is better to put in $ PATH.