Load load testing service
Our team was faced with shortcomings of load testing tools, and, in the end, it was decided to develop our own service. Main difficulties:
I hope the “bike” will be interesting to others - at first I will describe what is already working, then it will be possible to discuss further features.
For the test, open the form where we specify the email, fill in the URL, select the number of test flows, the region and start the test.
*** UPDATE ***
I see a lot of brave habravchan puts the task to 200 threads. If we assume that 1 page is issued for 1 second, then this corresponds to attendance> 100K visitors per hour. Normal projects, including ours, die from such tests.
')
After a minute, your result will be ready (for an example, look at the excellent result - test example.net ). As you can see, 200 threads allows you to generate more than 1000 requests per second - it all depends on the speed of communication with the service being tested, and. actually, the speed of response.
If you are ready to boast of your result on our website - you can click the Public result button. To show it to your colleagues just send a link.
What to test?
Static resources, images, scripts should be given from CDN. Testing their upload speed IMHO does not make sense, you only need to test the overall page loading speed, for example using the good old http://tools.pingdom.com/fpt/
Loadme focuses on testing the code of pages / methods api, etc ... Testing nginx giving 1x1.gif with the help of this tool is of course possible, but there is no practical use, and nginx won't even get warm from it.
To decide which pages are the bottleneck, it is best to use newrelic. Unlike the popular google analytics, it also allows you to track the statistics of bot requests, and build queries on the number of operations per page or another, as well as which of the pages most damaged the user experience by the apdex index .
As you know, a fly in the ointment spoils the barrel of tar, and if your application slows down on even relatively rare actions, this may well affect popular lightweight operations.
How do redirects work?
Redirects are executed; we actively use it to test one of our wikiart.org sites, implementing the “go to a random picture” function on it.
Why is it important to test multiple url?
To test the mutual influence of popular fast pages and slow (for example, search)
Why do you need $ RND?
The syntax is $ RND (from, to).
For example, http://someshop.com/search?from=$RND(0,1000)&to=$RND(1000,10000) will generate arbitrary requests for finding products at prices ranging from 0 to 1000 and ending with 1000 to 10,000. This makes it possible to estimate the real power of the search.
For example, the popular Ukrainian store Rozetka spends an average of 5 seconds searching for smartphones at a random price:
http://loadme.socialtalents.com/Result/ViewById/56108a645b5f1700481cc21d , which is very far from ideal result.
Amazon copes with this task fundamentally better - a significant amount of errors as a result, most likely, is protection from ddos
Post, put, delete requests
The right thing, clearly, is in the plans.
Authorization
Is it enough to support cookies in order for the first request to log in a random user test (which will require server support), and further work will be done on behalf of this user?
Step tests
Say, conduct a series of tests: 25%, 50%, 75% and 100%, and see the difference in speed.
Fixed load
Instead of the number of threads, let the user choose how many operations per second he wants to initiate.
Regular scheduled test
Repeat the test every day / week and send the report to email.
You can also provide a webhook to initiate an existing test from the code (for example, after an update)
Improved bandwidth visualization
The server may have behaved unevenly. Plans to add server bandwidth visualization in seconds.
Proof of domain ownership
The limit of no more than 200 agents on 1 domain exists exactly so that no one puts another site in ddos. For your site, you can create another subdomain and test it again.
In the future, however, it will be necessary to confirm domains with a CNAME record or file with a specific name.
Loadimpact.com - for a normal load test, at least 100 requests per second, 1500 so-called “virtual users” will be required - each of them loads the page once every 15 seconds. There is such a package at the moment $ 299 per month.
loader.io - great service, a paid package of only $ 99 per month. Very flexible URL settings - you can run methods, cookies, heders, but we didn’t have enough randomization of the test.
- If this is a service, the price is too high for a serious load.
- If it is a utility, the result depends on the speed of the computer / server channel from which the test was performed.
- Repeated requests do not reflect real speed, since caching is at various levels, starting from the CPU and ending with the database
I hope the “bike” will be interesting to others - at first I will describe what is already working, then it will be possible to discuss further features.
What has been done?
- You can test jobs from the list of url, up to 20 pieces
- Each url can contain one or more random parameters specified using the $ RND function
- The test is run from a variety of servers, each of which only has 8 threads.
- Testing can be conducted from 5 AWS regions - Dublin, Frankfurt, East / West USA, Tokyo
- We are ready to provide tests for up to 200 threads for free.
For the test, open the form where we specify the email, fill in the URL, select the number of test flows, the region and start the test.
*** UPDATE ***
I see a lot of brave habravchan puts the task to 200 threads. If we assume that 1 page is issued for 1 second, then this corresponds to attendance> 100K visitors per hour. Normal projects, including ours, die from such tests.
')
After a minute, your result will be ready (for an example, look at the excellent result - test example.net ). As you can see, 200 threads allows you to generate more than 1000 requests per second - it all depends on the speed of communication with the service being tested, and. actually, the speed of response.
If you are ready to boast of your result on our website - you can click the Public result button. To show it to your colleagues just send a link.
What to test?
Static resources, images, scripts should be given from CDN. Testing their upload speed IMHO does not make sense, you only need to test the overall page loading speed, for example using the good old http://tools.pingdom.com/fpt/
Loadme focuses on testing the code of pages / methods api, etc ... Testing nginx giving 1x1.gif with the help of this tool is of course possible, but there is no practical use, and nginx won't even get warm from it.
To decide which pages are the bottleneck, it is best to use newrelic. Unlike the popular google analytics, it also allows you to track the statistics of bot requests, and build queries on the number of operations per page or another, as well as which of the pages most damaged the user experience by the apdex index .
As you know, a fly in the ointment spoils the barrel of tar, and if your application slows down on even relatively rare actions, this may well affect popular lightweight operations.
How do redirects work?
Redirects are executed; we actively use it to test one of our wikiart.org sites, implementing the “go to a random picture” function on it.
Why is it important to test multiple url?
To test the mutual influence of popular fast pages and slow (for example, search)
Why do you need $ RND?
The syntax is $ RND (from, to).
For example, http://someshop.com/search?from=$RND(0,1000)&to=$RND(1000,10000) will generate arbitrary requests for finding products at prices ranging from 0 to 1000 and ending with 1000 to 10,000. This makes it possible to estimate the real power of the search.
For example, the popular Ukrainian store Rozetka spends an average of 5 seconds searching for smartphones at a random price:
http://loadme.socialtalents.com/Result/ViewById/56108a645b5f1700481cc21d , which is very far from ideal result.
Amazon copes with this task fundamentally better - a significant amount of errors as a result, most likely, is protection from ddos
Future plans
Post, put, delete requests
The right thing, clearly, is in the plans.
Authorization
Is it enough to support cookies in order for the first request to log in a random user test (which will require server support), and further work will be done on behalf of this user?
Step tests
Say, conduct a series of tests: 25%, 50%, 75% and 100%, and see the difference in speed.
Fixed load
Instead of the number of threads, let the user choose how many operations per second he wants to initiate.
Regular scheduled test
Repeat the test every day / week and send the report to email.
You can also provide a webhook to initiate an existing test from the code (for example, after an update)
Improved bandwidth visualization
The server may have behaved unevenly. Plans to add server bandwidth visualization in seconds.
Proof of domain ownership
The limit of no more than 200 agents on 1 domain exists exactly so that no one puts another site in ddos. For your site, you can create another subdomain and test it again.
In the future, however, it will be necessary to confirm domains with a CNAME record or file with a specific name.
Existing competitors
Loadimpact.com - for a normal load test, at least 100 requests per second, 1500 so-called “virtual users” will be required - each of them loads the page once every 15 seconds. There is such a package at the moment $ 299 per month.
loader.io - great service, a paid package of only $ 99 per month. Very flexible URL settings - you can run methods, cookies, heders, but we didn’t have enough randomization of the test.
Source: https://habr.com/ru/post/268503/
All Articles