Geekly Articles each Day
📜 ⬆️ ⬇️

KISS + Ruby = load server fast

I will describe the story here as on my knees made "load" testing service, and some thoughts about Ruby;)

Once upon a time there was one big system; it was not the first year, not the first release.
The system is a central server through which information runs and a set of desktop applications that generate and consume this information, so to speak, gateways and destination.

Once there was a trouble with the server, to locate the defect, it was necessary to load the server - to imitate the work of a large number of simultaneous active clients.
')


Clients communicate with servers through the Windows Communication Foundation aka WCF.

As in any serious system with customers, there was an emulator available, which through the GUI made it possible to pull all the methods and watch the answers.
In the emulator, there was even a special mode that could pretend to be living clients, like real ones (pull the right methods in the right order), even know how to do it multithreaded.

Actually this is a prolonged introduction, we turn to the details.
It was necessary to make a load of 6000 working customers.
Each of them shows activity every 10 seconds.
everyone has their own username / password.

One emulator was able to run 250 threads / clients. In total, 24 copies of the emulator were needed to run, on a pile of machines (depending on the capacity from 1 to 4 pieces per machine).
However, they said muzhuki, but it is necessary means it is necessary, launched, and the first stage was closed on this, confirmed the theory and postponed for the rarity of the situation.

Then after a while again there was a need for a load, and the person who let it all in - was absent attracted me, but since it was Friday night, etc., we decided to postpone until Monday.

Actually, I am not a developer - I am a tester / QA, but with different strange backgrounds.

I decided at home to see if it was possible to simplify this matter, whether it was possible to create a load with “more other” methods.

I took the emulator, took Fiddler2 (Fiddler is a Web Debugging Proxy) and looked at what was being done.

I saw there XML which runs both ways, and the magic word SOAP.

Oh, I said, a good excuse to finally get to know him better.

At the same time and check whether such things can be done on Ruby.
Why Ruby?
Well, firstly because it's beautiful, it's time,
secondly Ruby + WATIR after this bundle I got carried away with chopping (I think separately to write about it)
in the third - multi-platform, this was an important factor for me since at home, where the first phase of research took place - OS X (Hackintosh), at work, Windows, and of course, Linux also came in handy in real life.

I looked for a suitable library, found soap4r, Savon, chose the latter, because It seemed easier. Put the game started.

Surprisingly, the first working code came out quickly, the task was to get a request that looked like a request from a native emulator.
  1. def data_available ( client,id,password ) _xml= % Q |< ?xml version= "1.0" encoding= "utf-8" ? > < soap:Envelope xmlns:soap= "http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi= "http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd= "http://www.w3.org/2001/XMLSchema" > < soap:Body > < DataAvailable xmlns= "Company.ESD" > < ID > #{id}</deviceID> < Password > #{password}</Password> </ DataAvailable > </ soap:Body > </ soap:Envelope > | _xml. gsub! ( / \r | \n / , "" ) response = client. request "Company.ESD/Data/DataAvailable" , "xmlns" => "Company.ESD" do soap. xml =_xml end response end def do_test ( id,pwd ) client = Savon::Client . new do wsdl. endpoint = "https://superserver.super.com/DDService/Data.svc" wsdl. namespace = "https://superserver.super.com" http. read_timeout = 90 http. open_timeout = 90 end r=data_available ( client,id,pwd ) . to_hash end
  2. def data_available ( client,id,password ) _xml= % Q |< ?xml version= "1.0" encoding= "utf-8" ? > < soap:Envelope xmlns:soap= "http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi= "http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd= "http://www.w3.org/2001/XMLSchema" > < soap:Body > < DataAvailable xmlns= "Company.ESD" > < ID > #{id}</deviceID> < Password > #{password}</Password> </ DataAvailable > </ soap:Body > </ soap:Envelope > | _xml. gsub! ( / \r | \n / , "" ) response = client. request "Company.ESD/Data/DataAvailable" , "xmlns" => "Company.ESD" do soap. xml =_xml end response end def do_test ( id,pwd ) client = Savon::Client . new do wsdl. endpoint = "https://superserver.super.com/DDService/Data.svc" wsdl. namespace = "https://superserver.super.com" http. read_timeout = 90 http. open_timeout = 90 end r=data_available ( client,id,pwd ) . to_hash end
  3. def data_available ( client,id,password ) _xml= % Q |< ?xml version= "1.0" encoding= "utf-8" ? > < soap:Envelope xmlns:soap= "http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi= "http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd= "http://www.w3.org/2001/XMLSchema" > < soap:Body > < DataAvailable xmlns= "Company.ESD" > < ID > #{id}</deviceID> < Password > #{password}</Password> </ DataAvailable > </ soap:Body > </ soap:Envelope > | _xml. gsub! ( / \r | \n / , "" ) response = client. request "Company.ESD/Data/DataAvailable" , "xmlns" => "Company.ESD" do soap. xml =_xml end response end def do_test ( id,pwd ) client = Savon::Client . new do wsdl. endpoint = "https://superserver.super.com/DDService/Data.svc" wsdl. namespace = "https://superserver.super.com" http. read_timeout = 90 http. open_timeout = 90 end r=data_available ( client,id,pwd ) . to_hash end
  4. def data_available ( client,id,password ) _xml= % Q |< ?xml version= "1.0" encoding= "utf-8" ? > < soap:Envelope xmlns:soap= "http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi= "http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd= "http://www.w3.org/2001/XMLSchema" > < soap:Body > < DataAvailable xmlns= "Company.ESD" > < ID > #{id}</deviceID> < Password > #{password}</Password> </ DataAvailable > </ soap:Body > </ soap:Envelope > | _xml. gsub! ( / \r | \n / , "" ) response = client. request "Company.ESD/Data/DataAvailable" , "xmlns" => "Company.ESD" do soap. xml =_xml end response end def do_test ( id,pwd ) client = Savon::Client . new do wsdl. endpoint = "https://superserver.super.com/DDService/Data.svc" wsdl. namespace = "https://superserver.super.com" http. read_timeout = 90 http. open_timeout = 90 end r=data_available ( client,id,pwd ) . to_hash end
  5. def data_available ( client,id,password ) _xml= % Q |< ?xml version= "1.0" encoding= "utf-8" ? > < soap:Envelope xmlns:soap= "http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi= "http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd= "http://www.w3.org/2001/XMLSchema" > < soap:Body > < DataAvailable xmlns= "Company.ESD" > < ID > #{id}</deviceID> < Password > #{password}</Password> </ DataAvailable > </ soap:Body > </ soap:Envelope > | _xml. gsub! ( / \r | \n / , "" ) response = client. request "Company.ESD/Data/DataAvailable" , "xmlns" => "Company.ESD" do soap. xml =_xml end response end def do_test ( id,pwd ) client = Savon::Client . new do wsdl. endpoint = "https://superserver.super.com/DDService/Data.svc" wsdl. namespace = "https://superserver.super.com" http. read_timeout = 90 http. open_timeout = 90 end r=data_available ( client,id,pwd ) . to_hash end
  6. def data_available ( client,id,password ) _xml= % Q |< ?xml version= "1.0" encoding= "utf-8" ? > < soap:Envelope xmlns:soap= "http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi= "http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd= "http://www.w3.org/2001/XMLSchema" > < soap:Body > < DataAvailable xmlns= "Company.ESD" > < ID > #{id}</deviceID> < Password > #{password}</Password> </ DataAvailable > </ soap:Body > </ soap:Envelope > | _xml. gsub! ( / \r | \n / , "" ) response = client. request "Company.ESD/Data/DataAvailable" , "xmlns" => "Company.ESD" do soap. xml =_xml end response end def do_test ( id,pwd ) client = Savon::Client . new do wsdl. endpoint = "https://superserver.super.com/DDService/Data.svc" wsdl. namespace = "https://superserver.super.com" http. read_timeout = 90 http. open_timeout = 90 end r=data_available ( client,id,pwd ) . to_hash end
  7. def data_available ( client,id,password ) _xml= % Q |< ?xml version= "1.0" encoding= "utf-8" ? > < soap:Envelope xmlns:soap= "http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi= "http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd= "http://www.w3.org/2001/XMLSchema" > < soap:Body > < DataAvailable xmlns= "Company.ESD" > < ID > #{id}</deviceID> < Password > #{password}</Password> </ DataAvailable > </ soap:Body > </ soap:Envelope > | _xml. gsub! ( / \r | \n / , "" ) response = client. request "Company.ESD/Data/DataAvailable" , "xmlns" => "Company.ESD" do soap. xml =_xml end response end def do_test ( id,pwd ) client = Savon::Client . new do wsdl. endpoint = "https://superserver.super.com/DDService/Data.svc" wsdl. namespace = "https://superserver.super.com" http. read_timeout = 90 http. open_timeout = 90 end r=data_available ( client,id,pwd ) . to_hash end
  8. def data_available ( client,id,password ) _xml= % Q |< ?xml version= "1.0" encoding= "utf-8" ? > < soap:Envelope xmlns:soap= "http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi= "http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd= "http://www.w3.org/2001/XMLSchema" > < soap:Body > < DataAvailable xmlns= "Company.ESD" > < ID > #{id}</deviceID> < Password > #{password}</Password> </ DataAvailable > </ soap:Body > </ soap:Envelope > | _xml. gsub! ( / \r | \n / , "" ) response = client. request "Company.ESD/Data/DataAvailable" , "xmlns" => "Company.ESD" do soap. xml =_xml end response end def do_test ( id,pwd ) client = Savon::Client . new do wsdl. endpoint = "https://superserver.super.com/DDService/Data.svc" wsdl. namespace = "https://superserver.super.com" http. read_timeout = 90 http. open_timeout = 90 end r=data_available ( client,id,pwd ) . to_hash end
  9. def data_available ( client,id,password ) _xml= % Q |< ?xml version= "1.0" encoding= "utf-8" ? > < soap:Envelope xmlns:soap= "http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi= "http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd= "http://www.w3.org/2001/XMLSchema" > < soap:Body > < DataAvailable xmlns= "Company.ESD" > < ID > #{id}</deviceID> < Password > #{password}</Password> </ DataAvailable > </ soap:Body > </ soap:Envelope > | _xml. gsub! ( / \r | \n / , "" ) response = client. request "Company.ESD/Data/DataAvailable" , "xmlns" => "Company.ESD" do soap. xml =_xml end response end def do_test ( id,pwd ) client = Savon::Client . new do wsdl. endpoint = "https://superserver.super.com/DDService/Data.svc" wsdl. namespace = "https://superserver.super.com" http. read_timeout = 90 http. open_timeout = 90 end r=data_available ( client,id,pwd ) . to_hash end
  10. def data_available ( client,id,password ) _xml= % Q |< ?xml version= "1.0" encoding= "utf-8" ? > < soap:Envelope xmlns:soap= "http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi= "http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd= "http://www.w3.org/2001/XMLSchema" > < soap:Body > < DataAvailable xmlns= "Company.ESD" > < ID > #{id}</deviceID> < Password > #{password}</Password> </ DataAvailable > </ soap:Body > </ soap:Envelope > | _xml. gsub! ( / \r | \n / , "" ) response = client. request "Company.ESD/Data/DataAvailable" , "xmlns" => "Company.ESD" do soap. xml =_xml end response end def do_test ( id,pwd ) client = Savon::Client . new do wsdl. endpoint = "https://superserver.super.com/DDService/Data.svc" wsdl. namespace = "https://superserver.super.com" http. read_timeout = 90 http. open_timeout = 90 end r=data_available ( client,id,pwd ) . to_hash end
  11. def data_available ( client,id,password ) _xml= % Q |< ?xml version= "1.0" encoding= "utf-8" ? > < soap:Envelope xmlns:soap= "http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi= "http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd= "http://www.w3.org/2001/XMLSchema" > < soap:Body > < DataAvailable xmlns= "Company.ESD" > < ID > #{id}</deviceID> < Password > #{password}</Password> </ DataAvailable > </ soap:Body > </ soap:Envelope > | _xml. gsub! ( / \r | \n / , "" ) response = client. request "Company.ESD/Data/DataAvailable" , "xmlns" => "Company.ESD" do soap. xml =_xml end response end def do_test ( id,pwd ) client = Savon::Client . new do wsdl. endpoint = "https://superserver.super.com/DDService/Data.svc" wsdl. namespace = "https://superserver.super.com" http. read_timeout = 90 http. open_timeout = 90 end r=data_available ( client,id,pwd ) . to_hash end
  12. def data_available ( client,id,password ) _xml= % Q |< ?xml version= "1.0" encoding= "utf-8" ? > < soap:Envelope xmlns:soap= "http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi= "http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd= "http://www.w3.org/2001/XMLSchema" > < soap:Body > < DataAvailable xmlns= "Company.ESD" > < ID > #{id}</deviceID> < Password > #{password}</Password> </ DataAvailable > </ soap:Body > </ soap:Envelope > | _xml. gsub! ( / \r | \n / , "" ) response = client. request "Company.ESD/Data/DataAvailable" , "xmlns" => "Company.ESD" do soap. xml =_xml end response end def do_test ( id,pwd ) client = Savon::Client . new do wsdl. endpoint = "https://superserver.super.com/DDService/Data.svc" wsdl. namespace = "https://superserver.super.com" http. read_timeout = 90 http. open_timeout = 90 end r=data_available ( client,id,pwd ) . to_hash end
  13. def data_available ( client,id,password ) _xml= % Q |< ?xml version= "1.0" encoding= "utf-8" ? > < soap:Envelope xmlns:soap= "http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi= "http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd= "http://www.w3.org/2001/XMLSchema" > < soap:Body > < DataAvailable xmlns= "Company.ESD" > < ID > #{id}</deviceID> < Password > #{password}</Password> </ DataAvailable > </ soap:Body > </ soap:Envelope > | _xml. gsub! ( / \r | \n / , "" ) response = client. request "Company.ESD/Data/DataAvailable" , "xmlns" => "Company.ESD" do soap. xml =_xml end response end def do_test ( id,pwd ) client = Savon::Client . new do wsdl. endpoint = "https://superserver.super.com/DDService/Data.svc" wsdl. namespace = "https://superserver.super.com" http. read_timeout = 90 http. open_timeout = 90 end r=data_available ( client,id,pwd ) . to_hash end
  14. def data_available ( client,id,password ) _xml= % Q |< ?xml version= "1.0" encoding= "utf-8" ? > < soap:Envelope xmlns:soap= "http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi= "http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd= "http://www.w3.org/2001/XMLSchema" > < soap:Body > < DataAvailable xmlns= "Company.ESD" > < ID > #{id}</deviceID> < Password > #{password}</Password> </ DataAvailable > </ soap:Body > </ soap:Envelope > | _xml. gsub! ( / \r | \n / , "" ) response = client. request "Company.ESD/Data/DataAvailable" , "xmlns" => "Company.ESD" do soap. xml =_xml end response end def do_test ( id,pwd ) client = Savon::Client . new do wsdl. endpoint = "https://superserver.super.com/DDService/Data.svc" wsdl. namespace = "https://superserver.super.com" http. read_timeout = 90 http. open_timeout = 90 end r=data_available ( client,id,pwd ) . to_hash end
  15. def data_available ( client,id,password ) _xml= % Q |< ?xml version= "1.0" encoding= "utf-8" ? > < soap:Envelope xmlns:soap= "http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi= "http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd= "http://www.w3.org/2001/XMLSchema" > < soap:Body > < DataAvailable xmlns= "Company.ESD" > < ID > #{id}</deviceID> < Password > #{password}</Password> </ DataAvailable > </ soap:Body > </ soap:Envelope > | _xml. gsub! ( / \r | \n / , "" ) response = client. request "Company.ESD/Data/DataAvailable" , "xmlns" => "Company.ESD" do soap. xml =_xml end response end def do_test ( id,pwd ) client = Savon::Client . new do wsdl. endpoint = "https://superserver.super.com/DDService/Data.svc" wsdl. namespace = "https://superserver.super.com" http. read_timeout = 90 http. open_timeout = 90 end r=data_available ( client,id,pwd ) . to_hash end
  17. def data_available ( client,id,password ) _xml= % Q |< ?xml version= "1.0" encoding= "utf-8" ? > < soap:Envelope xmlns:soap= "http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi= "http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd= "http://www.w3.org/2001/XMLSchema" > < soap:Body > < DataAvailable xmlns= "Company.ESD" > < ID > #{id}</deviceID> < Password > #{password}</Password> </ DataAvailable > </ soap:Body > </ soap:Envelope > | _xml. gsub! ( / \r | \n / , "" ) response = client. request "Company.ESD/Data/DataAvailable" , "xmlns" => "Company.ESD" do soap. xml =_xml end response end def do_test ( id,pwd ) client = Savon::Client . new do wsdl. endpoint = "https://superserver.super.com/DDService/Data.svc" wsdl. namespace = "https://superserver.super.com" http. read_timeout = 90 http. open_timeout = 90 end r=data_available ( client,id,pwd ) . to_hash end
  18. def data_available ( client,id,password ) _xml= % Q |< ?xml version= "1.0" encoding= "utf-8" ? > < soap:Envelope xmlns:soap= "http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi= "http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd= "http://www.w3.org/2001/XMLSchema" > < soap:Body > < DataAvailable xmlns= "Company.ESD" > < ID > #{id}</deviceID> < Password > #{password}</Password> </ DataAvailable > </ soap:Body > </ soap:Envelope > | _xml. gsub! ( / \r | \n / , "" ) response = client. request "Company.ESD/Data/DataAvailable" , "xmlns" => "Company.ESD" do soap. xml =_xml end response end def do_test ( id,pwd ) client = Savon::Client . new do wsdl. endpoint = "https://superserver.super.com/DDService/Data.svc" wsdl. namespace = "https://superserver.super.com" http. read_timeout = 90 http. open_timeout = 90 end r=data_available ( client,id,pwd ) . to_hash end
  19. def data_available ( client,id,password ) _xml= % Q |< ?xml version= "1.0" encoding= "utf-8" ? > < soap:Envelope xmlns:soap= "http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi= "http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd= "http://www.w3.org/2001/XMLSchema" > < soap:Body > < DataAvailable xmlns= "Company.ESD" > < ID > #{id}</deviceID> < Password > #{password}</Password> </ DataAvailable > </ soap:Body > </ soap:Envelope > | _xml. gsub! ( / \r | \n / , "" ) response = client. request "Company.ESD/Data/DataAvailable" , "xmlns" => "Company.ESD" do soap. xml =_xml end response end def do_test ( id,pwd ) client = Savon::Client . new do wsdl. endpoint = "https://superserver.super.com/DDService/Data.svc" wsdl. namespace = "https://superserver.super.com" http. read_timeout = 90 http. open_timeout = 90 end r=data_available ( client,id,pwd ) . to_hash end
  20. def data_available ( client,id,password ) _xml= % Q |< ?xml version= "1.0" encoding= "utf-8" ? > < soap:Envelope xmlns:soap= "http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi= "http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd= "http://www.w3.org/2001/XMLSchema" > < soap:Body > < DataAvailable xmlns= "Company.ESD" > < ID > #{id}</deviceID> < Password > #{password}</Password> </ DataAvailable > </ soap:Body > </ soap:Envelope > | _xml. gsub! ( / \r | \n / , "" ) response = client. request "Company.ESD/Data/DataAvailable" , "xmlns" => "Company.ESD" do soap. xml =_xml end response end def do_test ( id,pwd ) client = Savon::Client . new do wsdl. endpoint = "https://superserver.super.com/DDService/Data.svc" wsdl. namespace = "https://superserver.super.com" http. read_timeout = 90 http. open_timeout = 90 end r=data_available ( client,id,pwd ) . to_hash end
  21. def data_available ( client,id,password ) _xml= % Q |< ?xml version= "1.0" encoding= "utf-8" ? > < soap:Envelope xmlns:soap= "http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi= "http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd= "http://www.w3.org/2001/XMLSchema" > < soap:Body > < DataAvailable xmlns= "Company.ESD" > < ID > #{id}</deviceID> < Password > #{password}</Password> </ DataAvailable > </ soap:Body > </ soap:Envelope > | _xml. gsub! ( / \r | \n / , "" ) response = client. request "Company.ESD/Data/DataAvailable" , "xmlns" => "Company.ESD" do soap. xml =_xml end response end def do_test ( id,pwd ) client = Savon::Client . new do wsdl. endpoint = "https://superserver.super.com/DDService/Data.svc" wsdl. namespace = "https://superserver.super.com" http. read_timeout = 90 http. open_timeout = 90 end r=data_available ( client,id,pwd ) . to_hash end
  23. def data_available ( client,id,password ) _xml= % Q |< ?xml version= "1.0" encoding= "utf-8" ? > < soap:Envelope xmlns:soap= "http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi= "http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd= "http://www.w3.org/2001/XMLSchema" > < soap:Body > < DataAvailable xmlns= "Company.ESD" > < ID > #{id}</deviceID> < Password > #{password}</Password> </ DataAvailable > </ soap:Body > </ soap:Envelope > | _xml. gsub! ( / \r | \n / , "" ) response = client. request "Company.ESD/Data/DataAvailable" , "xmlns" => "Company.ESD" do soap. xml =_xml end response end def do_test ( id,pwd ) client = Savon::Client . new do wsdl. endpoint = "https://superserver.super.com/DDService/Data.svc" wsdl. namespace = "https://superserver.super.com" http. read_timeout = 90 http. open_timeout = 90 end r=data_available ( client,id,pwd ) . to_hash end
  24. def data_available ( client,id,password ) _xml= % Q |< ?xml version= "1.0" encoding= "utf-8" ? > < soap:Envelope xmlns:soap= "http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi= "http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd= "http://www.w3.org/2001/XMLSchema" > < soap:Body > < DataAvailable xmlns= "Company.ESD" > < ID > #{id}</deviceID> < Password > #{password}</Password> </ DataAvailable > </ soap:Body > </ soap:Envelope > | _xml. gsub! ( / \r | \n / , "" ) response = client. request "Company.ESD/Data/DataAvailable" , "xmlns" => "Company.ESD" do soap. xml =_xml end response end def do_test ( id,pwd ) client = Savon::Client . new do wsdl. endpoint = "https://superserver.super.com/DDService/Data.svc" wsdl. namespace = "https://superserver.super.com" http. read_timeout = 90 http. open_timeout = 90 end r=data_available ( client,id,pwd ) . to_hash end
  25. def data_available ( client,id,password ) _xml= % Q |< ?xml version= "1.0" encoding= "utf-8" ? > < soap:Envelope xmlns:soap= "http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi= "http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd= "http://www.w3.org/2001/XMLSchema" > < soap:Body > < DataAvailable xmlns= "Company.ESD" > < ID > #{id}</deviceID> < Password > #{password}</Password> </ DataAvailable > </ soap:Body > </ soap:Envelope > | _xml. gsub! ( / \r | \n / , "" ) response = client. request "Company.ESD/Data/DataAvailable" , "xmlns" => "Company.ESD" do soap. xml =_xml end response end def do_test ( id,pwd ) client = Savon::Client . new do wsdl. endpoint = "https://superserver.super.com/DDService/Data.svc" wsdl. namespace = "https://superserver.super.com" http. read_timeout = 90 http. open_timeout = 90 end r=data_available ( client,id,pwd ) . to_hash end
  26. def data_available ( client,id,password ) _xml= % Q |< ?xml version= "1.0" encoding= "utf-8" ? > < soap:Envelope xmlns:soap= "http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi= "http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd= "http://www.w3.org/2001/XMLSchema" > < soap:Body > < DataAvailable xmlns= "Company.ESD" > < ID > #{id}</deviceID> < Password > #{password}</Password> </ DataAvailable > </ soap:Body > </ soap:Envelope > | _xml. gsub! ( / \r | \n / , "" ) response = client. request "Company.ESD/Data/DataAvailable" , "xmlns" => "Company.ESD" do soap. xml =_xml end response end def do_test ( id,pwd ) client = Savon::Client . new do wsdl. endpoint = "https://superserver.super.com/DDService/Data.svc" wsdl. namespace = "https://superserver.super.com" http. read_timeout = 90 http. open_timeout = 90 end r=data_available ( client,id,pwd ) . to_hash end
  27. def data_available ( client,id,password ) _xml= % Q |< ?xml version= "1.0" encoding= "utf-8" ? > < soap:Envelope xmlns:soap= "http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi= "http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd= "http://www.w3.org/2001/XMLSchema" > < soap:Body > < DataAvailable xmlns= "Company.ESD" > < ID > #{id}</deviceID> < Password > #{password}</Password> </ DataAvailable > </ soap:Body > </ soap:Envelope > | _xml. gsub! ( / \r | \n / , "" ) response = client. request "Company.ESD/Data/DataAvailable" , "xmlns" => "Company.ESD" do soap. xml =_xml end response end def do_test ( id,pwd ) client = Savon::Client . new do wsdl. endpoint = "https://superserver.super.com/DDService/Data.svc" wsdl. namespace = "https://superserver.super.com" http. read_timeout = 90 http. open_timeout = 90 end r=data_available ( client,id,pwd ) . to_hash end
  28. def data_available ( client,id,password ) _xml= % Q |< ?xml version= "1.0" encoding= "utf-8" ? > < soap:Envelope xmlns:soap= "http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi= "http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd= "http://www.w3.org/2001/XMLSchema" > < soap:Body > < DataAvailable xmlns= "Company.ESD" > < ID > #{id}</deviceID> < Password > #{password}</Password> </ DataAvailable > </ soap:Body > </ soap:Envelope > | _xml. gsub! ( / \r | \n / , "" ) response = client. request "Company.ESD/Data/DataAvailable" , "xmlns" => "Company.ESD" do soap. xml =_xml end response end def do_test ( id,pwd ) client = Savon::Client . new do wsdl. endpoint = "https://superserver.super.com/DDService/Data.svc" wsdl. namespace = "https://superserver.super.com" http. read_timeout = 90 http. open_timeout = 90 end r=data_available ( client,id,pwd ) . to_hash end
  29. def data_available ( client,id,password ) _xml= % Q |< ?xml version= "1.0" encoding= "utf-8" ? > < soap:Envelope xmlns:soap= "http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi= "http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd= "http://www.w3.org/2001/XMLSchema" > < soap:Body > < DataAvailable xmlns= "Company.ESD" > < ID > #{id}</deviceID> < Password > #{password}</Password> </ DataAvailable > </ soap:Body > </ soap:Envelope > | _xml. gsub! ( / \r | \n / , "" ) response = client. request "Company.ESD/Data/DataAvailable" , "xmlns" => "Company.ESD" do soap. xml =_xml end response end def do_test ( id,pwd ) client = Savon::Client . new do wsdl. endpoint = "https://superserver.super.com/DDService/Data.svc" wsdl. namespace = "https://superserver.super.com" http. read_timeout = 90 http. open_timeout = 90 end r=data_available ( client,id,pwd ) . to_hash end
  31. def data_available ( client,id,password ) _xml= % Q |< ?xml version= "1.0" encoding= "utf-8" ? > < soap:Envelope xmlns:soap= "http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi= "http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd= "http://www.w3.org/2001/XMLSchema" > < soap:Body > < DataAvailable xmlns= "Company.ESD" > < ID > #{id}</deviceID> < Password > #{password}</Password> </ DataAvailable > </ soap:Body > </ soap:Envelope > | _xml. gsub! ( / \r | \n / , "" ) response = client. request "Company.ESD/Data/DataAvailable" , "xmlns" => "Company.ESD" do soap. xml =_xml end response end def do_test ( id,pwd ) client = Savon::Client . new do wsdl. endpoint = "https://superserver.super.com/DDService/Data.svc" wsdl. namespace = "https://superserver.super.com" http. read_timeout = 90 http. open_timeout = 90 end r=data_available ( client,id,pwd ) . to_hash end
  32. def data_available ( client,id,password ) _xml= % Q |< ?xml version= "1.0" encoding= "utf-8" ? > < soap:Envelope xmlns:soap= "http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi= "http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd= "http://www.w3.org/2001/XMLSchema" > < soap:Body > < DataAvailable xmlns= "Company.ESD" > < ID > #{id}</deviceID> < Password > #{password}</Password> </ DataAvailable > </ soap:Body > </ soap:Envelope > | _xml. gsub! ( / \r | \n / , "" ) response = client. request "Company.ESD/Data/DataAvailable" , "xmlns" => "Company.ESD" do soap. xml =_xml end response end def do_test ( id,pwd ) client = Savon::Client . new do wsdl. endpoint = "https://superserver.super.com/DDService/Data.svc" wsdl. namespace = "https://superserver.super.com" http. read_timeout = 90 http. open_timeout = 90 end r=data_available ( client,id,pwd ) . to_hash end


those. there is one client, well, it's time to make a lot of customers open the documentation find an example, use
run_client - the code of one "client"

  1. def run_test ( clnt_id )
  2. pwd = get_pwd ( clnt_id )
  3. id = get_id ( clnt_id )
  4. while true
  5. do_test ( id, pwd )
  6. sleep 9
  7. end
  8. end
  10. clients = Array . new
  11. 1. upto ( 10 ) do | client_id |
  12. client << Thread . new do
  13. puts "Start: # {client_id}"
  14. run_client ( t_cnt )
  15. end
  16. end
  17. clients. map { | t | t. join }


launched - Hurray, works.
let's try 1000, ops, on the screen start ~ 130 streams and brakes; (
it looks like Ruby doesn't really want to let in many threads
Well, in general, it is not a question, the result is important for us.
let's put 100 pieces in one file, and there will be 10 such files, so we have 1000 clients.
we try - hooray works !!!, but let's say the percents are, we say gently, not podsaski, we start thinking what to do, we need even more!

here we remember that the client is active every 10 seconds
those. we can start up one thread that can easily process 10 consecutive clients, each by second, in the result, and we get the activity of each client every 10 seconds.
Hurray, it was possible to reduce the number of flows by 10 times, already happy, that is, 1000 clients need only 100 threads, and this is already real ...

got such a launcher
  1. def do_for_time ( time_in_sec, & block )
  2. time_end = Time . now + time_in_sec - 0.01
  3. yield
  4. time_rest = time_end - Time . now
  5. sleep time_rest if time_rest > = 0.1
  6. end
  8. def run_10 ( start_id )
  9. clinets = Array . new
  10. 1. upto ( 10 ) do | nn |
  11. id = get_id [ start_id + nn ]
  12. pwd = get_pwd [ start_id + nn ]
  13. clinets << [ id, pwd ]
  14. end
  16. clnt = HTTPClient. new
  17. while true do
  18. clinets. each do | client |
  19. id = client [ 0 ]
  20. pwd = client [ 1 ]
  21. do_for_time ( 1 ) do
  22. data_available ( clnt, id, pwd )
  23. update_status ( clnt, id, pwd )
  24. end
  25. end
  26. end
  27. end


By the way, the funny function is do_for_time (oh, I love blocks in ruby!)
we call it with the parameter how many seconds it should be executed, and we pass it the block that needs to be executed, and it together with the execution of the block will work as many seconds as requested (we assume that the function call of the block is much less than the waiting time)
, but we still load the percents, we need to do something about it.

I looked at the original requests again and understood, so why do we need SOAP?
in the end, this is nothing more than a POST request with early known XML!

said - done, rewrote our functions, threw out SOAP, got
  1. def check_data ( http_client, id, pwd )
  2. uri = 'https://superserver.super.com/DDService/Data.svc'
  3. soap = % Q | < ? xml version = "1.0" encoding = "utf-8" ? > < soap: Envelope xmlns: soap = "http://schemas.xmlsoap.org/soap/envelope/" xmlns: xsi = "http://www.w3.org/2001/XMLSchema-instance" xmlns: xsd = "http://www.w3.org/2001/XMLSchema" > < soap: Body > < DataAvailable xmlns = "company.ESD" > < ID > # {id} </ ID> <Password> # {pwd} < / Password> </ DataAvailable> </ soap: Body> </ soap: Envelope>
  4. |
  5. headers = {
  6. 'Content-Type' => 'text / xml; charset = UTF-8' ,
  7. 'SOAPAction' => 'company.EDC / IData /' ,
  8. }
  10. http_client. post uri, soap, headers
  11. end


Expectedly new version of a lot of food is much less proca, because did not engage in unnecessary nonsense.

3000 customers live quite easily ...

I stopped home games here, because rested in the non-metric channel ADSL, and yes it is necessary to rest;)

on Monday continued at work, on the fast channel.
, and here miracles begin, sad
screw Ruby 1.8.7 guzzles 100% of 2 nuclear cpu at 1000 clients, opanka…

for checking in a virtual machine (on the same machine) I put ubuntu, I give it two cores, and I let the same thing, and ...
In general, wonders, in VIRTUALKA under linux we have 5000 clients.

Further experiments:
by staging jruby, he has the right threads according to the description, we let 1000 - 20% of the load - URA
we start the second file with 1000 and the opunky, loading 2x cores 80%, is tolerable, but not very clear; (

as a result, 2000 by car came up to our task (it was necessary to start up on a remote car, the virtual machine could not be installed there)
The task was solved, the server was loaded.
Played with SOAP, funny, not very difficult.
Time spent, sumno hours 8.

findings:
1. “Keep it simple, Stupid!” - in our case, simplification helped solve the problem, avoiding complicated SOAP to just http requests saved the situation.
2. RUBY can be used for such tasks, I did not have to fight with the language, he just allowed me to do what I wanted;) (almost without creating problems)
3. Using scripting language - greatly speeds up parsing time with a problem, because makes it easy to experiment, it's quick and easy.
4. multiplatform - taxis
5. Now you can rewrite it all in C # if necessary, maybe there will be many clients on one machine, but it is not necessary yet, the task turned out to be a one-time task.
6. i love ruby!
______________________
The text was prepared in the Blog Editor from © SoftCoder.ru

Source: https://habr.com/ru/post/113420/

More articles:


All Articles