How to choose a programming language?
This is exactly the question that the Mail.Ru Mail team asked before writing the next service. The main purpose of this choice is the high efficiency of the development process within the framework of the chosen language / technology. What influences this indicator?
- Performance;
- Availability of debugging and profiling tools;
- A large community that allows you to quickly find answers to questions;
- The presence of stable libraries and modules necessary for the development of web applications;
- The number of developers on the market;
- The possibility of development in modern IDE;
- The threshold of entry into the language.
In addition, the developers welcomed the laconic and expressive language. Conciseness, of course, also affects the effectiveness of the development, as the lack of kilogram weights on the probability of success of a marathon runner.
Initial data
Challengers
Since many server microtasks are often born in the client side of the mail, the first applicant is, of course, Node.js with its native JavaScript and V8 from Google.
After discussion and on the basis of preferences within the team, the remaining participants of the competition were identified: Scala , Go and Rust .
')
As a performance test, it was proposed to write a simple HTTP server that receives HTML from the general template service and gives it to the client. Such a task is dictated by the current realities of the work of the mail - all template-based client-side occurs on V8 using the fest template engine.
When testing, it turned out that all applicants work with approximately the same performance in this formulation - it was all about V8 performance. However, the implementation of the task was not superfluous - the development in each of the languages ​​made it possible to make up a significant part of the subjective assessments, which in one way or another could affect the final choice.
So, we have two scenarios. The first is just the root URL greeting:
GET / HTTP/1.1 Host: service.host HTTP/1.1 200 OK Hello World! The second is a customer greeting by his name, passed in the URL path:
GET /greeting/user HTTP/1.1 Host: service.host HTTP/1.1 200 OK Hello, user Environment
All tests were conducted on a VirtualBox virtual machine.
Host MacBook Pro:
- 2.6 GHz Intel Core i5 (dual core);
- CPU Cache L1: 32 KB, L2: 256 KB, L3: 3 MB;
- 8 GB 1600 MHz DDR3.
VM:
- 4 GB RAM;
- VT-x / AMD-v, PAE / NX, KVM.
Software:
- CentOS 6.7 64bit;
- Go 1.5.1;
- Rustc 1.4.0;
- Scala 2.11.7, sbt 0.13.9;
- Java 1.8.0_65;
- Node 5.1.1;
- Node 0.12.7;
- nginx 1.8.0;
- wrk 4.0.0.
In addition to standard modules, in examples on Rust hyper was used, on Scala - spray . In Go and Node.js, only native packages / modules were used.
Measurement tools
The performance of the services was tested using the following tools:
This article discusses the benchmarks wrk and ab.
results
Performance
wrk
The following is the data of the five-minute test, with 1000 connections and 50 threads:
wrk -d300s -c1000 -t50 --timeout 2s service.host
| Label | Average Latency, ms | Request, # / sec |
|---|---|---|
| Go | 104.83 | 36,191.37 |
| Rusty | 0.02906 | 32,564.13 |
| Scala | 57.74 | 17,182.40 |
| Node 5.1.1 | 69,37 | 14 005.12 |
| Node 0.12.7 | 86.68 | 11,125.37 |
wrk -d300s -c1000 -t50 --timeout 2s service.host/greeting/hello
| Label | Average Latency, ms | Request, # / sec |
|---|---|---|
| Go | 105.62 | 33,196.64 |
| Rusty | 0.03207 | 29 623.02 |
| Scala | 55,8 | 17 531.83 |
| Node 5.1.1 | 71.29 | 13 620.48 |
| Node 0.12.7 | 90.29 | 10,681.11 |
So good looking, but, unfortunately, implausible figures in the results of Average Latency for Rust indicate one feature that is present in the hyper module. The thing is that the -c parameter in wrk indicates the number of connections that wrk will open on each thread and will not close, i.e. keep-alive connections. Hyper doesn’t work with keep-alive, one or two .
Moreover, if you display the distribution of requests on threads sent by wrk through a Lua script, we will see that only one thread sends all requests.
For those interested in Rust, it is also worth noting that these features led to this .
Therefore, in order for the test to be reliable, it was decided to conduct a similar test by placing in front of the nginx service, which will keep connections with wrk and proxy them to the required service:
upstream u_go { server 127.0.0.1:4002; keepalive 1000; } server { listen 80; server_name go; access_log off; tcp_nopush on; tcp_nodelay on; keepalive_timeout 300; keepalive_requests 10000; gzip off; gzip_vary off; location / { proxy_pass http://u_go; } } wrk -d300s -c1000 -t50 --timeout 2s nginx.host/service
| Label | Average Latency, ms | Request, # / sec |
|---|---|---|
| Rusty | 155.36 | 9,196.32 |
| Go | 145.24 | 7,333.06 |
| Scala | 233.69 | 2 513.95 |
| Node 5.1.1 | 207.82 | 2 422.44 |
| Node 0.12.7 | 209.5 | 2,410.54 |
wrk -d300s -c1000 -t50 --timeout 2s nginx.host/service/greeting/hello
| Label | Average Latency, ms | Request, # / sec |
|---|---|---|
| Rusty | 154.95 | 9 039,73 |
| Go | 147.87 | 7,427.47 |
| Node 5.1.1 | 199.17 | 2,470.53 |
| Node 0.12.7 | 177.34 | 2,363.39 |
| Scala | 262.19 | 2 218.22 |
As can be seen from the results, overhead with nginx is significant, but in our case we are interested in the performance of services that are on equal footing, regardless of the delay in nginx.
ab
The Apache ab utility, unlike wrk, does not keep keep-alive connections, so nginx is not useful to us here. Let's try to fulfill 50,000 requests in 10 seconds, with 256 possible parallel requests.
ab -n50000 -c256 -t10 service.host
| Label | Completed requests, # | Time per request, ms | Request, # / sec |
|---|---|---|---|
| Go | 50 000.00 | 22.04 | 11,616.03 |
| Rusty | 32,730.00 | 78.22 | 3 272.98 |
| Node 5.1.1 | 30,069.00 | 85.14 | 3,006.82 |
| Node 0.12.7 | 27 103.00 | 94.46 | 2 710.22 |
| Scala | 16 691.00 | 153.74 | 1 665,17 |
ab -n50000 -c256 -t10 service.host/greeting/hello
| Label | Completed requests, # | Time per request, ms | Request, # / sec |
|---|---|---|---|
| Go | 50 000.00 | 21.88 | 11,697.82 |
| Rusty | 49 878.00 | 51.42 | 4,978.66 |
| Node 5.1.1 | 30 333.00 | 84.40 | 3 033.29 |
| Node 0.12.7 | 27 610.00 | 92.72 | 2 760,99 |
| Scala | 27 178.00 | 94.34 | 2 713.59 |
It should be noted that Scala-application is characterized by some “warm-up” due to possible JVM optimizations that occur during the operation of the application.
As you can see, without nginx, hyper in Rust still does not do well even without keep-alive connections. And the only one who managed to process 50,000 requests in 10 seconds was Go.
Source
Node.js
var cluster = require('cluster'); var numCPUs = require('os').cpus().length; var http = require("http"); var debug = require("debug")("lite"); var workers = []; var server; cluster.on('fork', function(worker) { workers.push(worker); worker.on('online', function() { debug("worker %d is online!", worker.process.pid); }); worker.on('exit', function(code, signal) { debug("worker %d died", worker.process.pid); }); worker.on('error', function(err) { debug("worker %d error: %s", worker.process.pid, err); }); worker.on('disconnect', function() { workers.splice(workers.indexOf(worker), 1); debug("worker %d disconnected", worker.process.pid); }); }); if (cluster.isMaster) { debug("Starting pure node.js cluster"); ['SIGINT', 'SIGTERM'].forEach(function(signal) { process.on(signal, function() { debug("master got signal %s", signal); process.exit(1); }); }); for (var i = 0; i < numCPUs; i++) { cluster.fork(); } } else { server = http.createServer(); server.on('listening', function() { debug("Listening %o", server._connectionKey); }); var greetingRe = new RegExp("^\/greeting\/([az]+)$", "i"); server.on('request', function(req, res) { var match; switch (req.url) { case "/": { res.statusCode = 200; res.statusMessage = 'OK'; res.write("Hello World!"); break; } default: { match = greetingRe.exec(req.url); res.statusCode = 200; res.statusMessage = 'OK'; res.write("Hello, " + match[1]); } } res.end(); }); server.listen(8080, "127.0.0.1"); } Go
package main import ( "fmt" "net/http" "regexp" ) func main() { reg := regexp.MustCompile("^/greeting/([az]+)$") http.ListenAndServe(":8080", http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { switch r.URL.Path { case "/": fmt.Fprint(w, "Hello World!") default: fmt.Fprintf(w, "Hello, %s", reg.FindStringSubmatch(r.URL.Path)[1]) } })) } Rusty
extern crate hyper; extern crate regex; use std::io::Write; use regex::{Regex, Captures}; use hyper::Server; use hyper::server::{Request, Response}; use hyper::net::Fresh; use hyper::uri::RequestUri::{AbsolutePath}; fn handler(req: Request, res: Response<Fresh>) { let greeting_re = Regex::new(r"^/greeting/([az]+)$").unwrap(); match req.uri { AbsolutePath(ref path) => match (&req.method, &path[..]) { (&hyper::Get, "/") => { hello(&req, res); }, _ => { greet(&req, res, greeting_re.captures(path).unwrap()); } }, _ => { not_found(&req, res); } }; } fn hello(_: &Request, res: Response<Fresh>) { let mut r = res.start().unwrap(); r.write_all(b"Hello World!").unwrap(); r.end().unwrap(); } fn greet(_: &Request, res: Response<Fresh>, cap: Captures) { let mut r = res.start().unwrap(); r.write_all(format!("Hello, {}", cap.at(1).unwrap()).as_bytes()).unwrap(); r.end().unwrap(); } fn not_found(_: &Request, mut res: Response<Fresh>) { *res.status_mut() = hyper::NotFound; let mut r = res.start().unwrap(); r.write_all(b"Not Found\n").unwrap(); } fn main() { let _ = Server::http("127.0.0.1:8080").unwrap().handle(handler); } Scala
package lite import akka.actor.{ActorSystem, Props} import akka.io.IO import spray.can.Http import akka.pattern.ask import akka.util.Timeout import scala.concurrent.duration._ import akka.actor.Actor import spray.routing._ import spray.http._ import MediaTypes._ import org.json4s.JsonAST._ object Boot extends App { implicit val system = ActorSystem("on-spray-can") val service = system.actorOf(Props[LiteActor], "demo-service") implicit val timeout = Timeout(5.seconds) IO(Http) ? Http.Bind(service, interface = "localhost", port = 8080) } class LiteActor extends Actor with LiteService { def actorRefFactory = context def receive = runRoute(route) } trait LiteService extends HttpService { val route = path("greeting" / Segment) { user => get { respondWithMediaType(`text/html`) { complete("Hello, " + user) } } } ~ path("") { get { respondWithMediaType(`text/html`) { complete("Hello World!") } } } } Generalization
We present the success criteria defined in the beginning of the article in the form of a table. All applicants have debugging and profiling facilities, so there are no corresponding columns in the table.
| Label | Performance Rate 0 | Community size 1 | Packages count | IDE Support | Developers 5 |
|---|---|---|---|---|---|
| Go | 100.00% | 12,759 | 104 383 2 | + | 315 |
| Rusty | 89.23% | 3 391 | 3,582 | + 4 | 21 |
| Scala | 52.81% | 44,844 | 172 593 3 | + | 407 |
| Node 5.1.1 | 41.03% | 102 328 | 215,916 | + | 654 |
| Node 0.12.7 | 32.18% | 102 328 | 215,916 | + | 654 |
0 Performance was calculated based on the five-minute wrk tests without nginx, using the RPS parameter.
1 The size of the community was estimated on an indirect basis - the number of questions with the corresponding tag on StackOverflow .
2 The number of packages indexed by godoc.org .
3 Very roughly - search by languages ​​Java, Scala on github.com .
4 Under many favorite Idea plug-in is still not.
5 According to hh.ru.
Obviously, such graphs of the number of questions per tag per day can speak about the size of the community:
Go
Rusty
Scala
Node.js
For comparison, PHP:
findings
Understanding that benchmarks of performance is a rather shaky and ungrateful thing, it is difficult to draw any definite conclusions based only on such tests. Of course, everything is dictated by the type of task that needs to be addressed, the requirements for the program indicators and other environmental nuances.
In our case, on the basis of the criteria defined above and, in one way or another, subjective views, we chose Go.
The content of subjective assessments was deliberately omitted in this article in order not to make another round and not to provoke holivar. Moreover, if such estimates were not taken into account, then according to the criteria mentioned above, the result would remain the same.
Source: https://habr.com/ru/post/273341/
All Articles