Especially unscientific: Tarantool 1.6 vs Golang (by speed)

I recently read about Tarantool, it became interesting. The idea is good - the code next to the database is stored in such a fast Redis-like environment.

And I thought about something - we are now actively using Golang at work, in fact, the thought came that a lot of things were written on Go, including and embedded databases. And what if you compare, for example, Go + LevelDB (actually, it would be possible to any other) against Tarantool. I also tested Go + RocksDB, but it turned out to be a bit more complicated , and the result is about the same on small data.

I tested a simple task — an HTTP server; when prompted, write the key to the database, retrieve it by name (without any checks on the race), send back a simple JSON from this value.

Compared: go+leveldb , tarantool , go+go-tarantool , nginx upstream tnt_pass

Looking ahead - in my unscientific test I won Go + LevelDB by using all the processor cores. Most likely, if you run a few Tarantulas and a balancer, there may be some gain, but not to say that it is significant ... But, really, there will have to be replication or something like that.

But, in general, Tarantool is a very impressive thing.

Please note: I compare a very specific case , this does not mean that in all other cases, Go / LevelDB will win or lose.

And also: instead of LevelDB, it's probably better to use RocksDB.

So the result (briefly)

4-10 = 4 streams, 10 simultaneous connections
10-100 = 10 threads, 100 connections

Please note Tarantool takes only 1 CPU thread (or rather, by the form 2), and was tested on a 4-line CPU. Go uses all kernels and threads by default.

nginx lua tnt_pass is taken from the dedokOne comment ( result )

wrk -t 4 -c 10 (4 streams, 10 connections):

Golang:

  Latency Distribution 50% 269.00us 99% 1.64ms Requests/sec: 25637.26 

Tarantool:

  Latency Distribution 50% 694.00us 99% 1.43ms Requests/sec: 10377.78 

But, Tarantula took about only half of the cores, so, probably, their speed is about the same.

Under a large load ( wrk -t 10 -c 100 ) Tarantula remained in place on the RPS (but latency sagged much more noticeably than Golang, especially the upper part), and Golang even cheered up (but latency also sagged, of course).

Go:

  Latency Distribution 50% 2.85ms 99% 8.12ms Requests/sec: 33226.52 

Tarantool:

  Latency Distribution 50% 8.69ms 99% 73.09ms Requests/sec: 10763.55 

Tarantool has its advantages: secondary index, replication ...

Go also has a huge ecosystem of libraries ( about 100 thousand by my calculations, among them and the implementations of embedded (and not very) databases - the sea ), and, as an example, the same bleve gives a full-text search (which, as I understand it, , not in Tarantool).

Feels like the Tarantula ecosystem is poorer. At least everything that is offered - msgpack, http server, client, json, LRU cache, ... in Go is implemented in numerous variants ..

That is, in general, there is no insane speed win.

So far, my personal choice remains in the direction of Go, because there is no feeling that the Tarantool ecosystem will shoot so much in the near future, and Go has been actively developing for a long time.

The code in Tarantool, of course, is shorter, but mainly due to the fact that errors are processed by the language. In Go, you can also cut all the err and will remain about the same.

Maybe someone has other opinions?

Even in the comments we noticed about atomic code updates in Tarantool, but since we are talking about HTTP requests - we ( end of the current place of work) use endless for go and according to our tests (and we have thousands of requests there per second) - we update Go code without losing HTTP requests. An example at the end of the article.

And if more about the test:

  ➜ ~ go version go version go1.6 darwin/amd64 ➜ ~ tarantool --version Tarantool 1.6.8-525-ga571ac0 Target: Darwin-x86_64-Release 

Golang:

 ➜ ~ wrk -t 4 -c 10 -d 5 --latency http://127.0.0.1:8081/ Running 5s test @ http://127.0.0.1:8081/ 4 threads and 10 connections Thread Stats Avg Stdev Max +/- Stdev Latency 346.71us 600.80us 26.94ms 97.89% Req/Sec 6.54k 0.88k 13.87k 73.13% Latency Distribution 50% 269.00us 75% 368.00us 90% 493.00us 99% 1.64ms 130717 requests in 5.10s, 15.08MB read Requests/sec: 25637.26 Transfer/sec: 2.96MB 

Tarantool:

 ➜ ~ wrk -t 4 -c 10 -d 5 --latency http://127.0.0.1:8080/ Running 5s test @ http://127.0.0.1:8080/ 4 threads and 10 connections Thread Stats Avg Stdev Max +/- Stdev Latency 767.53us 209.64us 4.04ms 87.26% Req/Sec 2.61k 437.12 3.15k 45.59% Latency Distribution 50% 694.00us 75% 0.90ms 90% 1.02ms 99% 1.43ms 52927 requests in 5.10s, 8.58MB read Requests/sec: 10377.78 Transfer/sec: 1.68MB 

Under greater load:

Go:

 ➜ ~ wrk -t 10 -c 100 -d 5 --latency http://127.0.0.1:8081/ Running 5s test @ http://127.0.0.1:8081/ 10 threads and 100 connections Thread Stats Avg Stdev Max +/- Stdev Latency 3.04ms 1.48ms 25.53ms 80.21% Req/Sec 3.34k 621.43 12.52k 86.20% Latency Distribution 50% 2.85ms 75% 3.58ms 90% 4.57ms 99% 8.12ms 166514 requests in 5.01s, 19.21MB read Requests/sec: 33226.52 Transfer/sec: 3.83MB 

Tarantool:

 ➜ ~ wrk -t 10 -c 100 -d 5 --latency http://127.0.0.1:8080/ Running 5s test @ http://127.0.0.1:8080/ 10 threads and 100 connections Thread Stats Avg Stdev Max +/- Stdev Latency 10.65ms 14.24ms 269.85ms 98.43% Req/Sec 1.09k 128.17 1.73k 94.56% Latency Distribution 50% 8.69ms 75% 10.50ms 90% 11.36ms 99% 73.09ms 53943 requests in 5.01s, 8.75MB read Requests/sec: 10763.55 Transfer/sec: 1.75MB 

Test sources:

Go:

 package main import ( "encoding/json" "fmt" "io" "net/http" "github.com/syndtr/goleveldb/leveldb" ) var db *leveldb.DB func hello(w http.ResponseWriter, r *http.Request) { err := db.Put([]byte("foo"), []byte("bar"), nil) if err != nil { w.WriteHeader(500) io.WriteString(w, err.Error()) return } res, err := db.Get([]byte("foo"), nil) if err != nil { w.WriteHeader(500) io.WriteString(w, err.Error()) return } result, err := json.Marshal(string(res)) if err != nil { w.WriteHeader(500) io.WriteString(w, err.Error()) return } w.Write(result) } func main() { var err error db, err = leveldb.OpenFile("level.db", nil) if err != nil { panic(err) } http.HandleFunc("/", hello) fmt.Println("http://127.0.0.1:8081/") http.ListenAndServe("127.0.0.1:8081", nil) } 

Tarantool:

 #!/usr/bin/env tarantool box.cfg{logger = 'tarantool.log'} space = box.space.data if not space then space = box.schema.create_space('data') space:create_index('primary', { parts = {1, 'STR'} }) end local function handler(req) space:put({'foo','bar'}) local val = space:get('foo') return req:render({ json = val[2] }) end print "http://127.0.0.1:8080/" require('http.server').new('127.0.0.1', 8080) :route({ path = '/' }, handler) :start() 

Golang (atomic code replacement, without loss of compounds):

 package main import ( "encoding/json" "fmt" "io" "net/http" "syscall" "io/ioutil" "time" "github.com/fvbock/endless" "github.com/gorilla/mux" "github.com/syndtr/goleveldb/leveldb" ) var db *leveldb.DB func hello(w http.ResponseWriter, r *http.Request) { if db == nil { // ()  ,      panic("DB is not yet initialized") } err := db.Put([]byte("foo"), []byte("bar"), nil) if err != nil { w.WriteHeader(500) io.WriteString(w, err.Error()) return } res, err := db.Get([]byte("foo"), nil) if err != nil { w.WriteHeader(500) io.WriteString(w, err.Error()) return } result, err := json.Marshal(string(res)) if err != nil { w.WriteHeader(500) io.WriteString(w, err.Error()) return } w.Write(result) } func main() { var err error mux1 := mux.NewRouter() mux1.HandleFunc("/", hello).Methods("GET") fmt.Println("http://127.0.0.1:8081/") server := endless.NewServer("127.0.0.1:8081", mux1) server.BeforeBegin = func(add string) { ioutil.WriteFile("server.pid", []byte(fmt.Sprintf("%d", syscall.Getpid())), 0755) db, err = leveldb.OpenFile("level.db", nil) for err != nil { time.Sleep(10 * time.Millisecond) db, err = leveldb.OpenFile("level.db", nil) } } server.ListenAndServe() if db != nil { db.Close() } } 

After that, you can do a go build run and try to do a go build; kill -1 $(cat server.pid) during load go build; kill -1 $(cat server.pid) go build; kill -1 $(cat server.pid) - no data loss was observed in my tests.

In the comments recommended to try go + go-tarantool

I tried:

Lower load

 ➜ ~ wrk -t 4 -c 10 -d 5 --latency http://127.0.0.1:8081/ Running 5s test @ http://127.0.0.1:8081/ 4 threads and 10 connections Thread Stats Avg Stdev Max +/- Stdev Latency 799.14us 502.56us 25.22ms 95.74% Req/Sec 2.55k 248.65 2.95k 85.22% Latency Distribution 50% 727.00us 75% 843.00us 90% 1.02ms 99% 2.03ms 51591 requests in 5.10s, 5.95MB read Requests/sec: 10115.52 Transfer/sec: 1.17MB 

Huge pressure:

 ➜ ~ wrk -t 10 -c 100 -d 5 --latency http://127.0.0.1:8081/ Running 5s test @ http://127.0.0.1:8081/ 10 threads and 100 connections Thread Stats Avg Stdev Max +/- Stdev Latency 7.49ms 4.00ms 65.06ms 81.21% Req/Sec 1.38k 357.31 8.40k 94.61% Latency Distribution 50% 6.78ms 75% 8.86ms 90% 11.77ms 99% 22.74ms 69091 requests in 5.10s, 7.97MB read Requests/sec: 13545.12 Transfer/sec: 1.56MB 

Source:

tarantool.lua:

 #!/usr/bin/env tarantool box.cfg{ listen = '127.0.0.1:3013', logger = 'tarantool.log' } space = box.space.data if not space then box.schema.user.grant('guest', 'read,write,execute', 'universe') space = box.schema.create_space('data') space:create_index('primary', { parts = {1, 'STR'} }) end print(space.id) print('Starting on 3013') 

main.go:

 package main import ( "encoding/json" "fmt" "io" "log" "net/http" "time" "github.com/tarantool/go-tarantool" ) var client *tarantool.Connection func hello(w http.ResponseWriter, r *http.Request) { spaceNo := uint32(512) _, err := client.Replace(spaceNo, []interface{}{"foo", "bar"}) if err != nil { w.WriteHeader(500) io.WriteString(w, err.Error()) return } indexNo := uint32(0) resp, err := client.Select(spaceNo, indexNo, 0, 1, tarantool.IterEq, []interface{}{"foo"}) if err != nil { w.WriteHeader(500) io.WriteString(w, err.Error()) return } first := resp.Data[0].([]interface{}) result, err := json.Marshal(first[1]) if err != nil { w.WriteHeader(500) io.WriteString(w, err.Error()) return } w.Write(result) } func main() { var err error server := "127.0.0.1:3013" opts := tarantool.Opts{ Timeout: 500 * time.Millisecond, } client, err = tarantool.Connect(server, opts) if err != nil { log.Fatalf("Failed to connect: %s", err.Error()) } http.HandleFunc("/", hello) fmt.Println("http://127.0.0.1:8081/") http.ListenAndServe("127.0.0.1:8081", nil) }