We at PushAll process several thousand requests per second to get delivery statistics and open notifications and to send content alerts. A regular database like MySQL cannot cope with such a flow of queries and cannot respond so quickly.

Trying to transfer more and more operations to fast NoSQL repositories like Redis, we want to know how to use it more efficiently and whether we will have problems with a large number of connections.
We also use PHP forks to work with, and we wondered how Redis would behave if we made several thousand connections in several threads simultaneously. We decided to share our tests with the community.

Iron

We are testing on one of the VPS PushAll:
')
CPU: Intel Xeon E5-1650v2 3.5 Ghz - 2 cores.
RAM: 3 Gb DDR3 1866Mhz

PHP7.
Redis 3.0.7

Test conditions

We wrote a PHP multithreaded bot that:

• Makes forks in a cycle - 100 forks without any delays
• Each fork in its cycle creates a connection with Redis 1000 times and produces an increment
• The parent process waits for 3 seconds and takes the value, if not wait - Redis will not return the full value of the encerement

We also tested the version with 1000 forks and how the results will differ when using UNIX-socket and TCP.

100 forks, 1000 connections each, TCP

# time php benchmark.php End:100000 real 0m8.666s user 0m0.063s sys 0m0.073s 

100 forks, 1000 connections each, UNIX socket

 # time php benchmark.php End:100000 real 0m6.021s user 0m0.023s sys 0m0.067s 

TCP socket is on average 30% slower. (remember, it’s not Redis performance itself that is being tested anymore, but how it handles connections)

1000 forks, 1000 connections each, TCP + UNIX

Raise the stakes

TCP:

 # time php benchmark.php End:903505 real 1m7.659s user 0m0.073s sys 0m0.753s 

For 3 seconds, Redis did not have time to process them to the end - this is one of the details that must be taken into account. If you read the values ​​too quickly, you can catch a moment when they are still old.

What is most interesting, when conducting the same test, but for a unix-socket, we get errors:

 Fatal error: Uncaught RedisException: Redis server went away in .... 

That is, in spite of the fact that it is faster, a unix-socket can handle a slightly smaller number of requests. Or, alternatively, it is possible that because of the fact that it is so fast, the Redis server itself cannot cope.

We conducted similar tests for php-fpm - there also a TCP socket gave fewer errors with a bundle with NGINX than a UNIX socket. The difference in speed was insignificant there.

Attaching the script:

 <?php declare(ticks = 1); for($i=0; $i < 1000; $i++){ $pid = pcntl_fork(); if ($pid == -1) { die('could not fork'); } else if ($pid) { //parent } else { //child for ($a=0; $a < 1000; $a++) { $redis = new Redis(); //$redis->connect('127.0.0.1:6379'); $redis->connect('/run/redis/redis.sock'); $redis->incr('pushall:benchmark'); $redis->close(); } exit; } } pcntl_wait($status); //wait sleep(3); $redis = new Redis(); $redis->connect('/run/redis/redis.sock'); echo 'End:'.$redis->get('pushall:benchmark')."\r\n"; $redis->setTimeout('pushall:benchmark', 1); $redis->close(); 

UPD pconnect

It turns out pconnect works in forks (strange)

Took the case of 100 TCP processes:
pconnect

 # time php benchmark.php End:100000 real 0m4.679s user 0m0.023s sys 0m0.080s 

connect

 # time php benchmark.php End:100000 real 0m9.100s user 0m0.037s sys 0m0.103s 

For comparison, UNIX socket on 100 forks:
pconnect

 # time php benchmark.php End:100000 real 0m4.393s user 0m0.023s sys 0m0.073s 

connect

 # time php benchmark.php End:100000 real 0m6.002s user 0m0.027s sys 0m0.057s 

And, interestingly, when using pconnect, the difference between TCP and UNIX sockets is not so big 5-10%. At the same time, even having done everything that I was offered in the comments - I could not get the unix-sockets to work with 1000 forks.

UPD 2 Pconnect + 1000 forks

UNIX socket
pconnect

 # time php benchmark.php End:1000000 real 0m35.445s user 0m0.050s sys 0m0.637s 

connect - crashes in Fatal error: Uncaught RedisException: Redis server went away in ...

Tcp
pconnect

 # time php benchmark.php End:989596 real 0m43.711s user 0m0.050s sys 0m0.623s 

 # time php benchmark.php End:903505 real 1m7.659s user 0m0.073s sys 0m0.753s 

The difference is 20%.

Ps. Habr, why is there a MongoDB hub and MySQL, but not Redis?