Increase Redis performance with a simple cluster

Starting familiarity with the nosql-base Redis, in almost every article devoted to it, we find the statement that this base works incredibly fast. The speed of work is really amazing, thanks to the storage of data in RAM.

But imagine a situation where Redis is straining from the load. This situation is not uncommon. What, then, is to be done?
')

Replication

The first thing that comes to mind is to add another database, creating a master-slave link, and redirect read operations to it.

Pros:

You can add a database and configure replication quickly. Install Redis, execute several commands, wait for the data to be synchronized and use the replica.

The number of read operations increases in proportion to the number of slave databases.

Minuses:

It is necessary to refine the code - create a proxy class for distributing read / write operations to the required databases.

The number of write operations will remain unchanged. We can only read from slave-bases. Write operations can be performed only on the master database. If Redis screamed from too many writing commands, we will not get any speeding up.

Cluster

The second obvious solution is to create a cluster from Redis databases.

Cluster advantages:

The number of read and write operations increases in proportion to the number of databases in the cluster.

The cluster is easily configured for the new database.

It is quite easy to add replication for the Redis-databases included in the cluster. Thereby increasing the maximum number of read operations.

The size of the base is significantly larger. Agree that 8 servers with 32 GB of memory are much better than one.

If necessary, simply increase the size of the base. Replacing 8 servers from 32 GB of memory to 8 servers of 64 GB will immediately increase the maximum base size by 128 GB (Do not forget that for fast Redis operation, the amount of RAM must be at least 2 times larger than the base size). This is much better than replacing one server with another with a lot of RAM.

Cluster Cons:

It is necessary to refine the code - to create a proxy class for distributing read / write operations between the cluster nodes.

If such a solution is implemented on an existing database, then it is necessary to migrate data to the databases included in the cluster. This task is non-trivial and requires a lot of time and effort. Each portable database has its own logic and therefore for each case it is necessary to create an individual solution. The easiest way to create a cluster from databases initially and use it in work is to transfer a ready base to a cluster.

I have been using Redis for two years as the main database on high-load projects. (Games for mobile devices). All this time, the creators of Redis promise that they are about to release a cluster solution for Redis. They have been waiting for the promised three years, and those who are not waiting write their decisions, including me.

I managed to create my own cluster. I hope that my experience will be useful to people seeking to increase the performance of Redis in their projects. I have to say that I use 8 nodes by default. If the performance of 1 redis-base was not enough, then it was not possible to fully load 8 redis-bases.

Two years of cluster work under load in several projects showed its consistency.

Those interested can download and use the sample code from the github - github.com/dfer/myredis

PS

Two years ago, when I was thinking about how to solve the Redis base performance problem and create a redis cluster, the following 2 articles helped me:

oldblog.antirez.com/post/redis-presharding.html

blog.zawodny.com/2011/02/26/redis-sharding-at-craigslist

I hope that my solution and the articles mentioned above can be useful to people who are faced with poor Redis performance in their projects.