Dump memcached to disk

The memcached interaction API is represented in all popular languages, so this is the most used tool in caching tasks. In the case when nothing other than caching is required, it seems to be the most justified.
One of the problems I encountered when working with memcached is the inability to reset its state to disk. Existing solutions were either not a simple cache (representing a database rather), or were not as stable and supported. In addition, there was a desire to “code”, so I could miss out on some of the ready-made options.

Why this may be needed

The reason why you usually do not need persistent cache - data is easy to get again. However, if there is a possibility that during a rush hour the cache server with hitrate = 60% will reboot and the service will not physically be able to cope with the increased load, then this probability should be taken into account.

Fork and dump

The presented fork, memcached-dd , solves exactly the indicated problem - it produces a dump of records to disk. The use is simple - just run the modified memcached with the file name in the -F parameter:

memcached -F /tmp/memcache.dump -m 64 -p 11211 -l 127.0.0.1 

When you start the file /tmp/memcache.dump will be loaded (if it exists). There will also be a recording of a new dump. Manage dump records, etc. it is supposed from the outside, initiation - by signal SIGUSR2 . It is worth noting that the dump will first write the .tmp file and only then, upon successful recording, be renamed.
')

Features of the dump

• Dump is performed in a separate thread and does not block memcached work.
• If TTL is used, when loading the dump, the TTL will be equal to it at the time of unloading.
• Overdue data does not fall into dump (*)

Usage example

Let's see how to use memcached with a Perl script that loads “autogenerated” lines into it.

 use Cache::Memcached; $memd = new Cache::Memcached { 'servers' => [ "127.0.0.1:11211" ] }; $| = 1; for (my $i = 0; $i <= 10000; $i++) { $memd->set( "key_$i", "x"x100 . " [$i]" ); # my $val = $memd->get( "xkey_$i"); if ($i % 1000 == 0) { print "\r$i..."; } } 

Use this script after running memcached

 $ ./memcached -P /tmp/memcached.pid -F /tmp/memcached.dump -m 128 -p 11211 -l 127.0.0.1 #   $ perl ./load.pl #   $ kill -USR2 `cat /tmp/memcached.pid` 1Mb dumped: 10001 items (0 expired during dump, 0 nuked by flush) Moving temprorary /tmp/memcached.dump.tmp -> /tmp/memcached.dump 

So, we have the file /tmp/memcached.dump with 10001 entries. Restart memcached with the same -F flag and check for the existence of the key # 1:

 $ ./memcached -P /tmp/memcached.pid -F /tmp/memcached.dump -m 128 -p 11211 -l 127.0.0.1 $ echo get key_1 | nc 127.0.0.1 11211 VALUE key_1 0 104 xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx [1] END 

As you can see, the contents were successfully restored from the dump.

This solution allows you to use the memcache API, i.e. use dump without modifying the communication scheme. I hope that this feature will be useful in special scripts of working with memcached.