We write a simple cache manager in memory on Go

In the process of working on small projects, it is often necessary to cache data and it happens that it is not possible to use Redis or Memcache. In such situations, a simple and fairly effective way is suitable without the use of additional tools - caching in RAM.
In this article I will tell you where to start, to write the cache manager in memory on Go.

Attention! This article is intended for novice developers solely for academic purposes and is not considered such tools as Redis, Memcache, etc.
In addition, we will not go into the problem of memory allocation.

For simplicity, we restrict ourselves to three main methods: setting a Set , getting a Get and deleting Delete .

Data will be stored in key / value format.

Structure

The first thing to do is create a structure describing our storage container:

 type Cache struct { sync.RWMutex defaultExpiration time.Duration cleanupInterval time.Duration items map[string]Item } 

• sync.RWMutex - for secure data access during read / write (more about mutexes https://gobyexample.com/mutexes ),
• defaultExpiration - default cache lifetime (this parameter can be redefined for each item)
• cleanupInterval - the interval at which the cache cleaning mechanism is started (Garbage Collector, hereinafter referred to as GC)
• items - cache items (in key / value format)

Now we describe the structure for the element:

 type Item struct { Value interface{} Created time.Time Expiration int64 } 

• Value - the value. Since it can be any (number / line / array, etc.), you must specify interface{} as the type
• Created - cache creation time,
• Expiration - expiration time (in UnixNano) - by it we will check the relevance of the cache

Initialization of storage

Start by initializing a new storage container:

 func New(defaultExpiration, cleanupInterval time.Duration) *Cache { //  (map)   (string)/(Item) items := make(map[string]Item) cache := Cache{ items: items, defaultExpiration: defaultExpiration, cleanupInterval: cleanupInterval, } //     0,  GC (  ) if cleanupInterval > 0 { cache.StartGC() //     } return &cache } 

Initializing a new cache instance takes two arguments: defaultExpiration and cleanupInterval

• defaultExpiration - the default cache lifetime, if set to less than or equal to 0, the cache lifetime is indefinite.
• cleanupInterval - the interval between deleting expired cache. If the value is set to be less than or equal to 0, cleaning and deletion of expired cache does not occur.

At the output we get a container with the structure Cache

Be careful when setting these parameters, too small or too large values ​​can lead to undesirable consequences, for example, if you set cleanupInterval = 1 * time.Second search for expired keys will occur every second, which will negatively affect the performance of your program. Conversely, setting cleanupInterval = 168 * time.Hour - unused elements will accumulate in the memory.

Setting values

After the container is created, it would be good to be able to write data to it, for this we will write the implementation of the Set method.

 func (c *Cache) Set(key string, value interface{}, duration time.Duration) { var expiration int64 //     0 -   - if duration == 0 { duration = c.defaultExpiration } //     if duration > 0 { expiration = time.Now().Add(duration).UnixNano() } c.Lock() defer c.Unlock() c.items[key] = Item{ Value: value, Expiration: expiration, Created: time.Now(), } } 

Set adds a new item to the cache or replaces an existing one. In this case, the test for the existence of keys does not occur. It takes as arguments: a key-identifier in the form of a string key , a value and a cache lifetime; duration .

Getting values

Using Set we recorded data in the repository, now we implement a method for getting them Get

 func (c *Cache) Get(key string) (interface{}, bool) { c.RLock() defer c.RUnlock() item, found := c.items[key] //    if !found { return nil, false } //     ,      if item.Expiration > 0 { //        nil if time.Now().UnixNano() > item.Expiration { return nil, false } } return item.Value, true } 

Get returns a value (or nil ) and the second bool parameter is true if the key is found and false if the key is not found or the cache is outdated.

Deleting cache

Now that we have an install and a get, you need to be able to delete the cache (if we don’t need it anymore) for this we write the Delete method

 func (c *Cache) Delete(key string) error { c.Lock() defer c.Unlock() if _, found := c.items[key]; !found { return errors.New("Key not found") } delete(c.items, key) return nil } 

Delete deletes the item by key, if the key does not exist returns an error.

Garbage collection

We have add, get and delete. It remains to implement the search for expired keys, followed by cleaning (GC)
To do this, we write the StartGC method, which starts when a new instance of the New cache is initialized and runs until the program is completed.

 func (c *Cache) StartGC() { go c.GC() } func (c *Cache) GC() { for { //     cleanupInterval <-time.After(c.cleanupInterval) if c.items == nil { return } //           if keys := c.expiredKeys(); len(keys) != 0 { c.clearItems(keys) } } } // expiredKeys   ""  func (c *Cache) expiredKeys() (keys []string) { c.RLock() defer c.RUnlock() for k, i := range c.items { if time.Now().UnixNano() > i.Expiration && i.Expiration > 0 { keys = append(keys, k) } } return } // clearItems     ,    "" func (c *Cache) clearItems(keys []string) { c.Lock() defer c.Unlock() for _, k := range keys { delete(c.items, k) } } 

Usage example

 import ( memorycache "github.com/maxchagin/go-memorycache-example" ) //      -  5       10  cache := memorycache.New(5 * time.Minute, 10 * time.Minute) //     "myKey"    5  cache.Set("myKey", "My value", 5 * time.Minute) //     "myKey" i := cache.Get("myKey") 

What's next?

Now we have a cache manager with minimal functionality, it will be enough for the simplest tasks. If this is not enough (and in 95% of cases it is), as the next step, you can independently implement the methods:

Count - getting the number of items in the cache
GetItem - getting item cache
Rename - rename key
Copy - copy item
Increment - increment
Decrement - decrement
Exist - check item for existence
Expire - cache check for expiration
FlushAll - clear all data
SaveFile - save data to file
LoadFile - load data from file

This is not a complete list, but for the basic functionality is likely enough.

Sources c example on github

If you need a ready-made cache manager in memory, I recommend to pay attention to the following projects:
The patrickmn go-cache implementation
Beego memorycache