LUA in nginx: hot cache in memory

I decided to replenish the piggy bank of articles on Habré about such a wonderful PL as lua, a couple of examples of its use under the hood of nginx. Smashed into two independent posts, the second here .

In this post, nginx is used as a “hot cache” of some constantly updated data requested by clients over an interval with optional grouping (some analogue of BETWEEN and GROUP BY / AGGREGATE from SQL). Data is loaded into the cache by lua + nginx from Redis. The initial data in Redis is added every second, and customers want them from so far (interval in seconds, minutes, hours ...) with aggregation by N (1 <= N <= 3600) seconds, sorted by date and in json format.
With a good hitrate on an existing machine, it turns out to provide 110-130k “hotelok” per second, but with a bad one - only 20-30k. That, in general, is also acceptable for us at the same instance of nginx.


From a certain source, data that is added to Redis ZSET comes every second. The important point is to link the data to the time - the sample will go on time intervals. One client came - “give me one of these so-by-seconds”, another came - “and I’m having this interval, but let's get aggregated for hour”, the third needed one last second, fourth for a day with aggregation of 27 seconds d ... Knocking on data directly in Redis is unreal. It is very problematic to cache the prepared data in advance, because The required intervals and aggregation step are generally in each client / request and can vary arbitrarily. The server must be ready to quickly respond to any reasonable request.
')
Initially, there was an idea to perform aggregation on the Redis side, calling through EVAL redis-lua code from nginx-lua code. This “We need to go deeper technology” didn’t work because of the single-stream nature of Redis itself: it’s quick to give out “raw data” much faster than to group and push the finished result.

Data in Redis is stored element-wise already in json format of the form:

ZADD ns:zs:key 1386701764 "{\"data100500\":\"hello habr\",\"dt\":\"10.12.2013 10:05:00\",\"smth\":\"else\"}" 

The key is the timestamp, in dt, the string equivalent of the "filler" version.
Accordingly, sampling range:

 ZREVRANGEBYSCORE ns:zs:data:sec 1386701764 1386700653 WITHSCORES 

And on lua via resty Redis:

 local redis = require 'redis' local R, err = redis:new() R:connect('12.34.56.78', 6379) R:zrevrangebyscore('ns:zs:data:sec', to, from, 'WITHSCORES') --  .. 

We build nginx ( lua-nginx-module + lua-resty-redis ), fluently customize:

 http { lua_package_path '/path/to/lua/?.lua;;'; init_by_lua_file '/path/to/lua/init.lua'; lua_shared_dict ourmegacache 1024m; server { location = /data.js { content_by_lua_file '/path/to/lua/get_data.lua'; } } } 

For a variety of boundaries of the requested interval and the aggregation step, we will receive from the GET parameters of the query from , to, and step . With this agreement, the approximate format of the request to the service will be as follows:

/data.js?step=300&from=1386700653&to=1386701764

 local args = ngx.req.get_uri_args() local from = tonumber(args.from) or 0 ... 

So, we have element-wise json records stored in Redis, which we can get from there. How best to cache and give them to customers?

• You can store second records in a table separately. However, as practice has shown, the implementation of several dozen queries to the table has a very negative impact on performance. And if a request arrives for a day, then a response with a short timeout can be avoided;
• Records can be stored in blocks, combining through a common separator or serializing them at least in the same json. And when a query needs to be broken down by separator or deserialized. So-so option;
• Store data hierarchically, with partial repetitions at different levels of aggregation. Cache blocks of different sizes are used: 1 second (single entry), 10 seconds, 1 minute, 10 minutes, an hour. Each block contains data of all its seconds. The most important thing is that the contents of the block do not change and are not given in pieces: either as a whole or in any way.

The latter option is chosen, which consumes more memory but significantly reduces the number of table accesses. Cache blocks of different sizes are used: 1 second (single entry), 10 seconds, 1 minute, 10 minutes, an hour. Each block contains data of all its seconds. Each block is aligned to the boundary of its interval, for example, the first element of the 10 second interval always has a timestamp having a decimal remainder of 9 (sorting in descending order as customers want), and the hour block contains 59:59, 59: 58, ... 00:00 elements. When combining elements, they are immediately glued together with a separator - a comma, which allows you to give these blocks to the client in one action: '[', block, ']', and also quickly combine them into larger pieces.

To cover the requested interval, a partition is performed into the maximum possible blocks with smaller blocks along the edges. Since we have single blocks, it is always possible to fully cover the required interval. To query the interval 02: 29: 58 ... 03:11:02 we get the layout of the cache:

 1sec - 03:11:02
 1sec - 03:11:01
 1sec - 03:11:00
 1min - 03:10:59 .. 03:10:00
 10min - 03:09:59 .. 03:00:00
 30min - 02:59:59 .. 02:30:00
 1sec - 02:29:59
 1sec - 02:29:58

This is just an example. Real calculations are performed on timestamps.
It turns out that we need 8 requests to the local cache. Or to Redis, if they are locally / not yet available. And in order not to break behind the same data from different workers / connects, you can use the atomicity of shared dict operations to implement locks (where key is a string cache key containing information about the interval and aggregation step):

 local chunk local lock_ttl = 0.5 --     ,   local key_lock = key .. ':lock' local try_until = ngx.now() + lock_ttl local locked while true do locked = cache:add(key_lock, 1, lock_ttl) chunk = cache:get(key) if locked or chunk or (try_until < ngx.now()) then break end ngx.sleep(0.01) -- ,   nginx evloop end if locked then --   . ,   elseif chunk then --    ,        end if locked then cache:delete(key_lock) end 

Having the necessary layout for caches, the ability to select the desired range from Redis, and the aggregation logic (it is very specific here, I don’t give an example), we get an excellent caching server, which, after warming up, knocks on Redis only once a second for a new element + for the old, if they have not been selected yet or were thrown out on the LRU. And don't forget about the limited pool of connections in Redis.
In our case, the warm-up looks like a short-term jump in incoming traffic of the order of 100-110Mb / s for a few seconds. On cpu, on a machine with nginx, warming up is almost not noticeable at all.

The image in the cap is taken from here .