📜 ⬆️ ⬇️

Nginx + Lua + Redis. Effectively process the session and give the data

image
Suppose you have data that you want to cache and give away without using heavy languages ​​like php , while checking that the user is authenticated and has the right to access the data. Today I will tell you how, using the nginx lua redis bundle , to accomplish this task, unload the server and increase the speed at which the server returns information tenfold.

First you need to build nginx with the nginx_lua_module module .

Installation Instructions
Install the lua compiler (version 2.0 or 2.1)

Download luaJit and collect it.
make && sudo make install 

')
To build nginx with the nginx devel kit, you need the http_rewrite_module , which in turn requires the pcre library. Therefore, install it
 sudo apt-get update sudo apt-get install libpcre3 libpcre3-dev 


Download dependent modules and nginx itself
nginx devel kit
nginx lua module
nginx

Configure and install nginx
 export LUAJIT_LIB=/usr/local/lib //    lua export LUAJIT_INC=/usr/local/include/luajit-2.1 //  luaJit ./configure --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nginx --group=nginx --with-ld-opt="-Wl,-rpath,/path/to/lua/lib" //    Lua --add-module=/path/to/ngx_devel_kit //  nginx devel kit --add-module=/path/to/lua-nginx-module //   nginx lua module --without-http_gzip_module make -j2 sudo make install 


Download the lua library to work with redis lua redis lib and copy it to the lua library folder with the command
 sudo make install 


Let's connect the lua redis library to the nginx configuration

 http { ... lua_package_path lua_package_path "/path/to/lib/lua/resty/redis.lua;;"; //    lua redis ... } 


Everything. Now you can write scripts on lua that will be executed nginx


In order to quickly and efficiently give cached data, we will put the most frequently used ones in redis right after warming up the cache, and we will put less used ones on request. We will give data using lua on the side of nginx . Php will not be involved in this bundle, which will speed up data output and will take up much less memory from the server.

To do this, write the Lua script.

search.lua
 local string = ngx.var.arg_string --    GET  if string == nil then ngx.exec("/") --   ,    end local path = "/?string=" .. string local redis = require "resty.redis" --      redis local red = redis:new() red:set_timeout(1000) -- 1 sec local ok, err = red:connect("127.0.0.1", 6379) if not ok then ngx.exec(path) --     redis,    end res, err = red:get("search:" .. string); --    redis if res == ngx.null then ngx.exec(path) --   ,    else ngx.header.content_type = 'application/json' ngx.say(res) --   ,    end 



Connect this file to nginx.conf and reboot nginx

 location /search-by-string { content_by_lua_file lua/search.lua; } 


Now when you query / search-by-string? String = smth lua will connect to redis and try to find the data using the search: smth key . If there is no data, the request will process php . But if the data is already cached and are in redis , they will immediately be given to the user.

But what if we need to give data only if the user is authenticated and at the same time has a certain role?

In this case, you can store the session in redis and before giving the content, check the user's role according to the session.

Since I work with the Symfony2 framework, then a small nginx-session-handler bundle was written for it, with which you can store the session in redis exactly as it suits us.

In redis, the data will be stored as a hash value:
phpsession - session key prefix
php-session - php session itself
user-role - user role.

Now you need to write a lua script to process this data:

session.lua
 local redis = require "resty.redis" --      redis local red = redis:new() red:set_timeout(1000) -- 1 sec local ok, err = red:connect("127.0.0.1", 6379) if not ok then ngx.exit(ngx.HTTP_INTERNAL_SERVER_ERROR) --    , end --      500 local phpsession = ngx.var.cookie_PHPSESSID --  id   cookie  local ROLE_ADMIN = "ROLE_ADMIN" -- ,     if phpsession == ngx.null then ngx.exit(ngx.HTTP_FORBIDDEN) --   cookie  (  ), end --      403 local res, err = red:hget("phpsession:" .. phpsession, "user-role") --    --  redis  id  if res == ngx.null or res ~= ROLE_ADMIN then ngx.exit(ngx.HTTP_FORBIDDEN) --   (   )  end --     ,   , --      403 



We retrieve the session id of the user from the cookie, trying to get the role of the user by its session id from the redis at the request of HGET phpsession: id user-role . If the user has expired the session, he is not authenticated, or he does not have the role of ROLE_ADMIN, the server will return the code 403.

We add this session processing script to our data retrieval script and now only authenticated users with the ROLE_ADMIN role can receive data.

In fact, a session processing script will be required for multiple nginx locations. In order not to write the same code in different places, we will include this file where we need it.

First, let's rewrite our session processing script.

session.lua
 local _M = {} --  function _M.handle() --            local redis = require "resty.redis" local red = redis:new() red:set_timeout(1000) -- 1 sec local ok = red:connect("127.0.0.1", 6379) if not ok then ngx.exit(ngx.HTTP_INTERNAL_SERVER_ERROR) end local phpsession = ngx.var.cookie_PHPSESSID local ROLE_ADMIN = "ROLE_ADMIN" if phpsession == ngx.null then ngx.exit(ngx.HTTP_FORBIDDEN) end local res = red:hget("phpsession:" .. phpsession, "user-role") if res == ngx.null or res ~= ROLE_ADMIN then ngx.exit(ngx.HTTP_FORBIDDEN) end end return _M --     



Now you need to build the session.o file from session.lua using the luaJit compiler and build nginx with this file.

Compile the session.o file by executing the lua compiler command
 /path/to/luajit/bin/luajit -bg session.lua session.o 


Add a line to the configuration to build nginx

 --with-ld-opt="/path/to/session.o" 


and collect nginx (how to build nginx described above)

After that, you can connect the file to any lua script and call the handle () function to process the user session

 local session = require "session" session.handle() 


At the end of a small test for comparison.

computer configuration
Processor: Intel Xeon CPU X3440 @ 2.53GHz × 8
Memory: 7.9 GiB

Tests that extract data from redis using php or lua

ab -n 100 -c 100 php
Server Software: nginx / 1.9.4

Concurrency Level: 100
Time taken for tests: 3.869 seconds
Complete requests: 100
Failed requests: 0
Requests per second: 25.85 [# / sec] (mean)
Time per request: 3868.776 [ms] (mean)
Time per request: 38.688 [ms] (mean, across all concurrent requests)
Transfer rate: 6.66 [Kbytes / sec] received

Connection Times (ms)
min mean [± sd] median max
Connect: 1 3 1.1 3 5
Processing: 155 2116 1053.7 2191 3863
Waiting: 155 2116 1053.7 2191 3863
Total: 160 2119 1052.6 2194 3864

Percentage of the requests served within a certain time (ms)
50% 2194
66% 2697
75% 3015
80% 3159
90% 3504
95% 3684
98% 3861
99% 3864
100% 3864 (longest request)

ab -n 100 -c 100 lua
Server Software: nginx / 1.9.4

Concurrency Level: 100
Time taken for tests: 0.022 seconds
Complete requests: 100
Failed requests: 0
Requests per second: 4549.59 [# / sec] (mean)
Time per request: 21.980 [ms] (mean)
Time per request: 0.220 [ms] (mean, across all concurrent requests)
Transfer rate: 688.66 [Kbytes / sec] received

Connection Times (ms)
min mean [± sd] median max
Connect: 2 4 0.9 4 6
Processing: 3 13 1.6 13 14
Waiting: 3 13 1.6 13 14
Total: 9 17 1.3 18 18

Percentage of the requests served within a certain time (ms)
50% 18
66% 18
75% 18
80% 18
90% 18
95% 18
98% 18
99% 18
100% 18 (longest request)

The difference in the "number of requests per second" is 175 times.

The same test with other parameters
ab -n 10000 -c 100 php
Server Software: nginx / 1.9.4

Concurrency Level: 100
Time taken for tests: 343.082 seconds
Complete requests: 10,000
Failed requests: 0
Requests per second: 29.15 [# / sec] (mean)
Time per request: 3430.821 [ms] (mean)
Time per request: 34.308 [ms] (mean, across all concurrent requests)
Transfer rate: 7.51 [Kbytes / sec] received

Connection Times (ms)
min mean [± sd] median max
Connect: 0 0 0.3 0 4
Processing: 167 3414 197.5 3408 4054
Waiting: 167 3413 197.5 3408 4054
Total: 171 3414 197.3 3408 4055

Percentage of the requests served within a certain time (ms)
50% 3408
66% 3438
75% 3458
80% 3474
90% 3533
95% 3633
98% 3714
99% 3866
100% 4055 (longest request)

ab -n 10000 -c 100 lua
Server Software: nginx / 1.9.4

Concurrency Level: 100
Time taken for tests: 0.899 seconds
Complete requests: 10,000
Failed requests: 0
Requests per second: 11118.29 [# / sec] (mean)
Time per request: 8.994 [ms] (mean)
Time per request: 0.090 [ms] (mean, across all concurrent requests)
Transfer rate: 1682.94 [Kbytes / sec] received

Connection Times (ms)
min mean [± sd] median max
Connect: 0 0 0.4 0 5
Processing: 1 9 3.4 7 19
Waiting: 1 9 3.5 7 18
Total: 2 9 3.4 7 21

Percentage of the requests served within a certain time (ms)
50% 7
66% 13
75% 13
80% 13
90% 13
95% 13
98% 13
99% 15
100% 21 (longest request)

The difference in the "number of requests per second" is 381 times.

Hope my article was helpful. If you have any wishes, remarks or you know how to do better - write.

Source: https://habr.com/ru/post/270463/


All Articles