Applications for Tarantool. Part 1. Stored procedures

Hi, Habr! Today I want to share with you the experience of writing applications for Tarantool 1.7. This series of articles will be useful to those who are already going to use Tarantool in their projects, or to those who are looking for a new solution to optimize projects.

The whole cycle is devoted to the consideration of an existing application on Tarantool. This part will describe the installation of Tarantool, data storage and access to them, as well as some tricks of writing stored procedures.

Tarantool is a NoSQL database that stores data in memory or on disk (depending on the storage subsystem). The storage is persistent due to the well thought-out write ahead log mechanism. LuaJIT (Just-In-Time Compiler) is built into Tarantool, allowing you to execute Lua code. You can also write stored procedures in C.

Content of the cycle “Applications for Tarantool”

- Part 1. Stored procedures
- Part 2. OAuth2 authentication
- Part 3. Testing and running.

Why build your apps for Tarantool

There are two reasons:

1. This will speed up the service. Data processing on the storage side reduces the amount of data transferred, and combining multiple requests into one stored procedure will save on network delays.
2. Ready applications can be reused . Now the Tarantool ecosystem is actively developing, there are new opensource applications on Tarantool, some of which are eventually transferred to Tarantool itself. Such modules allow you to create new services faster.

Of course, this approach has its drawbacks. Tarantool cannot utilize all the multi-core processor resources, so to scale up the service, you will have to take care of sharding the storage, as well as the appropriate project architecture. However, with an increase in the number of requests, this approach will allow for easy scaling of the load.

Consider how one of the Tarantool apps was created. It implements an API for registering and authenticating users. Application functionality:

• registration and authentication by email in two stages: creating an account, confirming an account with a password;
• registration through social networks (FB, VK, Google+);
• the ability to recover a password.

As an example of writing a stored procedure Tarantool, we analyze the first stage of registration by email - obtaining a confirmation code. To revive the examples, you can use the source code, which is available on github .

Go!

Tarantool installation

Read how to install Tarantool in the documentation . For example, for Ubuntu you need to run in a terminal:

curl http://download.tarantool.org/tarantool/1.7/gpgkey | sudo apt-key add - release=`lsb_release -c -s` sudo apt-get -y install apt-transport-https sudo rm -f /etc/apt/sources.list.d/*tarantool*.list sudo tee /etc/apt/sources.list.d/tarantool_1_7.list <<- EOF deb http://download.tarantool.org/tarantool/1.7/ubuntu/ $release main deb-src http://download.tarantool.org/tarantool/1.7/ubuntu/ $release main EOF sudo apt-get update sudo apt-get -y install tarantool 

We will verify that the installation was successful by calling tarantool in the console and starting the interactive mode.

 $ tarantool version 1.7.3-202-gfe0a67c type 'help' for interactive help tarantool> 

Here you can try your hand at programming on Lua.
If there is no strength, then gather them in this small tutorial .

Register by email

Go ahead. Let's write the first script, allowing to create space (space) with users. Space is an analogue of tables for data storage. The data itself is stored as a tuple. Space should contain one primary (primary) index, it can also have several secondary (secondary) indices. An index can be one key or several at once. Tuple is an array in which records are stored. Consider the scheme of space authentication service:

As can be seen from the diagram, we use two types of indices: hash and tree. Hash-index allows you to find tuples by the complete coincidence of the primary key and must be unique. The tree-index supports non-unique keys, search by the first part of the composite index and allows you to optimize the sorting operations by key, since the values ​​in the index are stored in an orderly manner.

The space session holds the key (session_secret) that the session cookie is signed with. Keeping session keys allows users to log on to the service side, if needed. Session has an optional link to space social. This is necessary to validate the sessions of users entering through social networks (checking the validity of the stored OAuth2-token).

Let's move on to writing an application. To begin, consider the structure of the future project:

 tarantool-authman ├── authman │ ├── model │ │ ├── password.lua │ │ ├── password_token.lua │ │ ├── session.lua │ │ ├── social.lua │ │ └── user.lua │ ├── utils │ │ ├── http.lua │ │ └── utils.lua │ ├── db.lua │ ├── error.lua │ ├── init.lua │ ├── response.lua │ └── validator.lua └── test ├── case │ ├── auth.lua │ └── registration.lua ├── authman.test.lua └── config.lua 

Modules in Lua are imported from the paths specified in the package.path variable.
In our case, the modules are imported relative to the current directory, i.e. tarantool-authman. However, if necessary, the import path can be supplemented:

 lua --        (  ) package.path = "/some/other/path/?.lua;" .. package.path 

Before we create the first space, we will render the necessary constants in the model. Each space and each index must define its name. It is also necessary to determine the storage order of the fields in the tuple. This is the model of user authman / model / user.lua:

 --   —  Lua- local user = {} --     — model,         --          lua- function user.model(config) local model = {} --     model.SPACE_NAME = 'auth_user' model.PRIMARY_INDEX = 'primary' model.EMAIL_INDEX = 'email_index' --      (tuple) --    Lua   1 (!) model.ID = 1 model.EMAIL = 2 model.TYPE = 3 model.IS_ACTIVE = 4 --  : email-    model.COMMON_TYPE = 1 model.SOCIAL_TYPE = 2 return model end --   return user 

In the case of users, we need two indexes. Unique by id and non-unique by email, since, by registering via social networks, two different users can receive the same email or not receive email at all. The uniqueness of email for users who have not registered through social networks, will provide the application logic.

The authman / db.lua module contains a method for creating spaces:

 local db = {} --      model --     config  nil —   local user = require('authman.model.user').model() --   db,   (space)   function db.create_database() local user_space = box.schema.space.create(user.SPACE_NAME, { if_not_exists = true }) user_space:create_index(user.PRIMARY_INDEX, { type = 'hash', parts = {user.ID, 'string'}, if_not_exists = true }) user_space:create_index(user.EMAIL_INDEX, { type = 'tree', unique = false, parts = {user.EMAIL, 'string', user.TYPE, 'unsigned'}, if_not_exists = true }) end return db 

We take uuid as the user id, the hash index type, we look for a complete match. The index for email search consists of two parts: (user.EMAIL, 'string') - email, (user.TYPE, 'unsigned') - user type. Types were previously defined in the model. The composite index allows you to search not only for all fields, but also for the first part of the index, so you can search only by email (without the user type).

Now we will launch the interactive console Tarantool in the directory with the project and try using the authman / db.lua module.

 $ tarantool version 1.7.3-202-gfe0a67c type 'help' for interactive help tarantool> db = require('authman.db') tarantool> box.cfg({listen=3331}) tarantool> db.create_database() 

Great, the first space is created! Warning: before accessing box.schema.space.create, you must configure and start the server using the box.cfg method. Now consider a few simple actions inside the created space:

 --   tarantool> box.space.auth_user:insert({'user_id_1', 'exaple_1@mail.ru', 1}) --- - ['user_id_1', 'exaple_1@mail.ru', 1] ... tarantool> box.space.auth_user:insert({'user_id_2', 'exaple_2@mail.ru', 1}) --- - ['user_id_2', 'exaple_2@mail.ru', 1] ... --  Lua- ()   tarantool> box.space.auth_user:select() --- - - ['user_id_2', 'exaple_2@mail.ru', 1] - ['user_id_1', 'exaple_1@mail.ru', 1] ... --      tarantool> box.space.auth_user:get({'user_id_1'}) --- - ['user_id_1', 'exaple_1@mail.ru', 1] ... --      tarantool> box.space.auth_user.index.email_index:select({'exaple_2@mail.ru', 1}) --- - - ['user_id_2', 'exaple_2@mail.ru', 1] ... --       tarantool> box.space.auth_user:update('user_id_1', {{'=', 2, 'new_email@mail.ru'}, }) --- - ['user_id_1', 'new_email@mail.ru', 1] ... 

Unique indexes limit the insertion of non-unique values. If you need to create records that may already be in space, use the operation upsert (update / insert). A complete list of available methods can be found in the documentation .

Update the user model by adding functionality that allows us to register it:

  function model.get_space() return box.space[model.SPACE_NAME] end function model.get_by_email(email, type) if validator.not_empty_string(email) then return model.get_space().index[model.EMAIL_INDEX]:select({email, type})[1] end end --   -- ,     ,  function model.create(user_tuple) local user_id = uuid.str() local email = validator.string(user_tuple[model.EMAIL]) and user_tuple[model.EMAIL] or '' return model.get_space():insert{ user_id, email, user_tuple[model.TYPE], user_tuple[model.IS_ACTIVE], user_tuple[model.PROFILE] } end --  ,    ,     --  ,    GET-   -- activation_secret —        function model.generate_activation_code(user_id) return digest.md5_hex(string.format('%s.%s', config.activation_secret, user_id)) end 

In the above code snippet, two standard Tarantool modules, uuid and digest, are used, as well as one custom module, the validator. Before use, they must be imported:

 --   Tarantool local digest = require('digest') local uuid = require('uuid') --    (   ) local validator = require('authman.validator') 

Variables are declared with the local operator, which limits the scope of the variable to the current block. Otherwise, the variable will be global, which should be avoided due to a possible name conflict.

And now let's create the main authman / init.lua module. In this module all api application methods will be collected.

 local auth = {} local response = require('authman.response') local error = require('authman.error') local validator = require('authman.validator') local db = require('authman.db') local utils = require('authman.utils.utils') --     — api,       function auth.api(config) local api = {} --  validator      --       config = validator.config(config) --       local user = require('authman.model.user').model(config) --  space db.create_database() --  api         function api.registration(email) --    email —      email = utils.lower(email) if not validator.email(email) then return response.error(error.INVALID_PARAMS) end -- ,       email local user_tuple = user.get_by_email(email, user.COMMON_TYPE) if user_tuple ~= nil then if user_tuple[user.IS_ACTIVE] then return response.error(error.USER_ALREADY_EXISTS) else local code = user.generate_activation_code(user_tuple[user.ID]) return response.ok(code) end end --    space user_tuple = user.create({ [user.EMAIL] = email, [user.TYPE] = user.COMMON_TYPE, [user.IS_ACTIVE] = false, }) local code = user.generate_activation_code(user_tuple[user.ID]) return response.ok(code) end return api end return auth 

Fine! Now users can create accounts.

 tarantool> auth = require('authman').api(config) --  api     tarantool> ok, code = auth.registration('example@mail.ru') --       email    tarantool> code 022c1ff1f0b171e51cb6c6e32aefd6ab 

That's all. In the next part, we will look at using ready-made modules, networking and the implementation of OAuth2 in tarantool-authman.