Erlang for web development (2) -> DB and deploy;

In the first article, we met Erlang and the n2o framework. In this part we will continue to do our blog:

we will add authorization through facebook, for this we will call functions on the server from the client;
We will save comments and posts in the NoSQL database;
Let's deploy our blog on DigitalOcean and measure performance ( spoiler - 1300 requests per second ).

The code from the articles https://github.com/denys-potapov/n2o-blog-example , the finished project can be viewed at http://46.101.118.21:8001/ .

')

Configuration files

For authorization, we need to store facebook_app_id somewhere, in Erlang applications, the configuration is stored in sys.config, we will add our facebook_app_id there

[{n2o, [ {port,8001}, {route,routes}, {log_modules,sample} ]}, {sample, [ {facebook_app_id, "631083680327759"} ]} ].

Now we can get with the value in application: get_env (sample, facebook_app_id, "")

Calling the server code

For authorization via social networks in n2o projects there is an avz library that supports Twitter, Google, Facebook, Github and Microsoft authorization. But, avz requires a certain scheme in the database, which we do not yet have, and therefore we release the authorization ourselves.

The wf: wire function (#api {name = login}) allows you to bind a call to the login function on the client to an event
api_event (login, Response, Term) on the server.

Add a login.erl file:

 -module(login). -compile(export_all). -include_lib("n2o/include/wf.hrl"). -include_lib("nitro/include/nitro.hrl"). -include_lib("records.hrl"). main() -> wf:wire(#api{name=login}), #dtl{file="login", bindings=[{app_id, application:get_env(sample, facebook_app_id, "")}]}. api_event(login, Response, Term) -> {Props} = jsone:decode(list_to_binary(Response)), User = binary_to_list(proplists:get_value(<<"name">>, Props)), wf:user(User), wf:redirect("/").

In the main / 0 function, we declare a login event, which we then handle in api_event. We decode the json string, authorize the user and direct it to the main page. In priv / templates / login.html code that is copied from the sample to facebook, in which the main magic in the login call (response) .

priv / templates / login.html

 {% extends "base.html" %} {% block title %}Login{% endblock %} {% block content %} <h1>Login</h1> <p id="status"></p> <button id="login" class="btn btn-primary" onclick="onLoginClick();"> Login with facebook </button> <script> // This is called with the results from from FB.getLoginStatus(). function statusChangeCallback(response) { console.log('statusChangeCallback'); if (response.status === 'connected') { // Logged into your app and Facebook. FB.api('/me', function(response) { login(response); }); } else if (response.status === 'not_authorized') { document.getElementById('status').innerHTML = 'Please log ' + 'into this app.'; } else { document.getElementById('status').innerHTML = 'Please log ' + 'into Facebook.'; } } window.fbAsyncInit = function() { FB.init({ appId : '{{ app_id }}', cookie : true, version : 'v2.2' // use version 2.2 }); FB.getLoginStatus(function(response) { statusChangeCallback(response); }); }; // Load the SDK asynchronously (function(d, s, id) { var js, fjs = d.getElementsByTagName(s)[0]; if (d.getElementById(id)) return; js = d.createElement(s); js.id = id; js.src = "//connect.facebook.net/en_US/sdk.js"; fjs.parentNode.insertBefore(js, fjs); }(document, 'script', 'facebook-jssdk')); function onLoginClick() { FB.login(function(response) { statusChangeCallback(response); }, {scope: 'public_profile,email'});<source lang="html"> {% extends "base.html" %} {% block title %}Login{% endblock %} {% block content %} <h1>Login</h1> <p id="status"></p> <button id="login" class="btn btn-primary" onclick="onLoginClick();"> Login with facebook </button> <script> // This is called with the results from from FB.getLoginStatus(). function statusChangeCallback(response) { console.log('statusChangeCallback'); if (response.status === 'connected') { // Logged into your app and Facebook. FB.api('/me', function(response) { login(response); }); } else if (response.status === 'not_authorized') { document.getElementById('status').innerHTML = 'Please log ' + 'into this app.'; } else { document.getElementById('status').innerHTML = 'Please log ' + 'into Facebook.'; } } window.fbAsyncInit = function() { FB.init({ appId : '{{ app_id }}', cookie : true, version : 'v2.2' // use version 2.2 }); FB.getLoginStatus(function(response) { statusChangeCallback(response); }); }; // Load the SDK asynchronously (function(d, s, id) { var js, fjs = d.getElementsByTagName(s)[0]; if (d.getElementById(id)) return; js = d.createElement(s); js.id = id; js.src = "//connect.facebook.net/en_US/sdk.js"; fjs.parentNode.insertBefore(js, fjs); }(document, 'script', 'facebook-jssdk')); function onLoginClick() { FB.login(function(response) { statusChangeCallback(response); }, {scope: 'public_profile,email'}); }; </script> {% endblock %}

Component Update on Client

Now we will try to update the component from the server on the client. To do this, we will make a header on the main (index.erl), on which there will be an exit button. The header will be updated after the session data has been cleared:

 buttons() -> case wf:user() of undefined -> #li{body=#link{body = "Login", url="/login"}}; _ -> [ #p{class=["navbar-text"], body="Hello, " ++ wf:user()}, #li{body=#link{body = "New post", url="/new"}}, #li{body=#link{body = "Logout", postback=logout}} ] end. header() -> #ul{id=header, class=["nav", "navbar-nav", "navbar-right"], body = buttons()}. main() -> #dtl{file="index", bindings=[{posts, posts()}, {header, header()}]}. event(logout) -> wf:user(undefined), wf:update(header, header()).

In the event (logout) event, we clear the session data and update the component.

Database and dependencies

To access the database, we will use kvs . kvs allows you to store linked lists and supports different backends (Mnesia, Riak, KAI, Redis, MongoDB). Further in the example I will use mnesia , because it comes in the package and does not need to be customized.

The dependencies in the Erlang projects are in the rebar.config file, we add kvs there:

 {kvs, ".*", {git, "git://github.com/synrc/kvs", {tag, "2.9"} }}

In sys.config, we indicate which backend and which scheme we use. The scheme is needed only for mnesia, for other backends it is not needed.

 {kvs, {dba,store_mnesia}, {schema,[sample]} ]}

The schema is described by the metainfo / 0 function in sample.erl:

 metainfo() -> #schema{name=sample,tables=[ #table{name=id_seq,fields=record_info(fields,id_seq),keys=[thing]}, #table{name=post,fields=record_info(fields,post)} ]}.

We indicate that we have two tables: post, which contains entries of type post, and id_seq, in which kvs stores autoincrement values.

Right there in sample.erl in the init / 1 function we add a connection to kvs.

 init([]) -> case cowboy:start_http(http,3,port(),env()) of {ok, _} -> ok; {error,_} -> halt(abort,[]) end, sup(), kvs:join().

Now if we restart the applications, we should see our tables.

 2> kvs:dir(). [{table,post},{table,id_seq},{table,schema}]

Read and write

In the /src/new.erl module, we will have one event (post) event, which writes the post to the database with the kvs: put / 1 function:

 -module(new). -compile(export_all). -include_lib("n2o/include/wf.hrl"). -include_lib("nitro/include/nitro.hrl"). -include_lib("records.hrl"). main() -> case wf:user() of undefined -> wf:header(<<"Location">>, wf:to_binary("/login")), wf:state(status,302), []; _ -> #dtl{file="new", bindings=[{button, #button{id=send, class=["btn", "btn-primary"], body="Add post",postback=post,source=[title,text]} }]} end. event(post) -> Id = kvs:next_id("post",1), Post = #post{id=Id,author=wf:user(),title=wf:q(title),text=wf:q(text)}, kvs:put(Post), wf:redirect("/post?id=" ++ wf:to_list(Id)).

/priv/templates/new.html

 {% extends "base.html" %} {% block title %}New Post{% endblock %} {% block content %} <h1>Add new post</h1> <h3>Title</h3> <input id="title" class="form-control"> <h3>Body</h3> <textarea id="text" maxlength="1000" class="form-control" rows=10> </textarea> {{ button }} {% endblock %}

Now in the post.erl file we replace the function of getting the post, if the post is not found we give out a 404 error

 main() -> case kvs:get(post, post_id()) of {ok, Post} -> #dtl{file="post", bindings=[ {title, wf:html_encode(Post#post.title)}, {text, wf:html_encode(Post#post.text)}, {author, wf:html_encode(Post#post.author)}, {comments, comments()}]}; _ -> wf:state(status,404), "Post not found" end.

In the index.erl homepage module we get all posts by calling kvs: all (post):

 posts() -> [ #panel{body=[ #h2{body = #link{body = wf:html_encode(P#post.title), url = "/post?id=" ++ wf:to_list(P#post.id)}}, #p{body = wf:html_encode(P#post.text)} ]} || P <- kvs:all(post)].

Containers and iterators

The concept of containers and iterators is used to store linked lists in kvs. The iterator stores pointers to doubly linked lists, and the container stores pointers to the head and tail of the list.

Update our records to records.hrl and add an iterator comment and container post:

 -record(post, {?CONTAINER, title, text, author}). -record(comment, {?ITERATOR(post), text, author}).

We update the scheme:

 metainfo() -> #schema{name=sample,tables=[ #table{name=id_seq,fields=record_info(fields,id_seq),keys=[thing]}, #table{name=post,container=true,fields=record_info(fields,post)}, #table{name=comment,container=post,fields=record_info(fields,comment)} ]}.

Re-create the database schema:

 2> kvs:destroy(). ok 3> kvs:join(). ok

In the post.erl module, we update the comment logic:

 comments() -> case wf:user() of undefined -> #link{body = "Login to add comment", url="/login"}; _ -> [ #textarea{id=comment, class=["form-control"], rows=3}, #button{id=send, class=["btn", "btn-default"], body="Post comment",postback=comment,source=[comment]} ] end. event(init) -> [event({client,Comment}) || Comment <- kvs:entries(kvs:get(post, post_id()),comment,undefined) ], wf:reg({post, post_id()}); event(comment) -> Comment = #comment{id=kvs:next_id("comment",1),author=wf:user(),feed_id=post_id(),text=wf:q(comment)}, kvs:add(Comment), wf:send({post, post_id()}, {client, Comment}); event({client, Comment}) -> wf:insert_bottom(comments, #blockquote{body = [ #p{body = wf:html_encode(Comment#comment.text)}, #footer{body = wf:html_encode(Comment#comment.author)} ]}).

In the comments () function, we check whether the user has autologized. In event (init) we select all comments that relate to this post and pass them on to the event ({client, Comment}) event, that is, comments are loaded after the page loads.

In the event event (comment), we not only output a comment, but also save it to the database.

Creating your own items

For paginated navigation, we add our pagination element to DSL. In the file /apps/sample/include/elements.hrl we add an entry in which we indicate which module is responsible for displaying this element:

 -include_lib("nitro/include/nitro.hrl"). -record(pagination, {?ELEMENT_BASE(element_pagination), active, count, url}).

The output module element_pagination.erl itself is quite simple:

 -module(element_pagination). -compile(export_all). -include_lib("nitro/include/nitro.hrl"). -include_lib("elements.hrl"). link(Class, Body, Url) -> #li{class=[Class], body=#link{body=Body, url=Url}}. disabled(Body) -> link("disabled", Body, "#"). left_arrow(#pagination{active = 1}) -> disabled("«"); left_arrow(#pagination{active = Active, url = Url}) -> link("", "«", Url ++ wf:to_list(Active - 1)). right_arrow(#pagination{active = Count, count = Count}) -> disabled("»"); right_arrow(#pagination{active = Active, url = Url}) -> link("", "»", Url ++ wf:to_list(Active + 1)). left(0, P) -> [left_arrow(P)]; left(I, P) -> S = wf:to_list(I), left(I - 1, P) ++ [link("", S, P#pagination.url ++ S)]. right(I, P = #pagination{count = Count}) when I > Count -> [right_arrow(P)]; right(I, P) -> S = wf:to_list(I), [link("", S, P#pagination.url ++ S) | right(I + 1, P)]. render_element(P = #pagination{}) -> wf:render(#nav{body=#ul{class=["pagination"], body=[ left(P#pagination.active - 1, P), link("active", wf:to_list(P#pagination.active), "#"), right(P#pagination.active + 1, P) ]}}).

How to do it

Kvs is designed to store linked lists, and therefore not well suited for page navigation.

A sharp comment by the author kvs about page navigation in the modern web

But, for the purity of the experiment, we will add page navigation. Add a container feed, which will store posts

 -record(feed, {?CONTAINER}). -record(post, {?ITERATOR(feed), title, text, author}). -record(comment, {?ITERATOR(feed), text, author}).

And update the scheme:

 metainfo() -> #schema{name=sample,tables=[ #table{name=id_seq,fields=record_info(fields,id_seq),keys=[thing]}, #table{name=feed,container=true,fields=record_info(fields,feed)}, #table{name=post,container=feed,fields=record_info(fields,post)}, #table{name=comment,container=feed,fields=record_info(fields,comment)} ]}.

Comments we will store in the feed container of the form {post, post_id ()}:

 Comment = #comment{id=kvs:next_id("comment",1),author=wf:user(),feed_id={post, post_id()},text=wf:q(comment)},

And we will receive comments from this container:

 [event({client,Comment}) || Comment <- kvs:entries(kvs:get(feed, {post, post_id()}),comment,undefined) ];

Limit the pagination on the main page. Once again I note that kvs is poorly suited for page-by-page navigation, and this code is just a demonstration of how using inappropriate tools leads to code obfuscation:

 -define(POST_PER_PAGE, 3). page() -> case wf:q(<<"page">>) of undefined -> 1; Page -> wf:to_integer(Page) end. pages() -> Pages = kvs:count(post) div ?POST_PER_PAGE, case kvs:count(post) rem ?POST_PER_PAGE of 0 -> Pages; _ -> Pages + 1 end. posts() -> [ #panel{body=[ #h2{body = #link{body = wf:html_encode(P#post.title), url = "/post?id=" ++ wf:to_list(P#post.id)}}, #p{body = wf:html_encode(P#post.author)} ]} || P <- lists:reverse(kvs:traversal(post, kvs:count(post) - (page() - 1) * ?POST_PER_PAGE, ?POST_PER_PAGE, #iterator.prev))].

Deploy and performance

Mad allows you to create a bundle - a single file in which the code and all the necessary files for the application (templates, statics) are stored. Create and upload to a remote server:

 mad deps compile plan bundle sample scp sample root@46.101.117.36:/var/www/sample/

Install Erlang on a remote server and run our application:

 wget https://packages.erlang-solutions.com/erlang/esl-erlang/FLAVOUR_1_general/esl-erlang_18.0-1~ubuntu~trusty_amd64.deb dpkg -i esl-erlang_18.0-1~ubuntu~trusty_amd64.deb escript sample

To test performance, I created the smallest droplet on DigitalOcean (512 MB Memory / 20 GB Disk). For the test, we will make 20 thousand requests, 50 in parallel:

 root @ ubuntu-1gb-fra1-01: ~ # ab -l -n 20000 -c 50 -g gnuplot.dat http://46.101.118.21:8001/
 ...
 Concurrency Level: 50
 Time taken for tests: 15.131 seconds
 Complete requests: 20000
 Failed requests: 0
 Total transferred: 78279988 bytes
 HTML transferred: 76399988 bytes
 Requests per second: 1321.80 [# / sec] (mean)
 Time per request: 37.827 [ms] (mean)
 Time per request: 0.757 [ms] (mean, across all concurrent requests)
 Transfer rate: 5052.26 [Kbytes / sec] received

 Connection Times (ms)
               min mean [+/- sd] median max
 Connect: 0 0 0.3 0 9
 Processing: 9 37 4.9 37 65
 Waiting: 9 37 4.9 37 65
 Total: 11 38 4.9 37 65

 Percentage of the requests served within a certain time (ms)
   50% 37
   66% 38
   75% 39
   80% 40
   90% 44
   95% 47
   98% 53
   99% 56
  100% 65 (longest request)

The server processed about 1300 requests per second, 95% of requests were completed in less than 50 ms, which is very good for hosting for $ 5 per month. Same as graphics:

Source: https://habr.com/ru/post/274107/

All Articles