Comet server on Erlang
An article for people just starting out to get acquainted with an erlang: how to write a simple comet server.
The ready code is here: github.com/maxlapshin/comet
')
The comet server will be written using cowboy, tinymq, and a piece of toad script.
The code is laid out on the githaba with tags. The main stages are tagged, you can roll back to see what exactly was done at one time or another.
The article itself is written in parallel with the code, which is noticeable from the history in the gita.
The logic is as follows: by http we post a message to the server, it enters the message queue, from where the client raises it through a long-poll request.
Erlang here is good in two things:
1) the ability to keep messages in memory
2) the ability to cheaply keep in memory millions (!) Of unanswered requests, i.e. the client is waiting for a response, the server will respond to it a little later.
At first, we need every kind of routine: rebar, dependencies.
Next we create rebar.config to install the dependencies.
During the writing of the article, changes were made to the tinymq API. It is recommended to lock a particular commit!
And pull out the dependencies:
Immediately create and Emakefile. Many refuse to use emake, and I don’t, and will later show why I do it.
Now it remains to create the backbone for our application and everything will be ok.
Add deps and ebin to .gitignore and compile what happened:
The moment is marked with a v1 tag.
In terms of how to run the application on the Erlang there is some confusion and vacillation. In the sense of infrastructure launch.
As a rule for development, I make a Makefile with a run rule in which I specify the following:
It is important to understand the following: Erlang does not have a development regime and production. All this biliberda, with which many frameworks fool their heads, is missing.
Now more about options.
Now, in order to start everything up, you need to add the function start / 0 to comet_app.erl:
After that, we compile everything:
Those. our empty application has started. Exit Ctrl + c or halt (). in the console.
V2 tag
With the infrastructure so far finished, let's move on to launching the web server.
From the start of our application, we will run a cowboy with several handlers, but for this you must first add another application mimetypes (https://github.com/spawngrid/mimetypes) in order to correctly render static files.
Add the line
We write the necessary code in comet_app.erl , add the minimum code to www / index.html and run.
make run and go to localhost : 8080 /
Tag v3
Add jquery, form, and separate sendmessage_handler handler for url / sendmessage
We write the module itself clearly according to the cowboy’s instructions.
With cowboy, it is important to remember one very, very important thing: all functions from the cowboy API return a response and a new state. This is done to parse all the data on request.
It is important that you always use the new state, otherwise there is a chance to continue to cool.
Now it is important to tell why Emakefile. There are different approaches to how to reload code in development. I like only regular
Other options are reloader or sync. They are all bad because they reload the devil code when, and what the hell. sync so simply knocks into the crust due to some problems with ssl. It is for working make: all ([load]). (this command should be run in the Erlang console) and did Emakefile
Next you need to enable the message queue. We will add it to the supervisor of our application comet_sup.
Now you can add directly the logic of the comet:
V4 tag
In a comet, the following situations are possible:
1) came, and there already for us there are messages in the queue
2) came, no messages, waiting for when they appear.
In order to understand that we have already seen the messages, you need to use sortable identifiers. The most reliable in our case is real time. This is exactly how tinymq works.
Add comet_handler, which will use special cowboy chips to implement a comet.
Let's take a closer look at what is happening.
1) the client comes to / comet, having or not having the last timestamp on hand
2) the handler checks with tinymq: poll if there are newer messages than this timestamp.
3) if there is, it responds with a simple mechanism through handle / 2
4) if there are no such messages, it subscribes to the delivery of messages through tinymq: pull and hangs in the loop / 3 loop
5) when the right message arrives, it packs it in JSON and sends it along with the new timestamp to the client
Mochijson2 module added for json packaging
V5 tag
It remains only to attach HTML for this.
There is a bunch of nuances that all go around the same thing: how not to dump the server with too frequent requests. Those. After an emergency disconnection, you need to wait a little. But we will leave it all for later, at first just subscribe to comets.
V6 tag
So, in rough, the minimal example of a working comet is ready. In this form, you can not run in production, you need to finish:
1) authorization
2) channel indication
After that, the code described here can be rolled out to production. Yes, erlang is good because as a rule a prototype can be thrown into battle.
Also in the next article we will talk about web sockets, launch into production, etc.
The ready code is here: github.com/maxlapshin/comet
Description
')
The comet server will be written using cowboy, tinymq, and a piece of toad script.
The code is laid out on the githaba with tags. The main stages are tagged, you can roll back to see what exactly was done at one time or another.
The article itself is written in parallel with the code, which is noticeable from the history in the gita.
The logic is as follows: by http we post a message to the server, it enters the message queue, from where the client raises it through a long-poll request.
Erlang here is good in two things:
1) the ability to keep messages in memory
2) the ability to cheaply keep in memory millions (!) Of unanswered requests, i.e. the client is waiting for a response, the server will respond to it a little later.
Sundry stuff to get started
At first, we need every kind of routine: rebar, dependencies.
mkdir comet cd comet git init wget https://github.com/basho/rebar/wiki/rebar chmod +x rebar Next we create rebar.config to install the dependencies.
{lib_dirs, ["apps", "deps"]}. {sub_dirs, [ "apps/comet" ]}. {erl_opts, [ debug_info, warn_format, warn_export_vars, warn_obsolete_guard, warn_bif_clash ]}. {deps, [ {cowboy, "0.6.*", {git, "git://github.com/extend/cowboy.git", {tag, "0.6.1"}}}, {tinymq, ".*", {git, "git://github.com/evanmiller/tinymq.git", "386813add4"}} ]}.
During the writing of the article, changes were made to the tinymq API. It is recommended to lock a particular commit!
And pull out the dependencies:
./rebar get-deps
Immediately create and Emakefile. Many refuse to use emake, and I don’t, and will later show why I do it.
{"apps/comet/src/*", [debug_info, {outdir, "apps/comet/ebin"}, {i, "apps/comet/src"}]}. {"deps/cowboy/src/*", [debug_info, {outdir, "deps/cowboy/ebin"}, {i, "deps/cowboy/src"}]}. {"deps/tinymq/src/*", [debug_info, {outdir, "deps/tinymq/ebin"}, {i, "deps/tinymq/src"}]}.
Now it remains to create the backbone for our application and everything will be ok.
mkdir -p apps/comet cd apps/comet ../../rebar create-app appid=comet cd -
Add deps and ebin to .gitignore and compile what happened:
./rebar compile
The moment is marked with a v1 tag.
Launch
In terms of how to run the application on the Erlang there is some confusion and vacillation. In the sense of infrastructure launch.
As a rule for development, I make a Makefile with a run rule in which I specify the following:
ERL_LIBS=apps:deps erl +K true -name comet@127.0.0.1 -boot start_sasl -s comet_app -sasl errlog_type error
It is important to understand the following: Erlang does not have a development regime and production. All this biliberda, with which many frameworks fool their heads, is missing.
Now more about options.
ERL_LIBS=apps:deps specifies erlanga directories for searching libraries. A library is a directory in which there is ebin.
+K true indicates that epoll / kqueue can be used.
-name comet@127.0.0.1 - long name with explicit indication of host
-boot start_sasl - remember and repeat
-s comet_app - run the start / 0 function from the comet_app module (it is not currently available)
-sasl errlog_type error allows you to remove a squall from unnecessary messages that everything is fine.
Now, in order to start everything up, you need to add the function start / 0 to comet_app.erl:
-export([start/0]). start() -> application:start(comet).
After that, we compile everything:
> ./rebar compile > make run ERL_LIBS=apps:deps erl +K true -name comet@127.0.0.1 -boot start_sasl -s comet_app -sasl errlog_type error Erlang R15B (erts-5.9) [source] [64-bit] [smp:4:4] [async-threads:0] [hipe] [kernel-poll:true] Eshell V5.9 (abort with ^G) (comet@127.0.0.1)1>
Those. our empty application has started. Exit Ctrl + c or halt (). in the console.
V2 tag
Web server launch
With the infrastructure so far finished, let's move on to launching the web server.
From the start of our application, we will run a cowboy with several handlers, but for this you must first add another application mimetypes (https://github.com/spawngrid/mimetypes) in order to correctly render static files.
Add the line
{mimetypes, ".*", {git, "git://github.com/spawngrid/mimetypes.git"}} to rebar.config and pull out: ./rebar get-deps
We write the necessary code in comet_app.erl , add the minimum code to www / index.html and run.
make run and go to localhost : 8080 /
Tag v3
We write messages to queue
Add jquery, form, and separate sendmessage_handler handler for url / sendmessage
We write the module itself clearly according to the cowboy’s instructions.
With cowboy, it is important to remember one very, very important thing: all functions from the cowboy API return a response and a new state. This is done to parse all the data on request.
It is important that you always use the new state, otherwise there is a chance to continue to cool.
Now it is important to tell why Emakefile. There are different approaches to how to reload code in development. I like only regular
make:all([load]).
Other options are reloader or sync. They are all bad because they reload the devil code when, and what the hell. sync so simply knocks into the crust due to some problems with ssl. It is for working make: all ([load]). (this command should be run in the Erlang console) and did Emakefile
Next you need to enable the message queue. We will add it to the supervisor of our application comet_sup.
Now you can add directly the logic of the comet:
{Post, Req2} = cowboy_http_req:body_qs(Req), {<<"body">>, Body} = lists:keyfind(<<"body">>, 1, Post), tinymq:push(<<"default_channel">>, Body),
V4 tag
We read messages from the queue
In a comet, the following situations are possible:
1) came, and there already for us there are messages in the queue
2) came, no messages, waiting for when they appear.
In order to understand that we have already seen the messages, you need to use sortable identifiers. The most reliable in our case is real time. This is exactly how tinymq works.
Add comet_handler, which will use special cowboy chips to implement a comet.
Let's take a closer look at what is happening.
1) the client comes to / comet, having or not having the last timestamp on hand
2) the handler checks with tinymq: poll if there are newer messages than this timestamp.
3) if there is, it responds with a simple mechanism through handle / 2
4) if there are no such messages, it subscribes to the delivery of messages through tinymq: pull and hangs in the loop / 3 loop
5) when the right message arrives, it packs it in JSON and sends it along with the new timestamp to the client
Mochijson2 module added for json packaging
V5 tag
It remains only to attach HTML for this.
HTML for comet
There is a bunch of nuances that all go around the same thing: how not to dump the server with too frequent requests. Those. After an emergency disconnection, you need to wait a little. But we will leave it all for later, at first just subscribe to comets.
V6 tag
What's next?
So, in rough, the minimal example of a working comet is ready. In this form, you can not run in production, you need to finish:
1) authorization
2) channel indication
After that, the code described here can be rolled out to production. Yes, erlang is good because as a rule a prototype can be thrown into battle.
Also in the next article we will talk about web sockets, launch into production, etc.
Source: https://habr.com/ru/post/150706/
All Articles