Erlang-like microservices in Clojure application: it’s just

As it is known in the circle of Erlang developers: only Erlang developers know how to "live" correctly and everyone else "live" is wrong . Without attempting to dispute this fact, let us give an example of a Clojure application in the Erlang style using the Otplike library.

For understanding the article, the reader may need to know the basics of Clojure (Are there still people who do not know Clojure? ...) and the basic principles of Erlang / OTP (processes, sending messages, gen_server and supervisor behaviors). In order to deal with everything else, the average Clojure developer has everything you need: code with examples , REPL and a "tambourine".

Why clojure?

In fact, there are many answers to the question "Why Clojure". We give our favorite:

№1. Clojure is a very effective language for prototyping. Compared to Java, writing a Clojure application layout is very simple: it is very easy to develop data models and assemble them together.

№2. Clojure is very easy to test the application: REPL + ease of layout is decided here. Whatever the test case in the application, it is sufficient to simply construct the context in which to test the desired case.

The first two points accelerate the development and support of the application (which falls under the specified conditions) by a factor of 2. But we just started listing ...

Number 3. Clojure is fully interoperable with Java / JVM. This means, in particular, that you can use classes in a Clojure application and export a Clojure application as classes (for example, we integrate a clojure library into a java application ). It also means that all the accumulated code of humanity for the JVM is available for the Clojure application. So, the Clojure language is not alternative and does not contradict the development of the JVM, but as an addition to the JVM (a very important addition, I would like to say).

So, we mentioned that through Clojure it is convenient to get to any part of the "heritage of mankind" in the JVM and it is convenient to test. And now, why all the same Clojure ...

№4. Clojure is a language designed to make complex things simple and, in our opinion, they succeeded thanks to the experience and genius of Rich Hickey, who formulated the main ideas of the language (which, in turn, can be read, for example, here: Why study Clojure? )

Well, personally, my favorite reason ....

№5. Clojure programming is fun , i.e. "lively", interesting and stress free. Just download the project, just read the code, just think, just write and just ship unicorns You give the result.

Why Erlang / OTP for Clojure?

In the last section, we found out that Clojure is a silver bullet.

However, for industrial programming in the era of multi-threaded applications, in addition to a convenient and cool language, a practical methodology is needed for developing these multi-threaded applications themselves.

The standard solution for multithreaded applications on Clojure at the moment is the core.async library (for example, Prepare multithreading with core.async ). Practical experience however shows that although the library itself is good, for practice, higher-level “building blocks” are needed.

And here we return to the happy Erlang developers and to their "charms". Erlang / OTP has absorbed considerable experience in developing multi-threaded applications, perhaps, like no other language. Having the implementation of the basic ideas of Erlang / OTP using the Otplike library, we get:

• Erlang-like processes, messaging between processes and links between processes;
• implementation of an OTP process like supervisor ( supervisor behaviour ): a process that monitors and overloads other processes, depending on the settings;
• OTP implementation of a process of the gen_server type ( gen_server behaviour ): a process implementing microservice;
• start processes on a timer ( scheduling ).

And not to be unfounded, we give ...

Minimalistic sample application with microservice

TODO task list service

We want to create a microservice for the TODO task list, which accepts the following commands:

1. create entity todo: create-todo [params] -> [:ok todo] | [:error reason] create-todo [params] -> [:ok todo] | [:error reason]
2. return todo by id: find-todo-by-id [id] -> [:ok todo] | [:error reason] find-todo-by-id [id] -> [:ok todo] | [:error reason]
3. terminate todo: terminate-todo [id] -> [:ok updated_todo] | [:error reason] terminate-todo [id] -> [:ok updated_todo] | [:error reason]
4. delete todo: delete-todo [id] -> [:ok nil] | [:error reason] delete-todo [id] -> [:ok nil] | [:error reason]
5. return the list of active todo: enumerate-active-todos [] -> [:ok todos] | [:error reason] enumerate-active-todos [] -> [:ok todos] | [:error reason]

Strangely enough, but from this description, the public API for the TODO service follows directly (you can see the complete code sample on GitHub here ):

 (defn create-todo [params] (call* [:create-todo params])) (defn find-todo-by-id [id] (call* [:find-todo-by-id id])) (defn terminate-todo [id] (call* [:terminate-todo id])) (defn delete-todo [id] (call* [:delete-todo id])) (defn enumerate-active-todos [] (call* :enumerate-active-todos)) 

Calling a call* means simply sending a message to the service.

The essence of the processing of messages for the gen_server service is that all these messages are processed sequentially, passing the value of the state service from one message processing to another. Even if many processes in parallel send their requests to our service, it will not break the consistency of our state , since all these requests will be executed sequentially. In practice, this simplifies the development of the service life cycle.

For the development of the Otplike service gen_server usual OTP callbacks are available for implementation:

1. initialization handler of the state service: init [args] -> [:ok initial_state]
2. handler for messages that must return the result to the client process (synchronous messages): handle-call [message from state] -> [:reply reply updated_state] | [:noreply updated_state] handle-call [message from state] -> [:reply reply updated_state] | [:noreply updated_state]
3. message handler without a result to the client process (asynchronous messages): handle-cast [message state] -> [:noreply updated_state]
4. system message handler: handle-info [message state] -> [:noreply updated_state]
5. service terminate [reason state] -> nil handler: terminate [reason state] -> nil

Advanced example supervisor for reloading services into REPL

Suppose we implemented a TODO service, and from the REPL we launched it in a new process. After that we made changes to the code of this service and now we want to run the modified code in order to test it. How do we do this?

One solution is to kill the process with the old code and run the new code in the new process.
However, these services we can have a lot and they can depend on one another. In addition, the order of launching services can also be important.

Therefore, there is another radical solution that comes from OTP: you need to have a supervisor process that will monitor our services and who will be able to overload them.
In addition, we want to be able to overload the supervisor himself so that we can bring our application from the REPL to a known initial state.

For these requirements, we get the following code for Otplike:

 ;;;;;;;;;;;;;;;;;;;;;;;;; supervision-tree (defn- app-sup [_config] [:ok [{:strategy :one-for-one} [{:id :todo-server :start [todo-server/start-link [{}]]}]]]) ;;;;;;;;;;;;;;;;;;;;;;;;; boot-proc (defn- start-app-sup-link [config] (supervisor/start-link :app-sup app-sup [config])) (defn- start-boot-sup-link [config] (supervisor/start-link :boot-sup (fn [cfg] [:ok [{:strategy :one-for-all} [{:id :app-sup :start [start-app-sup-link [cfg]]}]]]) [config])) (defn start [] (if-let [pid (process/whereis :boot-proc)] (log/info "already started" pid) (let [config (config/get-config)] (process/spawn-opt (process/proc-fn [] (match (start-boot-sup-link config) [:ok pid] (loop [] (process/receive! :restart (do (log/info "------------------- RESTARTING -------------------") (supervisor/terminate-child pid :app-sup) (log/info "--------------------------------------------------") (supervisor/restart-child pid :app-sup) (recur)) :stop (process/exit :normal))) [:error reason] (log/error "cannot start root supervisor: " {:reason reason}))) {:register :boot-proc})))) (defn stop [] (if-let [pid (process/whereis :boot-proc)] (process/! pid :stop) (log/info "already stopped"))) (defn restart [] (if-let [pid (process/whereis :boot-proc)] (process/! pid :restart) (start))) 

The full code can be viewed on GitHub here.

In the app-sup function, we list the child processes for our main supervisor.
And the rest of the code is the workaround for restarting the supervisor.

Well, finally ...

Testing

Head over to the REPL and see how our TODO service and application restart work.

We start the REPL from the console from the project root:

 lein repl 

We start the application:

 erl-like-app.server=> (erl-like-app.server/start) <proc1@1> 18-05-11 14:29:24 INFO - todo server initialized 

Create a TODO pair and mark the first TODO as done:

 erl-like-app.server=> (erl-like-app.todo.todo-server/create-todo {:title "task #1", :description "create task #2"}) [:ok {:title "task #1", :description "create task #2", :id "1", :created 1526049427586, :updated 1526049427586, :status :active}] erl-like-app.server=> (erl-like-app.todo.todo-server/create-todo {:title "task #2"}) [:ok {:title "task #2", :id "2", :created 1526049434985, :updated 1526049434985, :status :active}] erl-like-app.server=> (erl-like-app.todo.todo-server/terminate-todo "1") [:ok {:title "task #1", :description "create task #2", :id "1", :created 1526049427586, :updated 1526049443912, :status :terminated}] 

What remained active TODO:

 erl-like-app.server=> (erl-like-app.todo.todo-server/enumerate-active-todos) [:ok ({:title "task #2", :id "2", :created 1526049434985, :updated 1526049434985, :status :active})] 

What is the value of the state service:

 erl-like-app.server=> (erl-like-app.todo.todo-server/get-state) {:counter 2, :db {"1" {:title "task #1", :description "create task #2", :id "1", :created 1526049427586, :updated 1526049443912, :status :terminated}, "2" {:title "task #2", :id "2", :created 1526049434985, :updated 1526049434985, :status :active}}} 

Reload the application:

 erl-like-app.server=> (erl-like-app.server/restart) true 18-05-11 14:30:28 INFO - ------------------- RESTARTING ------------------- 18-05-11 14:30:28 INFO - todo server stopped 18-05-11 14:30:28 INFO - -------------------------------------------------- 18-05-11 14:30:28 INFO - todo server initialized 

What is the state value of the service now?

 erl-like-app.server=> (erl-like-app.todo.todo-server/get-state) {:counter 0, :db {}} 

Total

And what is the result:

• there is an Otplike library that opens a “door” in Erlang in Clojure, and
• There is an example of an application that opens the "door" in Otplike.

I can't know if you need this door, but from our experience I can say that behind this door you get a cool code (which means effective and easy).

Good coding!