Monolithic Message-Oriented Application (MMOA)

Just finished the project
you have full knowledge
how it should be implemented.
(C) Tom Demarco

I always wanted to try to write an application, the modules of which communicate with each other through messaging. In principle, it is quite even in the spirit of the classical understanding of its founders. However, before Erlang, I was not old enough and I was familiar only with Golang, so it was on him that I tried to create a slightly bizarre, but nonetheless curious, web application architecture.

I conditionally broke the application into parts, which I call services. Services receive and send messages, and this is the only way they interact with each other. Of course, there is a significant overhead in this - it will be faster to call a method with the transfer of a message than to send them through channels and a bus. In addition, in addition to the overhead projector, there is also a complication of architecture, this is also important. If for the reader these are important and critical requirements, then perhaps he should not continue to bother reading this article, because MMOA is just an experiment and a curious piece of work.

However, if you still decided to spend your time reading (for which many thanks), then in addition to the minuses, I will point out one implicit plus, which is in all this strange construction. The fact is that communication through messages implies a little more freedom and independence of the services included in the monolith. Probably not obvious. But whenever possible, I tried to ensure that services and channels of communication with them were formed outside the core of the application. In the example lying in the distribution kit, it is clearly seen that the service initialization code lies in the main package.
')
Well, if everything is exactly as I say, then what is actually next? Monolith - not a monolith, but what is the profit? (the last phrase was in rhyme, an unexpected surprise!). Here I am just like a cap, I will say that the service, which is fairly easy to select from the monolith, if necessary, can be distinguished from the monolith. Dedicated service can be called microservice, which is what it will be by and large. Why allocate service in microservice? This is an important question. Under normal conditions, this is not required, but if, for example, this service suddenly became so heavy that it’s time for him to move to his own server? Then it is possible.

However, the issue of allocating a service to microservice is quite hypothetical, but the desire to play around with messages and touch them is quite real)) Although MMOA can be called a framework (more likely a microform), however, for me it is in many ways more like a library or a toolkit with some implementation concepts or, frankly, hypotheses. If I am to avoid tautology, perhaps I will sometimes call MMOA in different ways, please understand and forgive.

General principles

The concept of an MMAA is as follows: fairly independent services are developed (for this framework I have divided into several libraries), these services are combined into a single application and in the process of work they exchange data with each other by sending messages. The delivery address of the message consists of the name of the service and the subject of the message. (Themes are in essence events of events in services, however in this case for messages I preferred to call them that way).

Initially, after reading smart books, I created a very complex and intricate architecture, but over time, the husks fell off and a small amount of ingredients were left. I had to cut it alive. From the remote I will probably mention balancing channels supporting queues with priorities. Well, let them rest in peace))

Fast start

As part of the distribution, there is a ready-made example that vividly demonstrates the work of MMOA on the example of a simple site, following my old tradition, dedicated to the Latin American dance “rumba”. Go to the examples folder, compile and run the application. The result of the work look in the browser at localhost . As a router, I used my Bxog development, but you can use any router of your choice. Templating is done by the standard html / template library.

Details

The demo application has two services:

• article - article service. In order not to complicate the example of working with a DBMS, articles are stored here in a plain text file, which is parsed when the application is loaded. At the request of the Record service gives the name and text of the article, nothing complicated. He also has a supported List topic, which gives a list of articles in the database.
  
  
• menu - this service should give an array id - the name, i.e. list of articles. But since the articles have their own service, the menu requests the list from the article and, upon receiving it, sends a response to the aggregator on its own behalf. This solution (not the most optimal in terms of performance) is intended to show the interaction of services between them. Initially, I just wanted to hardcorely put an array of key-values ​​into this service and give it on request, but it would not be interesting at all.

MMOA Composition

For convenience and ease of creating services as part of the application, some parts of the MMAA are separated into separate libraries.

tools

This library is required when creating both applications and services.

• config.go - the file contains the types used in the application, timer settings and statuses
• message.go is the main and only unit of information exchange between services, the content of the message is stored in MsgCtx, everything else is an envelope.
• themes.go is a structure listing all the services in the application and the messages they receive. The structure was created specifically for convenience (IDE will not let you write the name of a non-existent service or a message subject that it does not support).
• exchanger.go - all data structures are stored in this file, in the format of which services can exchange data between themselves

service

This library is used to create utility services and is required when writing custom services.

• logger.go - a simple library for logging to the console
• service.go - the basis of the service, listens to the input port, if there is a suitable waiting for the message, determines it there, if not, tries to call the method assigned to the message subject, if not, sends the message to the recycle bin.
• waiting.go - stores the unit waiting for the arrival of messages, if the unit is full, it is sent to the desired method, if the unit is outdated, it is sent to the basket, and waiting is deleted.
• aggregate.go - accumulates messages with a given CID (correlation identifier), after adding a message it gives the number of still expected messages.

support

This library contains the aggregator and basket service services, as well as a bus for sending messages.

• aggregator.go is an analogue of waiting with the difference that here aggregates accumulate messages for sending to the handler, and unlike waiting units come here from handlers.
• trash.go - cart, messages with the wrong address, incorrect, and also with expired units are sent here.
• bus.go — the bus accepts messages and forwards them to destination channels. If the recipient is missing, the message is sent to the basket.

Application root

These files are the core of the MMOA, and are not used outside of it.

• cid.go - correlation identifier, a correlation identifier that helps services identify messages.
• handler.go - the request handler, creates an aggregate for the request and sends messages with requests to the necessary services.
• controller.go - conducts the initial initialization of the application, creates a bus and service services - an aggregator and a basket.
• view.go - responsible for templating. Stores templates for responses of services and for the page.

How to add your service

For a bunch of the above words, it can be very incomprehensible how convenient / inconvenient to use the concept described by MMOA, so with a simple example I will describe the process of adding a new service. For example, we wanted the date to always be displayed on the page.

Update the list of services and topics

In the tools / services_themes.go add structure

// ThemeCalendar structure type ThemeCalendar struct { Date TypeTHEME } 

We add the Calendar ThemeCalendar line to the Themes structure and the Calendar TypeSERVICE line to the ListServices structure.

Create a service file

Create the directory example / calendar and the calendar.go file with the following contents:

 package calendar // Monolithic Message-Oriented Application (MMOA) // Calendar // Copyright 2016 Eduard Sesigin. All rights reserved. Contacts: <claygod@yandex.ru> import ( "time" "github.com/claygod/mmoa/service" "github.com/claygod/mmoa/tools" ) func NewServiceCalendar(chIn chan *tools.Message, chBus chan *tools.Message) *ServiceCalendar { the := tools.NewThemes() s := &ServiceCalendar{ service.NewService(the.Service.Calendar, chIn, chBus), } s.MethodWork = s.Work s.setEvents() s.Start() return s } type ServiceCalendar struct { service.Service } func (s *ServiceCalendar) setEvents() { s.Methods[s.The.Calendar.Date] = s.dateEvent } func (s *ServiceCalendar) dateEvent(msgIn *tools.Message) { t := time.Now() msgOut := tools.NewMessage().Cid(msgIn.MsgCid). From(s.Name).To(msgIn.AddsRe). Theme(msgIn.MsgTheme). Field("Day", t.Day()). Field("Month", int(t.Month())). Field("Year", t.Year()) s.ChBus <- msgOut } 

Templates

In the example / data directory, create a date.html file with the line {{.Day}}. {{. Month}}. {{. Year}} , and change the contents of the general page template to:

 <!DOCTYPE html> <html> <head> <title>{{.Title}}</title> <meta http-equiv="content-type" content="text/html; charset=utf-8"/> <link rel="stylesheet" href="file/twocolumn.css"> </head> <body> <div id="header"><h1>Rumba</h1></div> <div id="sidebar"> {{.Sitemap}} {{.Date}} </div> <div id="content"> {{.Record}} </div> </body> </html> 

Rule the application

Now it remains for us to add a new library to the import files, create a channel for it, create a new structure, and add a line to the handler's initialization:

 package main // Monolithic Message-Oriented Application (MMOA) // Application // Copyright 2016 Eduard Sesigin. All rights reserved. Contacts: <claygod@yandex.ru> import ( "github.com/claygod/Bxog" "github.com/claygod/mmoa" "github.com/claygod/mmoa/example/article" "github.com/claygod/mmoa/example/calendar" "github.com/claygod/mmoa/example/menu" "github.com/claygod/mmoa/tools" ) const chLen int = 1000 func main() { chBus := make(chan *tools.Message, chLen) chMenu := make(chan *tools.Message, chLen) chArticle := make(chan *tools.Message, chLen) chCalendar := make(chan *tools.Message, chLen) the := tools.NewThemes() app := mmoa.NewController(chBus) sm := menu.NewServiceMenu(chMenu, chBus) app.AddService(sm.Name, chMenu) sa := article.NewServiceArticle(chArticle, chBus) app.AddService(sa.Name, chArticle) sc := calendar.NewServiceCalendar(chCalendar, chBus) app.AddService(sc.Name, chCalendar) hPage := app.Handler("./data/template.html"). ContentType("text/html; charset=UTF-8"). Service(tools.NewPart(sm.Name).Theme(the.Menu.Sitemap).Template("./data/sitemap.html")). Service(tools.NewPart(sa.Name).Theme(the.Article.Record).Template("./data/record.html")). Service(tools.NewPart(sc.Name).Theme(the.Calendar.Date).Template("./data/date.html")). StatusCodeOf(the.Article.Record) m := bxog.New() m.Add("/:id", hPage.Do) m.Start(":80") } 

Performance

As we have already learned, despite the fact that the application is compiled, MMOA, this is not the best solution for tasks where speed is the main and decisive factor, since in the process, the application services send messages to each other through the channels, which is natural is a brake. In order to at least understand about how productive MMOA is, purely for reference, I spent a simple ab testing a running example from the example folder. My computer issued the following spherical horse in a vacuum:

• ab -n 10000 -c 1 -> 3127 r / s
• ab -n 30000 -c 100 -> 6373 r / s

Below, the benchmark shows quite well that an application running only with the article service works much faster than with the menu , which sends a request to the article , waits for it, receives an answer and only then sends its answer to the aggregator. (Note: in parallel mode, the difference is somewhat reduced.)

• BenchmarkOnlyArticle-4 50000 24722 ns / op
• BenchmarkArticleAndMenu-4 30000 43404 ns / op
• BenchmarkOnlyArticleParallel-4 100000 13831 ns / op
• BenchmarkArticleAndMenuParallel-4 100000 20752 ns / op

What was / could be

If you wish, you can add priorities to the messages. By the way, initially they were, but I found their functionality premature. In addition to services, the handler can be “out”, having simultaneously got rid of the controller. I also tried this option, but decided that it’s better to the detriment of flexibility that everything lies inside and does not hurt the eyes. Handler is generally worth it to simplify and give him a splitter. There were other ideas, but sometimes you just want to stop.

Conclusion

Most likely in MMOA, many readers have something familiar: patterns, microservices, SOA, MQ, etc. This is good and I dare to assure you, MMOA does not pretend to overthrow or appropriate other people's laurels. This is only a tool whose ideas you might be interested in. From myself I will add only one IMHO - in many respects the MMOA is written under the influence of Golang, which I consider to be quite interesting and very suitable for developing a wide variety of applications, and many thanks to the authors of the language for their work.

Links

Github