The term "module" (module) is taken from the article Modules vs. microservices . Similarly, to describe something in between microservices and monoliths, sometimes the terms microlith (monrolith) or monoservice are used. But, despite the fact that the term "module" is already loaded with well-known meaning, in my opinion, it fits better than other options. Update: In the lega commentary I used the term "embedded microservice" - it describes the essence of the approach better than the "module".

Monolith and microservices are very different approaches, so in any attempt to take the best from both, the balance is crucial - what to take and what not. Otherwise, you get a monster like OSGi .

I’m writing microservices since 2009, but I haven’t tried to use modules instead of microservices in real projects - everything described later is my assumption about what the above balance should be, and it needs both theoretical criticism and testing practice.

What is a module

A module is something like microservice, only implemented within a single application consisting of a group of such modules and a very small part, which is engaged in the initialization and launch of all these modules.

Although formally such an application can be called a monolith, this approach has much more in common with microservices, so it makes sense to compare it with the microservice approach. Like microservice, each module:

• Must have a carefully designed and documented API, and only it should be available to other code running in the same application (controlling this constraint requires support at the level of the language and / or development tools).
• It should be executed independently of the code of other modules and be able to efficiently interact with the API of other modules without using network protocols (this requires support at the language level: threads / gortines / actors, channels / messages, etc.).
• Must use its own data storage (if it needs one).
• May be in a separate repo or mono application.
• It can be used by different applications.
• It can provide access to its API over the network (only it is not necessary for the module to do this, and access may not be complete, as within the application, but partial).

Unlike microservices, the module:

• Normally, you cannot deploy independently (unless the language supports hot-swapping part of the code in a running application).
• You cannot restart regardless of the entire application.
• Cannot be scaled independently of other modules.
• It can not prevent access to "their" files from other modules if they decide to break the isolation between modules. Neither the language nor the tools can prevent such a violation, so here you have to rely solely on the discipline of the developers themselves and the review of the code.
• Perhaps, it can without any restrictions use libraries shared with other modules - but here I am not sure. The negative consequences of this will not be as significant as for ordinary microservices, but they will be. Especially if the module is launched in the future as a full-fledged microservice (and this, by and large, so far nothing interferes - the modules are quite easily turned into separate microservices if necessary).

Unlike conventional libraries, the module:

• Should not share any common data with the API code using it - all data should be transferred via the API either as a copy or if there is a lot of data and only read access is needed, as an immutable data structure (requires support for language level).
• It does not provide functions that other modules can call — i.e. does not execute any of its code in another thread of execution (thread, gorutin, etc.), thus isolating from the calling code even exceptions that may occur in the module code.
  
  • The module has a public function of initializing and launching the module, which is called when the application is started, but it is not intended to (re) call from other modules.
• It has its own, completely isolated from other modules:
  
  • Configuration (transmitted to him when the application starts).
    • It may include common logging settings for all modules.
  • Data storage (if it needs it).
    • Support for versioning the schema of these data and migrations during updates is also the responsibility of the module, although the launch of data migration of each module can be part of the application process common for all modules.

In most cases, modules do not need any service registry — instead of registering themselves and searching for other modules, they get the interfaces they need from other modules when they are started, when the starting application calls the initialization function of each module (in the order determined by the dependencies between the modules). As a side effect, this will allow you to immediately detect cyclic dependencies between modules, if they appear (and, if possible, change the architecture so as to get rid of them).

Where are the modules

There is a constant added complexity ( accidental complexity ), the same in every microservice (registration / discovery of services, connection and reconnection to them, authorization between services, (de) marshaling and traffic encryption, use of loopback requests interrupters, implementation of query tracing, etc.). There is a similar constant added operational complexity (the need to automate testing and rollout, the implementation of detailed monitoring, log aggregation, the use of service services for registering and finding services, for storing service configurations, for auditing, etc.). You can put up with it, because you can do it all once, as the number of microservices grows, this complexity does not grow, and the advantages of microservices more than compensate for these costs.

But there is a complexity that depends on the business logic of a particular application and increases as the application develops, from which I would like to get rid of at least in those applications that do not require scalability and high availability (or at least that part of the code of such applications that does not have explicit need to interact with external services):

• The need to separate idempotent and non idempotent queries.
• For all requests on the client:
  
  • The need to use timeouts (which may be different for different requests).
  • The need to use more asynchrony to speed up the work.
  • More situations of eventual consistency and the resulting difficulties.
    • Including the need to sometimes cache data from other services.
  • Need to repeat some queries:
    • Using a limit on the number or time of repetitions.
    • Using delays between repetitions.
• Additionally for non idempotent client requests:
  
  • The need to implement the ability to identify duplicates when retrying requests.
• Additionally for non idempotent queries on the server:
  
  • The need to define and process duplicate requests (differently, depending on the business logic of each request).
• In some cases, the need to perform queries in the same manner as in
  where they were shipped.

The correct modular approach allows you to retain many of the benefits of microservices (with the necessary support at the level of language and / or development tools), but besides the loss of scaling capabilities and high availability that are not necessary in this application, there are also other ones:

• A module crash causes the entire application to crash.
• The speed of assembly and testing decreases.
  
  • I'm not sure that the assembly speed is a real problem - there are many ways to speed it up.
  • In principle, when testing a branch / PR, it is possible to run only tests of a changed module, and tests of the whole project should be performed only before deployment - this should sufficiently level this problem.
• Starting speed decreases (warming up caches, etc.).
  
  • Here, the problem is not so much in real slowdown, but in the fact that when there is a lot of start-up services "smeared" across several services that (re) start up independently of each other. However, the apparent effect of a slower launch does not negate this fact.

Also, the modular approach has new advantages:

• The speed of interaction between service modules increases, which significantly reduces the need for asynchrony, eventual consistency and caching.
• As long as the module does not have an external (network) API, its API is much easier to modify and deploy these changes atomically with clients using this module (other modules of the same application).
• Module collaboration is easier to test than a group of microservices.
• It is easier to use monorepo (if you want), although this is not necessary.
  
  • Monorepo simplifies refactoring and changing the API, at least for now there is a guarantee that the module does not have a network interface and external clients.

Summary

In general, the approach looks quite tempting - we get the opportunity to write a monolithic application in the form of a handful of really well isolated parts (and the language and / or tools, rather than internal discipline), developed in microservice style (in including different teams in different repos), which can be turned into real microservices with a “flick of the wrist” if there is a real need for it ... But for now there is none - we can use the exchange of messages between fashion s within the application as a simple and very quick replacement of the RPC, eliminating the complexity of asynchrony, eventual consistency and handling of network errors.

The necessary support for this approach is currently far from being found in all languages, but in some there is: the author of the article "Modules vs. microservices" wrote about support for modularity in Java 9, Go has been supporting internal packages for a couple of years, in Erlang judging by article on the same topic Dawn of the Microlith - Monoservices with Elixir everything is fine, .... I'm not sure how much it is possible in scripting languages ​​to provide real isolation of modules, but there are attempts: micromono on NodeJS, in the lega comment link to the approach for Python, ...

If you have ideas on the topic (or even better - the experience of a real project on similar principles) or additional links to articles / projects on the topic - write in the comments, I will try to complement the article.