Small, and not so, reflections on SOA, or life in the MVC-beyond world (part 0)
Hello everyone, my name is Yura.
Today I would like to share with you my experience and ideas in the field of software design and development, dispel some prejudices and shed light on the current state of SOA solutions in our, not quite " reactive ", world. How to write less code, get less unhappy customers, gray hair, and more profit. Implement fully reusable solutions without side effects, and why so far nobody has done this â.â
(ภ︥ '-' ď¸ ) ภAnd so, Greetings. Thank you for deciding to give me a little free time, and not much .
I will try to present everything briefly and clearly, although I usually get the last sucks.
')
A couple of years ago, I ran into the problem of lack of tools for implementing backend and front-end solutions (web, desktop, mobile). Since that time a lot of liquid has flowed under the bridge, I got acquainted with a large number of existing approaches and I had ideas how to do everything a little better than it is now. After examining the current market position, I realized that existing solutions do not specifically give full control over applications and are either empty-cases, which in themselves are only a small part of the entire project architecture; their goal is to maintain customer interest and expert audience, which has a positive effect on the cost of the office, or a support drug - multilayered horror with various mutants of OASIS 'a specifications, where support and training of personnel should be robust. The most prominent example of the latter is the WSO2 stack. I consider it the most complete SOA solution on the market, albeit monstrous, curved and very redundant. Such things are difficult to compare Agile methodologies, since they all revolve around papers and business processes, and not around the needs of living people (the first paragraph of the Agile manifesto ). In these articles, I want to share my vision of designing SOA, how to adapt it to the requirements of modern markets and RAD, to describe the main advantages of such approaches.
It is now very fashionable to discuss various reactive trends - that the systems should be:
The picture isstolen from the reactive manifest . There, Elastic and Resilient, probably because Scalable and Fault tolerant are concepts that are too popular and do not already represent such marketing value as in the days of MongoDB. Especially brightly it all appears in the context of the so-called "functional reactive programming" (FRP).
In practice, it turns out that all of this is a solid marketing murmur to attract the attention of investors, and, in general, most of the solutions do not represent any practical value, since developers must implement all this âreactivityâ on their own, without any ready-made solutions or developments according to existing requirements. Even a full AAA is nowhere to be found.
Another thing is SOA - there it should all work right from the box, but no matter how.
It turns out that each vendor has its own SOA with chess and poetess, and they are not friends with each other, although everything is fine with OpenSource solutions, more on this later. All this quickly becomes obsolete, somehow new communities are not formed at all, and projects are developed only by the efforts of their owners. If this is SOA, and any service can be used at least 100,500 times in a whole bundle of projects, then why do we still not have a standardized repository of reusable services and domains ? The owners of existing solutions are simply not interested in this - after all, they need to suck out the money from support.
Especially fun discussing the last year microservices. I think this is a pretty stupid name, and they are not at all âmicroâ, as they imply deep encapsulation of several protocols. For this, run âmicroservicesâ within a single server, or virtual machines, only increase delays and scale vacuum. In general, in classic SOA, it makes no difference whether the service is part of a single system process, or it should be launched as a separate set of processes with its queues and protocols on different machines - all this is just a matter of registering the service and its dependencies. So I consider âmicroservicesâ, too, to be nothing more than a marketing ploy - an attempt to ârethinkâ obvious old ideas, in order to extract more money.
âOk, what's wrong with MVC?â - you will probably think ... but nothing, it exists, it is for simple graphical interfaces, without frills, and in inept hands even the banal CRUD is a ton of save-paste. MVC poorly controls the subject area, encourages state storage and code duplication, it badly separates the responsibilities of logic (segregation of commands and requests), denormalizes the model. Bicycles are another matter: ADR , any ECB , EBI with Clean Architecture and other variations on Hexagonal Architecture - they are an attempt to rework MVC, of ââcourse, not without their individual flaws and, of course, not everyone has enough experience and skills to implement such things, but they solved quite a lot of problems inherent in MVC. All this has been created, for the most part, over the past 3 years by PHP adherents and, respectively, one way or another, the features of this platform are reflected. Not that I really disliked PHP, but modern variations, combined with Python and Ruby, do not fully meet the demands of modern markets.
I understand that I can hurt the feelings of adherents of the cult of the â Golden Hammer, â but then it will be clear why everything is not so optimistic at all, and I will show what I have invented on this subject.
And then there is Flux , but this is also another marketing trick.
Flux in the common people is nothing more than the more complex abbreviation CQRS-ES , because Facebook wanted to simplify it so that it would be possible to attract attention and producehamsters of fans, to develop rather mediocre communities, and this is nothing more than PR. The goal of Facebook is not to provide a quality tool for the implementation of your projects, but to convey the idea to the user community so that they can make as many crutches as possible on GitHub.
It is the CQRS - ES patterns that is the basis of all âreactivitiesâ and is best suited for the implementation of âreactiveâ applications, but in its pure form it is useless. I believe that the problems of use and the benefits of CQRS-ES are most well revealed in the context of SOA design.
The main problem of modern SOA, and indeed of all existing architectures, is a rather poor abstraction of the domain and the definition of a domain model. I will consider this question from the point of view of synchronized discrete FSM finite automata and show you to define domains as accurately as possible (without redundancy), taking into account modern requirements for applications.
As Evans said:
At the end, the life cycle of developing the backend and frontend should look like this:
It remains only to design (optional) and the implementation of the missing functionality, by adding their services and domains to the system.
Itâs as if you would have your own Pocket Firebase , which besides automatically inserting all edits and migrations into the database, would also do Scaffolding for your mobile, desktop and web applications: automatically generated all views, templates, validation and API clients with push notifications, and asynchronous UI.
Due to the fact that most of any modern functionality is very stereotypical, there is no sense to rewrite everything from scratch from project to project. We need a platform so that people can share their work and standardize quality control and interaction between them. In this way, you can get a community of users who are implicitly motivated to improve the quality of products and the qualifications of its members, add gami cation with higher levels, achivka and comic stats ... well, you understand - there are some fantasies where to roam.
When developing such a solution, it is necessary to prevent the occurrence of two problem situations - this is âdevelopment by the committeeâ and âdevelopment by specifications,â which actually led to a terrifying inflation of the J2EE stack.
Now, almost no one is interested in automating the work of developers, although most of the code of existing applications can be generated automatically or abstracted by means of dynamic programming.
Yes ... speaking very roughly and simplistic: according to the database scheme, you can immediately generate REST endpoints, an API client and templates for your front on your favorite Angular / React / Ember, etc. Although I do not really digest the approaches of modern browser frameworks, more on that later. It remains to register user groups and their rights, to stretch the existing design. If you think this is a mystery - well, Tastypie already knows how to generate REST endpoints from the model, although there are not enough features and the performance is rather mediocre, even for Python, because of a bad design, but you can use it as an example.
This is an idea . It consists in the fact that applications should consist of reusable interchangeable "black boxes" - services that communicate over standardized protocols and intfeses among themselves.
There are several important conditions, services:
In principle, the first three conditions should be more or less clear. Services do not depend on each other, do not share common resources, and can migrate between processes of both local and remote machines. Can be reused and simultaneously used in several projects.
As for the states, everything is simple: the server does not have to store data about the user session, this is done so that the request can be processed by any instance of the running service, which is usually the least loaded at any given time. This is done by incorporating encrypted, salted session information into a user request. In order not to transfer this payload with each request - they usually create a separate short-term registration service for user sessions that temporarily caches them. Creating a global service for storing user sessions will lead to the appearance of a âbottleneckâ in which it will still be necessary to solve the problem of consensus, which will have avery negative impact on the speed of processing requests.
With abstractions, everything is rather tight - some domains or other (base scheme and logic) should be dynamically expandable depending on the specific needs of the dependent services and domains. Suppose there is an Accounting service (Accounting) and various services are registered in it â for some, only postpay is provided for billing, while others require prepayment. Add here more different discount and savings systems and get a quality headache for many months, because the whole logic of the work of these components is constantly changing. The normalization of the relational model plays an important role here - please note that there are six normal forms for a long time. Good abstraction and detailed elaboration of the subject area makes it possible to reduce the complexity of introducing a new, or changing the existing logic, but this will be discussed in the next part.
The possibility of separating access to services between several âsubordinateâ and âmanagersâ follows from the requirement for weak connectivity and the absence of shared states. Everything is straightforward as with multi-threaded programming and methods of preventing the states of "race". We should also mention the case of domain expansion: there is a compatibility problem when sharing an extended domain with several services. In general, roughly speaking, when one service needs a label with one number of fields and states (well, there is an enumeration value), and another with a completely different one, and how to âmake friendsâ of them so that there would be no problems when using this label together, or some other resource.
There are many system ârootâ services that are responsible for the work of all the others, and for their behavior in case of errors. One of them is the Registrar which allows you to register acap and launch new instances of services, run implementation and correctness tests, roll back to previous versions in case of errors. Just like in Erlang 'e: 3 In general, SOA is in many ways very similar to the OTP model, although there is also a lot missing.
I think on this note you can complete a review of the objectives and requirements of SOA.
For the needs of something else to add.
Here is a diagram of how I generally represent the structure of a service-oriented architecture.
Let's look at the purpose of root services.
Each of these services has its own design features (software) and I will try to describe all the details in the following articles, I hope I will not miss anything.
Any project should contain, at a minimum, the Registrar , the Archivist , the Resource Manager and the AAA , although you can certainly come up with better names. The latter two are missing for a number of reasons related to the complexity of the implementation of expandable domains (hi cocaine ).
Now there is no full-fledged SOA - everywhere somewhere something is missing.
On the code generation of hypermedia types, history is generally silent.
I worked with
Of course, in the case of the J2EE stack, there is terrifying redundancy and neither WSO nor JBoss are suitable for high loads. And in general, âweak bindingâ does not fit very well with modern j2ee EJB puffs.
Cocaine and Bluemix I, personally, attribute to SOA solutions only partially - there is no ready-made system for the implementation and standardization of the interaction of existing servers, complete DIY. In the case of cocaine, there is a locator , albeit rather rudimentary, but itâs good that at least something. In bluemix, you can implement a good SOI, but you also need to play a bit. From a SOI point of view, Amazonâs services are the most flexible solution.
All vendors offer to deploy infrastructure only on their PaaS'ah, this is especially noticeable in the case of Oracle and its Integration Cloud Service. It is necessary to understand that most users of SOA solutions need to have all the servers within the same data center, and communication delays are minimal - it is often necessary to organize CDNs and other distributed junk on my own, and an extra vendor lockin only links hands when choosing a data center. Both Amazon and Azure have already fallen off - it is better for serious and responsible customers to do risk management themselves, and for ordinary people itâs enough to âstart up, work - godrink some rest and wait until it falls off, calls start at 4 am ...â, usually at 95% cases so it happens. Yes, and cheaper to shoot a dozen bald dedikov with homemade SOI on the room digital ocean, linode, online.net or ovh and juggle them as needed.
BPEL / BPMN is a separate topic: they tried to write a language for describing business processes for secretaries, but they got a many-sided, incomprehensible something that is not envisaged by the standard to implement without editing the language specification itself is almost impossible. To do this, you need to contact OASIS, join the consortium, issue RFCs ... well, you understand that there is nothing in common with solving the tasks set, but only payment for the existence of many-sided monolithic standards that cannot be adapted to modern market requirements within an acceptable time frame. Easier, much easier, to take and write everything with your hands ... in any favorite language. The description of business processes can be very simplified if we consider it from the point of view of finite automata (FSM).
The main advantage of the existing solutions is the guarantee from the vendors, which are sufficient for use in honey. sector and partly in the foreign state. and the automotive sector - there are no such stringent market requirements and there is no need to follow the âlatest fashion trendsâ. Consequently, for the implementation of new projects, or even, I will not be afraid of this word, startups - these solutions and platforms are not suitable.
Also, old-kind SAP can be considered as a full-fledged SOA.
But, given the support policy, cost, continuous obsolescence, poor performance, and a rather mediocre attitude to security â I would not consider this solution at all.
Basically, they want to:
In the case of SOA in this case is almost the ideal solution, and sooner or later, starting development with some popular MVC framework, adding various message queues and tasks, implementing interprocess communication, developers from their bikes roll into SOA with CQRS-ES ' , .
In practice, it turns out that there is no quality control at all, there is no evolutionary refactoring and the risk assessment is rather mediocre. They do "just to work" and especially no one has any understanding in six months - problems are solved as they are received and, of course, this negatively affects the service itself and the developers themselves. In this case, SOA is a dandy because delicious reusable âblack cubesâ allow you to forget about many sample tasks and focus on the business logic of the application, and most of the risk management goes directly to the developer community.
Because it is difficult.
To drive into this all is not enough to just read a couple of books about DDD and Corporate Application Templates, you need to put it all into practice and fill a bunch of cones ... understand that not everything is âtrueâ that they write in books, and even that âtruthâ "In most existing OpenSource projects is not implemented and therefore can not be taken as the basis of their own.
The key feature of any SOA is the presence of an IPC bus, a modularity system, and a discovery service (registrar). Symfony2 and Meteor.js are a good example of how things might look like - their modularity allows you to directly introduce new views and the domain model, plug-ins of other frameworks most often simply extend the functionality of the framework itself, add some scaffolding, but directly to the implementation and logic Your applications have no direct relationship. In principle, if you add a bus here on some thread Gearmand / Beanstalk / RabbitMQ etc you can even get some more or less SOA. Although, IMHO, AMQP is a wild overkill for the legs grow with OASIS, and redundancy is the worst. I really like protobuf's approaches, although its performance could be better.
App Data over Avian Carriers with Quality of Service
I often saw people trying to implement XMPP as a transport protocol for their mobile applications, sometimes even BOSH was used for websites with an exclamation âthis is ejabberd! Well this is erlang! Well this highold! It is not common for humans to measure the overhead of communication and serialization-deserialization of objects, although it so happens that this is about 20% of the entire time the application runs under load. Even a banal change of the network IP socket to a file UNIX socket can save quite a lot of processor time due to working off the IP stack, and since all the application traffic still passes through the OS firewall and other obscenities.
Scaling / dev / null
Often noticed enthusiastic posts like " here, we switched from the rails to Go and greatly reduced the server park. âI liked the phraseâ And we've never had another colossal clusterf ** k since. âYes, it is true: if you cannot load your equipment by 100% when processing requests - you scale the vacuum. The situation with C # / Java is actually not very different. The banal datemappers are nothing more than bytecode enhancing which additionally hides a few holes in abstractions in the form of multi-level caches and entity managers. If you rewrite everything with your hands many times faster, compare at least DataNucleus and old-ext first Hibernate. Why this is happening - the topic of a separate article, but if there were good source code preprocessors, such as sishnyh ones, such crutches would not be needed, and you wouldnât have to dig into byte code, bothering with compatibility. I personally have so far focused on Rust'e - a good balance of security and performance, and there are not so many implicit obscurities in Go (I'm talking about interfaces and âduck typingâ).
There are quite a few examples of indecency and unsuccessful attempts, but they crawled in the right direction, although they usually did not crawl.
80% of the projects I had to deal with do not have a normal normalized relational model. The first step is to normalize your modelku right up to the 6th form , and adopt readable rules for naming tablets and columns.
Because it
For complex and large queries, it is worth using materialized views - this will save a little time for the profiler. Before you think about denormalizing a model, think about storing intermediate results in temporal representations.
With a normalized database, you can issue such a router for your CRUDs:
In the case of folded keys, you can come up with something like $ id1_id2_id3 (at your discretion), and, for example, take the key columns in alphabetical order. Of course, you will first need to read the database schema, and use it to validate the router, otherwise you will get a bunch of blind SQL injections.
Well, test, test, profile, reduce test execution time, refactor and test again.
By the same principle, the aforementioned TastyPie works.
With JS frontend, I personally, for the time being, also do not understand what and where.
On the one hand, we have quite a lot of all sorts of standards, preprocessors, postprocessors, on the other hand, they do not simplify development at all. Most of the existing solutions require duplication and transfer of your templates to the browser, after which you still have to arrange EventSourcing with your hands, partially duplicate your model in JS and get rid of validation ... and nothing works âfrom the boxâ.
Often, watching the development of existing tools from the outside, I got the impression that the farther we go - the more difficult they become, and less and less they are aimed at solving our practical tasks. At the same time, all existing frontend projects collect quite large investments, we are talking about tens of millions of U.Dmurdsk e., And try to attract and sharpen attention.
I stayed on React.js with âisomorphicâ renderers in the node, even though I am already pretty nauseated by this. The problem is that you need to render simultaneously on the server and on the client for a normal UX, in any case, and various webkit surrogates like prerender.io do not have any reasonable performance.
Iâm one of those who donât really understand why I need to sculpt node.js where Iâm not hitting, simply because modern JS frontends arenât particularly rendered in other environments. In principle, nothing prevents you from overtaking html in json - transfer templates and changes on pages on web sockets and longpolling and when clicking on links. After that, apply a âpatchâ to the current DOM tree - then you will not have to bother with routing and models on the pages, and the rules for form validation can be described in terms of data- * attributes and regular expressions. Immediately, there is no need for any VirtualDOMs, since all changes in collections and templates can be traced already on the server side and send rendered patches.
Well, in general, you wrote your templates on the usual jinja / twig'ah and other ctpp, added one JS to the page and a specialized controller for collection management, that's all. True, such a thing will work fully (with push notifications) only if there is an EventSourcing. Thus, you donât need to duplicate the model and view from the browser side, and this is a rather large piece of work. I hope the idea is clear.
You can transfer HTML pages from the template engine in the form of a similar JSON
I do not know of any existing tool for the JS-frontend, as part of modern MVC frameworks, that would perform browser templating and collection management in a similar way. Although the management of collections and ES already exists in Meteor.js, but this is rather an exception, and Yandex had something similar somewhere ...
I hope it was not too boring, I was able to tell something new, and push you to think.
I wish you a pleasant time of day.
If necessary, I will add something here, correct errors and clarify.
Send all comments to the PM, do not litter the comments.
Today I would like to share with you my experience and ideas in the field of software design and development, dispel some prejudices and shed light on the current state of SOA solutions in our, not quite " reactive ", world. How to write less code, get less unhappy customers, gray hair, and more profit. Implement fully reusable solutions without side effects, and why so far nobody has done this â.â
(ภ︥ '-' ď¸ ) ภAnd so, Greetings. Thank you for deciding to give me a little free time, and not much .
I will try to present everything briefly and clearly, although I usually get the last sucks.
')
Estimated content
0 Introduction 0.1 What is a Service Oriented Architecture? 0.2 What are the advantages and disadvantages of existing solutions? 0.3 What are the requirements of the current market? 0.4 Why has nobody done this yet? 1 Service 1.1 Domains and relational model 1.2 Solving the problem of consensus 1.3 Scaling and sharding 1.4 Fault tolerance 1.5 Logging and monitoring 1.6 Search and registration 1.7 AAA 2 Submission 2.1 Hypermedia types and scaffolding 2.2 CQRS-ES all the things 2.3 Templates 2.4 Protocols and gateways 2.5 Counters, reports and statistics 3. Support 3.1 Documentation 3.2 Backward Compatibility 3.3 Migrations 3.4 Correctness 3.5 Solution Life Cycle
Introduction
A couple of years ago, I ran into the problem of lack of tools for implementing backend and front-end solutions (web, desktop, mobile). Since that time a lot of liquid has flowed under the bridge, I got acquainted with a large number of existing approaches and I had ideas how to do everything a little better than it is now. After examining the current market position, I realized that existing solutions do not specifically give full control over applications and are either empty-cases, which in themselves are only a small part of the entire project architecture; their goal is to maintain customer interest and expert audience, which has a positive effect on the cost of the office, or a support drug - multilayered horror with various mutants of OASIS 'a specifications, where support and training of personnel should be robust. The most prominent example of the latter is the WSO2 stack. I consider it the most complete SOA solution on the market, albeit monstrous, curved and very redundant. Such things are difficult to compare Agile methodologies, since they all revolve around papers and business processes, and not around the needs of living people (the first paragraph of the Agile manifesto ). In these articles, I want to share my vision of designing SOA, how to adapt it to the requirements of modern markets and RAD, to describe the main advantages of such approaches.
It is now very fashionable to discuss various reactive trends - that the systems should be:
- Responsive
- Fault tolerant
- Scalable
- Managed streams of events (messages)
The picture is
In practice, it turns out that all of this is a solid marketing murmur to attract the attention of investors, and, in general, most of the solutions do not represent any practical value, since developers must implement all this âreactivityâ on their own, without any ready-made solutions or developments according to existing requirements. Even a full AAA is nowhere to be found.
Another thing is SOA - there it should all work right from the box, but no matter how.
It turns out that each vendor has its own SOA with chess and poetess, and they are not friends with each other, although everything is fine with OpenSource solutions, more on this later. All this quickly becomes obsolete, somehow new communities are not formed at all, and projects are developed only by the efforts of their owners. If this is SOA, and any service can be used at least 100,500 times in a whole bundle of projects, then why do we still not have a standardized repository of reusable services and domains ? The owners of existing solutions are simply not interested in this - after all, they need to suck out the money from support.
Especially fun discussing the last year microservices. I think this is a pretty stupid name, and they are not at all âmicroâ, as they imply deep encapsulation of several protocols. For this, run âmicroservicesâ within a single server, or virtual machines, only increase delays and scale vacuum. In general, in classic SOA, it makes no difference whether the service is part of a single system process, or it should be launched as a separate set of processes with its queues and protocols on different machines - all this is just a matter of registering the service and its dependencies. So I consider âmicroservicesâ, too, to be nothing more than a marketing ploy - an attempt to ârethinkâ obvious old ideas, in order to extract more money.
âOk, what's wrong with MVC?â - you will probably think ... but nothing, it exists, it is for simple graphical interfaces, without frills, and in inept hands even the banal CRUD is a ton of save-paste. MVC poorly controls the subject area, encourages state storage and code duplication, it badly separates the responsibilities of logic (segregation of commands and requests), denormalizes the model. Bicycles are another matter: ADR , any ECB , EBI with Clean Architecture and other variations on Hexagonal Architecture - they are an attempt to rework MVC, of ââcourse, not without their individual flaws and, of course, not everyone has enough experience and skills to implement such things, but they solved quite a lot of problems inherent in MVC. All this has been created, for the most part, over the past 3 years by PHP adherents and, respectively, one way or another, the features of this platform are reflected. Not that I really disliked PHP, but modern variations, combined with Python and Ruby, do not fully meet the demands of modern markets.
I understand that I can hurt the feelings of adherents of the cult of the â Golden Hammer, â but then it will be clear why everything is not so optimistic at all, and I will show what I have invented on this subject.
And then there is Flux , but this is also another marketing trick.
Flux in the common people is nothing more than the more complex abbreviation CQRS-ES , because Facebook wanted to simplify it so that it would be possible to attract attention and produce
It is the CQRS - ES patterns that is the basis of all âreactivitiesâ and is best suited for the implementation of âreactiveâ applications, but in its pure form it is useless. I believe that the problems of use and the benefits of CQRS-ES are most well revealed in the context of SOA design.
The main problem of modern SOA, and indeed of all existing architectures, is a rather poor abstraction of the domain and the definition of a domain model. I will consider this question from the point of view of synchronized discrete FSM finite automata and show you to define domains as accurately as possible (without redundancy), taking into account modern requirements for applications.
As Evans said:
Difficulties are not there where all these frameworks, databases, queues, and other things from which engineers are so dragged, difficulties are always hidden in the description and modeling of the subject area - the creation of domains.Not that I would like to turn your ideas about the architecture of applications so dramatically, but this is necessary in view of the fact that ~ 80% consist, a little more than completely, with crutches, with polished ivory, encrusted with diamonds, because the budget allows and âwhy and no: we are paid money, and it works â why learn something new? â
Why do I feel this way?
Killjoys S01E06
Killjoys S01E06
âOk, I'm tired of reading, what is your idea?â
At the end, the life cycle of developing the backend and frontend should look like this:
It remains only to design (optional) and the implementation of the missing functionality, by adding their services and domains to the system.
Itâs as if you would have your own Pocket Firebase , which besides automatically inserting all edits and migrations into the database, would also do Scaffolding for your mobile, desktop and web applications: automatically generated all views, templates, validation and API clients with push notifications, and asynchronous UI.
Due to the fact that most of any modern functionality is very stereotypical, there is no sense to rewrite everything from scratch from project to project. We need a platform so that people can share their work and standardize quality control and interaction between them. In this way, you can get a community of users who are implicitly motivated to improve the quality of products and the qualifications of its members, add gami cation with higher levels, achivka and comic stats ... well, you understand - there are some fantasies where to roam.
When developing such a solution, it is necessary to prevent the occurrence of two problem situations - this is âdevelopment by the committeeâ and âdevelopment by specifications,â which actually led to a terrifying inflation of the J2EE stack.
Now, almost no one is interested in automating the work of developers, although most of the code of existing applications can be generated automatically or abstracted by means of dynamic programming.
Yes ... speaking very roughly and simplistic: according to the database scheme, you can immediately generate REST endpoints, an API client and templates for your front on your favorite Angular / React / Ember, etc. Although I do not really digest the approaches of modern browser frameworks, more on that later. It remains to register user groups and their rights, to stretch the existing design. If you think this is a mystery - well, Tastypie already knows how to generate REST endpoints from the model, although there are not enough features and the performance is rather mediocre, even for Python, because of a bad design, but you can use it as an example.
What is Service Oriented Architecture?
This is an idea . It consists in the fact that applications should consist of reusable interchangeable "black boxes" - services that communicate over standardized protocols and intfeses among themselves.
There are several important conditions, services:
- Must be run under any conditions and scale on demand.
- Must be orthogonal (loosely coupled) and fault tolerant
- Must Have Reusability
- Should not store states
- Should be as abstract as possible.
- Must be able to be used in several business processes at the same time, without side effects
- Should be able to dynamically register and implement on the fly
In principle, the first three conditions should be more or less clear. Services do not depend on each other, do not share common resources, and can migrate between processes of both local and remote machines. Can be reused and simultaneously used in several projects.
As for the states, everything is simple: the server does not have to store data about the user session, this is done so that the request can be processed by any instance of the running service, which is usually the least loaded at any given time. This is done by incorporating encrypted, salted session information into a user request. In order not to transfer this payload with each request - they usually create a separate short-term registration service for user sessions that temporarily caches them. Creating a global service for storing user sessions will lead to the appearance of a âbottleneckâ in which it will still be necessary to solve the problem of consensus, which will have a
With abstractions, everything is rather tight - some domains or other (base scheme and logic) should be dynamically expandable depending on the specific needs of the dependent services and domains. Suppose there is an Accounting service (Accounting) and various services are registered in it â for some, only postpay is provided for billing, while others require prepayment. Add here more different discount and savings systems and get a quality headache for many months, because the whole logic of the work of these components is constantly changing. The normalization of the relational model plays an important role here - please note that there are six normal forms for a long time. Good abstraction and detailed elaboration of the subject area makes it possible to reduce the complexity of introducing a new, or changing the existing logic, but this will be discussed in the next part.
The possibility of separating access to services between several âsubordinateâ and âmanagersâ follows from the requirement for weak connectivity and the absence of shared states. Everything is straightforward as with multi-threaded programming and methods of preventing the states of "race". We should also mention the case of domain expansion: there is a compatibility problem when sharing an extended domain with several services. In general, roughly speaking, when one service needs a label with one number of fields and states (well, there is an enumeration value), and another with a completely different one, and how to âmake friendsâ of them so that there would be no problems when using this label together, or some other resource.
There are many system ârootâ services that are responsible for the work of all the others, and for their behavior in case of errors. One of them is the Registrar which allows you to register a
I think on this note you can complete a review of the objectives and requirements of SOA.
For the needs of something else to add.
Here is a diagram of how I generally represent the structure of a service-oriented architecture.
Let's look at the purpose of root services.
- Registrar - as mentioned above, is engaged in registration and start / stop services. It also stores addresses and descriptions of protocols for connection, checks the correct operation of services after updates, and can roll back to a previous working version automatically, without pausing work.
- The resource manager is one of the most important services, it coordinates the use of existing resources: the allocation of physical resources for the operation of services, and their monitoring for load distribution. It can be implemented based on Provisioning and DevOps solutions such as OpenStack . Resource managers are often referred to as SOI .
- Archivist - a service for logging events and collecting statistics, preparing reports.
- Courier - is engaged in the delivery and routing of messages.
- The warden - monitors the performance of all existing services, makes a restart on demand and blocking malicious users.
- AAA is 3 services responsible for
- Authentication (Authentication) - confirmation of the user's identity.
These are logins / passwords, biometrics, OAuth, sending SMS for login, etc. - Authorization (Authorization) - the separation of user rights.
These are usually different named groups for which different actions are available. - Accounting (Accounting) - purchase / sale of various services through payment services, the history of flow accounts, discount and savings systems. Often, methods are implemented to account for the various goods in storage areas, their reservation and availability.
- Authentication (Authentication) - confirmation of the user's identity.
Each of these services has its own design features (software) and I will try to describe all the details in the following articles, I hope I will not miss anything.
Any project should contain, at a minimum, the Registrar , the Archivist , the Resource Manager and the AAA , although you can certainly come up with better names. The latter two are missing for a number of reasons related to the complexity of the implementation of expandable domains (hi cocaine ).
What are the advantages and disadvantages of existing solutions?
Now there is no full-fledged SOA - everywhere somewhere something is missing.
On the code generation of hypermedia types, history is generally silent.
I worked with
Of course, in the case of the J2EE stack, there is terrifying redundancy and neither WSO nor JBoss are suitable for high loads. And in general, âweak bindingâ does not fit very well with modern j2ee EJB puffs.
Cocaine and Bluemix I, personally, attribute to SOA solutions only partially - there is no ready-made system for the implementation and standardization of the interaction of existing servers, complete DIY. In the case of cocaine, there is a locator , albeit rather rudimentary, but itâs good that at least something. In bluemix, you can implement a good SOI, but you also need to play a bit. From a SOI point of view, Amazonâs services are the most flexible solution.
All vendors offer to deploy infrastructure only on their PaaS'ah, this is especially noticeable in the case of Oracle and its Integration Cloud Service. It is necessary to understand that most users of SOA solutions need to have all the servers within the same data center, and communication delays are minimal - it is often necessary to organize CDNs and other distributed junk on my own, and an extra vendor lockin only links hands when choosing a data center. Both Amazon and Azure have already fallen off - it is better for serious and responsible customers to do risk management themselves, and for ordinary people itâs enough to âstart up, work - go
BPEL / BPMN is a separate topic: they tried to write a language for describing business processes for secretaries, but they got a many-sided, incomprehensible something that is not envisaged by the standard to implement without editing the language specification itself is almost impossible. To do this, you need to contact OASIS, join the consortium, issue RFCs ... well, you understand that there is nothing in common with solving the tasks set, but only payment for the existence of many-sided monolithic standards that cannot be adapted to modern market requirements within an acceptable time frame. Easier, much easier, to take and write everything with your hands ... in any favorite language. The description of business processes can be very simplified if we consider it from the point of view of finite automata (FSM).
The main advantage of the existing solutions is the guarantee from the vendors, which are sufficient for use in honey. sector and partly in the foreign state. and the automotive sector - there are no such stringent market requirements and there is no need to follow the âlatest fashion trendsâ. Consequently, for the implementation of new projects, or even, I will not be afraid of this word, startups - these solutions and platforms are not suitable.
Also, old-kind SAP can be considered as a full-fledged SOA.
But, given the support policy, cost, continuous obsolescence, poor performance, and a rather mediocre attitude to security â I would not consider this solution at all.
What are the requirements of the modern market?
Basically, they want to:
- The application could withstand very large load jumps.
- Content was updated in real time and used on various devices.
- There was no need to bother with long-term support
- It was not necessary to often purchase new equipment
- BusFactor was over 30%
In the case of SOA in this case is almost the ideal solution, and sooner or later, starting development with some popular MVC framework, adding various message queues and tasks, implementing interprocess communication, developers from their bikes roll into SOA with CQRS-ES ' , .
In practice, it turns out that there is no quality control at all, there is no evolutionary refactoring and the risk assessment is rather mediocre. They do "just to work" and especially no one has any understanding in six months - problems are solved as they are received and, of course, this negatively affects the service itself and the developers themselves. In this case, SOA is a dandy because delicious reusable âblack cubesâ allow you to forget about many sample tasks and focus on the business logic of the application, and most of the risk management goes directly to the developer community.
Why has nobody done this yet?
Because it is difficult.
To drive into this all is not enough to just read a couple of books about DDD and Corporate Application Templates, you need to put it all into practice and fill a bunch of cones ... understand that not everything is âtrueâ that they write in books, and even that âtruthâ "In most existing OpenSource projects is not implemented and therefore can not be taken as the basis of their own.
The key feature of any SOA is the presence of an IPC bus, a modularity system, and a discovery service (registrar). Symfony2 and Meteor.js are a good example of how things might look like - their modularity allows you to directly introduce new views and the domain model, plug-ins of other frameworks most often simply extend the functionality of the framework itself, add some scaffolding, but directly to the implementation and logic Your applications have no direct relationship. In principle, if you add a bus here on some thread Gearmand / Beanstalk / RabbitMQ etc you can even get some more or less SOA. Although, IMHO, AMQP is a wild overkill for the legs grow with OASIS, and redundancy is the worst. I really like protobuf's approaches, although its performance could be better.
App Data over Avian Carriers with Quality of Service
I often saw people trying to implement XMPP as a transport protocol for their mobile applications, sometimes even BOSH was used for websites with an exclamation âthis is ejabberd! Well this is erlang! Well this highold! It is not common for humans to measure the overhead of communication and serialization-deserialization of objects, although it so happens that this is about 20% of the entire time the application runs under load. Even a banal change of the network IP socket to a file UNIX socket can save quite a lot of processor time due to working off the IP stack, and since all the application traffic still passes through the OS firewall and other obscenities.
Scaling / dev / null
Often noticed enthusiastic posts like " here, we switched from the rails to Go and greatly reduced the server park. âI liked the phraseâ And we've never had another colossal clusterf ** k since. âYes, it is true: if you cannot load your equipment by 100% when processing requests - you scale the vacuum. The situation with C # / Java is actually not very different. The banal datemappers are nothing more than bytecode enhancing which additionally hides a few holes in abstractions in the form of multi-level caches and entity managers. If you rewrite everything with your hands many times faster, compare at least DataNucleus and old-ext first Hibernate. Why this is happening - the topic of a separate article, but if there were good source code preprocessors, such as sishnyh ones, such crutches would not be needed, and you wouldnât have to dig into byte code, bothering with compatibility. I personally have so far focused on Rust'e - a good balance of security and performance, and there are not so many implicit obscurities in Go (I'm talking about interfaces and âduck typingâ).
There are quite a few examples of indecency and unsuccessful attempts, but they crawled in the right direction, although they usually did not crawl.
Ok, what do we do with our MV-something?
80% of the projects I had to deal with do not have a normal normalized relational model. The first step is to normalize your modelku right up to the 6th form , and adopt readable rules for naming tablets and columns.
Because it
- Allow to speed up most queries.
- Prevents redundancy and reduces the size of your base.
- Simplify support and documentation of your solution.
- Simplify implementation of existing CRUD logic
For complex and large queries, it is worth using materialized views - this will save a little time for the profiler. Before you think about denormalizing a model, think about storing intermediate results in temporal representations.
With a normalized database, you can issue such a router for your CRUDs:
GET /plural($tableName) // GET /plural($tableName)/page/$pageNumber // GET /plural($tableName)/page/$pageNumber/size/$pageSize // GET /plural($tableName)/where/$fieldName/$value // GET /$tableName/$id // $id GET /$tableName/$id/plural($referencedTableName) // $id POST /$tableName/$id // ID PUT /$tableName // PUT /$tableName/$id/$referencedTableName/$referenced_id // ManyToMany DELETE /$tableName/$id // $id DELETE /$tableName/$id/$referencedTableName/$referenced_id // ManyToMany In the case of folded keys, you can come up with something like $ id1_id2_id3 (at your discretion), and, for example, take the key columns in alphabetical order. Of course, you will first need to read the database schema, and use it to validate the router, otherwise you will get a bunch of blind SQL injections.
Well, test, test, profile, reduce test execution time, refactor and test again.
By the same principle, the aforementioned TastyPie works.
Ok, and what about the JS frontend?
With JS frontend, I personally, for the time being, also do not understand what and where.
On the one hand, we have quite a lot of all sorts of standards, preprocessors, postprocessors, on the other hand, they do not simplify development at all. Most of the existing solutions require duplication and transfer of your templates to the browser, after which you still have to arrange EventSourcing with your hands, partially duplicate your model in JS and get rid of validation ... and nothing works âfrom the boxâ.
Often, watching the development of existing tools from the outside, I got the impression that the farther we go - the more difficult they become, and less and less they are aimed at solving our practical tasks. At the same time, all existing frontend projects collect quite large investments, we are talking about tens of millions of U.Dmurdsk e., And try to attract and sharpen attention.
I stayed on React.js with âisomorphicâ renderers in the node, even though I am already pretty nauseated by this. The problem is that you need to render simultaneously on the server and on the client for a normal UX, in any case, and various webkit surrogates like prerender.io do not have any reasonable performance.
Iâm one of those who donât really understand why I need to sculpt node.js where Iâm not hitting, simply because modern JS frontends arenât particularly rendered in other environments. In principle, nothing prevents you from overtaking html in json - transfer templates and changes on pages on web sockets and longpolling and when clicking on links. After that, apply a âpatchâ to the current DOM tree - then you will not have to bother with routing and models on the pages, and the rules for form validation can be described in terms of data- * attributes and regular expressions. Immediately, there is no need for any VirtualDOMs, since all changes in collections and templates can be traced already on the server side and send rendered patches.
Well, in general, you wrote your templates on the usual jinja / twig'ah and other ctpp, added one JS to the page and a specialized controller for collection management, that's all. True, such a thing will work fully (with push notifications) only if there is an EventSourcing. Thus, you donât need to duplicate the model and view from the browser side, and this is a rather large piece of work. I hope the idea is clear.
You can transfer HTML pages from the template engine in the form of a similar JSON
{ "action": "append|remove|replace", "selector": "#HaveANiceDay", "tagName": { // optional "attrs": [ { "name": "value" }, "name", ], "content": { {"otherTagName": { ... }}, }, }, } I do not know of any existing tool for the JS-frontend, as part of modern MVC frameworks, that would perform browser templating and collection management in a similar way. Although the management of collections and ES already exists in Meteor.js, but this is rather an exception, and Yandex had something similar somewhere ...
OK. Quite so more than enough for the introduction.
I hope it was not too boring, I was able to tell something new, and push you to think.
I wish you a pleasant time of day.
If necessary, I will add something here, correct errors and clarify.
Send all comments to the PM, do not litter the comments.
Source: https://habr.com/ru/post/263589/
All Articles