Three microservice stories, or MSA for Enterprise
The idea of microservice is to build an application as a set of small services with dedicated functionality, each of which works in its own process. This approach has a number of advantages, but this is not the topic of today's story, but how the idea of microservice architecture looks from the point of view of Russian corporate business and IT managers in enterprises.
Together with Igor Bespalchuk we will try to look at this trend from three different angles, which is very useful for understanding the nature of what we are dealing with, and, as a result, in order to draw the right conclusions and make the right decision.
Microservices is one of the most important and significant components of the Web-scale architecture, which has the greatest implications for the reworking of device techniques and patterns in Enterprise. It is difficult now to say where the technology itself is now - perhaps at the highest peak, and we have to be disappointed ten more times. But, nevertheless, this is no reason not to study it right now .
About the speaker: Igor Bespalchuk is an application architect for the CUSTIS group of companies, which develops customized software for large corporate parties and government agencies. Prior to that, Igor also worked with corporate companies and retail companies, so he had the opportunity to observe how development processes take place in the corporate sector and how the management system is structured, as well as these or other changes gradually seep through there.
')
Further - decoding of the report of Igor on Web-Scale IT Conference 2017.
I first encountered microservice architecture in November 2012, when I came across a wonderful presentation by James Lewis from QCon "Services: Java, the Unix Way" . He talked about how they worked on a large, complex, loaded project with a huge amount of data: they sawed into separate functional elements, and decided to implement them as separate services. It was so surprising and unusual, including for our practice of architectural design, that just stirred the imagination. At the same time, this overlapped strongly with the concepts of Domain Driven Design (DDD), which we used for quite a long time in our practice, but in monolithic systems.
And in 2014, a large article “Microservices” appeared on the Martin Fowler website, where he clarified what microservices are and why they are needed, the article was translated into Russian .
In short, we used to do diverse functionality together, there was one system or application, and it was executed as a single process. If it was necessary to scale the system, this was done by scaling horizontally, but uniformly across all the functionality.
If it is super-short, then the central idea of microservices is that we start putting a separate kind of functionality into a separate process - the Bounded Context , as it is called in DDD. If necessary, we can scale these pieces independently, because they are separated into a separate process. You can run 5, 10 or 20 - as many as needed, product catalogs, user authorization blocks, or something else.
It turns out very small, compared to how it was previously designed, services.
This approach has several advantages , which we will talk about later, but in general this is not the topic of today's story.
In 2014–2015, I decided to look for whether there is a vivid experience in the use of such architectures in the Russian corporate sector. Then there was nothing among my friends and colleagues - it was interesting to theoretically chat, but no one had any practice.
In 2016, "something" began to be. In some banks, it was possible to hear that "we are trying to use microservice architecture, we have Docker and so on." It was interesting what would come of it.
In 2017, we at CUSTIS decided to hold the “Microservices for Enterprise” meeting . It was a half-closed event, mostly called by familiar architects from Enterprise.
At this meeting, it was found that there is still a lot of misunderstanding , especially from IT managers. They do not understand why it is needed at all. It seems to them that this is some kind of “another invention of programmers”, from which only extra trouble. Therefore, I once again tried to look at this phenomenon from the point of view, which, perhaps, would be more understandable not by techies, but by people in the position of IT director or development director. Well, if you are a technical specialist, perhaps this story will help you explain to the management the meaning and benefits of the microservice approach.
It may be a long time not to dwell on the fact that interest in the network to the topic of microservices has increased tremendously in recent years.
Since 2014, several international conferences have been held specifically on microservices.
It turns out a huge amount of literature on this topic.
Information is just a storehouse, but, nevertheless, in the Russian corporate party, if you look at the general mass, they are just beginning to talk about this. Of course, everything is heterogeneous - somewhere the practice of applying more, somewhere less, somewhere quite still neither sleep nor spirit.
Therefore, I am trying to fill this gap.
My story will be built as three different stories of development . Together we will try to look at this trend from three different angles. It seems to me that this is very useful for understanding the nature of what we are dealing with, and, as a result, in order to draw the right conclusions and make the right decisions regarding the trend of microservices.
In the first story, we will go from outside to inside, as it should be in systems engineering, and we will talk about the market and the need - about Enterprise and the Web as two worlds .
In 2014, Gartner released an analytical report , which said that by 2017, 50% of the main companies must use a web-scale architecture, but it was generally not very clear what it is. When we decided to arrange the Web-scale IT Conference, we tried to define what Web-scale technologies are, what Enterprise is and somehow see the difference.
Then for myself, I painted something like this. Conditional “Web” and conditional “Enterprise” are two large industries, or two large segments of the industry. From the IT point of view, today they look very different due to the fact that they had significantly different development paths:
Initially, these two worlds were almost completely separated. The Internet appeared as something separate, and for 10–20 years experienced tremendous evolutionary pressure on this island of its own.
The clash of
What are the interests and concerns of the classical business, which seem to be justified? I like the metaphor of the two collided continents.
There was a mainland Enterprise - the market of classic services with automation, with its unhurried evolution. And suddenly it turns out that another continent goes to a collision - another world, in which an evolutionary process also takes place, its own at its own pace. And it may turn out that for us, the inhabitants of the familiar Enterprise-world, the results of this evolutionary process will not be very friendly - as in the picture.
We already see how Google is engaged in cars, Airbnb - hotels, Amazon - offline-shops, etc.
Thus, it turns out that if we, as IT departments of corporations or as contractors of the Enterprise world, now don’t pay attention to these factors, we can regret very much later.
Analyzing the technical side of how the Internet giants work, they call so many different patterns and mechanisms that are part of the Web-scale architecture.
In my opinion, microservices here is one of the most important, complex, and significant components, having the greatest implications for reworking the techniques and patterns in Enterprise.
Of course, like any innovation, the theme of microservices cannot escape its period of hyip. There are companies that will happily come running and implement for a coin - everything is twisted and excessive expectations arise.
It is difficult now to say where the technology itself is now - perhaps at the highest peak of expectations, and we have to be disappointed ten more times. But, nevertheless, this is no reason not to study it right now .
Summarizing the first story about markets and needs:
These are the advantages that microservice architecture can potentially bring to Enterprise. If in some area of your business you intend or think that you have to compete with Internet companies, then it may well be that you need to borrow their technologies and in these areas try to apply yourself, grow competencies and set the same kind of tasks - increase stress and variability. Otherwise, then just eat.
3. They are already here.
If you yourself are from the world of Enterprise, then you can see how everything is alive here, dynamically and not like what we are dealing with in a classic enterprise.
We again go outside - inward: from the market, that is, from the environment of a separate enterprise, we turn to the device engineering in this separate enterprise. Let's talk about how the architectural styles of enterprise software have changed, how IT was built at the enterprise.
Like any systems and patterns, the development of IT in an enterprise goes from problem to problem. We solve one problem at the expense of a certain pattern and often bring some other problems, this is the inevitable way. At the same time, we always move from the simpler to the more complex , these are the general laws of development.
At one time, my colleagues and I discussed how it is, we always want to make it easier for people. Why is the opposite all the more complicated and complicated?
That is something for which we had to fight for a lot of labor hours, at some point we just start to receive, as an infrastructure , sometimes indifferent to our applied tasks. This “sediment” is constantly increasing, but the complexity of the systems only grows with time.
A long time ago (surely you, like me, did not find these times) all the IT in enterprises was done on one big computer. From the infrastructure there was only hardware, operating system and files. Everything else that is needed for calculations (data storage, calculation logic, command line interface, etc.), our predecessors had to write by hand.
As time went on, IT capabilities were changing, and now a personal computer is on each desk, shared data is somehow connected to the network and stored on a common file server. There are more opportunities in the infrastructure: we no longer figure out how to access the network, there are tools for working with files on the file server, the first databases and so on.
But of course, we still write more complex tasks with our hands: we need to store something in the database; still need to cheat business logic; to make a more complex graphical interface and so on.
The next step is relational databases. There was a long period of time when IT in enterprises was built on a client-server architecture , as an architectural style. This was supported by all vendors: here is the database, here is a quick development tool.
Here, part of the complexity of what we wrote in the previous step with our hands, “falls into some sediment,” into the infrastructure. We get it out of the box, but there is always enough work. We are writing data schemes of an increasingly complex structure, query languages are becoming more and more abstract, SQL has appeared. We develop business logic, user interfaces, etc.
After some time, the three-tier architecture captured the minds and interests of the developers of corporate software. For display, for example, there are slightly more blanks in the form of a browser and a library of components. Nevertheless, a lot of manual labor still remains, a huge number of integration tasks arise. The application server is largely engaged in integration, but it is easier to scale.
This process continues - service-oriented architecture . Corporate IT, at some point begins to go on the Web, even if in a B2B system, specialized Web servers and corporate buses appear to integrate a variety of solutions among themselves, systems for orchestrating complex interaction processes between all these systems. And every time history repeats itself - we get something out of the box, but still we finish a lot with our hands.
After such a retrospective, you can see that every time we have more and more ambitious tasks, their complexity and non-functional requirements increase. As a result, IT systems begin to granulate:
It is easy to assume that this process will not suddenly stop at three-tier and service-oriented architectures. My hypothesis is that it continues just in microservice architecture.
What do we have at the moment?
The variety of services included in even one application is amazing and even scary.
Common services that are needed by different applications are now provided from the cloud, for example, there you can find authorization forms. It explodes the brain a little bit, but it follows the outline of the very same principles of development - it stands out, specializes and belongs to some infrastructure .
A huge number of opportunities that previously had to be done by hands for each responsible system, now more and more often I want to do universally for any services. Again, I really want this to be provided out of the box - everything that concerns automatic scaling, monitoring systems, high availability, logging, which should already be done centralized, because otherwise it is impossible to climb on tens and hundreds of machines to understand what happened
That is, at the same time there is a huge demand for infrastructure , and little by little, with delay, the market of vendors providing it pulls up.
In fact, it is surprising how many infrastructure tasks arise. This is really a huge amount of work, for example, Avito says that to make a monitoring system for microservices is 1 person-year of work.
I think that in five years we will still see good infrastructures to support microservices out of the box from different manufacturers. But for now, it's still pretty damp. Although without this, it is nowhere to go - the infrastructure is getting fatter, services and functions are becoming more granular .
There is an interesting moment. When users now come to you and begin to interact with a particular device, it is even difficult to say what an “application” is. Previously, there were different applications in the enterprise, and their boundaries were clearly understood.
Now a user logs in from one device, and then - from another - is this the same application or not? We look deep into - there are some App Gw, which in turn interact with different services. Where is the app? There is a feeling that after some time the concept of the application may become very blurred .
Perhaps only the boundaries of business functions will somehow hold back this confusion. And then, in the presence of a large number of infrastructure services, it is foolish to produce them. Everything that is in your systems will not be tied tightly to business functions, to all appearances, will be done in the general infrastructure of the enterprise. But now it is not so, and in the next 5-10 years it will still be wrong, probably.
A very interesting point in this whole process - is it necessary for you, a specific medium or even a large enterprise, in general, all of these are microservices and the attendant complication? Maybe you can wait and not go to this mountain?
I think, fortunately or unfortunately, but to jump out of this process will not work . This is a trend that is stronger than us and bears everyone. We still have a choice - either try to wait aside, or still integrate.
The trouble is that emerging new infrastructure styles across the enterprise are starting to push you to new architectures, even if there is no need for this in your enterprise specifically. For example, you don't need super scalable systems and big data, you have a relatively small business that works great. With this, perhaps, in practice, calmly copes two-tier application on Delphi.
But the important thing is that the entire focus of attention of vendors and researchers who are developing technologies is now directed here - this is the point. It seems that this does not concern you directly, but it turns out that the entire industry is unfolding here.
The vector of aspirations of your workers is also directed in this direction, because it is fashionable, it is HYIP, it is books, conferences and everything else. If you don’t try to integrate into this movement somehow, after a while you realize that you will have to work with very specific frames that the new doesn’t care about at all. I'm not saying that these are bad workers. But some very good people will wash out of you, if you are not at least to some extent in line with the trend.
Therefore, I believe that sooner or later you will have to join, even if you really do not want to, therefore it is better to do this in a manageable manner within your company. That is, to lead, if it is impossible to fight .
Summing up a summary of the history of the evolution of architectural styles in the enterprise, we can fix the following:
And we go further inland. We talked about the markets and why it is worth thinking about it, and how it fits into the overall trend of Web-scale. Now I want to talk about how the roles of architects themselves and their specialization will change in connection with the trend of microservice architecture.
A long time ago, the development team was engaged in the development team and at most one manager-manager, who managed both work and resources. If the system was large, then suddenly it turned out that if you didn’t care about the architecture (there wasn’t such a word in relation to software before) and didn’t manage it, then it will grow “somehow by itself”, and then, zakostenev, will begin to manage your project for you.
It became clear that the position of the architect is still needed , I repeat - on relatively large systems.
As part of a large enterprise, it is still a bit more complicated.
At first, they were treated with separate independent samopisnymi systems. Then it suddenly became clear that there are many such systems, often they are supplied by vendors already ready, and they need to be integrated with each other. Here, the profession of a software architect is no longer very necessary, but such a specific position of a person is needed who would implement the next system, finish something small to it and tie it all together. The position of the integration architect (specialist in integration solutions) appears and the concept of a solution arises.
This is an interesting specialization, when a solution architect can be a person who has grown from analysts, and does not know how a particular software is programmed, but does a great, complex and useful work.
It is very interesting, as in this period, before microservices, work is usually carried out with the infrastructure of the enterprise. It is almost completely fenced off by a fence, some people are perfectly at their own pace, in their life cycle of changes they provide a uniform, standard and very stable infrastructure for all enterprise applications:
This is the philosophy of IT-services in large enterprises - for many years everything has been aimed at improving the efficiency of work, and therefore everything is standardized - it is simply impossible to order something unusual.
This is also a certain management system - we separately manage applications and separately - infrastructure, and for each part there are separate chiefs.
In this picture, which was held at enterprises for quite a long time, unpleasant moments arise:
The first.It is very difficult to bring new technologies , because they are working in the infrastructure block, trying to reduce costs and make everything standard.
And the second. It is more profitable for vendor to bring you as large a product as possible, which contains “everything you need for your business”. And then the functional architecture of applications in your enterprise will eventually begin to be managed by several large vendors , and not by you.
In the functions of your application will be what came up with vendors. Or you need to spend a large amount of effort, time and precious management resources to rebuild your own IT.
This is a very unpleasant par for some market players. If you are a second-tier business, this may suit you. If you are trying to work in leadership positions, you need to compete, which is greatly hampered by the fact that any template changes are a terrible pain.
If the trend of microservices really develops as we can assume, we will see that large products will be divided into a fairly large number of individual services, managed quasi-independent, approximately, as in the next picture.
Of course, the concept of a solution will remain, which describes the complete assembly of several services aimed at solving a single task.
The position of the integration architect will, of course, remain, but perhaps more will shift towards technical architecture. If now the integration architect often deals with information architecture at the enterprise, decides what data goes where, then, most likely, these functions will be divided, because the workload will increase - the more microservices, the more integration work.
Of course, the position of the architect of a separate service will remain. It is not going anywhere - from vendors, and from the enterprise. And, in my opinion, the demand for it, on the one hand, will increase, because the "cubes" are becoming more and more, and on the other hand, it will become slightly easier to ensure this position, as the "cubes" become smaller.
Previously, to make the architecture of a large monolithic ERP system needed a man with just a huge experience. In a situation when services are more isolated from each other, and the microservice architecture outlines their boundaries well and prescribes contracts, this position will become easier to ensure. People with lower qualifications will be able to fill this need . In my opinion, this is great.
Strong changes will occur in infrastructure management, in what the term DevOps can often be heard for now .. Many people say that this is a connection between programmers and administrators. I think this does not capture the essence. In fact, the task itself has changed: it’s not just someone who provides the entire infrastructure, and it works stably and equally. For each such solution, each service now requires something of its own — its own databases, different monitoring tools, and so on. There is a huge need for infrastructure that does not come out of the box , but it must be assembled by hand and often another, new, different one.
That is, in principle, the nature of this work and the nature of managing people is changing. You can no longer manage this via ITSM. You often need to manage this in project form, because the project includes a huge piece of completely unique infrastructure for a specific solution.
Therefore, DevOps appears as an infrastructure developer , and an infrastructure architect may appear nearby .
It is interesting that lately I sometimes see that the architect is also called the person who in this situation of the boiling variety of technologies is called and said:
That is, it is not even quite an architect of software in the full sense. This is a technological frame specialist, one might say, a technologist. His task is to understand and make a large variety of raw open-source technologies work. Apparently, such specialization also stands out now, perhaps, not in the form of a full-time employee, but the person who came, helped, started and further advises.
Briefly summarize the entire report. I tried to give an idea of how the MSA trend looks when viewed from the “Enterprise world” from three sides and in historical perspective from the past to the future, in terms of:
It seems to me that such a three-dimensional vision will help CIOs better position themselves in the MSA trend and understand what a specific company should do with it - to fence off or, conversely, to integrate.
news
Together with Igor Bespalchuk we will try to look at this trend from three different angles, which is very useful for understanding the nature of what we are dealing with, and, as a result, in order to draw the right conclusions and make the right decision.
Microservices is one of the most important and significant components of the Web-scale architecture, which has the greatest implications for the reworking of device techniques and patterns in Enterprise. It is difficult now to say where the technology itself is now - perhaps at the highest peak, and we have to be disappointed ten more times. But, nevertheless, this is no reason not to study it right now .
About the speaker: Igor Bespalchuk is an application architect for the CUSTIS group of companies, which develops customized software for large corporate parties and government agencies. Prior to that, Igor also worked with corporate companies and retail companies, so he had the opportunity to observe how development processes take place in the corporate sector and how the management system is structured, as well as these or other changes gradually seep through there.
')
Further - decoding of the report of Igor on Web-Scale IT Conference 2017.
My introduction to the MSA theme
I first encountered microservice architecture in November 2012, when I came across a wonderful presentation by James Lewis from QCon "Services: Java, the Unix Way" . He talked about how they worked on a large, complex, loaded project with a huge amount of data: they sawed into separate functional elements, and decided to implement them as separate services. It was so surprising and unusual, including for our practice of architectural design, that just stirred the imagination. At the same time, this overlapped strongly with the concepts of Domain Driven Design (DDD), which we used for quite a long time in our practice, but in monolithic systems.
And in 2014, a large article “Microservices” appeared on the Martin Fowler website, where he clarified what microservices are and why they are needed, the article was translated into Russian .
In short, we used to do diverse functionality together, there was one system or application, and it was executed as a single process. If it was necessary to scale the system, this was done by scaling horizontally, but uniformly across all the functionality.
If it is super-short, then the central idea of microservices is that we start putting a separate kind of functionality into a separate process - the Bounded Context , as it is called in DDD. If necessary, we can scale these pieces independently, because they are separated into a separate process. You can run 5, 10 or 20 - as many as needed, product catalogs, user authorization blocks, or something else.
It turns out very small, compared to how it was previously designed, services.
This approach has several advantages , which we will talk about later, but in general this is not the topic of today's story.
In 2014–2015, I decided to look for whether there is a vivid experience in the use of such architectures in the Russian corporate sector. Then there was nothing among my friends and colleagues - it was interesting to theoretically chat, but no one had any practice.
In 2016, "something" began to be. In some banks, it was possible to hear that "we are trying to use microservice architecture, we have Docker and so on." It was interesting what would come of it.
In 2017, we at CUSTIS decided to hold the “Microservices for Enterprise” meeting . It was a half-closed event, mostly called by familiar architects from Enterprise.
At this meeting, it was found that there is still a lot of misunderstanding , especially from IT managers. They do not understand why it is needed at all. It seems to them that this is some kind of “another invention of programmers”, from which only extra trouble. Therefore, I once again tried to look at this phenomenon from the point of view, which, perhaps, would be more understandable not by techies, but by people in the position of IT director or development director. Well, if you are a technical specialist, perhaps this story will help you explain to the management the meaning and benefits of the microservice approach.
Online interest
It may be a long time not to dwell on the fact that interest in the network to the topic of microservices has increased tremendously in recent years.
Since 2014, several international conferences have been held specifically on microservices.
It turns out a huge amount of literature on this topic.
Information is just a storehouse, but, nevertheless, in the Russian corporate party, if you look at the general mass, they are just beginning to talk about this. Of course, everything is heterogeneous - somewhere the practice of applying more, somewhere less, somewhere quite still neither sleep nor spirit.
Therefore, I am trying to fill this gap.
My story will be built as three different stories of development . Together we will try to look at this trend from three different angles. It seems to me that this is very useful for understanding the nature of what we are dealing with, and, as a result, in order to draw the right conclusions and make the right decisions regarding the trend of microservices.
The first story. Enterprise and Web as two worlds
In the first story, we will go from outside to inside, as it should be in systems engineering, and we will talk about the market and the need - about Enterprise and the Web as two worlds .
In 2014, Gartner released an analytical report , which said that by 2017, 50% of the main companies must use a web-scale architecture, but it was generally not very clear what it is. When we decided to arrange the Web-scale IT Conference, we tried to define what Web-scale technologies are, what Enterprise is and somehow see the difference.
Ways of development
Then for myself, I painted something like this. Conditional “Web” and conditional “Enterprise” are two large industries, or two large segments of the industry. From the IT point of view, today they look very different due to the fact that they had significantly different development paths:
- Enterprise companies have evolved from the classic business, which mainly provides material goods and services, and IT exists as a means of automating this business.
- On the Web from the very beginning, both the product and the service often had a digital form, or, as in the case of Amazon, a very significant share in the service itself is digital.
Initially, these two worlds were almost completely separated. The Internet appeared as something separate, and for 10–20 years experienced tremendous evolutionary pressure on this island of its own.
Evolutionary Web Pressure
- Lack of physical growth restrictions
- The explosive growth of new types of services , including digital.
- Strong competition for an unlimited amount of customers , which covers the whole world, regardless of territorial location.
- Requirements for UI / UX, load and scaling, developability.
- Frequent change of technology, so a stable homogeneous infrastructure and architectural style does not have time to form.
- The wave of development of Open Source, is not formed the cult of the heavy vendor.
- Result: some survived, giving rise to a number of technical and organizational patterns that respond to these requirements.
The clash of continents markets
What are the interests and concerns of the classical business, which seem to be justified? I like the metaphor of the two collided continents.
There was a mainland Enterprise - the market of classic services with automation, with its unhurried evolution. And suddenly it turns out that another continent goes to a collision - another world, in which an evolutionary process also takes place, its own at its own pace. And it may turn out that for us, the inhabitants of the familiar Enterprise-world, the results of this evolutionary process will not be very friendly - as in the picture.
We already see how Google is engaged in cars, Airbnb - hotels, Amazon - offline-shops, etc.
Thus, it turns out that if we, as IT departments of corporations or as contractors of the Enterprise world, now don’t pay attention to these factors, we can regret very much later.
Analyzing the technical side of how the Internet giants work, they call so many different patterns and mechanisms that are part of the Web-scale architecture.
In my opinion, microservices here is one of the most important, complex, and significant components, having the greatest implications for reworking the techniques and patterns in Enterprise.
Summary of the first story
Of course, like any innovation, the theme of microservices cannot escape its period of hyip. There are companies that will happily come running and implement for a coin - everything is twisted and excessive expectations arise.
It is difficult now to say where the technology itself is now - perhaps at the highest peak of expectations, and we have to be disappointed ten more times. But, nevertheless, this is no reason not to study it right now .
Summarizing the first story about markets and needs:
- MSA is one of the technical patterns that emerged in the course of a tough competitive development in the "parallel world" of Web technologies.
- In many ways, in this “parallel world”, those who learned to provide survive:
- retaining the online client with high quality service;
- high loads and data volumes that are completely uncharacteristic of Enterprise;
- rapid variability to technology , vendor independence.
These are the advantages that microservice architecture can potentially bring to Enterprise. If in some area of your business you intend or think that you have to compete with Internet companies, then it may well be that you need to borrow their technologies and in these areas try to apply yourself, grow competencies and set the same kind of tasks - increase stress and variability. Otherwise, then just eat.
3. They are already here.
If you yourself are from the world of Enterprise, then you can see how everything is alive here, dynamically and not like what we are dealing with in a classic enterprise.
The second story. Enterprise Architectural Styles
We again go outside - inward: from the market, that is, from the environment of a separate enterprise, we turn to the device engineering in this separate enterprise. Let's talk about how the architectural styles of enterprise software have changed, how IT was built at the enterprise.
The development of architectural styles
Like any systems and patterns, the development of IT in an enterprise goes from problem to problem. We solve one problem at the expense of a certain pattern and often bring some other problems, this is the inevitable way. At the same time, we always move from the simpler to the more complex , these are the general laws of development.
At one time, my colleagues and I discussed how it is, we always want to make it easier for people. Why is the opposite all the more complicated and complicated?
The understanding was born that the complexity of systems, including technical ones, never decreases , as it may sometimes seem - it “ precipitates ” in the form of infrastructure.
That is something for which we had to fight for a lot of labor hours, at some point we just start to receive, as an infrastructure , sometimes indifferent to our applied tasks. This “sediment” is constantly increasing, but the complexity of the systems only grows with time.
A long time ago (surely you, like me, did not find these times) all the IT in enterprises was done on one big computer. From the infrastructure there was only hardware, operating system and files. Everything else that is needed for calculations (data storage, calculation logic, command line interface, etc.), our predecessors had to write by hand.
As time went on, IT capabilities were changing, and now a personal computer is on each desk, shared data is somehow connected to the network and stored on a common file server. There are more opportunities in the infrastructure: we no longer figure out how to access the network, there are tools for working with files on the file server, the first databases and so on.
But of course, we still write more complex tasks with our hands: we need to store something in the database; still need to cheat business logic; to make a more complex graphical interface and so on.
The next step is relational databases. There was a long period of time when IT in enterprises was built on a client-server architecture , as an architectural style. This was supported by all vendors: here is the database, here is a quick development tool.
Here, part of the complexity of what we wrote in the previous step with our hands, “falls into some sediment,” into the infrastructure. We get it out of the box, but there is always enough work. We are writing data schemes of an increasingly complex structure, query languages are becoming more and more abstract, SQL has appeared. We develop business logic, user interfaces, etc.
After some time, the three-tier architecture captured the minds and interests of the developers of corporate software. For display, for example, there are slightly more blanks in the form of a browser and a library of components. Nevertheless, a lot of manual labor still remains, a huge number of integration tasks arise. The application server is largely engaged in integration, but it is easier to scale.
This process continues - service-oriented architecture . Corporate IT, at some point begins to go on the Web, even if in a B2B system, specialized Web servers and corporate buses appear to integrate a variety of solutions among themselves, systems for orchestrating complex interaction processes between all these systems. And every time history repeats itself - we get something out of the box, but still we finish a lot with our hands.
Separation of functions
After such a retrospective, you can see that every time we have more and more ambitious tasks, their complexity and non-functional requirements increase. As a result, IT systems begin to granulate:
- Decentralization becomes profitable.
- Increased autonomy of individual functions. If one client computer crashes, the database remains in order.
- Scaling performance. No one is satisfied with inelastic systems that cannot be easily scaled.
- Specialization of computing functions.
- Integration of shared . There is a separate specialization in the integration of all the numerous cut pieces.
It is easy to assume that this process will not suddenly stop at three-tier and service-oriented architectures. My hypothesis is that it continues just in microservice architecture.
What do we have at the moment?
- Clients for whom we are fighting, and employees with increased requirements for usability, come with different devices (browsers, mobile devices, etc.)
- Application services , in which we implement business tasks, still remain.
- Databases specialize and share. We are no longer satisfied with one universal database, which does everything badly. We, at a minimum, want one to be very fast and the other to be very large.
- Specialized nodes appear in order to adapt the application for different platforms ( Application-level gateway ).
- Practices related to orchestration and messaging are still needed.
The variety of services included in even one application is amazing and even scary.
Common services that are needed by different applications are now provided from the cloud, for example, there you can find authorization forms. It explodes the brain a little bit, but it follows the outline of the very same principles of development - it stands out, specializes and belongs to some infrastructure .
A huge number of opportunities that previously had to be done by hands for each responsible system, now more and more often I want to do universally for any services. Again, I really want this to be provided out of the box - everything that concerns automatic scaling, monitoring systems, high availability, logging, which should already be done centralized, because otherwise it is impossible to climb on tens and hundreds of machines to understand what happened
That is, at the same time there is a huge demand for infrastructure , and little by little, with delay, the market of vendors providing it pulls up.
In fact, it is surprising how many infrastructure tasks arise. This is really a huge amount of work, for example, Avito says that to make a monitoring system for microservices is 1 person-year of work.
I think that in five years we will still see good infrastructures to support microservices out of the box from different manufacturers. But for now, it's still pretty damp. Although without this, it is nowhere to go - the infrastructure is getting fatter, services and functions are becoming more granular .
There is an interesting moment. When users now come to you and begin to interact with a particular device, it is even difficult to say what an “application” is. Previously, there were different applications in the enterprise, and their boundaries were clearly understood.
Now a user logs in from one device, and then - from another - is this the same application or not? We look deep into - there are some App Gw, which in turn interact with different services. Where is the app? There is a feeling that after some time the concept of the application may become very blurred .
Perhaps only the boundaries of business functions will somehow hold back this confusion. And then, in the presence of a large number of infrastructure services, it is foolish to produce them. Everything that is in your systems will not be tied tightly to business functions, to all appearances, will be done in the general infrastructure of the enterprise. But now it is not so, and in the next 5-10 years it will still be wrong, probably.
A very interesting point in this whole process - is it necessary for you, a specific medium or even a large enterprise, in general, all of these are microservices and the attendant complication? Maybe you can wait and not go to this mountain?
Common boat problem
I think, fortunately or unfortunately, but to jump out of this process will not work . This is a trend that is stronger than us and bears everyone. We still have a choice - either try to wait aside, or still integrate.
The trouble is that emerging new infrastructure styles across the enterprise are starting to push you to new architectures, even if there is no need for this in your enterprise specifically. For example, you don't need super scalable systems and big data, you have a relatively small business that works great. With this, perhaps, in practice, calmly copes two-tier application on Delphi.
But the important thing is that the entire focus of attention of vendors and researchers who are developing technologies is now directed here - this is the point. It seems that this does not concern you directly, but it turns out that the entire industry is unfolding here.
The vector of aspirations of your workers is also directed in this direction, because it is fashionable, it is HYIP, it is books, conferences and everything else. If you don’t try to integrate into this movement somehow, after a while you realize that you will have to work with very specific frames that the new doesn’t care about at all. I'm not saying that these are bad workers. But some very good people will wash out of you, if you are not at least to some extent in line with the trend.
Therefore, I believe that sooner or later you will have to join, even if you really do not want to, therefore it is better to do this in a manageable manner within your company. That is, to lead, if it is impossible to fight .
Summary of the second story
Summing up a summary of the history of the evolution of architectural styles in the enterprise, we can fix the following:
- MSA - the next step in the development of architectural styles of complex software systems of the enterprise.
- MSA continues the logic that was before - the general movement towards specialization, granularization and the allocation of common infrastructures.
- Like all previous steps, MSA solves some of the problems that arise in the previous styles, and generates a number of new ones. Nothing can be done with this and you will have to live with it - to master it and once again pass through it.
- Free breakfast, of course, does not happen!
The third story. The role and specialization of the architect
And we go further inland. We talked about the markets and why it is worth thinking about it, and how it fits into the overall trend of Web-scale. Now I want to talk about how the roles of architects themselves and their specialization will change in connection with the trend of microservice architecture.
A long time ago, the development team was engaged in the development team and at most one manager-manager, who managed both work and resources. If the system was large, then suddenly it turned out that if you didn’t care about the architecture (there wasn’t such a word in relation to software before) and didn’t manage it, then it will grow “somehow by itself”, and then, zakostenev, will begin to manage your project for you.
It became clear that the position of the architect is still needed , I repeat - on relatively large systems.
As part of a large enterprise, it is still a bit more complicated.
At first, they were treated with separate independent samopisnymi systems. Then it suddenly became clear that there are many such systems, often they are supplied by vendors already ready, and they need to be integrated with each other. Here, the profession of a software architect is no longer very necessary, but such a specific position of a person is needed who would implement the next system, finish something small to it and tie it all together. The position of the integration architect (specialist in integration solutions) appears and the concept of a solution arises.
This is an interesting specialization, when a solution architect can be a person who has grown from analysts, and does not know how a particular software is programmed, but does a great, complex and useful work.
It is very interesting, as in this period, before microservices, work is usually carried out with the infrastructure of the enterprise. It is almost completely fenced off by a fence, some people are perfectly at their own pace, in their life cycle of changes they provide a uniform, standard and very stable infrastructure for all enterprise applications:
“Here is the Oracle database, here are the virtual machines — work, and we all support it in order, in strict standards.”
This is the philosophy of IT-services in large enterprises - for many years everything has been aimed at improving the efficiency of work, and therefore everything is standardized - it is simply impossible to order something unusual.
This is also a certain management system - we separately manage applications and separately - infrastructure, and for each part there are separate chiefs.
In this picture, which was held at enterprises for quite a long time, unpleasant moments arise:
The first.It is very difficult to bring new technologies , because they are working in the infrastructure block, trying to reduce costs and make everything standard.
And the second. It is more profitable for vendor to bring you as large a product as possible, which contains “everything you need for your business”. And then the functional architecture of applications in your enterprise will eventually begin to be managed by several large vendors , and not by you.
In the functions of your application will be what came up with vendors. Or you need to spend a large amount of effort, time and precious management resources to rebuild your own IT.
This is a very unpleasant par for some market players. If you are a second-tier business, this may suit you. If you are trying to work in leadership positions, you need to compete, which is greatly hampered by the fact that any template changes are a terrible pain.
If the trend of microservices really develops as we can assume, we will see that large products will be divided into a fairly large number of individual services, managed quasi-independent, approximately, as in the next picture.
Of course, the concept of a solution will remain, which describes the complete assembly of several services aimed at solving a single task.
The position of the integration architect will, of course, remain, but perhaps more will shift towards technical architecture. If now the integration architect often deals with information architecture at the enterprise, decides what data goes where, then, most likely, these functions will be divided, because the workload will increase - the more microservices, the more integration work.
Of course, the position of the architect of a separate service will remain. It is not going anywhere - from vendors, and from the enterprise. And, in my opinion, the demand for it, on the one hand, will increase, because the "cubes" are becoming more and more, and on the other hand, it will become slightly easier to ensure this position, as the "cubes" become smaller.
Previously, to make the architecture of a large monolithic ERP system needed a man with just a huge experience. In a situation when services are more isolated from each other, and the microservice architecture outlines their boundaries well and prescribes contracts, this position will become easier to ensure. People with lower qualifications will be able to fill this need . In my opinion, this is great.
Strong changes will occur in infrastructure management, in what the term DevOps can often be heard for now .. Many people say that this is a connection between programmers and administrators. I think this does not capture the essence. In fact, the task itself has changed: it’s not just someone who provides the entire infrastructure, and it works stably and equally. For each such solution, each service now requires something of its own — its own databases, different monitoring tools, and so on. There is a huge need for infrastructure that does not come out of the box , but it must be assembled by hand and often another, new, different one.
That is, in principle, the nature of this work and the nature of managing people is changing. You can no longer manage this via ITSM. You often need to manage this in project form, because the project includes a huge piece of completely unique infrastructure for a specific solution.
Therefore, DevOps appears as an infrastructure developer , and an infrastructure architect may appear nearby .
It is interesting that lately I sometimes see that the architect is also called the person who in this situation of the boiling variety of technologies is called and said:
“Bind us, please, all this together so that it“ just starts to work. ” And then our applied will understand.
That is, it is not even quite an architect of software in the full sense. This is a technological frame specialist, one might say, a technologist. His task is to understand and make a large variety of raw open-source technologies work. Apparently, such specialization also stands out now, perhaps, not in the form of a full-time employee, but the person who came, helped, started and further advises.
findings
Briefly summarize the entire report. I tried to give an idea of how the MSA trend looks when viewed from the “Enterprise world” from three sides and in historical perspective from the past to the future, in terms of:
- market needs in the worlds of Web and Enterprise, where it all comes from and what threatens us if we don’t master it;
- architectural styles of enterprise software systems - and why you most likely will not be able to jump out of this boat;
- specializations of the role of the architect , who are also rapidly changing and dividing.
It seems to me that such a three-dimensional vision will help CIOs better position themselves in the MSA trend and understand what a specific company should do with it - to fence off or, conversely, to integrate.
news
Conference of conferences Russian-Internet technologies are just around the corner, this year we decided to distribute conference reports at the junction of Web-Scale IT enterprise and web technologies to other conferences - technical reports will go to RIT ++ Backend Conf , management at Whale Rider . At the moment there are several interesting applications, for example:
- Organization's willingness to change or where dreams lead / Denis Kochergin (Fair Masters)
- How big IT team can change immediately on all fronts and not die? / Alexander Andronov (Dodo Pizza)
- Who lives in the clouds or what is a cloud-native infrastructure / Anton Weiss (Otomo)
- Own cloud available to everyone: the life cycle of a typical microservice and its dependencies in kubernetes / Maxim Vikharev (Alytics)
The status of the reports will be updated during April, but we promise to select the coolest ones, so it’s not necessary to wait - you can already book tickets .
Source: https://habr.com/ru/post/353172/
All Articles