Information Systems Integration

It's no secret that "everything is already done before us." Only a little “assembled fragments” remained to solve the task. And here it turns out that integrating fragmented parts is not often more difficult than writing them. Why is this happening? What can you do about it?



All programmers love to make systems from scratch, when a thought can freely invent any form and means, when it is possible to make decisions without looking at what is going on. Of course, an integrally designed system, provided that it was made by an expert, always looks monolithic and pleases the eye. But after all, the reality is fluid and over time, any conceptual idyll is disrupted in the course of business development, changes in processes, absorption or merging of enterprises, the introduction of new systems, changes in hardware or software platforms, and even legislation.



Who supported and implemented the system, and even more so, was engaged in refining, reengineering and integration, he knows that more than two thirds of all efforts in IT (attention, time and money) go to the "gluing together" of the incompatible and trying to "make friends" modules written different people, at different times, in different languages ​​and technologies, under different platforms.



Let's list and analyze the factors affecting integration:

• Acceleration of processes . The development of the organization requires more and more often to change the data structures, business processes, not to mention the design and user interface, which is simply constantly in change. Here, just in such dynamic areas, where “variability” is the very essence and nature of the system, the task of integration is aggravated and turns into a serious problem.
• Distribution . Organizations are becoming increasingly large, and the tasks to be solved are more and more complex, a logical, organizational and geographical distribution appears.
• Heterogeneity In a large project, there is almost never an opportunity to stick to platforms and tools from one manufacturer, so you have to take into account and maintain the features of several platforms.
• Heredity . The inability to completely abandon Legacy systems, obsolete technologies, old hardware, and, by the way, sometimes give quite good indicators of reliability and performance, but they do not contribute to integration.
• Randomness It is not always possible to fully formalize, specify and structure the data, and part of the model remains “loosely coupled”, not amenable or weakly amenable to machine processing, analysis, indexing, and calculation.
• Conditionality Unfortunately, information systems are limited not only by the technical framework, but also by the habits of people (which are difficult to retrain), the peculiarities of the legislation (which is simply not ready for the emergence of such systems), many other factors independent of the developers.
• Interactivity . The consumer of information constantly raises his expectations about the speed of reaction of the system, speed and efficiency of information delivery. Most processes tend to perform in real time.
• Mobility . The user of systems began to move faster, and interaction with him is conducted through public communication channels in transport, at home and on the street, in public places and everywhere.
• Safety While the data was stored on the media inside the protected premises, no one was particularly worried about encryption, but now the network packets are flying in the air and this cannot be ignored.
• High load . The complexity of the integration is influenced by: the number of users in the system, the intensity of the data processing flow, the volume of data and the computation resource intensiveness.
• Continuity of work cycle . The integration and upgrade of systems should almost always be carried out without stopping their operation, smoothly, gradually and imperceptibly for the organization and its clients.
• Intersystem integration . The tasks of docking are not limited to the framework of the organization, it is increasingly necessary to integrate with partners, customers, suppliers, contractors and even government agencies.

These are the realities, I am even ready to argue that there is no more complex task in IT than the integration of systems. Let us now analyze the task from a different angle, select the parameters responsible for the complexity of integration and offer options for minimizing the negative impact of these parameters:

• Conceptual difference - it is based on the fact that the developers of different systems initially made different decisions, assumptions and assumptions that are not conceptually intertwined with each other. It is solved by the introduction of another layer of abstraction, which conceptually does not contradict both approaches. In this case, there are two options for implementation: (a) when the resulting system becomes centralized, and two or more integrable systems turn into subsystems, and (b) when we use the broker’s architecture (non-center intermediary), while the systems remain independent, and the broker provides an interlayer between them.
• Technological difference - when we have incompatible data exchange formats, interaction protocols and interfaces. It is solved by writing envelopes, interlayers, brokers and other lotions, not quite beautiful, but fairly reliable.
• License incompatibility. I will not dwell on this in more detail, since I am not a specialist in this matter, but the solution can be individual in each case, at the organizational level.

Our common task now looks like this: it is necessary to integrate N information systems characterized by the factors described above, with minimization of the number of layers, converters, brokers and interfaces between them. If we solve the problem head-on, then between N systems there will be N (N-1) / 2 connections, that is, with two-way interaction of N (N-1) interfaces. If we consider that under the interface we can understand anything here, from a web service to an offline process that runs, for example, once a day and performs a number of complex database synchronization operations (requests, processing, export, downloading via FTP, signal transfer the other part of the system to accept the transferred data and complete its part of the work, and then notify the results and transfer the necessary data back). In general, such options can never be completely eliminated, the only question is their proper implementation.

')

But I will cite the whole arsenal of tools for solving the problem, from those I have learned from others, I used myself and observed in practice:

• Standardization - it is necessary and important to use as many international, state and industry standards as possible, and if some are not enough, but they are clearly asking, then corporate standards need to be introduced, and it often makes sense to promote them in relevant organizations for early dissemination and promotion. .
• Broker-level integration . Advantages : universality - it is almost always possible to create an additional software module that will be addressed to both systems in different ways (for example, to one through a database, and to another through RPC). Disadvantages : complexity, complexity, and therefore the high cost of development, implementation and ownership.
• Data-level integration — that is, multiple applications can access a single database or multiple databases connected by replications. Advantages : low cost of integration, and when using one DBMS this becomes a very tempting solution. Disadvantages : if the database is not shielded by stored procedures and does not have the necessary integrity constraints (in the form of specifying cascade operations and triggers), then different applications can cause the data in conflicting states. If the base is shielded and integrity is ensured, then in this case, in applications running in parallel with one DB, there will be duplicate parts of the code that perform the same or similar operations. In addition, when changing the structure of the database, we will separately rewrite the code of all applications that work with it.
• Integration at the service level is a beautiful integration based on fixing interfaces and data formats from two sides and allowing for quick development of inter-corporate business logic. There are also disadvantages: nevertheless, there is a fixation, and if the structures or processes change, then problems and narrowly specialized, private solutions are formed.
• User - level integration is an extreme case, not automated integration, when users move data between systems through copy-paste, files, mail, and other outrages. We do not consider such methods, but they, unfortunately, are often used in the period when the software systems are not ready, and the development of the company does not allow waiting.
• Dynamic interpretation of meta - information - we'll talk about this in a separate article.

This is almost an exhaustive review of classical methods, please add if I missed something. But for non-classical integration methods, I am preparing another publication. Thanks for attention.