Software integration. Process description by business consultant

Synergy (Greek. Συνεργία - cooperation, assistance, assistance, complicity, complicity; from Greek. Σύν - together, Greek. Ἔργον - business, work, work (impact)) - a summing effect of the interaction of two or more factors, characterized by that their action significantly exceeds the effect of each individual component in the form of their simple sum [1], emergence.

Wikipedia.

In the process of working as a business consultant, in order to increase the efficiency of the enterprise systems, I almost always propose to carry out integration between various customer software. Because by integrating various systems it is possible to achieve a synergy effect.

I constantly have to face the same problems and solutions, many of which have to be explained in each new project to customers, some to programmers. Therefore, I believe that it is worth talking in detail about the integration process. In most of the examples, I chose various cases of integration of 1C and CRM, since today this very issue, as my practice shows, is the most relevant. Although this article is suitable for the integration of almost any software. So, let's begin.
')
Integration is a very important part of the work on automating business processes, as it is constantly required. In different situations, there is a need to promptly exchange data between different 1C configurations, between 1C software products and the site, between 1C and CAD systems, as well as billing systems, etc. Also, quite often it is required to integrate various web services with each other, for example, an online store and a CRM system. In general, it is impossible in most cases to combine the work of various departments of the company and automate the workflow without using integration.

What is integration?

Wikipedia gives us the definition:

Integration (from the Latin. Integratio - "connection") - the process of combining parts into a whole. Depending on the context, it may mean:

• Web integration - the integration of heterogeneous web applications and systems into a single web-based environment.
• Data Integration - combining data from different sources and providing data to users in a unified way.

.

I think Vicki is absolutely right in this case. And it can be supplemented with only one definition:

The integration of software systems and products is the exchange of data between systems with possible subsequent processing.

The point of integration is to ensure that the data that the user enters into one system is automatically transferred to another. The product in which the user enters data is called the source. And the receiver is the receiver, respectively.

Quite often, the data is transferred to both sides, for example, after conversion in the receiving system, the results are sent back to the source. And because integration is both one-way and two-way.

For example, if you combine the configuration of 1C: Trading with 1C: Accounting, you may need to transfer data for all sales to accounting, and get back information about payment for these sales.

Personally, I divide the integration process into the following stages:

1. Determine which product is the source, which - the receiver.
2. We match the objects between the source and the receiver.
3. Choosing a protocol for integration
4. We carry out post-processing of data (after transfer to one of the parties)

I always adhere to this sequence when planning integration work. This helps to work systematically, not to miss a single important point and to carry out the integration in such a way that it would be convenient for the client to work in the integrated system.

Important: when integrating various software solutions, you need to understand their functionality well.

Once I myself made such a mistake, and took up the integration of software products that I did not know well enough. Therefore, I can say for sure: studying a software product in the process of integration is not entirely correct. With this approach, most often there are many errors and problems, for example, the transfer of the wrong data or malfunctions. I recommend that you first study well a software product, understand that it can, how certain functions are implemented in it, and only then engage in integration.

In principle, in the process of integration you may need a more complex exchange, and you will have to enter, for example, three- or four-sided integration. But, in fact, these processes are no different from the usual one-or two-way process. Therefore, I will speak about one-sided integration. And at the end I will say a few words about the features of the two-way. All other areas you can always build by analogy.

Select the source and receiver

For each case of data integration, it is important to clearly define which system will be the source and which will be the receiver.

For example, you have a CRM system and a 1C: Trade program. In both systems, there is such a thing as a contact person. In principle, you can enter it from one side and the other. In this case, it is obvious that the source should be assigned to CRM, since this requires the logic of working with any CRM system.

Similarly in other cases. You need to understand in which system the user will enter data, and which one will become the recipient of this data through integration. This is necessarily agreed with the client (user), except when the source is obvious. in this case, it is necessary to inform the client that data of a certain type should be entered through the source system.

Comparison of objects (data)

Each time you work with data, you need to very well understand what you are unloading, in what form, and also where you will unload this data. In some cases, you will have a string variable in the source, and two or more objects in the receiver. In others, it is important to simply choose the right receiver object.

For example, in almost any CRM contact person and client are the same. On the other hand, in 1C contact person can be a client, partner, supplier. And it is very important to understand exactly where to write the data of this contact person. It is also important to compare all the data before working directly with the code. Tables or flowcharts are perfect for this.

Once I, like many, neglected this stage of work. Now I know that these actions will avoid a huge number of errors. On whatever side the programmer works - on the side of the source program or the receiver, such a label helps a lot. The programmer must clearly understand what data will be taken from the source, where they need to be transferred, and how they will be processed.

For example, when you unload a contact person from CRM, you must clearly associate this contact with a partner or buyer.
It is also very important to understand what transformations will be required for the uploaded data. For example, the data needed for the integration is stored in the source as a listing as text. And in the receiver (let it be 1C), a similar listing has a reference type. Consequently, you will need to convert the text into a link, and already transfer the link to a document.

And here a problem arises: matching rules are required.

You must clearly consider and write the matching rules. Moreover, these rules need to notify your customers. It is important to understand that the client does not see the logic of data exchange, he does not understand the features of integration.

Of course, you will definitely introduce a restriction of access rights, add other protection options. But, as practice shows, this does not guarantee that the user will make a mistake, because of which integration will stop working or will not work correctly. This may be someone from the staff who has administrative rights, or a guest expert who modifies, for example, the printed form of the document, but is not aware of the features of integration.

As a result, there are a variety of incidents. For example, you use the word "dealer" as a search keyword when matching. The client for some reason changes it in the source program to the word "dealers". It would seem a trifle! But this trifle will lead to the fact that the search in 1C will stop working.

I solved this problem this way:

1. Be sure to leave to the client the detailed description of the rules of comparison and explanation of which parameters and data are unacceptable.
2. I provide error notification options. Those. Not only fix the problem in the error log, but also notify the user about the failure in some way: via SMS, email, pop-up notifications in 1C. And sometimes in all these ways at once.

Why did I come to this type of work?

Integration is a complex process, and problems due to the human factor arise quite often, it is practically impossible to protect oneself from them. There are also software failures, especially for such complex systems with a large number of bugs, like 1C software products. And for business it is very important that the data exchange takes place in a timely manner, and if there is a problem, it is also important to resolve it promptly.

For example, in my practice there was a situation when I carried out the integration of 1C and Oracle, and the latter was the source program. Next, on the Oracle side, we changed one of the fields. As a result, orders stopped loading in 1C in general, and the server did not issue an error notification. Found a problem in a week.

On the one hand, this is an obvious flaw in the sales department of my client, since for a week not to receive a single order and not to worry about this, to put it mildly, it is strange. On the other hand, I consider the lack of error notification to be a flaw. Of course, as a result, the errors were corrected, the system continued to work without failures, but now I always add several options for error notification during data transfer.

The most common solutions:

1. With the help of SMS, email, pop-up notifications in 1C, the person who processes orders should receive information about the failure.
2. For control, a similar notice (most often by email) is sent to the head of the department or director of the company.
3. Be sure to keep a log file of errors so that the specialist was able to view all the details.

In some cases, it is also worthwhile to add a notice of failure to others, this issue is resolved with the customer individually.

It is also worth the error log file to keep the history in as much detail as possible and as long as possible. Do not forget that you are dealing with data that is in one database but is missing in another. And without a detailed report it will be very difficult for you to understand what exactly happened in the process of data transfer.

Data exchange: write yourself or use a typical solution?

Personally, I prefer to always develop a solution for the customer. Here you can argue, you can discuss various options, but there is a fact: typical data exchanges are always very overloaded with opportunities that your client does not need. As a result, the exchange process is significantly slowed down, and the number of possible errors grows significantly.

In addition, when choosing a typical software solution, you are very dependent on the software vendor. For any bug fix you have to wait for the release of the next version of the program. You will also have to adjust to updates for all changes in the work that the developer made.

Therefore, when choosing between self-writing data exchange and a typical solution that is not 100% suitable for this situation, it is better to write the exchange yourself.

In some cases, when a typical solution really satisfies the client’s needs 100%, and the speed of work for it is not critical, I also use ready-made products. For example, when uploading a nomenclature and photos to a site, I often use a ready-made data exchange from Bitrix. But only for unloading. To work with orders, I use samopisny exchange.

Connection Method: REST API, SOAP, or Direct Connection to the Receiver Base

In most cases, the choice of data exchange protocol directly depends on the system you are integrating. In most cases, the programmer has to take into account the requirements of both systems, and therefore there is no choice as such. In cases where the system can work with multiple protocols, choose the one that is more convenient for you. In my experience, for small and medium enterprises this issue is not fundamental.

Client Access Questions: Why Does Exchange Not Work?

I believe that all possible access restrictions should be learned at the initial stage of integration. Thus, you are guaranteed to avoid a very common problem:

You have implemented the integration, checked everything, tested, made sure that the system works. After which the user discovers that data is not being exchanged.

The most common situations are:

• Access restriction by IP.
• Restriction of user rights.
• Restriction on the number of calls to the source or receiver

.
In the first two cases, restrictions are usually associated with the information security policy of the enterprise, and they are resolved at the administrative level. For users who need to work with the exchange of data, the system administrator will configure the rights listed by you. Similarly for IP restriction.

In the case of working with a CRM system, restrictions are usually due to a paid package of services. It is enough to notify the client about the presence of such a restriction, and, if necessary, to help pay for and configure the advanced package.

1C identifiers and errors associated with them

When integrating with 1C, very often data exchange errors occur due to the wrong choice of MD (unique identifier). The essence of the problem lies in the fact that the objects in 1C have two types of MDs: one is unique within the selected type of objects. The second is used to work with the entire database.

If you search the entire directory using an identifier that is designed to work within a particular data type, an error will occur. The object may not be found at all, or the system will find several different objects at once. This feature of 1C should be treated very carefully.

Another problem: there is no possibility to bind to a unique identifier.

For example, the source system is the site, and it does not have a separate field for customer information, it goes in the general order text. In this case, you will have to choose some other identification option, for example, by email.

When integrating, it is very important to select one of the fields in the source, which will become a unique identifier.

I think it is good practice to duplicate this identifier in two systems. For example, if I upload information from CRM to 1C, then I copy the identifier field from CRM to 1C system. In the future, all search and integration is performed on this field quickly and easily.

In principle, this is not a mandatory action. Moreover, you will even store redundant data, since you have the necessary information in one of the systems, but such duplication improves the reliability of both programs and is a convenient solution for integration and subsequent data processing.

For example, by identifier that is identical to the source, the search will be easier and faster, since it will not require additional processing. In addition, if something happens to the database of one of the systems, thanks to duplicate identifiers, it will be much easier to compare the data.

Upload format

For data exchange uses a variety of formats. This could be JSON, XML, CSV, TXT, direct access to the database, etc. I have no definite preferences in this matter. I believe that here it is necessary to proceed from the rational requirements of the project.

Post processing

So, the data exchange was successful. What's next? I believe that this is not the final integration, as the user is not enough data appeared in the system. Usually he needs to perform some actions with this data. What exactly the client needs, should be clarified with him. But you should always remember that you work for the user in order to make it convenient for him.

For online stores, the integration often requires:

• Alert manager on the receipt of the order, for example, using sms
• Notifying users of new orders or other relevant information by email
• Beep and / or pop-up window in 1C with a reminder that there are new requests or requests

Post-processing is required, first of all, in order for the obtained data to go through a full life cycle and, therefore, take part in some subsequent business processes. Therefore, after downloading, notifications or specific processes, such as order processing, should be launched.

In addition to the actions that need to be performed at the receiver, it is also often necessary, after completing a successful data transfer, to perform certain actions at the source. What exactly is required, you will also tell the user.

For example, this could be a customer notification that his order was successfully unloaded and sent for processing. And here you can also use sms, email or simply change the status of the order in the system.

Integration testing

From my point of view, integration is a part (sometimes a special case) of software implementation. And here, as well as for any other work on software implementation, testing will be required by the programmer, then personally by the consultant, as well as various testing options with users. I wrote about this in detail in the article Implementing a software product. Features of the business consultant. Part III. The final .

Difference between one-way and two-way integration

In fact, there are no fundamental differences with unilateral, bilateral or multilateral integration. The essence of the process remains the same, just at different points in time, the receiver and source switch roles. The only important rule that I have introduced for myself and I also advise you: in a two-way exchange, you must store a unique identifier for all systems that participate in the integration. And I think that it is also worth duplicating it in both systems.

Today I tried to briefly describe the features of the integration process that I personally encounter in practice. I hope that the article turned out to be useful for you, and if there are any questions, I, as always, are ready to answer them.