How we developed the integration of an online store with 1C: Enterprise and why it became popular. Part 2
I will continue the story on the integration of online stores with back office and the approach to their solution in our implementation of the integration of 1C: Bitrix online store with 1C: Trade management.
In the first post on this topic, we talked about how the tasks faced by the owners of online stores were analyzed, the concept of data exchange was selected and the exchange protocol between 1C and the online store was developed.
In the continuation of the topic, I would like to tell in more detail how the exchange and its configuration look like as a result, both from the back office 1C and from the Internet site. And also show how problems were solved to optimize work with large amounts of data.
I recall that in general, the following integration scheme was chosen:
')
1) Site and 1C work independently and exchange data in CommerceML format
2) The site does not have direct access to 1C
3) 1C regularly accesses the public script on the site, giving or accepting data
3) The initiator of the exchange and connection is always 1C.
Consider the implementation of this concept in practice.
So, as I said earlier, the “1C: Trade Management” configuration includes a specialized module for exchanging data with sites. This is a universal module, working according to the protocol described in the previous post and using the CommerceML format.
The module is in the standard 1C: Trade management for both version 10.3 and version 11 (platform 8.2). In addition to 1C: Trade management, exchange functionality with sites also exists in other configurations, including localized ones. We have brought information about the support of the exchange functionality with sites in 1C configurations into a special table - 1c.1c-bitrix.ru/ecommerce/require_1C.php .
Consider the functionality of the module, part of the 1C: Trade Management 11.
The first tab of the interface module contains the basic interaction settings.
The Data exchange mode section allows you to specify which exchange mode will be used in this profile. Or, more simply, what business problem will be solved in an exchange session: unloading goods, exchanging orders, or both.
In the section Assignment , the parameters of the site with which the exchange will be made are indicated: the URL of the exchange script, the account that is allowed to be exchanged. You can immediately check the success of the connection.
Change control allows you to use internal 1C journals, which record changes in objects related to the exchange with the site and include only those data that have been modified in 1C since the previous exchange session. Of course, this reduces the amount of data, reduces the load on 1C and on the server of the online store.
In the Automatic exchange section, you can very flexibly set a schedule according to which the exchange functionality will be launched automatically, without user intervention.
Let us consider in more detail the solution of the first integration problem -
For this, it is necessary to put the option “Unload goods” in the exchange mode, as a result of which the tab with the same name will be available:
The main and most important section here is the directory table , which allows you to create what will be uploaded to the site.
In the “default” variant, the entire directory of the nomenclature is unloaded as it is: all groups and all products, without any restrictions. Of course, this option will not please everyone, and a flexible choice is available - what exactly and under what conditions should be uploaded to the site.
For example, you can select groups and subgroups of the catalog, unload goods with non-zero residues, with certain types of prices (retail, wholesale, dealer), from specific warehouses and the like.
Further, by default, all products are uploaded to the XML file in a single directory (<Directory> tag). Each catalog has its own classifier, including the structure of groups and subgroups for products, as well as properties for all products in this catalog.
If different types of products with their own group tree and their own set of properties (characteristics) are uploaded to the site, then each such type is recommended to be downloaded to a separate directory. So, in the system 1C-Bitrix: Site Management, this will lead to the fact that each directory will be uploaded to a separate information block, which has its own groups, properties, rights, settings, etc. As a result, you can make a different presentation in the online store for different types of products: hierarchy, filter by properties, product comparison, etc.
Directory settings in this case will be something like this:
There is another possibility of forming a catalog structure that is uploaded to the site - the use of types of nomenclature. Types of nomenclature is essentially an alternative category heading for the types of goods. If the structure of the nomenclature directory groups is not suitable for setting up uploads to the site, then you can create item types, attach products to them and upload them to the site using a new rubricator.
Putting the checkboxes “Upload image files” and “Upload other files” makes sense, if 1C has high-quality product descriptions, images, related files (instructions, etc.). Many owners of online stores do not store in 1C descriptions of goods, properties, quality descriptions. In 1C, they are not so necessary for them, and the site usually has an HTML editor for descriptions, and various utilities for image processing, watermarking. In addition, it is more convenient to organize remote work of content editors when working with a website than to let them inside 1C. And, nevertheless, there are those who prefer to keep the entire narrative in one place - in 1C, and use the site as a visualization tool. There is another task - when the same goods are uploaded to several online stores at the same time. Then it is better to make descriptions in 1C once, and then automatically unload them into several sites.
Which directory management model to choose is the end user decides. The main thing is that the integration functionality allows it.
If you now start the upload (click the "Execute data exchange" button), then 1C begins to prepare the unloaded data.
2 files are generated: import.xml and offers.xml (for each catalog being uploaded). The first file contains information about the product catalog (structure, products, properties), and the second - trade offers for this product. If one product has options depending on characteristics (color, size, etc.), then all of them will be offers in the offers file. Details are in CommerceML 2.04 standard.
If images or files are associated with products, they are located in the import_files subfolder, and data from XML refers to them.
For example, 2 product catalogs from 1C were unloaded:
Now you need to transfer these files to the server. In the exchange protocol (see Part 1) there is a special mode for this: http: // <website> /bitrix/admin/1c_exchange.php?type=catalog&mode=file&filename= <file name>.
If the amount of data is large (initial unloading of a large reference book of the nomenclature occurs), then of course I would like to optimize the transfer of files.
Firstly, ZIP compression is used . As we remember, before starting the file transfer, 1C requests the exchange parameters from the site, among which there is a flag supporting ZIP compression. All files are packaged and one zip-archive is obtained, which is sent to the site. If compression is not supported, all files from the folder (including images) are sent sequentially.
Secondly, many hosting providers limit the amount of data transmitted by a POST request. And even a zip-archive can easily fail if it is tens of megabytes. In this case, 1C can break up the downloaded files into chunks , if their size exceeds the maximum allowed. On the side of the site, numbered portions are received and then assembled back into a single file. The maximum size of a portion of the data 1C also receives from the site in exchange initialization mode.
Now the received files need to be processed on the site side and loaded into the site database.
There is also a moment of hosting restrictions. Processing (parsing XML and loading into the database) is done by a PHP script, the time of which is usually limited. Of course, at the beginning of the script is set_time_limit (0), but it does not work everywhere. Therefore, in the settings 1C-Bitrix, you can specify the maximum time that the script can work out - by default 30 seconds. Then, when this limit is reached, the script will reload and continue importing from the previous position in the file.
In addition, the import script can compare the checksums of the XML description of the goods in the file being processed with the checksum of this product in the database since the previous download. If the checksums match, then the product is skipped and not updated. This is necessary for those cases when the catalog is already uploaded to the site, and full upload is still done from 1C. Then the data import will be performed significantly faster.
Of course, it is always recommended to set up only the changed positions in 1C, then full unloading will be done only 1 time, and then small portions of data will go, which will be uploaded to the site very quickly.
The import parameters of the catalog from 1C are configured in 1C-Bitrix in the administrative part on the special page "Integration with 1C"
I will not go further into the details of import settings, of which there are many: image processing, CNC generation for products, data processing that are not in the import file, but are available on the site, and the like. In general, catalogs from 1C are most fully reflected in the catalogs on the site.
Unloaded:
Due to optimization mechanisms, the issue of initial unloading and processing of large amounts of data is solved. The mode of unloading only the changed positions in 1C, allows almost not to think about the load for subsequent exchange sessions.
I would also like to say that in 1C profiles of settings for uploading goods to the site can be many. They form the so-called exchange nodes:
This gives even greater flexibility, because, for example, you can update different categories of goods at different times, upload products to different sites and different CMS systems, etc. In addition, at different times, not related to the unloading of goods, you can also exchange orders: for example, take orders from the site every 5 minutes.
But I will tell you about the exchange of orders in the third and final part.
In the first post on this topic, we talked about how the tasks faced by the owners of online stores were analyzed, the concept of data exchange was selected and the exchange protocol between 1C and the online store was developed.
In the continuation of the topic, I would like to tell in more detail how the exchange and its configuration look like as a result, both from the back office 1C and from the Internet site. And also show how problems were solved to optimize work with large amounts of data.
I recall that in general, the following integration scheme was chosen:
')
1) Site and 1C work independently and exchange data in CommerceML format
2) The site does not have direct access to 1C
3) 1C regularly accesses the public script on the site, giving or accepting data
3) The initiator of the exchange and connection is always 1C.
Consider the implementation of this concept in practice.
So, as I said earlier, the “1C: Trade Management” configuration includes a specialized module for exchanging data with sites. This is a universal module, working according to the protocol described in the previous post and using the CommerceML format.
The module is in the standard 1C: Trade management for both version 10.3 and version 11 (platform 8.2). In addition to 1C: Trade management, exchange functionality with sites also exists in other configurations, including localized ones. We have brought information about the support of the exchange functionality with sites in 1C configurations into a special table - 1c.1c-bitrix.ru/ecommerce/require_1C.php .
Consider the functionality of the module, part of the 1C: Trade Management 11.
The first tab of the interface module contains the basic interaction settings.
The Data exchange mode section allows you to specify which exchange mode will be used in this profile. Or, more simply, what business problem will be solved in an exchange session: unloading goods, exchanging orders, or both.
In the section Assignment , the parameters of the site with which the exchange will be made are indicated: the URL of the exchange script, the account that is allowed to be exchanged. You can immediately check the success of the connection.
Change control allows you to use internal 1C journals, which record changes in objects related to the exchange with the site and include only those data that have been modified in 1C since the previous exchange session. Of course, this reduces the amount of data, reduces the load on 1C and on the server of the online store.
In the Automatic exchange section, you can very flexibly set a schedule according to which the exchange functionality will be launched automatically, without user intervention.
Let us consider in more detail the solution of the first integration problem -
Unloading goods (nomenclature directory) from 1C to the website of an online store
For this, it is necessary to put the option “Unload goods” in the exchange mode, as a result of which the tab with the same name will be available:
The choice of products for unloading and setting unloading in 1C
The main and most important section here is the directory table , which allows you to create what will be uploaded to the site.
In the “default” variant, the entire directory of the nomenclature is unloaded as it is: all groups and all products, without any restrictions. Of course, this option will not please everyone, and a flexible choice is available - what exactly and under what conditions should be uploaded to the site.
For example, you can select groups and subgroups of the catalog, unload goods with non-zero residues, with certain types of prices (retail, wholesale, dealer), from specific warehouses and the like.
Further, by default, all products are uploaded to the XML file in a single directory (<Directory> tag). Each catalog has its own classifier, including the structure of groups and subgroups for products, as well as properties for all products in this catalog.
If different types of products with their own group tree and their own set of properties (characteristics) are uploaded to the site, then each such type is recommended to be downloaded to a separate directory. So, in the system 1C-Bitrix: Site Management, this will lead to the fact that each directory will be uploaded to a separate information block, which has its own groups, properties, rights, settings, etc. As a result, you can make a different presentation in the online store for different types of products: hierarchy, filter by properties, product comparison, etc.
Directory settings in this case will be something like this:
There is another possibility of forming a catalog structure that is uploaded to the site - the use of types of nomenclature. Types of nomenclature is essentially an alternative category heading for the types of goods. If the structure of the nomenclature directory groups is not suitable for setting up uploads to the site, then you can create item types, attach products to them and upload them to the site using a new rubricator.
Putting the checkboxes “Upload image files” and “Upload other files” makes sense, if 1C has high-quality product descriptions, images, related files (instructions, etc.). Many owners of online stores do not store in 1C descriptions of goods, properties, quality descriptions. In 1C, they are not so necessary for them, and the site usually has an HTML editor for descriptions, and various utilities for image processing, watermarking. In addition, it is more convenient to organize remote work of content editors when working with a website than to let them inside 1C. And, nevertheless, there are those who prefer to keep the entire narrative in one place - in 1C, and use the site as a visualization tool. There is another task - when the same goods are uploaded to several online stores at the same time. Then it is better to make descriptions in 1C once, and then automatically unload them into several sites.
Which directory management model to choose is the end user decides. The main thing is that the integration functionality allows it.
Unloading process
If you now start the upload (click the "Execute data exchange" button), then 1C begins to prepare the unloaded data.
2 files are generated: import.xml and offers.xml (for each catalog being uploaded). The first file contains information about the product catalog (structure, products, properties), and the second - trade offers for this product. If one product has options depending on characteristics (color, size, etc.), then all of them will be offers in the offers file. Details are in CommerceML 2.04 standard.
If images or files are associated with products, they are located in the import_files subfolder, and data from XML refers to them.
For example, 2 product catalogs from 1C were unloaded:
Now you need to transfer these files to the server. In the exchange protocol (see Part 1) there is a special mode for this: http: // <website> /bitrix/admin/1c_exchange.php?type=catalog&mode=file&filename= <file name>.
If the amount of data is large (initial unloading of a large reference book of the nomenclature occurs), then of course I would like to optimize the transfer of files.
Firstly, ZIP compression is used . As we remember, before starting the file transfer, 1C requests the exchange parameters from the site, among which there is a flag supporting ZIP compression. All files are packaged and one zip-archive is obtained, which is sent to the site. If compression is not supported, all files from the folder (including images) are sent sequentially.
Secondly, many hosting providers limit the amount of data transmitted by a POST request. And even a zip-archive can easily fail if it is tens of megabytes. In this case, 1C can break up the downloaded files into chunks , if their size exceeds the maximum allowed. On the side of the site, numbered portions are received and then assembled back into a single file. The maximum size of a portion of the data 1C also receives from the site in exchange initialization mode.
Now the received files need to be processed on the site side and loaded into the site database.
There is also a moment of hosting restrictions. Processing (parsing XML and loading into the database) is done by a PHP script, the time of which is usually limited. Of course, at the beginning of the script is set_time_limit (0), but it does not work everywhere. Therefore, in the settings 1C-Bitrix, you can specify the maximum time that the script can work out - by default 30 seconds. Then, when this limit is reached, the script will reload and continue importing from the previous position in the file.
In addition, the import script can compare the checksums of the XML description of the goods in the file being processed with the checksum of this product in the database since the previous download. If the checksums match, then the product is skipped and not updated. This is necessary for those cases when the catalog is already uploaded to the site, and full upload is still done from 1C. Then the data import will be performed significantly faster.
Of course, it is always recommended to set up only the changed positions in 1C, then full unloading will be done only 1 time, and then small portions of data will go, which will be uploaded to the site very quickly.
Data import settings on the site side
The import parameters of the catalog from 1C are configured in 1C-Bitrix in the administrative part on the special page "Integration with 1C"
I will not go further into the details of import settings, of which there are many: image processing, CNC generation for products, data processing that are not in the import file, but are available on the site, and the like. In general, catalogs from 1C are most fully reflected in the catalogs on the site.
Unloaded:
- Directory group structure
- List of products group
- Properties of goods (numeric, string, type “list”, date-time, boolean)
- Product Description (including HTML description)
- Related Images and Files
- Price set (retail, wholesale, dealer, etc.)
- Remains in warehouses
- A set of proposals for the goods, differing characteristics
Due to optimization mechanisms, the issue of initial unloading and processing of large amounts of data is solved. The mode of unloading only the changed positions in 1C, allows almost not to think about the load for subsequent exchange sessions.
I would also like to say that in 1C profiles of settings for uploading goods to the site can be many. They form the so-called exchange nodes:
This gives even greater flexibility, because, for example, you can update different categories of goods at different times, upload products to different sites and different CMS systems, etc. In addition, at different times, not related to the unloading of goods, you can also exchange orders: for example, take orders from the site every 5 minutes.
But I will tell you about the exchange of orders in the third and final part.
Source: https://habr.com/ru/post/131344/
All Articles