How to independently serve 2500 accounting bases
In this article, the Button will honestly tell about the pitfalls of administering accounting tools and will give practical advice with useful links.
If your company is engaged in accounting outsourcing, first of all you need to decide on the program in which the accountant will work. Offers on the market are diverse, we used the following criteria:
Our selection list includes Kontur. Elba, 1C Enterprise Accounting 3.0 and Xero. The decision was made to use 1C Enterprise Accounting 3.0, because Elba is only suitable for small companies, Xero is comfortable and technologically advanced, but still English, it is impossible to set up accounting and tax accounting according to Russian standards.
')
The advantage of 1C products is that at the moment it is the most popular accounting software product in our country. It is difficult to find an accountant who has never worked in this program. Since the Button does not specialize in servicing customers of a particular industry, it is important that the program is suitable for individual entrepreneurs and limited liability companies with any taxation system, and 1C allows it.
Over time, we abandoned the use of file databases and deployed our cloud using 1C Fresh technology, but we will not get ahead of ourselves.
The first customers came to Button in 2013, for each they created a file base BP 3.0. When the number of bases approached a hundred, hell came for the system administrator. It took several days, sometimes nights, to add a new user to all databases or install an update. I had to invent software solutions, inventing updates and processing to add users. It was all too slow and hopeless.
An alternative to file bases was the use of the data sharing mechanism ( Multitenancy ) of the 1C platform. Its essence lies in the fact that one information base 1C is divided into independent areas with the help of separators. The split mode is enabled with the “Split Enabled” constant. By the number of the separator the user connects to a separate area. Within one such information database, data can be unseparated, that is, common to all areas, for example, a classifier of banks or units of measurement, and separated, unique, existing within one area, for example, counterparties and documents.
It is much easier to administer one database instead of hundreds. The cloud service 1cfresh.com works on this principle, and the technology of publishing solutions for 1C Fresh can be purchased from 1C partners.
Delivery 1C Fresh includes two specialized configurations for information database management.
More information about the composition of the delivery of 1C Fresh, we wrote in the article on Infostarte .
When the application service was installed and the first databases were published, the speed of the “cloud accounting” work compared to the file one was about 10 times higher. For the first time in a long time, quarterly reporting was quite easy, since 1C worked without freezing and reloading.
A few months later, our Fresh began to work unstably. The reason: mass connection of clients. We had published two accounting bases and two ZUP 3.0 bases, the number of areas in each of which exceeded 300 pieces. The situation was aggravated by the fact that the Fresh cluster was not split up into nodes, all databases were spinning on the same database server. Our hardware could not cope with the load and, as a result, the service manager lost contact with the regions. As the saying goes, neither download currency rates nor synchronize databases. It is good that our colleagues from 1C supported us, penetrated into our specifics and found errors in administration, helped to correctly build the logic of using the cloud.
We advise everyone, before deploying your service, to get systematized knowledge on the course “ Operation of large systems ” at the 1C training center.
The number of areas in one database is a compromise, which is due to many factors, it can be very different, we chose 200 for ourselves. The cluster itself should consist of a service manager, service agent and nodes (database servers, application servers, web services), when the number of regions becomes more than two hundred, the next node unfolds.
When setting up a PostgreSQL server, you should definitely read the 1C recommendations (information is available on kb.1c.ru ) and use for example such a service pgtune.leopard.in.ua . And yes, for PostgreSQL, the most delicious thing is memory!
Now the architecture of our Fresh-cloud includes 7 nodes, 7 bases 1C BP 3.0, 2 bases 1C UNF and three bases ZUP 3.0 are published. The service is accompanied by five system administrators and two 1C programmers. Such a team is more than enough, with such a composition it is possible to maintain a base of 2000+ clients.
The configurations of our machines are different and change periodically, in response to changing conditions. The application server typically has 4-6 processors (cores) and 16GB of memory, using PostgreSQL 9.4.2-1.1C as the database server, located on Linux machines. Resources for these machines are allocated depending on the size of the bases, the average number of connections and peak loads. If the application server hosts several large databases, resources can reach 6-8 cores and 30 GB of memory.
With the help of Zabbix monitoring, we control the availability of information databases, the load on nodes and 1C servers. There is a TV in the office, and when everything is bad we immediately see it:
Every day, 95 accountants are actively working in the bases. Clients Buttons that want to self-record documents in 1C get access to their database via the Internet. It is possible to connect to the web-client shopping equipment (CMC, barcode scanner). For a thin client, an optional document scanner connection is available. It is very convenient to scan and attach files to documents directly from 1C.
All reporting from us is sent from 1C, through the Kaluga Astral certification center, we use CryptoPro keys. We are actively implementing Dyadok electronic document management, this functionality is also available in the service model.
There are several options for data exchange:
Information base published in a web-service is called an application. For the new client, an area is created in BP 3.0, ZUP 3.0 and, if necessary, the UNF. At the moment, synchronization of data between applications occurs every 10 minutes. Synchronization between applications in the service can be controlled using the service manager, but we brought the data to monitor. The system administrator sees online in which areas the data exchange is stable, in which it does not, and errors need to be corrected.
Many customers of Buttons use 1C solutions for wholesale or retail trade (1C “Trade Management” 10.3 or 11.X, 1C ”Retail”), which they store on their servers and independently administer. In this case, it is also possible to set up data exchange with accounting in our cloud. Such synchronization is called “Data exchange in the service model”. It is configured automatically according to the schedule, but it often “breaks” after the update due to the fact that the client on his side does not update the configuration on time. We control on monitoring, if the exchange has not passed more than two weeks, contact the client.
Now online stores and various online trading platforms are becoming more popular. For the exchange of data from 1C, the CommercelML 2 exchange protocol is used. The 1C website has a list of online store platforms with which it is possible to configure automatic synchronization. Recently there was a download from the site in BP 3.0, but a more detailed setting of the download is implemented in 1C UNF.
To integrate with button services, we do not change existing configurations, but use COM (external connection) technology and external processing. Since the COM-connection technology is outdated and available only on the Windows platform, it plans to redo the integration using the OData mechanism. In detail about the integration methods we recommend to look here .
The developers have made several dozen robots that help to enter documents and control errors, without changing the configuration. Read more in our blog . Written external processing, with which you can quickly handle large retail databases, download data from electronic wallets, write off materials into production and many other interesting things.
Total, we have gone from the use of file databases 1C, to a full-fledged cloud service. 1C Fresh gave us the opportunity to promptly administer about 2500 databases, manage the access of employees and customers, and integrate with external and internal services. Perhaps, the 1C Fresh technology was conceived by the developers for several other uses than in the Button, but after three years, we believe that this is one of the most important tools for our company.
How to choose the main tool for an accountant
If your company is engaged in accounting outsourcing, first of all you need to decide on the program in which the accountant will work. Offers on the market are diverse, we used the following criteria:
- The solution must be scalable. There is a “Do more with less” value in the Button. This means that the growth of the server infrastructure should not be directly proportional to the growth of the client base. Adopting a new system administrator for new 100 customers is uneconomical.
- The solution should have an intuitive interface and be popular with accountants.
- One should not forget about integration with various business solutions and Button applications. The client can keep operational records in various configurations of 1C (for example, Trade Management, Retail, Small Company Management), use online trading platforms, for example, My warehouse, or various solutions for catering establishments (Iiko, StoreHouse and so on.), And also open your website. In any case, the data must be promptly transferred to the accounting system automatically in order not to do double work.
- At any time, it should be possible to promptly transfer the accounting base to the client, so that the new accountant can continue to keep records in it.
Our selection list includes Kontur. Elba, 1C Enterprise Accounting 3.0 and Xero. The decision was made to use 1C Enterprise Accounting 3.0, because Elba is only suitable for small companies, Xero is comfortable and technologically advanced, but still English, it is impossible to set up accounting and tax accounting according to Russian standards.
')
The advantage of 1C products is that at the moment it is the most popular accounting software product in our country. It is difficult to find an accountant who has never worked in this program. Since the Button does not specialize in servicing customers of a particular industry, it is important that the program is suitable for individual entrepreneurs and limited liability companies with any taxation system, and 1C allows it.
Over time, we abandoned the use of file databases and deployed our cloud using 1C Fresh technology, but we will not get ahead of ourselves.
How to reduce the number of bases served
The first customers came to Button in 2013, for each they created a file base BP 3.0. When the number of bases approached a hundred, hell came for the system administrator. It took several days, sometimes nights, to add a new user to all databases or install an update. I had to invent software solutions, inventing updates and processing to add users. It was all too slow and hopeless.
An alternative to file bases was the use of the data sharing mechanism ( Multitenancy ) of the 1C platform. Its essence lies in the fact that one information base 1C is divided into independent areas with the help of separators. The split mode is enabled with the “Split Enabled” constant. By the number of the separator the user connects to a separate area. Within one such information database, data can be unseparated, that is, common to all areas, for example, a classifier of banks or units of measurement, and separated, unique, existing within one area, for example, counterparties and documents.
It is much easier to administer one database instead of hundreds. The cloud service 1cfresh.com works on this principle, and the technology of publishing solutions for 1C Fresh can be purchased from 1C partners.
Delivery 1C Fresh includes two specialized configurations for information database management.
- Service Manager. This is the control center of data areas, through it all the standard admin works are performed. You can load data into a new area (a standard mechanism from the file base forms a dump from files in xml format, which is then loaded into the area), give rights to a list of users, schedule an update, control data synchronization between applications. It is possible to add external reports and processing to specific areas. Also, the so-called “Supplied Data” is updated from the service manager (exchange rates, FIAS, banks classifier).
- Service agent Service configuration designed to run scripts (both predefined and added by the administrator without the need to edit the configuration).
More information about the composition of the delivery of 1C Fresh, we wrote in the article on Infostarte .
When the application service was installed and the first databases were published, the speed of the “cloud accounting” work compared to the file one was about 10 times higher. For the first time in a long time, quarterly reporting was quite easy, since 1C worked without freezing and reloading.
How not to do, or learn from the mistakes of others
A few months later, our Fresh began to work unstably. The reason: mass connection of clients. We had published two accounting bases and two ZUP 3.0 bases, the number of areas in each of which exceeded 300 pieces. The situation was aggravated by the fact that the Fresh cluster was not split up into nodes, all databases were spinning on the same database server. Our hardware could not cope with the load and, as a result, the service manager lost contact with the regions. As the saying goes, neither download currency rates nor synchronize databases. It is good that our colleagues from 1C supported us, penetrated into our specifics and found errors in administration, helped to correctly build the logic of using the cloud.
We advise everyone, before deploying your service, to get systematized knowledge on the course “ Operation of large systems ” at the 1C training center.
The number of areas in one database is a compromise, which is due to many factors, it can be very different, we chose 200 for ourselves. The cluster itself should consist of a service manager, service agent and nodes (database servers, application servers, web services), when the number of regions becomes more than two hundred, the next node unfolds.
When setting up a PostgreSQL server, you should definitely read the 1C recommendations (information is available on kb.1c.ru ) and use for example such a service pgtune.leopard.in.ua . And yes, for PostgreSQL, the most delicious thing is memory!
How we all arranged
Now the architecture of our Fresh-cloud includes 7 nodes, 7 bases 1C BP 3.0, 2 bases 1C UNF and three bases ZUP 3.0 are published. The service is accompanied by five system administrators and two 1C programmers. Such a team is more than enough, with such a composition it is possible to maintain a base of 2000+ clients.
The configurations of our machines are different and change periodically, in response to changing conditions. The application server typically has 4-6 processors (cores) and 16GB of memory, using PostgreSQL 9.4.2-1.1C as the database server, located on Linux machines. Resources for these machines are allocated depending on the size of the bases, the average number of connections and peak loads. If the application server hosts several large databases, resources can reach 6-8 cores and 30 GB of memory.
With the help of Zabbix monitoring, we control the availability of information databases, the load on nodes and 1C servers. There is a TV in the office, and when everything is bad we immediately see it:
Every day, 95 accountants are actively working in the bases. Clients Buttons that want to self-record documents in 1C get access to their database via the Internet. It is possible to connect to the web-client shopping equipment (CMC, barcode scanner). For a thin client, an optional document scanner connection is available. It is very convenient to scan and attach files to documents directly from 1C.
All reporting from us is sent from 1C, through the Kaluga Astral certification center, we use CryptoPro keys. We are actively implementing Dyadok electronic document management, this functionality is also available in the service model.
How to exchange data in the "clouds"
There are several options for data exchange:
- between applications in the service;
- with an external base 1C in the service model;
- with a website or other program for business.
Information base published in a web-service is called an application. For the new client, an area is created in BP 3.0, ZUP 3.0 and, if necessary, the UNF. At the moment, synchronization of data between applications occurs every 10 minutes. Synchronization between applications in the service can be controlled using the service manager, but we brought the data to monitor. The system administrator sees online in which areas the data exchange is stable, in which it does not, and errors need to be corrected.
Many customers of Buttons use 1C solutions for wholesale or retail trade (1C “Trade Management” 10.3 or 11.X, 1C ”Retail”), which they store on their servers and independently administer. In this case, it is also possible to set up data exchange with accounting in our cloud. Such synchronization is called “Data exchange in the service model”. It is configured automatically according to the schedule, but it often “breaks” after the update due to the fact that the client on his side does not update the configuration on time. We control on monitoring, if the exchange has not passed more than two weeks, contact the client.
Now online stores and various online trading platforms are becoming more popular. For the exchange of data from 1C, the CommercelML 2 exchange protocol is used. The 1C website has a list of online store platforms with which it is possible to configure automatic synchronization. Recently there was a download from the site in BP 3.0, but a more detailed setting of the download is implemented in 1C UNF.
Integration with internal services
To integrate with button services, we do not change existing configurations, but use COM (external connection) technology and external processing. Since the COM-connection technology is outdated and available only on the Windows platform, it plans to redo the integration using the OData mechanism. In detail about the integration methods we recommend to look here .
The developers have made several dozen robots that help to enter documents and control errors, without changing the configuration. Read more in our blog . Written external processing, with which you can quickly handle large retail databases, download data from electronic wallets, write off materials into production and many other interesting things.
Conclusion
Total, we have gone from the use of file databases 1C, to a full-fledged cloud service. 1C Fresh gave us the opportunity to promptly administer about 2500 databases, manage the access of employees and customers, and integrate with external and internal services. Perhaps, the 1C Fresh technology was conceived by the developers for several other uses than in the Button, but after three years, we believe that this is one of the most important tools for our company.
Source: https://habr.com/ru/post/324662/
All Articles