Go application in production systems. Validators
Good afternoon, Habr!
I often start the morning with viewing Habr and finally decided to contribute to this interesting learning process. If everything works out, then this is my first article from the cycle of using the GO language on production systems. I want to tell you some of the subtleties of creating applications and servers, the convenience of the language and the speed of development on it. Perhaps, for professionals, this article will seem boring and not interesting, but reading the literature I did not find the general picture of the solution of production problems. Although most problems are solved the same way. In this article I will explain the general principles of building a server and, as an example, I will use the server to validate and view the postal codes of Post Russia. This article will highlight more general, methodological problems and subtleties that I encountered while creating this system. I will not describe the general settings, there are many of these in the network, I just want to focus on the little things that prevented the launch of the project.
Reading Habr, I often faced the task of validation - checking the correctness of user input. Each author solved this problem in his own way. I used external service or wrote my server. I propose another solution to this problem - the creation of a directory service. Directories are servers that contain information of a certain direction and which can be separated into a separate, independent subsystem. Typically, reference books are used to assist the user in completing and validating the information entered. As an example of such reference books, one can cite the services of requests for regions, requests for cities, KLADR, postal codes, directories of GAIS (state automated information systems). Almost all existing systems exchange REST data using either json or soap. I will try to tell a template for the development of such directories that will allow you to quickly create such systems. And put on github the sources of my internal project.
')
And so let's start in order. Our current system is built on products from the company 1C, namely the BEADS and the corporate portal. To unify the OS, we have chosen from the recommended installation for 1C - namely, OS Linux CentOS. Launch scripts of 1C web environment are run on this OS. Windows OS was not considered. I can not say that I like the choice, I would choose debian, but it happened. Unification was necessary for us, because historically we have a large zoo OS, various builds of OS Linux, OS FreeBSD, OS VxWorks. And, in my opinion, the fastest Linux OS CRUX.
How many projects created by the system on 1C has grown into a large portal and it became necessary to allocate from the system and / or supplement the system with various services - directories. Analyzing the offers and possibilities of the already created services, as well as our needs in these reference books, we came to the conclusion that the reference book:
According to the above criteria and its ease of entry, Go was chosen to write reference books. Additionally:
That is, Go is well suited for creating directories. From myself I add, php and Go are a bit similar, which allows you to maintain systems written in these languages. Also, Go already has a large number of solutions, which allows you to quickly build these solutions for your tasks, that is, as in the designer of the cubes to assemble a model of the house. And so we proceed to practice.
I want to draw your attention, Go is divided into two categories: the first one is the compilation language itself and the libraries included in the build ( GOROOT ), the second is the additional libraries that you put and where you will create your project ( GOPATH ). Setting up the environment and preparing the development folder, I will use / home / gouser / . Add / etc / profile or to custom .profile .
Prepare a folder for your project:
Then everything is simple for your project, create a folder in src, create files with the go extension and compile the project:
Both ready examples and libraries can be found at libs.garden . Installing Found Examples
After installing Go, we will proceed to the analysis of the example. Download and install to compile it can be commands:
It is necessary to create a reference book for working with postal codes by Russian post, which will solve the following tasks:
First you need to initialize the database, and then create a query server for this database. Need a database of postal codes. The search is vinfo.russianpost.ru/database/ops.html . Base in FoxPro and in zip archive. The database contains the following accounting objects:
For initialization on libs.garden we find components and examples of working with database and zip. We take httprouter as a request router . We connect to the project.
When examining the database file, it was found that not all accounting objects are specified , that is, there are zero values ​​in the sample.
Accounting objects stand in the hierarchy when the database is initialized, if the accounting object is zero, then the name is taken from the parent. Server ideology - requests come to main (here a request router) and are redirected to handlers (here all data checks, conversions, etc.). All requests to the database and calculations are made from handlers and the handlers carry out the output of the information received. This decomposition allows you to separate queries in the database, calculations, verification of the entered information and the output of the found. In principle, everything.
The functional guide turned out to be more complicated than KLADR . It not only allows you to create a validator or directory on the site, but also to create an algorithm for regional business.
If there is a distributed regional business selling services or goods. Dealers in the regions are registered on the portal, indicate the area where they will provide the service or sell the goods and they are assigned an array of postal codes. When requesting a service or product, the client indicates the postal code, and his application is sent to his regional dealer (here you can build a complex algorithm). The list of REST requests is on github in the project documentation.
This handbook can be connected to any project or framework, but since we have bitrix, I put a module to add a validator to web forms of standard components.
PS Server installation is specially divided into 3 stages. This is done due to the obsolescence of the site’s vinfo.russianpost.ru site and due to licensing restrictions. The base itself is not my property; using it you accept the licensing policy above the voiced site.
PSS Servers created in my projects for the corporate segment and under the conditions should not go online. The code can be modified for your specific tasks. We do not use external access for our projects. The following projects that I would like to describe are a full-fledged KLADR and a streaming radio protocol analyzer. Next Bitrix24 + asterisk. Please comment on who is more interesting.
I often start the morning with viewing Habr and finally decided to contribute to this interesting learning process. If everything works out, then this is my first article from the cycle of using the GO language on production systems. I want to tell you some of the subtleties of creating applications and servers, the convenience of the language and the speed of development on it. Perhaps, for professionals, this article will seem boring and not interesting, but reading the literature I did not find the general picture of the solution of production problems. Although most problems are solved the same way. In this article I will explain the general principles of building a server and, as an example, I will use the server to validate and view the postal codes of Post Russia. This article will highlight more general, methodological problems and subtleties that I encountered while creating this system. I will not describe the general settings, there are many of these in the network, I just want to focus on the little things that prevented the launch of the project.
Instead of intro
Reading Habr, I often faced the task of validation - checking the correctness of user input. Each author solved this problem in his own way. I used external service or wrote my server. I propose another solution to this problem - the creation of a directory service. Directories are servers that contain information of a certain direction and which can be separated into a separate, independent subsystem. Typically, reference books are used to assist the user in completing and validating the information entered. As an example of such reference books, one can cite the services of requests for regions, requests for cities, KLADR, postal codes, directories of GAIS (state automated information systems). Almost all existing systems exchange REST data using either json or soap. I will try to tell a template for the development of such directories that will allow you to quickly create such systems. And put on github the sources of my internal project.
')
The choice of systems. Immediately the question arises why GO? Why Linux? And which Linux?
And so let's start in order. Our current system is built on products from the company 1C, namely the BEADS and the corporate portal. To unify the OS, we have chosen from the recommended installation for 1C - namely, OS Linux CentOS. Launch scripts of 1C web environment are run on this OS. Windows OS was not considered. I can not say that I like the choice, I would choose debian, but it happened. Unification was necessary for us, because historically we have a large zoo OS, various builds of OS Linux, OS FreeBSD, OS VxWorks. And, in my opinion, the fastest Linux OS CRUX.
How many projects created by the system on 1C has grown into a large portal and it became necessary to allocate from the system and / or supplement the system with various services - directories. Analyzing the offers and possibilities of the already created services, as well as our needs in these reference books, we came to the conclusion that the reference book:
- Standalone server service;
- Data exchange occurs via REST;
- Has the ability to scale horizontally;
- Must quickly give data, albeit redundant, processing on 1C products;
- Quickly rebuild logic depending on new business tasks;
According to the above criteria and its ease of entry, Go was chosen to write reference books. Additionally:
- Go language with the compiler and allows you to include code snippets in C;
- The program is a single file with all libraries (there are no problems with the compatibility of libraries after assembly);
- a large community (many examples and ready-made solutions);
That is, Go is well suited for creating directories. From myself I add, php and Go are a bit similar, which allows you to maintain systems written in these languages. Also, Go already has a large number of solutions, which allows you to quickly build these solutions for your tasks, that is, as in the designer of the cubes to assemble a model of the house. And so we proceed to practice.
GO. Installing and searching for libraries or solutions
wget https://dl.google.com/go/go1.12.1.linux-amd64.tar.gz tar -C /usr/local -xvzf go1.12.1.linux-amd64.tar.gz I want to draw your attention, Go is divided into two categories: the first one is the compilation language itself and the libraries included in the build ( GOROOT ), the second is the additional libraries that you put and where you will create your project ( GOPATH ). Setting up the environment and preparing the development folder, I will use / home / gouser / . Add / etc / profile or to custom .profile .
export GOPATH=/home/gouser export GOROOT= /user/local/go/ export PATH=$PATH:/usr/local/go/bin Prepare a folder for your project:
mkdir -p /home/gouser/{bin,pkg,src} Then everything is simple for your project, create a folder in src, create files with the go extension and compile the project:
go build Recommendations for creating a project. General recommendations
- For a project on github, it is more understandable to create a folder for the project src / github.com / <project name>;
- to draft a project in the form of a package, to comment on each procedure — a way for very conveniently getting ready program documentation in godoc;
- Global variables to put in a separate module and connect it if necessary (like a template alone);
Both ready examples and libraries can be found at libs.garden . Installing Found Examples
go get < > , github.com/labstack/echo PostCode. Link to the project
After installing Go, we will proceed to the analysis of the example. Download and install to compile it can be commands:
go get github.com/julienschmidt/httprouter go get github.com/LindsayBradford/go-dbf/godbf go get github.com/go-sql-driver/mysql go get github.com/julienschmidt/httprouter cd /home/gouser/go/src/github.com/ git clone https://github.com/Theo730/postcode.git postcode Formulation of the problem
It is necessary to create a reference book for working with postal codes by Russian post, which will solve the following tasks:
- Validation of postal codes;
- The possibility of obtaining a list of all accounting objects (regions, hotels, etc.);
- Getting all the indices in the object of registration of cities, districts, regions ...
- From the above, exchange REST data in the form of json.
Research task
First you need to initialize the database, and then create a query server for this database. Need a database of postal codes. The search is vinfo.russianpost.ru/database/ops.html . Base in FoxPro and in zip archive. The database contains the following accounting objects:
- region;
- autonomous obla;
- area;
- locality;
- postcode.
For initialization on libs.garden we find components and examples of working with database and zip. We take httprouter as a request router . We connect to the project.
When examining the database file, it was found that not all accounting objects are specified , that is, there are zero values ​​in the sample.
Solution and implementation
Accounting objects stand in the hierarchy when the database is initialized, if the accounting object is zero, then the name is taken from the parent. Server ideology - requests come to main (here a request router) and are redirected to handlers (here all data checks, conversions, etc.). All requests to the database and calculations are made from handlers and the handlers carry out the output of the information received. This decomposition allows you to separate queries in the database, calculations, verification of the entered information and the output of the found. In principle, everything.
Instead of the total
The functional guide turned out to be more complicated than KLADR . It not only allows you to create a validator or directory on the site, but also to create an algorithm for regional business.
If there is a distributed regional business selling services or goods. Dealers in the regions are registered on the portal, indicate the area where they will provide the service or sell the goods and they are assigned an array of postal codes. When requesting a service or product, the client indicates the postal code, and his application is sent to his regional dealer (here you can build a complex algorithm). The list of REST requests is on github in the project documentation.
And a little bitrix
This handbook can be connected to any project or framework, but since we have bitrix, I put a module to add a validator to web forms of standard components.
PS Server installation is specially divided into 3 stages. This is done due to the obsolescence of the site’s vinfo.russianpost.ru site and due to licensing restrictions. The base itself is not my property; using it you accept the licensing policy above the voiced site.
PSS Servers created in my projects for the corporate segment and under the conditions should not go online. The code can be modified for your specific tasks. We do not use external access for our projects. The following projects that I would like to describe are a full-fledged KLADR and a streaming radio protocol analyzer. Next Bitrix24 + asterisk. Please comment on who is more interesting.
Source: https://habr.com/ru/post/445044/
All Articles