Recognition of guilloche elements on the example of the passport of the Russian Federation
Guilloche is a special technology for the protection of banknotes, documents, securities and other types of printed products (tickets, excise stamps, certificates and many other documents on a national scale).
Document protection is provided by applying to the forms of complex compositions of various guilloche elements. The guilloche element is an intricate pattern of a set of repeatedly intersecting the thinnest lace lines ( Figure 1 ). Typically, these elements are represented by various kinds of protective nets, rosettes, borders, vignettes and corners. Guilloche can be both symmetrical and asymmetrical in its design.
According to existing regulations, guilloche elements should occupy at least 70% of the area of ​​securities.
Moreover, most of this area should contain multi-color guilloche compositions.
')
a) a banknote in denominations of 5 rubles of the sample of 1997 b) image of the passport page of a citizen of the Russian Federation
Original image
When recognizing document images, one often has to deal with various guilloche elements. At the same time, being usually a part of the background, the guilloche only complicates the process of recognizing meaningful information (complicating the algorithm of “suppressing” the background, which is an integral part of any recognition system). However, in a number of tasks, guilloche elements themselves are significant objects and must be recognized.
For example, such a situation arises when recognizing the pages of a citizen of the Russian Federation passport ( Figure 1b ). For additional protection, the passport page number is printed as a guilloche rosette ( Figure 1B ). However, when setting up algorithms for recognition of such graphics, a number of difficulties arise. When trying to build a feature vector of a recognizable object using a high resolution image, due to the peculiarities of guilloche elements (which, we recall, are a drawing of the thinnest lines), its value will change significantly with minor geometric distortions (such as, for example, shift or rotation).
On the other hand, an attempt to construct a feature vector in a lower resolution image is unsuccessful due to the lack of contrast of the object being recognized. In addition, additional difficulties for recognition create possible additional interference. For example, the passport page numbers, which are already poorly recognizable, are often overlapped by various stamps and seals.
In this article we will tell readers about the technique of recognizing guilloche elements on the example of the pages of a citizen of the Russian Federation passport. The technique described below, after minor modifications, can be used to process and recognize guilloche applied to other documents ( Figure 2 ).
a) foreign passport of a citizen of the Russian Federation b) Identity card of a soldier of the Russian Federation
Original image
So, on the image of the passport page you need to determine the number. In this case, the exact orientation of the page is unknown, and there may also be interference in the form of stamps and seals ( Figure 3a ).
Immediately before recognition, it is necessary to “prepare” the original image: suppress noise and determine the correct orientation of the image. Note that both recognizable objects (passport page numbers) and possible interference are mostly representable as a composition of lines, the geometric characteristics of which are known (in our case only thickness is important). When working with such objects, morphological filtering is often used.
Morphology (mathematical morphology) is a tool for selecting and analyzing graphic elements with a known geometric structure in an image. The main morphological operations are dilatation and erosion. Dilatation of the image f (x, y) along a rectangular structural element e = {(x, y): | x | ≤a, | y | ≤b} is denoted by f
e and is defined as follows:
(f e) (x, y) = max {f (x + s, y + t)}
| s | ≤a, | t | ≤b
Similarly, erosion of the image f (x, y) by the structural element e is denoted by f
e and is defined as follows:
(f e) (x, y) = min {f (x + s, y + t)}
| s | ≤a, | t | ≤b
Consider two more important morphological operations: opening and closing. The opening of the image f (x, y) by the structural element e is denoted by f
e and is determined by the equality f e = (f e) e. Similarly, the closure of the image f (x, y) by the primitive e is denoted by f 
e is defined as f e = (f e) e. The closure smoothes the contours of the object, "breaks" the narrow necks and eliminates the protrusions of a small width. The opening also tends to smooth out the contour sections, but, unlike the opening, "floods" narrow gaps and long indentations of small width, as well as fills the contour gaps.
Let the characteristic thickness of the lace lines of the guilloche t g , the characteristic distance between the two lines of the guilloche d g and the thickness of the lines of the stamps t s . Note that for the pages of the passport the following relationship holds true:
t g <d g <t s
Step 1. Using the opening operation, we merge the lines forming the guilloche with each other. This will increase the contrast of recognizable objects, while practically does not change the type of interference (since d g <t s ). We introduce the structural element e 1 = {(x, y): | x | ≤d g , | y | ≤d g }. Then the contrast is set as follows ( Figure 3b ):
f 1 (x, y) = (f e 1 ) (x, y)
Step 2. Knowing the thickness of the lines of the stamps, with the help of the circuit we suppress possible interference. We introduce the structural elements for “removing” the horizontal and vertical lines of stamps as e 2 h = {(x, y): | x | ≤0, | y | ≤t s } and e 2 v = {(x, y): | x | ≤t s , | y | ≤0}, respectively. Then the background image (that is, the image of the passport page “without interference”) is obtained as follows ( Figure 3c ):
f 2h (x, y) = (f 1 e 2h ) (x, y)
f 2v (x, y) = (f 1 e 2v ) (x, y)
f 2 (x, y) = max {f 2h (x, y), f 2v (x, y)}
Step 2 *. After receiving the background image as a bonus, it is easy to obtain an image f 2 * (x, y) containing only noise ( Figure 3c ). It is obtained by subtracting the background from the original image and can be useful for analyzing the filling of the page (are there any stamps on the current passport page).
After removing the noise image is considered to be prepared to determine the orientation and page number. Note that the preferential orientation of the page can be found by detecting and analyzing the location of the corner page mark ( Figure 3e ), which for the RF passport should always be at the bottom. Note that the detection algorithm must be resistant to minor object distortion and noise.
In addition, the adjustment of the algorithm should be carried out on a small training set. Perhaps one of the most well-known algorithms that satisfy these requirements is the method of Viola and Jones.
a) the original image (upside down); b) morphological filtration of the “opening” type; c) selected background image; d) selected image of seals; e) determining the orientation of the page; e) recognition of the page number
The algorithm for detecting objects from Viola and Jones was originally designed to search for faces in real-time images. He considers the problem of detection as a problem of binary classification at each point, that is, for each possible shift in the image with the help of a previously trained recognizer (classifier) ​​the hypothesis of the presence of the desired object in this position is checked. Key features of the algorithm are:
The same algorithm is also used to recognize the page number of the passport through preliminary training of individual classifiers for each guilloche rosette corresponding to the page numbers.
Step 3. Using a pre-trained classifier, we determine the possible location of the corner marks on the background image f 2 (x, y). If the found label is located on top, then rotate the image 180 degrees around the center ( Figure 3d ).
Step 4. Having determined the orientation of the page and having rotated the image if necessary, we apply each of the trained classifiers of guilloche rosette numbers to the background image f 2 (x, y). The categorizer, which provides a greater number of positives in the central area of ​​the image, will determine the desired page number ( Figure 3e ).
To test the algorithm described above, two data sets were prepared. The first one was used to train the recognizers of the angular elements of the passport page numbers (they consisted of 333 images, that is, 37 different images of each page of the passport). The second data set was used to check the performance and assess the quality of detection and consisted of 616 different images (the distribution by page types is uneven, exact information is given in Table 1 ).
As a result, the average quality of detection was 92.86%. Note that many errors were obtained when recognizing page 5 (despite the larger training sample compared to other pages). This is primarily due to the dense filling. Some of the unrecognized pages are also difficult to recognize even by humans ( Figure 4 ).
a) the original image; b) background image
Document protection is provided by applying to the forms of complex compositions of various guilloche elements. The guilloche element is an intricate pattern of a set of repeatedly intersecting the thinnest lace lines ( Figure 1 ). Typically, these elements are represented by various kinds of protective nets, rosettes, borders, vignettes and corners. Guilloche can be both symmetrical and asymmetrical in its design.
According to existing regulations, guilloche elements should occupy at least 70% of the area of ​​securities.
Moreover, most of this area should contain multi-color guilloche compositions.
')
Picture 1
Examples of documents containing guilloche elements:a) a banknote in denominations of 5 rubles of the sample of 1997 b) image of the passport page of a citizen of the Russian Federation
Original image
(3 158.74 KB, PNG-24)
When recognizing document images, one often has to deal with various guilloche elements. At the same time, being usually a part of the background, the guilloche only complicates the process of recognizing meaningful information (complicating the algorithm of “suppressing” the background, which is an integral part of any recognition system). However, in a number of tasks, guilloche elements themselves are significant objects and must be recognized.
For example, such a situation arises when recognizing the pages of a citizen of the Russian Federation passport ( Figure 1b ). For additional protection, the passport page number is printed as a guilloche rosette ( Figure 1B ). However, when setting up algorithms for recognition of such graphics, a number of difficulties arise. When trying to build a feature vector of a recognizable object using a high resolution image, due to the peculiarities of guilloche elements (which, we recall, are a drawing of the thinnest lines), its value will change significantly with minor geometric distortions (such as, for example, shift or rotation).
On the other hand, an attempt to construct a feature vector in a lower resolution image is unsuccessful due to the lack of contrast of the object being recognized. In addition, additional difficulties for recognition create possible additional interference. For example, the passport page numbers, which are already poorly recognizable, are often overlapped by various stamps and seals.
In this article we will tell readers about the technique of recognizing guilloche elements on the example of the pages of a citizen of the Russian Federation passport. The technique described below, after minor modifications, can be used to process and recognize guilloche applied to other documents ( Figure 2 ).
Figure 2
Examples of documents containing guilloche elements:a) foreign passport of a citizen of the Russian Federation b) Identity card of a soldier of the Russian Federation
Original image
(3,638.15 KB, PNG-24)
Passport Page Recognition Algorithm
So, on the image of the passport page you need to determine the number. In this case, the exact orientation of the page is unknown, and there may also be interference in the form of stamps and seals ( Figure 3a ).
Immediately before recognition, it is necessary to “prepare” the original image: suppress noise and determine the correct orientation of the image. Note that both recognizable objects (passport page numbers) and possible interference are mostly representable as a composition of lines, the geometric characteristics of which are known (in our case only thickness is important). When working with such objects, morphological filtering is often used.
Morphology (mathematical morphology) is a tool for selecting and analyzing graphic elements with a known geometric structure in an image. The main morphological operations are dilatation and erosion. Dilatation of the image f (x, y) along a rectangular structural element e = {(x, y): | x | ≤a, | y | ≤b} is denoted by f
e and is defined as follows:
(f
e) (x, y) = max {f (x + s, y + t)}
| s | ≤a, | t | ≤b
Similarly, erosion of the image f (x, y) by the structural element e is denoted by f
e and is defined as follows:
(f
e) (x, y) = min {f (x + s, y + t)}
| s | ≤a, | t | ≤b
Consider two more important morphological operations: opening and closing. The opening of the image f (x, y) by the structural element e is denoted by f
e and is determined by the equality f e = (f e) e. Similarly, the closure of the image f (x, y) by the primitive e is denoted by f e is defined as f e = (f e) e. The closure smoothes the contours of the object, "breaks" the narrow necks and eliminates the protrusions of a small width. The opening also tends to smooth out the contour sections, but, unlike the opening, "floods" narrow gaps and long indentations of small width, as well as fills the contour gaps.
Let the characteristic thickness of the lace lines of the guilloche t g , the characteristic distance between the two lines of the guilloche d g and the thickness of the lines of the stamps t s . Note that for the pages of the passport the following relationship holds true:
t g <d g <t s
Step 1. Using the opening operation, we merge the lines forming the guilloche with each other. This will increase the contrast of recognizable objects, while practically does not change the type of interference (since d g <t s ). We introduce the structural element e 1 = {(x, y): | x | ≤d g , | y | ≤d g }. Then the contrast is set as follows ( Figure 3b ):
f 1 (x, y) = (f
e 1 ) (x, y)
Step 2. Knowing the thickness of the lines of the stamps, with the help of the circuit we suppress possible interference. We introduce the structural elements for “removing” the horizontal and vertical lines of stamps as e 2 h = {(x, y): | x | ≤0, | y | ≤t s } and e 2 v = {(x, y): | x | ≤t s , | y | ≤0}, respectively. Then the background image (that is, the image of the passport page “without interference”) is obtained as follows ( Figure 3c ):
f 2h (x, y) = (f 1
e 2h ) (x, y)
f 2v (x, y) = (f 1
e 2v ) (x, y)
f 2 (x, y) = max {f 2h (x, y), f 2v (x, y)}
Step 2 *. After receiving the background image as a bonus, it is easy to obtain an image f 2 * (x, y) containing only noise ( Figure 3c ). It is obtained by subtracting the background from the original image and can be useful for analyzing the filling of the page (are there any stamps on the current passport page).
After removing the noise image is considered to be prepared to determine the orientation and page number. Note that the preferential orientation of the page can be found by detecting and analyzing the location of the corner page mark ( Figure 3e ), which for the RF passport should always be at the bottom. Note that the detection algorithm must be resistant to minor object distortion and noise.
In addition, the adjustment of the algorithm should be carried out on a small training set. Perhaps one of the most well-known algorithms that satisfy these requirements is the method of Viola and Jones.
Figure 3
Recognition of the passport page of a citizen of the Russian Federation:a) the original image (upside down); b) morphological filtration of the “opening” type; c) selected background image; d) selected image of seals; e) determining the orientation of the page; e) recognition of the page number
The algorithm for detecting objects from Viola and Jones was originally designed to search for faces in real-time images. He considers the problem of detection as a problem of binary classification at each point, that is, for each possible shift in the image with the help of a previously trained recognizer (classifier) ​​the hypothesis of the presence of the desired object in this position is checked. Key features of the algorithm are:
- The use of rectangular Haar signs as basic features when building a classifier.
- The use of the so-called image integral, which allows you to quickly calculate the elementary features.
- Using an effective AdaBoost machine learning algorithm for training classifiers.
- Using the cascade structure of classifiers in order to reduce the total image processing time.
The same algorithm is also used to recognize the page number of the passport through preliminary training of individual classifiers for each guilloche rosette corresponding to the page numbers.
Step 3. Using a pre-trained classifier, we determine the possible location of the corner marks on the background image f 2 (x, y). If the found label is located on top, then rotate the image 180 degrees around the center ( Figure 3d ).
Step 4. Having determined the orientation of the page and having rotated the image if necessary, we apply each of the trained classifiers of guilloche rosette numbers to the background image f 2 (x, y). The categorizer, which provides a greater number of positives in the central area of ​​the image, will determine the desired page number ( Figure 3e ).
Experimental Results
To test the algorithm described above, two data sets were prepared. The first one was used to train the recognizers of the angular elements of the passport page numbers (they consisted of 333 images, that is, 37 different images of each page of the passport). The second data set was used to check the performance and assess the quality of detection and consisted of 616 different images (the distribution by page types is uneven, exact information is given in Table 1 ).
Table 1. Recognition statistics.
| Number of images | Detection quality | |
|---|---|---|
| Page four | 34 | 88.86% |
| Page five | 129 | 89.15% |
| Page 6 | 90 | 91.11% |
| Page 7 | 88 | 97.73% |
| Page eight | 58 | 91.38% |
| Page 9 | 59 | 94.92% |
| Page ten | 57 | 91.23% |
| Page eleven | 53 | 100.00% |
| Page 12 | 48 | 91.67% |
| Total | 616 | 92.86% |
Figure 4
Examples of unrecognizable images of passport pages:a) the original image; b) background image
The full text of the article is published in the collection of works of the Institute of System Analysis of the Russian Academy of Sciences. Processing information and graphic resources. 2013. Vol. 63, Issue 3. P. 106–110. “Recognition of guilloche elements: the definition of the pages of the passport of the Russian Federation // Usilin SA, Nikolaev DP, Sholomov DL, Arlazarov VV
Recommended literature
- GOST R 54109-2010. Protective technology. Printing products protected. General technical requirements. 2010
- Gonzalez R., Woods R. Digital Image Processing. 2006. p. 1072.
- Soille P. Morphological Image Analysis: Principles and Applications // Morphological Image Analysis Principles and Applications. Springer-Verlag, 1999. p. 391.
- Viola P., Jones M. Robust Real-time Object Detection // International Journal of Computer Vision. 2002
- Freund Y., Schapire RE A decision-theoretic generalization of online learning // Journal of Computer and System Sciences. 1997. Vol. 55, No. 1. P. 119–139.
Source: https://habr.com/ru/post/204658/
All Articles