Origami and augmented reality (continuation 2)
Earlier we considered these issues of using augmented reality to create origami:
This article focuses on the technical details of using barcodes to generate hints in augmented reality.
Suppose we have a virtual origami model, a real sheet of paper and a webcam. On the sheet image received from the camera, the program should draw a hint on the current step of folding the model, for example, an arrow from one corner of the sheet to another.
To create a model, you need someone who initially folded origami using a computer, you can read more here: all the components .
The problem is to correctly determine where our objects are on the sheet. The sheet can be arbitrarily oriented and have an arbitrary shape at some step. All the details of the folded monochrome sheet of a conventional web-camera "see" is almost impossible. The use of a color printer for marking areas of a sheet is not available to everyone.
')
Barcodes are placed on certain areas of the sheet. The program finds them, easily identifies, determines the orientation of the sheet and the required offset to the objects that make up the tooltip. How are we going to have barcodes on the sheet? Consider the following factors:
To fulfill the first point, it is sufficient to use such useful information obtained from the model as the triangulation of an area. Since folds can only pass along the borders of triangles, it is sufficient to store barcodes inside them, and preferably with some indentation from the edge. We can always enter a circle in a triangle, easily find its center and radius, and place a barcode inside it.
The sizes of the triangles included in the partitioning area impose a limit on the maximum size of the barcode. Of course, it should not occupy a half-sheet, but at the same time it should be visible to the camera. To fulfill the second requirement, we also need information about the visibility of the triangles at each step. Fortunately, it is also contained in the model.
For the last item, a frequency dictionary of the appearance of triangles in sight is also required. It is logical to place the barcodes in the most common triangles and reuse, then their number will decrease dramatically.
At each step, we give each triangle an estimate that meets our requirements, and choose the one that received the best one. Each criterion has an adjustable degree of influence (weight), so that the system can be optimally adjusted.
It is also possible to choose to print from one or both sides of the sheet, it depends on whether we can see a hint from any side, or only from a certain one. Barcodes encode their unique identifiers, and they are used as a key to search for information about the prompt (type of action, offset to objects, and so on). Despite all the precautions taken, the barcode can still be somewhat damaged, so Reed-Solomon coding is used, which allows to restore the original data in case of partial damage.
About tags and orientation in space (in our opinion, the most interesting detail) we will tell in the next article.
This article focuses on the technical details of using barcodes to generate hints in augmented reality.
Task
Suppose we have a virtual origami model, a real sheet of paper and a webcam. On the sheet image received from the camera, the program should draw a hint on the current step of folding the model, for example, an arrow from one corner of the sheet to another.
To create a model, you need someone who initially folded origami using a computer, you can read more here: all the components .
Restrictions
The problem is to correctly determine where our objects are on the sheet. The sheet can be arbitrarily oriented and have an arbitrary shape at some step. All the details of the folded monochrome sheet of a conventional web-camera "see" is almost impossible. The use of a color printer for marking areas of a sheet is not available to everyone.
')
Appearance of the solution
Barcodes are placed on certain areas of the sheet. The program finds them, easily identifies, determines the orientation of the sheet and the required offset to the objects that make up the tooltip. How are we going to have barcodes on the sheet? Consider the following factors:
- the barcode should not lie on the lines of the folds, as it can jam and be damaged;
- the barcode must be visible at all steps of folding the model in which it is used to display the tooltip;
- the barcode should be located at a sufficiently large distance from the objects of the hint, since the user will most likely take hold of them with his hands to fold and close it;
- At the same time, the barcode should not be too far, so that the errors in the orientation determination do not strongly influence the points obtained after the shift;
- barcode should not be too large or too small;
- the less they are, the better, the paint is worth the money.
To fulfill the first point, it is sufficient to use such useful information obtained from the model as the triangulation of an area. Since folds can only pass along the borders of triangles, it is sufficient to store barcodes inside them, and preferably with some indentation from the edge. We can always enter a circle in a triangle, easily find its center and radius, and place a barcode inside it.
The sizes of the triangles included in the partitioning area impose a limit on the maximum size of the barcode. Of course, it should not occupy a half-sheet, but at the same time it should be visible to the camera. To fulfill the second requirement, we also need information about the visibility of the triangles at each step. Fortunately, it is also contained in the model.
For the last item, a frequency dictionary of the appearance of triangles in sight is also required. It is logical to place the barcodes in the most common triangles and reuse, then their number will decrease dramatically.
At each step, we give each triangle an estimate that meets our requirements, and choose the one that received the best one. Each criterion has an adjustable degree of influence (weight), so that the system can be optimally adjusted.
It is also possible to choose to print from one or both sides of the sheet, it depends on whether we can see a hint from any side, or only from a certain one. Barcodes encode their unique identifiers, and they are used as a key to search for information about the prompt (type of action, offset to objects, and so on). Despite all the precautions taken, the barcode can still be somewhat damaged, so Reed-Solomon coding is used, which allows to restore the original data in case of partial damage.
About tags and orientation in space (in our opinion, the most interesting detail) we will tell in the next article.
Source: https://habr.com/ru/post/117520/
All Articles