Red arrows of extra reality

My readers already know that the manual direction finder and augmented reality are made for each other. A simple background of the video can significantly simplify the work of the operator. Nevertheless, there are people who have difficulty in finding sources of radio-radiation with an amplitude direction finder with a simple video background. Simplification of the process requires a closer integration of the direction finder with the "heavy" additional reality. Despite the emergence of technologies such as ARKit and ARCore, the most accessible remains Google Tango.

Augmented reality for a beautiful picture, so inside not only technical details, but also video clips of the direction finder in one of the most beautiful places in St. Petersburg.

Technology Google Tango many times described on Habré, the best descriptions here and here . In our case, of the number of components of the technology, only the ability to track movements and orientations (Motion Tracking) is used. The possibility of separating flat and other surfaces, we actually do not need. After all, the source, in this case, the WiFi device, can be located behind the wall and at the same time be quite a bearing. Although it is clear that if there is nothing in front of the Tango camera, then she (Tanga - she) cannot track the movements and orientation. Maybe in the future it will be possible to automatically build a 3D map of the surrounding space and save it along with the video as a result of the work, but for now this is fantastic.

It is possible to briefly justify the use of the technology of visual inertial odometry (Visual-Inertial Odometry) for the orientation of the direction finder with their better accuracy compared to inertial systems only. Coordinate and orientation data supplied
devices with Tango are local, they are tied to the starting point. But, within the session, they, in contrast to inertial data, have much greater accuracy.

The WiFi hotspot direction finder described earlier was assigned as the test subject. It is designed to search for illegal access points (rogue AP) and devices of the Internet of things based on WiFi. Although Bluetooth is pushing new technologies, such as Mesh , thanks to the World Forge of All Existing, the use of WiFi-based things is very economically justified. This is confirmed by articles on Habré: here and here .

Direction finder on a regular Android works like this:

If anyone is interested, see the source on github.com/xnzr and various videos . Unfortunately, I am not a professional in the video, and the videos are not very high quality, for which I ask forgiveness in advance. But to show the work of this direction finder is only possible on video.

Let's start with iron. The first Tango tablet was from Google, and it was called Google Tango Tablet. First there was the epic story of getting this miracle to Russia. Officially it was impossible. I was forced to create a fake account on Gmail from Finland, to order in the Google store from Finland to the mailbox in Finland. And I thought it was lucky that I got the device with such a little blood. And you say - sanctions. Then there were still no sanctions, and the curtain was already there. From the other side.

But this is not the end. It turned out that the USB port of this miracle does not pull even the direction finder. I had to invent an external removable power bank to power the direction finder and a special cable for it. It looked awful (there is no direction finder in the photo, only four of its 3M Dual-Lock adhesives are visible):

Well, to hell with emotions. Then everything went well. Work on the software was delayed, as often happens. And soon (Oh, thank you, great Google and its best friend Lenovo) appeared Lenovo Phab 2 Pro! The only available in Russia and, apparently, the latest tablet with Google Tango. The gadget is sturdy, powerful and large, which is important for the direction finder. At the same time, it is cheaper than gadgets with ARCore support, on average, by 10 tr. The direction finder sits on it beautifully:

Therefore, the Google Tango Tablet was placed in sarcophagus Museum of hand-held DR-finders until better times. And may the power bank be with him.

Now about the software. It is, as always, available in source, at github.com/xnzr/xnzr-tango . The point is quite simple: if the threshold for the difference in the levels of two antennas, meaning that the source is close to the normal to the plane of the direction finder, is exceeded, then a red arrow is stuck into the space. These arrows are not erased, but are accumulated in space, which allows not to track the binding of the direction-finding parameter to the terrain by the brain, but to be guided by the arrows hanging in space.

The arrows are displayed by the SagittaStorage and OpenGlCylinder classes.

When the direction finder reports that it is directed to the WiFi source, we request the current “pose” of our Tango device (or rather, not the pose itself, but only the transition matrix):

TangoSupport.TangoMatrixTransformData depthTarea = TangoSupport.getMatrixTransformAtTime(pointCloud.timestamp, TangoPoseData.COORDINATE_FRAME_START_OF_SERVICE, TangoPoseData.COORDINATE_FRAME_CAMERA_DEPTH, TangoSupport.TANGO_SUPPORT_ENGINE_OPENGL, TangoSupport.TANGO_SUPPORT_ENGINE_TANGO, 0); if (depthTarea.statusCode == TangoPoseData.POSE_VALID) { mRenderer.addPeleng(depthTarea.matrix); } 

And we put in CopyOnWriteArrayList (this is a multi-threaded version of ArrayList).

Each frame we in turn apply the saved matrices to the cylinder and draw the cylinder in all saved "poses":

  public void draw(GL10 gl) { for (float[] mm : mModelMatrixList) { mSagittaObject.setModelMatrix(mm); mSagittaObject.draw(gl); } } 

As I wrote above, the only way to show the work of such a direction finder is video. As a WiFi hotspot was my smartphone with the Internet sharing feature enabled. The direction finder accepts Beacon Frame packets and delivers the data to the tablet, which displays them in red arrows of augmented reality (KSDR). Links to test videos are below. The videos were shot in the Summer Garden on a weekday afternoon, so there were not many people who saved me from unnecessary adventures and gave an amazing background with elements of antique nude . You can even watch the kids.

First one
second one
third
and more
and more
and more .

Those who do not like to watch the video, I recommend to see the photo (in full resolution, a large amount of files): 1 , 2 , 3 , 4 , 5 , 6 , 7 , 8 and 9 .

Naturally, it is possible in the software to find the intersection of red arrows and build and display the area of ​​space where the source is most likely located. We can say that today there is not just a new interface with an augmented reality style for the direction finder, but a base for fully automatic search of radio emission sources with display of results in 3D. In the future, it will also be possible to make more vivid the presentation of the results of my phase satellite finder on satellite navigation, described here and here .

I want to note that Mikhail Lukin, who is also Ktator , helped me in the software part (for which I thank him so much), which I hope will continue publishing on Habré on augmented reality for radio direction finders. I will try to focus on the gland.