JavaScript Augmented Reality - JSARToolkit test

JSARToolkit is a JavaScript library ported with FLARToolkit (Flash) and intended to track AR Markers on video. ARToolKit converts data from markers to 3D coordinates, using them you can superimpose images or 3D objects on a flat surface.

You've probably already seen JSARToolkit in action on the Ilmari Heikkinen demo - Remixing Reality .
Ilmari's demo is part of Mozillov's “ Web O 'Wonder ”, a site showing new technologies that will be added to Firefox 4 .

HTML5 Clip Studies

The customer gave us a task - to evaluate the possibility of using JSARToolkit for online HTML5 clips. (We were asked to consider only those users who used the latest version of Firefox and Chrome)
Here are some of the questions we would like to answer:
- Will the processing be fast on slow computers?
- How many AR Markers can we track at the same time?
- How fast can you move the marker so that it becomes not traceable?
- What is the greatest distance at which the camera can track the marker?
')
Answers to these questions, source code and demos can be found below.

Video recording

I used Flip Ultra HD to record a test video. Flip Ultra HD video quality is acceptable, considering that this is not a professional camera. We knew that we would not get anything good from such a camera, but that was enough for a test. The main problem with such a low-quality camera is the inability to switch the shutter speed.

This meant that we could not do anything with blurred AR Tags if we moved too fast. We were surprised at how quickly we lose the marker, when we move from side to side.
However, we are confident that when shooting in a well-lit studio using a camera with a high shutter speed, there will be very few unreadable AR tags.

Printing AR Markers

I typed a few markers that came with JSARToolkit and started shooting test footage in the kitchen. I didn’t even expect that everything will work the first time, every video added to the library turned out to be working.

Some of the results were not qualitative, but I want to repeat that the quality of our camera was poor compared to professional ones. We shot the video without focusing on individual markers, which could add to the accuracy of their tracking.

Recoding video to VP8 WebM

We recorded video in H.264 Mpeg format. For everything to work with HTML5 video, we had to convert the video to WebM. The first video decoder I tried was Ffmpeg2Theora, judging by the name, it should have encoded the video in WebM. However, I realized that there are problems with Ffmpeg2Theora. When I encoded a video under Linux, it could not play in Windows and vice versa.

After the tests of other encoders, I stopped at Miro Video Converter. Unfortunately, Miro cannot handle multiple videos, but the video it produces works on all operating systems and browsers.

Create wrapper

I wanted to write a simple JSARToolkit based API that could be reused. Something that I could connect to another JavaScript library, for example Popcorn.js. The code I found in the Ilmari demo was task specific. Since the code was not full of comments, I had problems understanding what was happening. I had to experiment. I have come to a simple way of writing a wrapper for JSARToolkit.

The first step in using the JSARToolkit wrapper is to install your tracker. An example of how this can be done:

//        var myTracker = jsartoolkit.tracker({ src : 'my-video.webm', //    autoplay : true, //      repeat : true, //   volume : 0, //    target : doc.getElementById('DOMTarget'), // DOM element     canvas width : 720, //   height : 360, //   threshold : 100, //    ratio : 0.5, //    canvas   (1 = 11  ) debug : false //     -     }); 

Once the tracker was created, the next step was to add content to the markers.
We added a static image, and then a 3D object exported from Blender3D:

  //     myTracker.marker(0).image('my-image_01.png'); //   Blender3D myTracker.marker(2).model('HTML5_Logo001'); 

This example shows how to update the properties of the marker after it has been created:

  //    0 myTracker.marker(0) .scale(1) .axis(0, 0, 1) .angle(0) .position(0,0,0) ; 

You can also add more complex behavior using JSARToolkit-Wrapper. The following example demonstrates how to update the properties of a marker in real time. This code causes the first marker to spin and pulsate:

  //    Marker_0   var interval = global.setInterval( function(){ var date = + new Date(), scl = 1.5 + (Math.sin( date/200 ) * 0.5), axs = Math.cos( date/300 ), posX = Math.sin( date/300 ), posY = Math.cos( date/300 ) ; myTracker.marker(0) .scale(scl) .axis(0, axs, 0) .position(posY, posX, 0) .angle(date / 230) ; }, 20); 

To access the video tracker, you can do something like this:

  //        currentTime myTracker.video.currentTime = 1; 

Answers on questions

Will the processing be fast on slow computers?
Video processing and marker positioning is very fast. I almost did not notice the difference in tracking 1 marker and 100 markers. The main load - the imposition of content on the video.

How many AR-Markers can we track at the same time?
I tracked 100 markers at the same time without any problems.

How fast can you move a marker so that it becomes not traceable?
It all depends on your camera and how you shoot. If you use a camera with a high shutter speed, such as those used to record sports, we will get a minimum of blurry markers (if we get at all) and they will be tracked very well.

What is the longest distance a camera can track a marker?
Again, everything depends on several factors - on the speed of movement of the camera / objects and on the lighting. In a well-lit room (not a professional studio). I managed to track Markers from 10 meters in the direction of the lens in a resolution of 720p. The higher the resolution of the camera, the higher the quality of tracking markers. One thing that is worth noting: you can shoot your video in 1080 resolution, cache the tracking results and reduce the number of processing on the client side. You can rotate the ratio if something is poorly tracked or the threshold if the frame was poorly lit.

Conclusion

pros

- Easy to implement
- Optimum tracking algorithm, does not load the processor
- You can track at least 100 markers
- Can be exported directly from Blender3D
- You can overlay any content: pictures, videos, 3D objects

Minuses

- Too many global variables in source library code
- It is necessary to tighten up export from Blender3D a little
- Simultaneous processing of multiple videos is not supported.
- Too many costly getElementById () calls

You need to do a great job if you want to use this library on a production or make it part of a large library. But after a huge number of tests and experiments, I will say that this code works amazingly!

Examples, Tests and references

JSARToolkit-Wrapper Demo
JSARToolkit Video Tests
JSARToolkit Marker Images
JSARToolkit-Wrapper on Github

You need the latest version of Firefox 4 or Chrome to view.

From translator

The author unfortunately has not yet posted a demo. The use of AR in the form in which it is presented in translation is not very broad, but with the advent of HTML5 Device Element ( another article ), everything can change.
AR Soon and in your browser;)