Geekly Articles each Day
📜 ⬆️ ⬇️

Lobby VR. Libraries for viewing panoramas 360 *



Hello! VR is now in trend, the topic is topical, various projects and ways to implement the display of three-dimensional images, it seems, will soon be a million. Today our guest is Mikhail Vaisman, the founder of the company for the development of mobile applications Trinity Digital. He volunteered to tell you about working with libraries to display 360-degree panoramas, for which we express our gratitude to him. Okay, you all came here for an interesting experience, and not for content before kat. ;)


What's going on here?


Hi, Habr! They already managed to introduce me, but the rules are rules. I am Mikhail Vaisman. I gained experience working with VR while working at Trinity Digital on the Airpano Virtual Travel project with the Doubble team. Airpano is an application with the ability to view VR photos and video panoramas. In the process of work, I ran into a problem: the VR theme is not the first year in the trend, but there are no ready-made libraries that allow high-quality display of VR content. I had to choose between writing my own solution from scratch and applying what is already on the market. We chose a compromise option - to take the open source solution on the market and refine our goals. The latter option has its advantages, but you should not forget about the minuses. We tried three libraries, and this is what came of it:
')

Krpano - WebView based library


We started working with the version implemented on the basis of WebView - from the krpano library. The reason for choosing were:



In reality, with the krpano library, everything was not so rosy, because krpano has a browser engine. You can start at least with the fact that not all devices have modern chrome engines that can most effectively implement the capabilities of this library. Most of the not-so-high-end devices worked slowly with panoramas solely due to the use of webview, and the library did not work on some of them. As a potential solution, we tried to integrate into the application engines from Chrome and Firefox (first one, then the other - for tests). To some extent this helped. This thing worked everywhere, that's just ...

As a result, the application occupied more than 100 megabytes - and this is not our files or code, but a library, plus a browser engine in the project. The most annoying thing is that even these perversions did not help to significantly optimize the quality and speed of work - they remained at an unacceptably low level. The crucial disadvantage of krpano was the need to program at the level of xml configurations (such configuration takes a long time and increases the development price for each panorama).

PanoramaGL - library based on OpenGL


After a month of attempts (or torture) with krpano, after trying several types of libraries, we stopped at panoramagl - this library worked faster than others, and did not have those critical flaws that we found in krpano. Plus Panoramagl was easier to implement and supported work on weak devices.

But here it was not without flaws: for example, panorama gl was an implementation based only on activity. We had to rewrite it, add support fragments.

In addition to the lack of support for the fragments, panoramagl found many functions that require adaptation and rewriting - in order to support our tasks: for example, the stereo mode to support the cardboard in panoramagl is not implemented. At the same time, it is impossible to integrate the Cardboard SDK into PanoramaGL - due to the fact that the latter uses OpenGL.

But with the help of this, it is enough just to make a basic, not quite correct version of the stereo mode - the picture is simply duplicated on two parts of the screen, without shifting the left part relative to the right.

This is how a correctly implemented stereo mode looks like - the right picture is scrolled relative to the left:



And this is what the stereo mode that we implemented on the basis in the PanoramaGL library looks like - the right picture is identical to the left one, there is no shift between the pictures:



Mindlight library Panframe


Even in the process of finalizing with the panoramagl file, we had the need to add video content to the application. In panoramagl there is no such possibility, and we again faced a dilemma: write the library itself to display video content, or use a ready-made solution. We stopped at the conditionally-paid library Panframe .

We chose it for supporting video and photo panoramas “out of the box”, as well as for good speed of video and photo panoramas. Everything worked fine even on typical "average", not to mention the flagships. Working with her is a pleasure.

Use the library to display the video



1. Specify permission:

<uses-permission android:name="android.permission.INTERNET"/> <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE"/>


2. Initialize the view:

 PFView vrView = PFObjectFactory.view(this); //   .     activity PFView vrView = PFObjectFactory.vr(this); //        .     activity.     PFAsset asset = PFObjectFactory.assetFromUri(this, Uri.parse(filename), this); //   : ,   ,  PFAssetObserver      vrView.displayAsset(asset); //    viewContainer.addView(vrView.getView(), 0); // view  layout. viewContainer — ViewGroup  layout asset.play();//


The argument list is simple:
PFView - view object with 360 content;

PFAssetObserver contains the signature of the onStatusMessage method with two parameters:

PFAsset - content interface with control method signatures:
  • void play ();
  • void stop ();
  • void pause ();
  • float getPlaybackTime ();
  • void setPLaybackTime (float var1);
  • float getDuration ();
  • String getUrl ();
  • PFAssetStatus getStatus ();
  • int getDownloadProgress ();
  • void release ();
  • void setVolume (float var1);


There is also a PFAssetStatus - enum with a set of statuses:
  • LOADED;
  • PLAYING;
  • PAUSED;
  • STOPPED;
  • COMPLETE;
  • DOWNLOADING;
  • DOWNLOADED;
  • DOWNLOADCANCELLED;
  • ERROR;
  • BUFFERING;
  • BUFFERFULL.



Image display example


With panoramic photos, everything is also not bad, just instead of PFAsset, it is enough to use the injectImage (Bitmap image) or injectImageFromResource (int id) method
Moreover, if you set setMode () with type 1, the data will not be displayed:

 vrView = PFObjectFactory.view(this); vrView.injectImageFromResource(R.raw.geysers_1); container.addView(vrView.getView(), 0);


The library also supports animated hotspot:
PFHotspot hp = vrView.createHotspot (BitmapFactory.decodeResource (getResources (), R.raw.hotspot)); //R.raw.hotspot - hotspot image id
hp.setCoordinates (60, 40, 0);
hp.setClickListener (this); // as a parameter, an object is passed that implements the PFHotspotClickListener interface

PFHotspot - interface with method signatures to manage
void setEnabled (boolean var1);
void setSize (float var1);
void setCoordinates (float var1, float var2, float var3); polar coordinates in degrees
void setTag (int var1);
int getTag ();
void setClickListener (PFHotspotClickListener var1);
void animate (); // animates hotspot throttling animation

PFHotspotClickListener - a simple interface with the onclick method with the PFHotspot parameter


Something else


PFView has a setMode method with the following parameters:
mode:
0 - spherical panorama;
1 is a flat image;
2 - for two cameras (for vr it is not necessary to specify separately).

aspect: The aspect ratio of the content for a flat image.

setFormat:
0 - spherical panorama;
1 - top-down formatted spherical panorama

To specify the navigation method (gyroscope or touch), use the vrView.setNavigationMode method with the PFNavigationMode parameter:

MOTION - Gyro Navigation
TOUCH - navigation tachem


Of the obvious disadvantages: the source code of the library is obfuscated - it will be difficult to read and expand the functionality; it is impossible to understand the text of errors. In combination
with slow answers from the library developer (We have been waiting for more than three months to answer the question: “How to add cardboard mode”) - the solution of panframe as the main photo and video library in the application turned out to be difficult in terms of expansion for further operation and product improvement.

Google's VR View


The library for displaying VR content from Google was introduced in March 2016. I followed the development of the library with interest, and at the Google I / O 2016 conference I paid special attention to lectures on the topic of VR, and also talked with Google developers about the library development plans. You can integrate the library into your product as follows.

Download Google VR from this link link:
developers.google.com/vr/android/get-started#using_android_studio
And also pick up the Release notes of the latest library versions:
developers.google.com/vr/android/release-notes#cardboard_sdk_for_android_v070

The Google VR SDK contains several aar libraries:


and several sample projects:


To use the VrPanoramaView in our project, perform the following steps:


Public class VrPanoramaView is inherited from VrWidgetView and contains VrPanoramaRenderer and VrPanoramaEventListener, public class VrWidgetView is inherited from FrameLayout and contains a GL-scene and control buttons.

We also need these guys:


The VrPanoramaRenderer class with packet access is inherited from VrWidgetView and implements the Bitmap download, after which it calls the panowidget.aar panorenderer native arm library.

Public abstract VrWidgetView implements the Renderer interface and calls native methods on Renderer events - the openGl interface with the signatures of the onSurfaceCreated, onSurfaceChanged, onDrawFrame methods.

Thus, you can quickly (in three steps) deploy an application that supports displaying a simple bitmap image with the ability to view in MONO and STEREO mode with the Google interface. However, if you need to implement dynamic content, for example, displaying markers on top of a Bitmap, reading a camera, subscribing to renderer events, then VrPanoramaView does not provide these capabilities, so in order to extend the functionality you will have to edit the panorenderer native library.

For our project in GVR there was not enough support for the transition between panoramas through the labels set on one or another point of the panoramas. VR View developers suggest sending feature requests for the features you need to the library repository on github. The main intrigue is whether google will develop a library one level higher than the MVP and maintain functionality for complex projects. We do not know the answer yet, but already now you can add well-working VR capabilities to your project in 10 minutes, a year ago it was hard to imagine.

findings




The VR library market is not keeping up with demand, but the ice has already started, the team has set off and everything. The process is unstoppable: the developers have become available the first good libraries for technical tasks in VR projects, ready to use out of the box. Below is a brief summary of the pros and cons of these solutions:

Library
pros
Minuses
KRpano
Quick integration into the project;

Cross platform
Slow work on old devices;

Complete failure of work on a number of devices.
PanoramaGL
Relatively fast integration into the project;

Support not only the most basic functionality, but also, for example, the transition between panoramas;

Not a bad speed of panoramas on slow devices.

Long not supported (the last commit was two years ago);

Incorrect stereo mode;

No support for Cardboard SDK.
Panframe
Support 360-degree video panoramas;

Good performance of panoramas on slow devices.
The closeness of the source code;

Conditionally-paid;

Slow / missing support answers.
VR View
Fast integration capability;

Basic MVP capabilities;

Good speed of panoramas on slow devices

Google support;

The ability to leave the feature request to add the necessary features.
The library does not yet support functions beyond MVP

(for example, image overlay on top of a panorama, transition between panoramas).


Conclusions - if you need to quickly integrate VR into your application, whether it is a specific project sharpened for VR, or any other application where it would be interesting to add VR - you can use VR View from Google.

If your project needs support for complex effects and advanced features (image overlay, transitions, etc.) - use the Panframe + Panoramagl configuration, but be prepared for the fact that you have to sweat a little and add these libraries to the needs of your project. The alternative is to wait for the development of GVR. Make the feature request of the functions you need - the likelihood is that they will be added either by Google itself in the next releases, or by third-party developers, Since the library is laid out as an open source project.

That's all, thank you for your attention, if you have questions - ask, I will be happy to answer.

Source: https://habr.com/ru/post/306246/

More articles:


All Articles