Unlimited XR technologies in the era of distributed computing

How the transformation of Wireless Edge will help in the development of augmented reality photo-realistic mobile systems.

Extended Reality (XR) already gives users revolutionary opportunities, but to achieve even greater realism and a new level of immersion, given the limitations associated with the performance and cooling of thin portable gadgets, is a rather trivial task.

Looking to the future: slim and stylish augmented reality glasses

With the transformation of Wireless Edge systems (wireless systems operating at the junction of the network and the gadget), a new era of distributed computing will begin, in which 5G technologies will be actively used, information processing on the devices themselves and edge cloud computing. And it is this transformation that should help find an optimized solution.
')

The best of both worlds

What if we could take all the advantages of mobile XR devices and combine them with the performance of XR-based PC systems? Mobile gadgets for augmented reality is the future of XR, because they can be used anywhere and anytime, without prior preparation and without wires. The implementation of PC-based XR, although it cannot be considered the future of augmented reality, has an important advantage - it is not limited in terms of power consumption and efficiency of cooling systems, which, in turn, allows for more extensive calculations. Thanks to 5G networks with lower latency and high bandwidth, we plan to take the best from both approaches. Distributing computing tasks using 5G technology will offer the best of two worlds - unlimited mobile XR-experience and photo-realistic graphics in thin and affordable XR-headsets. As a result, users will get “endless” opportunities in every sense, because they will be able to connect to augmented reality, wherever they want, and the degree of immersion in XR-applications will be even higher.


Technologies of boundless augmented reality offer to take the best from mobile XR-devices and devices with PC connection

Improving the efficiency of extended reality data processing on the device

Working with graphics in augmented reality systems requires a lot of computational power and is sensitive to response time. For a correct separation of calculations, a systematic approach is needed. Let's look at how the use of edge cloud computing can help more effectively complement processing on the device, creating systems of unlimited augmented reality with photo-realistic graphics (more detailed information is in our webinar ).

When a user of the XR system turns his head, the processing on the device determines the position of the head and sends this data to the cloud of edge computing via a 5G channel with minimal latency and high quality service. This system uses the received head position data to partially render the next frame of the image, encode the data and send it back to the XR headset. The headset then decrypts the last packet received and, using regularly updated data on changes in head position, continues to visualize and adjust the image to minimize the motion-to-photon latency (the delay between changing the user's head position and changing the image in the headset). Recall that in accordance with this indicator, all processing must be completed in a time not exceeding 20 milliseconds. Exceeding this threshold causes the user to experience discomfort and reduce the level of immersion in augmented reality.


Computations on the device are complemented with edge cloud computing and a 5G channel with low response time.

As you can see, to get a high-quality immersion effect in XR, you need a system solution with low response time and high reliability, so 5G networks with their minimum latency, high throughput are a crucial element of XR. As 5G-networks improve and their coverage grows, users will be able to enjoy photo-realistic graphics in XR applications in a large number of places and will be confident that the premium stand-alone XR experience will remain available thanks to efficient computing on the device itself.

And this is a key point that needs to be emphasized again: the processing of data on the device remains an extremely important factor in any scenarios. In offline mode, the on-board calculation on the device is responsible for generally all the calculations related to XR. When working together with the boundary cloud computing system, the on-board processing will provide the XR headset with the possibility of energy-efficient, high-performance visualization and tracking with low latency.

Creating “boundless” augmented reality

Qualcomm Technologies is already developing high-performance, stand-alone mobile XR solutions and remains the leader in promoting 5G technology in the world. But we cannot translate our ideas of “infinite” XR into reality alone. We actively cooperate with the largest representatives of the XR and 5G ecosystems, including OEMs and content creators, service providers and infrastructure, because the separate rendering architecture is a system solution.


The participants of the XR and 5G ecosystems should work together to make the technology of “limitless” XR a reality

As a result of synergy, all participants in the XR ecosystem will receive a large overall benefit from its active development, and the name of this benefit is “consumer acceptance growth”. For example, telecom operators will receive certain benefits from the transformation of Wireless Edge as a whole, but let's consider the benefits from the development of unlimited XR. First, with the advent of 5G networks, improved broadband accessibility will increase capacity, reduce response time and provide a guaranteed class of service, which will enable the emergence of richer and more interactive XR applications. Secondly, as operators increase their capacity for edge cloud computing, they will be able to offer completely new services to the mainstream user, for example, XR applications with photo-realistic graphics.

We believe that a great advantage will be a new revolutionary user experience, including joint interactive work in real time, multiplayer games with photo-realistic graphics, a new generation of video with support for six degrees of freedom, educational applications with immersion and personalized shopping which was not. These perspectives are inspiring, so we hope for fruitful collaboration with other ecosystem participants — this work will help turn our XR vision into reality.