A Hybrid framework for mobile augmented reality

Abstract

Augmented reality (AR) aims to combine real and virtual worlds, offering real-time 3D interactions between users and virtual information. Existed AR systems detect specific markers in environments and augmented them with virtual contents such as 3D objects and videos. With the explosive growth of powerful, less expensive mobile devices, AR systems are extended to mobile devices. However, they fail to achieve the accurate and real-time integration of the real world and virtual contents because of the insufficient detection accuracy to objects in real world and computational resource shortage. In this paper, we propose a hybrid AR framework that integrates cloud computing, 5G communication and deep learning technology to achieve accurate tracking and ultra-low latency mobile AR applications. Our AR framework contains three blocks, an image capturing block, a deep learning based objects tracking block and a WebGL based rendering block. We use cloud computing technology to deploy them on mobile devices and a cloud server separately according to calculation amounts and use 5G to achieve communication between the cloud server and mobiles devices. The experiment results show that the accuracy and running speed of our framework can meet the requirements of most mobile AR applications.