Stereo Vision Based Localization of Handheld Controller in Virtual Reality for 3D Painting Using Inertial System

Abstract

Google Tilt Brush is expensive for virtual drawing which needs further improvement on the functionalities of mechanisms rather than implementation expects addressed in this research. Several issues are addressed by this research in this context, i.e., noise removal from sensor data, double integration-based drift issues and cost. Recently, available smart phones do not have the ability to perform drawing within artificial settings handling cardboard and daydream of google without purchasing Oculus Rift and HTC Vive (Virtual Reality Headset) because of expensiveness for large number of users. In addition, various extrinsic hardwares, i.e., satellite localization hardware and ultrasonic localization applications are not used for drawing in virtual reality. Proposed methodology implemented extended Kalman filter and Butterworth filter to perform positioning using six degree of freedom using Microelectromechanical Applications (MEMS) software data. A stereo visual method using Simultaneous Localization and Mapping (SLAM) is used to estimate the measurement for positioning implicating mobile phone (i.e., android platform) for the hardware system to estimate drift. This research implemented Google Virtual Reality application settings Kit with Unity3D engine. Experimentation validation states that proposed method can perform painting using virtual reality hardware integrated with controller software implicating smartphone mobile without using extrinsic controller device, i.e., Oculus Rift and HTC Vive with satisfactory accuracy.