Development of pass-through augmented reality interface for human robot interaction

Abstract

In this technology driven world, human machine interaction is indispensable. Conventional human machine interface techniques pose the challenges of extensive training and increased cognitive load. Usage of hand gestures, face and speech recognition, eye ball tracking, etc. are in the upward trend to overcome these limitations and prove to be more intuitive and user friendly. In this paper, a vision-based pass-through augmented reality system is developed that enables the user to command the mobile robot for various mobility and navigation tasks. A pass-through augmented reality setup is achieved by a combination of a typical virtual reality headset and a stereo camera with depth perception. The stereo camera and a Leap Motion sensor mounted on a wearable VR headset acts as the vision system for the user and the gesture recognition system respectively. Necessary graphical user interface with the list of functions and information will appear on the VR headset display from which the user can select the option/command to be given to the mobile robot. The gesture made by the user in the GUI is detected and communicated to the robot through wireless means to perform the corresponding task.