See-through and spatial augmented reality - a novel framework for human-robot interaction

Abstract

Autonomous and semi-autonomous mobile robots have been deployed to cooperate with humans in many industrial applications. These tasks require human and robot to communicate and present information quickly and effectively. Recent human-robot interfaces usually use a setup including a camera and a projector attached to the mobile robot to project the information to the floor or to the wall during the interaction process. However, there are some limitations to these interfaces. First, using a projector for projecting information seems to be fine for an indoor application. On the contrary, it is very difficult or even impossible for users to view this source of information in outdoor contexts. This makes the current framework inappropriate for many outdoor industrial tasks. Secondly, as the projector is the only device for exchanging information between human and robot, the human-robot interacting process is insecure and people who work in the same environment can control the robot in the same manner as the main operator. Finally, the current interfaces normally use mouse, keyboard or a teach pendant to provide task information to the robot. This approach poses some difficulties if the main operator is working in an industrial context where he is supposed to wear protective equipment such as gloves or helmets which make it hard to control a mouse or to type on a keyboard. This work proposes a new interface framework for human - computer interaction in industry that can overcome the current limitations of previous works. The framework uses a laser-writer instead of a projector which is suitable for both indoor and outdoor applications. Furthermore, the combination of see-through head-mounted display augmented reality and spatial augmented reality would provide the system a novel way to enhance the security level of exchanging information since the system now can separate the information presenting to the main user and to people working in the same environment. Finally, a novel hand-held device is incorporated to the framework which provides various input modalities for users to interact with the mobile robot. The device will allows the elimination of mouse and keyboard or teach pendants in industrial contexts.