Multimodal human-machine interface for remote operation of robot systems

Abstract

The trend in robotization is to use industrial robot more and more frequently in small and medium sized enterprises (SMEs). Wide spread of the enterprises over considerable area and small number of applied robot units per enterprise, emerges need to provide an efficient, inexpensive and immediate support remotely. Since there are a lot of important information to be communicated to efficiently support an industrial robot system, traditional data acquisition and human-machine-interfaces (HMI) cannot provide complete and effectively comprehended overview of the state of the robot cell. The goal of the paper is to demonstrate new methodology and technology for remote HMI based upon Cognitive Informatics Communication (CogInfoCom 2011) that augments efficiency of communication between an industrial robot system and remote operator exercising control over the system. For the purposes of evaluation a remote operating laboratory is developed. The laboratory consists of multimodal HMI interfacing an operator with physically separated seven axes NACHI MR20 robot. By means of multi-camera vision, multi-speaker sound system, and advanced 3D simulation, the robot is controlled remotely. The results show that HMI based upon CogInfoCom can be beneficially applied to communicate complex states of an industrial robot system to a remote operator.