Human-computer interaction in immersive virtual maintenance

Abstract

Design of product maintainability, involving analysis, verification, and training, can be pre-conducted with the aid of virtual environment — virtual maintenance. Unlike non-immersive virtual simulation, immersive virtual simulation offers a more realistic and effective path for maintainability design. However, in the current maintenance simulation environment, the actions of human beings who act as the subject of recognition are always observed and analyzed externally, where little sensory information acquired from human beings can be adopted for further interaction. Meanwhile, current maintenance simulation is conducted based on personal experiences and skills where the users may be disoriented without a natural and reasonable interaction pattern and model. To solve the problem of human-computer interaction in virtual maintenance system, we adapt the driver class of data gloves and optical tracking devices to expatiate the method of maintenance action data acquisition and disposal strategy, combined with the virtual environment. With the use of gloves and a 3D data tracker for motion capture, gesture recognition, and mapping of command, we expound the realization of human-computer interaction methods in the DELMIA virtual environment. We operate a virtual model of parts and components maintenance via a virtual hand. The focus is to achieve a virtual entity that manipulates the process in direct and concrete ways, relying on optical tracking, data gloves, and other peripherals. Finally, a virtual maintenance case is presented to verify the proposed method.