Abstract
The more sensory channels are equipped in a virtual assembly system, the more real users feel in the whole system, however, most of the existing virtual assembly systems are based on the natural interaction method of one or two sensory channels. Thus, this paper proposes a novel virtual assembly system integrating multi-sensory channels, including gesture interaction, Chinese speech interaction, tactile interaction, 3D display and real-time display of real environment pictures in the virtual environment. To improve the operability of the virtual environment, we analyze the parallel virtual assembly sequence on the basis of two-hand interaction, and the assembly priority is prompted based on UI interface. For ease of operation, we present a method of viewpoint control based on gesture interaction and the coordinate threshold of spatial position. A hierarchical bounding box collision detection algorithm based on volume difference is proposed to improve the efficiency of collision feedback and collision avoidance. In addition, the power equipment models are exhibited in the virtual scene, as the exhibits of the virtual roaming process. Finally, to evaluate the training effect of this system, a comparative experiment is designed to compare the participants’ experience and the effect of assembly training. The experimental results show that the virtual assembly system of natural interaction with multi-sensory channels is flexible and immersive.
Highlights
With the continuous development of human-computer interaction technology, virtual reality technology has been integrated into various industries, such as military, medical, industrial production and manufacturing, interior design and so on [1]–[4]
RELATED WORK At present, the commonly used virtual reality interactive modes include the gesture interaction supported by Leap Motion, the somato-sensory interaction of the head-mounted display and virtual handle supported by Oculus Rift or HTC Vive, the posture recognition interaction supported by Kinect, the speech recognition interaction and so on
83% volunteers agree that the virtual assembly system of natural interaction with multi-sensory channels owns the strongest immersion, 87% volunteers agree that the multi-sensory channels virtual assembly system has the best operability, and 53% volunteers feel the weakest sense of vertigo in the process of experiment
Summary
With the continuous development of human-computer interaction technology, virtual reality technology has been integrated into various industries, such as military, medical, industrial production and manufacturing, interior design and so on [1]–[4]. The virtual assembly systems mainly complete the whole simulated operation process by one or two human-computer interaction modes, while people in the real environment can receive the environmental information through a variety of sensory. To solve the problems mentioned above, this paper takes the control cabinet of power transformer as the virtual object to assemble, and proposes a natural interaction method of multi-sensory channels for the virtual assembly system. This method will adopt various interactive ways, such as speech interaction, gesture interaction, embodied and tangible interaction, stereo vision, dynamic tracking, real-time video transmission of virtual reality combination, etc. As to enhance the sense of reality of the system, reduce the assembly time of learners and improve the learning efficiency
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
