A Natural Bare-Hand Interaction Method With Augmented Reality for Constraint-Based Virtual Assembly

Abstract

Traditional virtual assembly methods have a high requirement in hand-eye coordination because of the separation of feedback and operation regions. These methods are inconsistent with human interaction habits and lack of naturalness because of limited interaction space and indirect interaction mode. Therefore, we propose a natural bare-hand interaction method for virtual assembly, enabling operators to interact with virtual objects by using natural gestures even while in motion. The Leap Motion controller (LMC) fixed on the Augmented Reality (AR) glasses is used to track hands of the operator and the mobility of the interactive device relieves the location limitation of assembly processes. Furthermore, AR allows operators to perform bare-hand assembly in a realistic situation. The interval Kalman filter (IKF) is applied to estimate hands’ positions to improve the accuracy of measured gesture data. Moreover, constraint assisted technology is introduced to aid operators in learning and completing assembly tasks quickly and accurately, and the assembly sequence is generated to assist the decision-making process during the interaction. Experimental results show that the proposed method can be used by non-professional operators for assembly tasks and can potentially improve assembly efficiency. Based on operator ratings about assembly experience and significant difference analysis, the proposed method performed better at providing better interactive experiences.