AR-Based Assembly Assistance System with Efficient Evaluation of Misalignment Between Virtual and Real Objects

Abstract

Good assembly instructions improve the effectiveness of assembly operations and training. The development of augmented reality (AR) has enhanced the experience of assembly operations by showing computer-aided design (CAD) models at installation locations to an operator wearing an AR head-mounted display (HMD). In such operations, it is important to detect whether misassembly occurs. This research proposes two methods to construct an AR assembly assistance system using only an HMD of Microsoft HoloLens. One method is coordinate calibration to make the CAD models display correctly at desired locations by aligning a reference CAD model to the corresponding real object’s position. The other evaluates in real time whether misalignment has occurred by comparing the depth images of the CAD model and the real object for efficiency. With the preliminary development, we can use only an HMD to achieve the AR-based assembly instruction and verification to some extent.