Integrate augmented reality and force sensing devices to assist blind area assembly

Abstract

Blind area assembly is one of the challenges of manual assembly. The obscured pose of occluded components can lead to errors and reduce assembly efficiency. It has been observed that the sense of touch is commonly used to localize the assembly interface to complete the assembly in the blind area. Based on this, a blind area assembly assistance system was developed, which integrates augmented reality and force sensing devices. The system estimates the component poses during the tactile localizing stage based on the measured forces and moments applied to the product by the components, and then overlays virtual models of the components in situ to support assembly in the blind area. A user study was conducted to evaluate the performance and user experience of the system. The results showed that the system could successfully track and visualize target components during the tactile localizing stage, demonstrating great performance and user experience. The target component visualization can expedite assembly in the blind area, improve the system ease-of-use, and reduce the perceived workload. Finally, the limitations of the research are discussed and some directions for future work are provided.