Image-Based Visual Servoing System for Components Alignment Using Point and Line Features

Abstract

The design of visual servoing system for robotic high-precision assembly is of great challenge. For the difficulty of insufficient accuracy of target object feature extraction, a deep neural network combined with feature pyramid network (FPN) structure is proposed. This lightweight network requires only a small amount of labeled data to achieve significant segmentation results. The control laws of translation and orientation for component alignment are separately designed. The translation is controlled based on the interaction matrix of point features. The orientation is controlled based on the interaction matrix of line features. The relations between the cameras’ motion and the end-effector of the manipulator are calibrated via the manipulator’s active movements, which are <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$3 \times $ </tex-math></inline-formula> 3 transformation matrices. The depth information of feature points is integrated into the transformation matrix. The alignment pose error estimation is realized with the interaction matrices, transformation matrices, and point and line features. A robotic assembly system is developed to assemble two aviation circular connectors with six degree-of-freedoms (DOFs) in high precision in 3-D space. The experimental results verify the effectiveness of the proposed method.