3D Reconstruction of Grabbed Objects Using a Single Image Based on Palletizing Task

Abstract

In the palletizing task, the three-dimensional size of the stack directly affects the working distance (i.e., depth) of the manipulator in the Z-axis direction. In order to complete the palletizing task accurately and reliably and realize human-computer interaction, it is necessary to obtain the three-dimensional structural characteristics of the pallet in real time. Therefore, this paper proposes a three-dimensional reconstruction of the single image of the grab based on the palletizing task. Firstly this paper converts the RGB image of the stack to a grayscale image, then uses the Canny method to detect the edge of the corresponding grayscale image, and uses the Hough line detection algorithm to extract the straight line in the edge image. By analyzing the characteristics of the straight line cluster. A stack is divided into three groups of straight lines in the horizontal, vertical, and vertical directions according to the angle characteristics of the straight line. The RANSAC algorithm is used to analyze and obtain the best linear model of the straight line in each direction. And the minimum distance method is used to solve the vanishing points in the three directions of horizontal, vertical and vertical, and then the internal and external parameters of the camera are calibrated according to the attributes of the vanishing points. A fast and high-precision rectangle detection algorithm is used to detect the coordinates of the vertices of the cuboid. In particular, the size of each cuboid part is consistent and known, so this paper combines the camera’s internal and external parameters with the geometric characteristics of the stack can get the length, width, and height and visual model of a real-time changing stack during the palletizing task. The experimental results show that this algorithm can well build a three-dimensional wireframe model of the stack, and realize the three-dimensional visualization of the model. The maximum error of the reconstructed model is 0.950, the minimum error is 0, and the average error is 0.217; the maximum length The error is 0.560, the minimum error is 0, and the average error is 0.187; the maximum error of the width is 0.950, the minimum error is 0.064, and the average error is 0.461; the maximum error of the height is 0.420, the minimum error is 0, and the average error is 0.14; the robot Z axis The maximum error of the working distance (i.e. depth) in the direction is 0.24, the minimum error is 0, and the average error is 0.08, which meets our actual modeling requirements.