Uncalibrated stereo vision with deep learning for 6-DOF pose estimation for a robot arm system

Abstract

This paper proposes a novel method for six degrees of freedom pose estimation of objects for the application of robot arm pick and place. It is based on the use of a stereo vision system, which does not require calibration. Using both cameras, four corner points of the object are detected. A deep-neural-network (DNN) is trained for the prediction of the 6 DOF pose of the object from the four detected corner points’ coordinates in each image of both cameras. The stereo vision used is a low-end vision system placed in a custom-made setup. Before the training phase of the DNN, the robot is set to auto collect data in a predefined workspace. This workspace is defined dependently on the spatial feasibility of the robot arm and the shared field of view of the stereo vision system. The collected data represent images of a 2D marker attached to the robot arm gripper. The 2D marker is used for data collection to ease the detection of the four corner points. The proposed method succeeds in estimating the six degrees of freedom pose of the object, without the need for the determination of neither the intrinsic nor the extrinsic parameters of the stereo vision system. The optimum design of the proposed DNN is obtained after comparing different activation functions and optimizers associated with the DNN.The proposed uncalibrated DNN-based method performance is compared to that of the traditional calibration-based method. In the calibration-based method, the rotational matrix relating the robot coordinates to the stereo vision coordinates is computed using two approaches. The first approach uses Singular Value Decomposition (SVD) while the second approach uses a novel proposed modification of particle swarm optimization (PSO) called Hyper particle Scouts optimization (HPSO). HPSO outperforms other metaheuristic optimization algorithms such as PSO and genetic algorithm (GA).Exhaustive tests are performed, and the proposed DNN-based method is shown to outperform all tested alternatives.