Development of a vision based pose estimation system for robotic machining and improving its accuracy using LSTM neural networks and sparse regression

Abstract

In this work, an eye to hand camera based pose estimation system is developed for robotic machining and the accuracy of the estimated pose is improved using two different approaches, namely Long Short Term Memory (LSTM) neural networks and sparse regression. To improve the accuracy obtained from the Levenberg–Marquardt (LM) based pose estimation algorithm, two distinct supervised data driven approaches are proposed which can take the dynamics into account during robotic machining through utilization of the torque information available from the sensors at each joint. The first one is a LSTM neural network and the second one is a method based on sparse regression. The proposed methods are validated by an experimental study performed using a KUKA KR240 R2900 ultra robot while machining a NAS 979 part, during which the orientation of the cutting tool was fixed, and free form milling, during which the orientation of the cutting tool continuously changed. A target object to be tracked by the camera was designed with fiducial markers to guarantee trackability with ±90°in all directions. The design of these fiducial markers guarantee the detection of at least two distinct non-parallel markers from any view, thus preventing pose estimation ambiguities. Moreover, in order to reduce the errors due to the construction of the camera target and placement of the markers on it, this work proposes a method for optimizing the positions of the corners of the fiducial markers in the object frame using a laser tracker. The proposed methods were compared with an Extended Kalman Filter (EKF) and the experimental results show that both of the proposed approaches significantly improve the pose estimation accuracy and precision of the vision based system during robotic machining while proving much more effective than the EKF approach. The attainable absolute position errors were 5.47 mm, 2.9 mm and 2.05 mm on average for NAS 979 machining and 5.35 mm, 2.17 mm and 0.86 mm on average for free form machining when using the EKF, the proposed LSTM network and the proposed sparse regression approaches, respectively. Moreover, the proposed sparse regression based method provides parsimonious models and better results when compared with the proposed LSTM based approach.