A deep neural network approach for accurate 3D shape estimation of soft manipulator with vision correction

Abstract

Soft manipulator is a strong non-linear and uncertain system with infinite degrees of freedom. The real-time 3D shape estimation is the guarantee for control and application, but a single kind of sensor technology always has inherent limitations. To address the above issue, firstly, the constant curvature (CC) kinematics is proposed to roughly estimate the 3D shape of the soft manipulator, because this method becomes inaccurate when the manipulator is bent with large deformation. Secondly, a vision-based high-precision shape estimation method is developed. The self-organizing map (SOM) adaptive algorithm is introduced to identify the centerline of the manipulator from the image data. While vision sensing has strict requirements for the environment. Therefore, a learning approach based on a deep neural network (DNN) is designed to correct the CC kinematics with accurate visual estimation results. Finally, the performance of the trained DNN is evaluated on the test set and a real-time bending deformation experiment. The results indicate that the DNN approach has high accuracy and stability ability to learn the entire 3D shape of the soft manipulator in real time.