YOLOv4‐tiny‐based robust RGB‐D SLAM approach with point and surface feature fusion in complex indoor environments

Abstract

AbstractAlgorithm frameworks based on feature point matching are mature and widely used in simultaneous localization and mapping (SLAM). However, in the complex and changeable indoor environment, feature point matching‐based SLAM currently has two major problems, namely, decreased accuracy of pose estimation due to the interference caused by dynamic objects to the SLAM system and tracking loss caused by the lack of feature points in weak texture scenes. To address these problems, herein, we present a robust and real‐time RGB‐D SLAM algorithm that is based on ORBSLAM3. For interference caused by indoor moving objects, we add the improved lightweight object detection network YOLOv4‐tiny to detect dynamic regions, and the dynamic features in the dynamic area are then eliminated in the algorithm tracking stage. In the case of indoor weak texture scenes, while extracting point features the system extracts surface features at the same time. The framework fuses point and surface features to track camera pose. Experiments on the public TUM RGB‐D data sets show that compared with the ORB‐SLAM3 algorithm in highly dynamic scenes, the root mean square error (RMSE) of the absolute path error of the proposed algorithm improved by an average of 94.08%. Camera pose is tracked without loss over time. The algorithm takes an average of 34 ms to track each frame of the picture just with a CPU, which is suitably real‐time and practical. The proposed algorithm is compared with other similar algorithms, and it exhibits excellent real‐time performance and accuracy. We also used a Kinect camera to evaluate our algorithm in complex indoor environment, and also showed high robustness and real‐time. To sum up, our algorithm can not only deal with the interference caused by dynamic objects to the system but also stably run in the open indoor weak texture scene.