Deep Neural Network-Based Dynamical Object Recognition and Robust Multiobject Tracking Technique for Onboard Unmanned Aerial Vehicle’s Computer Vision-Based Systems

Abstract

Computer vision-based systems seem highly perspective for semantic analysis of the dynamical objects. However, considering dynamical object recognition and tracking from the unmanned aerial vehicle the task to design a robust model for data association is highly challenging due to additional issues, e.g., image degradation, non-fixed object camera distance and shooting focus, and real-time issues. Thus, we propose an accurate deep neural network-based dynamical object recognition and robust multi-object tracking technique based on bidirectional LSTM with the optimized motion and appearance gates as a multi-object tracking backbone, supported by an advanced single-shot detector network improved with residual prediction model and implemented DenseNet network as well as YOLOv4eff network as feature extraction. The technique has been trained on VisDrone 2022 and UAVDT datasets with the side-shoot dynamical objects at a height of up to 50 meters. The performance analysis on the test stage performed on 7 metrics demonstrate that the proposed technique surpasses by accuracy and robustness ability of other state-of-the-art techniques based on 2 cumulative MOTA and MOTP, as well as MT and IDsw. In particular, we have dramatically decreased the number of IDsw which implies a better capability to handle several occlusions, which is a desirable property in real-time multiple object tracking. We have pointed out the sensitivity of the tracking performance of our technique on the number of utilizing different sequence lengths and have defined an optimum. Finally, the applicability and reliability of the proposed technique for onboard UAV computer-based systems have been discussed.