EKF-Based Real-Time Self-Attitude Estimation With Camera DNN Learning Landscape Regularities

Abstract

This letter presents an EKF (extended Kalman filter) based real-time self-attitude estimation method with a camera DNN (deep neural network) learning landscape regularities. The proposed DNN infers the gravity direction from a single shot image. It outputs the gravity direction as a mean vector and a covariance matrix in order to express uncertainty of the inference. It is pre-trained with datasets collected in a simulator. Fine-tuning with datasets collected with real sensors is carried out after the pre-training. Data augmentation is processed during the training in order to provide higher general versatility. The proposed method integrates angular rates from a gyroscope and the DNN's outputs in an EKF. The covariance matrix output from the DNN is used as process noise of the EKF. Moreover, inferences with too large variance are filtered out before processing the integration in the EKF. Static validations are performed to show the DNN can infer the gravity direction with uncertainty expression. Dynamic validations are performed to show the DNN can be used in real-time estimation. Some conventional methods are implemented for comparison.