Q‐learning for noise covariance adaptation in extended KALMAN filter

Abstract

AbstractThe extended Kalman filter (EKF) is a widely used method in navigation applications. The EKF suffers from noise covariance uncertainty, potentially causing it to perform poorly in practice. This paper attempts to suppress the unfavorable effect of the noise covariance uncertainty to the EKF in the framework of reinforcement learning. The proposed state estimation algorithm combines the EKF and a Q‐learning method, where a covariance adaptation strategy is designed based on the Q‐values, leading to a gradual improvement in the estimation performance. The resultant algorithm is called the Q‐learning extended Kalman filter (QLEKF), which is less sensitive to the noise covariance uncertainty. To evaluate the estimation error behavior in nonlinear uncertain systems, the stability of the filtering algorithm is investigated. A numerical simulation for spacecraft autonomous navigation is implemented to demonstrate the efficiency of the QLEKF. It is shown that the presented algorithm outperforms a traditional EKF and an adaptive extended Kalman filter (AEKF) in estimation accuracy.