Temporal Difference Learning for Noise Covariance Adaptation

Abstract

When the noise covariance matrices in the filtering algorithm are not match with the practical system, the filtering performance may be degraded. To address this problem, a temporal difference learning Kalman filter (TDLKF) algorithm is presented for systems with noise covariance uncertainty. The key idea is to select appropriate noise parameters adaptively via trial-and-error interactions in an operational environment, such that the filtering performance is improved gradually. The adaptation scheme is implemented via temporal difference (TD) learning, which is an optimization-searching algorithm based on a value function updated iteratively as episode increases. The TDLKF is applied for an autonomous satellite constellation navigation system. The high performance of the algorithm is illustrated via simulation in comparison with the multiple-model adaptive estimation (MMAE) and the adaptive Kalman filter (AKF).