Transformer based Kalman Filter with EM algorithm for time series prediction and anomaly detection of complex systems

Abstract

Modeling time series signals is critical in modern intelligent systems, which are able to determine the health condition and detect potential anomalies of complex systems by comparing predictions and measured signals. The accurate prediction of sequential signals is still challenging due to the conditional independence of measurements. This study proposes a Transformer based Kalman Filter (TKF) for accurate time-varying signal prediction, which contributes to capturing temporal features among time-series data. Then, Expectation Maximization (EM) algorithm is adopted to estimate parameters for Kalman Filter with hidden states, which avoids parameters’ determination heavily rely on manual experience. We conduct experiments on real-world time-series signals collected from complex systems. The results show that the proposed TKF achieves higher prediction accuracy and has potential to be applied for anomaly detection of complex systems.