The Kalman Filter Revisited Using Maximum Relative Entropy

Renaldas Urniezius,Adom Giffin

doi:10.3390/e16021047

Abstract

In 1960, Rudolf E. Kalman created what is known as the Kalman filter, which is a way to estimate unknown variables from noisy measurements. The algorithm follows the logic that if the previous state of the system is known, it could be used as the best guess for the current state. This information is first applied a priori to any measurement by using it in the underlying dynamics of the system. Second, measurements of the unknown variables are taken. These two pieces of information are taken into account to determine the current state of the system. Bayesian inference is specifically designed to accommodate the problem of updating what we think of the world based on partial or uncertain information. In this paper, we present a derivation of the general Bayesian filter, then adapt it for Markov systems. A simple example is shown for pedagogical purposes. We also show that by using the Kalman assumptions or “constraints”, we can arrive at the Kalman filter using the method of maximum (relative) entropy (MrE), which goes beyond Bayesian methods. Finally, we derive a generalized, nonlinear filter using MrE, where the original Kalman Filter is a special case. We further show that the variable relationship can be any function, and thus, approximations, such as the extended Kalman filter, the unscented Kalman filter and other Kalman variants are special cases as well.

Highlights

We show that by using the Kalman assumptions or “constraints”, we can arrive at the Kalman filter using the method of maximum entropy (MrE), which goes beyond Bayesian methods
We derive the Bayesian filter and, show that by applying the Kalman assumptions, we arrive at a solution that is consistent with the original
A paper [22] used maximum relative entropy (MrE) with the Kalman assumptions, but did not explicitly state that there is a direct link between these two approaches

Summary

Introduction

The algorithm follows the logic that if the previous state of the system is known, it could be used as the best guess for the current state This information is used in two ways, the first is that prior to any measurement, the underlying dynamics of the system may be known. Given this knowledge and the previous state, the new state could be determined. We will show a simple example illustrating that the Kalman filter is a special case of the general Bayesian filter for pedagogical purposes. We will show how the same Kalman logic can be applied to non-linear dynamical systems using Bayes rule and avoid approximations that are usually applied in extended Kalman filter and the unscented Kalman filter

Bayesian Filter

Kalman Filter

A Simple Example

Maximum Relative Entropy

Maximum Relative Entropy and Kalman

Kalman Filter’s Updating Step

Kalman Filter Revisited

Nonlinear Filter

Generalized Univariate Nonlinear Filter for Monotonic Transitions

Generalized Multivariate Nonlinear Filter for Monotonic Transitions

Kalman Filter Revisited Using a Jacobian

Summary and Final Remarks