The multivariate Kalman filter

Abstract

Previously we explored the Kalman filter within the framework of a simple problem. In the bucketful of resistors problem, we were estimating a single static parameter. We pulled one resistor out of the bucket and attempted to estimate its resistance given some successive measurements. Obviously we need to go beyond that for most practical applications. We need to be able to handle cases where the parameter of interest is actually changing over time and we want to be able to form an estimate of its value over time as it is changing. Furthermore, we want to be able to handle cases where we are estimating multiple parameters simultaneously and those parameters are related to each other. It is not a simple matter of five parameters and thus the need for five single-parameter Kalman filters running at the same time. If we are trying to estimate multiple parameters that have some kind of relationship to each other, then we want to be able to exploit those relationships in the course of our estimation and thus we need to expand our Kalman filter accordingly.We are thus transitioning from univariate parameter estimation to multivariate parameter estimation and from static systems to dynamic systems. Along the way we will explore state variables, state vectors, and state transition matrices. We will learn about the fundamental equations upon which the Kalman filter is based: the so-called "system equation" and the "measurement equation." We will need to learn about covariance matrices and then eventually we will develop the full Kalman recursion. We will then take our bucketful of resistors example and modify it slightly in order to ease into these new topics.Up to this point, we have only considered the estimation of a single parameter (the univariate case) with a value that did not change over time (a static system). We need to generalize the Kalman filter to be able to handle dynamic multivariate cases. The single parameters are going to become vectors of parameters and the variances are going to become covariance matrices. We will model system dynamics within the filter (the state transition matrix) and will model complex relationships between the states and the measured quantities (the H matrix will no longer be equal to 1).