Stochastic state space models

Abstract

This chapter discusses stochastic state space models and how optimal predictors can be constructed to predict the future state of a stochastic dynamic system. Finally, we examine how such predictors converge over large time horizons to a stationary predictor. Stochastic state space model The stochastic state space model is the main model used throughout this book. We will also use the phrase partially observed Markov model interchangeably with state space model. We start by giving two equivalent definitions of a stochastic state space model. The first definition is in terms of a stochastic difference equation. The second definition is presented in terms of the transition kernel of a Markov process and the observation likelihood. Difference equation form of stochastic state space model Let k = 0, 1, …, denote discrete time. A discrete-time stochastic state space model is comprised of two random processes { x k } and { y k }: x k +1 = ϕ k ( x k , w k ), x 0 ∼ π 0 y k = ψ k ( x k , v k ). The difference equation (2.1) is called the state equation . It models the evolution of the state x k of a nonlinear stochastic system – nonlinear since ϕ k ( x , w ) is any nonlinear function; stochastic since the system is driven by the random process { w k } which denotes the “process noise” or “state noise”. At each time k , the state x k lies in the state space X = ℝ X . The initial state at time k = 0, x 0 is generated randomly according to prior distribution π 0 . This is denoted symbolically as x 0 ∼ π 0 . The observation equation (2.2) models a nonlinear noisy sensor that observes the state process { x k } corrupted by measurement noise { v k }. At each time k , the observation y k is a Y -dimensional vector valued random variable. Note that y k defined by (2.2) is a doubly stochastic process. It is a random function of the state x k which itself is a stochastic process evolving according to (2.1). It is assumed that the state noise process { w k } is an X -dimensional independent and identically distributed (i.i.d.) sequence of random variables. Also it is assumed that{ w k }, { v k } and x 0 are independent. The observation noise process { v k } is assumed to be a Y -dimensional i.i.d. sequence of random variables.