A Bayesian Theory of Change Detection in Statistically Periodic Random Processes

Abstract

A new class of stochastic processes called independent and periodically identically distributed (i.p.i.d.) processes is defined to capture periodically varying statistical behavior. A novel Bayesian theory is developed for detecting a change in the distribution of an i.p.i.d. process. It is shown that the Bayesian change point problem can be expressed as an optimal control problem of a Markov decision process (MDP) with periodic transition and cost structures. An optimal control theory is developed for periodic MDPs for discounted and undiscounted total cost criteria. A fixed-point equation is obtained that is satisfied by the optimal cost function. It is shown that a nonstationary but periodic policy is optimal. A value iteration algorithm is obtained to compute the optimal cost function. The results from the MDP theory are then applied to detect a change in the distribution of an i.p.i.d. process. It is shown that while a stopping rule based on a periodic sequence of thresholds is exactly optimal, a single-threshold policy is asymptotically optimal, as the probability of false alarm goes to zero. Numerical results are provided to demonstrate that the asymptotically optimal policy is not strictly optimal.