Posterior convergence and model estimation in Bayesian change-point problems

Heng Lian

doi:10.1214/09-ejs477

Abstract

We study the posterior distribution of the Bayesian multiple change-point regression problem when the number and the locations of the change-points are unknown. While it is relatively easy to apply the general theory to obtain the $O(1/\sqrt{n})$ rate up to some logarithmic factor, showing the parametric rate of convergence of the posterior distribution requires additional work and assumptions. Additionally, we demonstrate the asymptotic normality of the segment levels under these assumptions. For inferences on the number of change-points, we show that the Bayesian approach can produce a consistent posterior estimate. Finally, we show that consistent posterior for model selection necessarily implies that the parametric rate for posterior estimation stated previously cannot be uniform over the class of models we consider. This is the Bayesian version of the same phenomenon that has been noted and studied by other authors.

Highlights

We consider the regression problem of estimating a piece-wise constant function when the number of segments as well as the locations of its change-points is unknown
A more recent trend of analysis that dispenses with the usage of MCMC for the change-point problem starts with the paper [23] where a dynamic programming approach is utilized to marginalize over segment levels and change-point locations
We focus on√the case that θ0 is piece-wise constant and aim to achieve the parametric O(1/ n) rate of convergence and study the posterior consistency in the estimation of the number of change-points, which we refer to as the model selection problem

Summary

Introduction

We consider the regression problem of estimating a piece-wise constant function when the number of segments as well as the locations of its change-points is unknown. Consistency is proved for the case that the true regression function is in the Lipschitz class as well Another related work is [28] where the Bayesian density estimation problem is studied with density approximated by piece-wise constant functions. We focus on√the case that θ0 is piece-wise constant and aim to achieve the parametric O(1/ n) rate of convergence and study the posterior consistency in the estimation of the number of change-points, which we refer to as the model selection problem.

Main results

Unknown noise variance

Discussion