Exploring the Use of Subpopulation Membership in Bayesian Hierarchical Model Assessment

Abstract

We investigate whether the posterior predictive p-value can de- tect unknown hierarchical structure. We select several common discrepancy measures (i.e., mean, median, standard deviation, and χ 2 goodness-of-fit) whose choice is not motivated by knowledge of the hierarchical structure. We show that if we use the entire data set these discrepancy measures do not detect hierarchical structure. However, if we make use of the subpopu- lation structure many of these discrepancy measures are effective. The use of this technique is illustrated by studying the case where the data come from a two-stage hierarchical regression model while the fitted model does not include this feature.