Objective Testing Procedures in Linear Models: Calibration of thep‐values

Abstract

Abstract. An optimal Bayesian decision procedure for testing hypothesis in normal linear models based on intrinsic model posterior probabilities is considered. It is proven that these posterior probabilities are simple functions of the classicalF‐statistic, thus the evaluation of the procedure can be carried out analytically through the frequentist analysis of the posterior probability of the null. An asymptotic analysis proves that, under mild conditions on the design matrix, the procedure is consistent. For any testing hypothesis it is also seen that there is a one‐to‐one mapping – which we callcalibration curve– between the posterior probability of the null hypothesis and the classicalbip‐value. This curve adds substantial knowledge about the possible discrepancies between the Bayesian and thep‐value measures of evidence for testing hypothesis. It permits a better understanding of the serious difficulties that are encountered in linear models for interpreting thep‐values. A specific illustration of the variable selection problem is given.