Bayesian approach to estimation of accuracy in two-sided testing

Abstract

For a two-sided hypothesis testing, it is known that there does not exist UMP tests. Uniformly most powerful unbiased (UMPU) tests and likelihood ratio test are the common approaches to two-sided testing. The p-values of these tests are usually used as evidence against null hypothesis. However, there are criticisms of p-values as a measure of evidence against null hypothesis for the two-sided testing problem in literature. Thus, in this paper, evidence measures derived from Bayesian approach are proposed to be the replacements of p-values and are investigated from both decision and testing perspectives. From decision theoretic framework, the proposed evidence measures can be demonstrated as admissible estimators, however, p-value of UMPU test are not admissible estimators; from testing aspect, the tests derived from p-values and the proposed evidence measures are shown to be UMPU tests. The Bayes estimator is better than p-value from theoretical aspect and it has the same merit as the p-value in testing point of view. Therefore, evidence measures derived from Bayesian approach are recommended in this paper.