Combination of “combinations of p values”

Abstract

We investigate the impact of an uncertain number of false individual null hypotheses on commonly used p value combination methods. Under such uncertainty, these methods perform quite differently and often yield conflicting results. Consequently, we develop a combination of “combinations of p values” (CCP) test aimed at maintaining good power properties across such uncertainty. The CCP test is based on a simple union–intersection principle that exploits the weak correspondence between two underlying p value combination methods. Monte Carlo simulations show that the CCP test controls size and closely tracks the power of the best individual methods. We empirically apply the CCP test to explore the stationarity in real exchange rates and the information rigidity in inflation and output growth forecasts.