A scan procedure for multiple testing: Beyond threshold-type procedures

Abstract

In a multiple testing setting, we consider testing n null hypotheses, denoted by H1,…,Hn, with the corresponding p-values Pi for each Hi. We propose a new method which ‘scans’ all intervals in the unit interval [0,1], selects the longest interval with estimated false discovery rate not exceeding the target level, and then rejects a hypothesis if its p-value is contained in this interval. In contrast, the Benjamini–Hochberg method, which is a threshold-type procedure, does the same but over intervals with an endpoint at the origin. Following Storey et al. (2004), we show that our procedure controls the false discovery rate in an asymptotic sense. We also investigate its asymptotic false non-discovery rate, deriving conditions under which it outperforms the Benjamini–Hochberg procedure. This happens, for example, in power-law location models.