Control Of False Discovery Proportion Research Articles

Exceedance control of the false discovery proportion via high precision inversion method of Berk-Jones statistics

Exceedance control of the false discovery proportion (FDP) can provide an interpretable method for addressing the variability in the false discovery proportion estimates. Exceedance control of FDP can be viewed as constructing a confidence interval for FDP and as such inverting a hypothesis test is a viable method for achieving exceedance control. A novel powerful approach for exceedance control is presented based on using a directional Berk-Jones goodness-of-fit statistic. The approach employs a high-precision implementation procedure to accurately compute confidence envelopes for FDP. The procedure is compared against other methods and generalized to include other goodness-of-fit statistics that follow an isotropy condition.

False Discovery Rate Control Under General Dependence By Symmetrized Data Aggregation

We develop a new class of distribution-free multiple testing rules for false discovery rate (FDR) control under general dependence. A key element in our proposal is a symmetrized data aggregation (SDA) approach to incorporating the dependence structure via sample splitting, data screening, and information pooling. The proposed SDA filter first constructs a sequence of ranking statistics that fulfill global symmetry properties, and then chooses a data-driven threshold along the ranking to control the FDR. The SDA filter substantially outperforms the Knockoff method in power under moderate to strong dependence, and is more robust than existing methods based on asymptotic p-values. We first develop finite-sample theories to provide an upper bound for the actual FDR under general dependence, and then establish the asymptotic validity of SDA for both the FDR and false discovery proportion control under mild regularity conditions. The procedure is implemented in the R package sdafilter. Numerical results confirm the effectiveness and robustness of SDA in FDR control and show that it achieves substantial power gain over existing methods in many settings.

Inference on Multi-level Partial Correlations Based on Multi-subject Time Series Data

Partial correlations are commonly used to analyze the conditional dependence among variables. In this work, we propose a hierarchical model to study both the subject- and population-level partial correlations based on multi-subject time-series data. Multiple testing procedures adaptive to temporally dependent data with false discovery proportion control are proposed to identify the nonzero partial correlations in both the subject and population levels. A computationally feasible algorithm is developed. Theoretical results and simulation studies demonstrate the good properties of the proposed procedures. We illustrate the application of the proposed methods in a real example of brain connectivity on fMRI data from normal healthy persons and patients with Parkinson’s disease. Supplementary materials for this article are available online.

Controlling false discovery proportion in identification of drug-related adverse events from multiple system organ classes.

Analyzing safety data from clinical trials to detect safety signals worth further examination involves testing multiple hypotheses, one for each observed adverse event (AE) type. There exists certain hierarchical structure for these hypotheses due to the classification of the AEs into system organ classes, and these AEs are also likely correlated. Many approaches have been proposed to identify safety signals under the multiple testing framework and tried to achieve control of false discovery rate (FDR). The FDR control concerns the expectation of the false discovery proportion (FDP). In practice, the control of the actual random variable FDP could be more relevant and has recently drawn much attention. In this paper, we proposed a two-stage procedure for safety signal detection with direct control of FDP, through a permutation-based approach for screening groups of AEs and a permutation-based approach of constructing simultaneous upper bounds for false discovery proportion. Our simulation studies showed that this new approach has controlled FDP. We demonstrate our approach using data sets derived from a drug clinical trial.

Predictor ranking and false discovery proportion control in high-dimensional regression

We propose a ranking and selection procedure to prioritize relevant predictors and control false discovery proportion (FDP) in variable selection. Our procedure utilizes a new ranking method built upon the de-sparsified Lasso estimator. We show that the new ranking method achieves the optimal order of minimum non-zero effects in ranking relevant predictors ahead of irrelevant ones. Adopting the new ranking method, we develop a variable selection procedure to asymptotically control FDP at a user-specified level. We show that our procedure can consistently estimate the FDP of variable selection as long as the de-sparsified Lasso estimator is asymptotically normal. In simulations, our procedure compares favorably to existing methods in ranking efficiency and FDP control when the regression model is relatively sparse.

New procedures controlling the false discovery proportion via Romano–Wolf’s heuristic

The false discovery proportion (FDP) is a convenient way to account for false positives when a large number $m$ of tests are performed simultaneously. Romano and Wolf [ Ann. Statist. 35 (2007) 1378–1408] have proposed a general principle that builds FDP controlling procedures from $k$-family-wise error rate controlling procedures while incorporating dependencies in an appropriate manner; see Korn et al. [ J. Statist. Plann. Inference 124 (2004) 379–398]; Romano and Wolf (2007). However, the theoretical validity of the latter is still largely unknown. This paper provides a careful study of this heuristic: first, we extend this approach by using a notion of “bounding device” that allows us to cover a wide range of critical values, including those that adapt to $m_{0}$, the number of true null hypotheses. Second, the theoretical validity of the latter is investigated both nonasymptotically and asymptotically. Third, we introduce suitable modifications of this heuristic that provide new methods, overcoming the existing procedures with a proven FDP control.

Sample Size Calculation for Controlling False Discovery Proportion

The false discovery proportion (FDP), the proportion of incorrect rejections among all rejections, is a direct measure of abundance of false positive findings in multiple testing. Many methods have been proposed to control FDP, but they are too conservative to be useful for power analysis. Study designs for controlling the mean of FDP, which is false discovery rate, have been commonly used. However, there has been little attempt to design study with direct FDP control to achieve certain level of efficiency. We provide a sample size calculation method using the variance formula of the FDP under weak-dependence assumptions to achieve the desired overall power. The relationship between design parameters and sample size is explored. The adequacy of the procedure is assessed by simulation. We illustrate the method using estimated correlations from a prostate cancer dataset.

Exceedance Control of the False Discovery Proportion

Multiple testing methods to control the false discovery rate, the expected proportion of falsely rejected null hypotheses among all rejections, have received much attention. It can be valuable to control not the mean of this false discovery proportion (FDP), but rather the probability that the FDP exceeds a specified bound. In this article we construct a general class of methods for exceedance control of FDP based on inverting tests of uniformity. The method also produces a confidence envelope for the FDP as a function of rejection threshold. We discuss how to select a procedure with good power.