False Discovery Rate Control Procedure Research Articles

An adaptive approach for online monitoring of large-scale data streams

In this article, we propose an adaptive top-r method to monitor large-scale data streams where the change may affect a set of unknown data streams at some unknown time. Motivated by parallel and distributed computing, we propose to develop global monitoring schemes by parallel running local detection procedures and then use the Benjamin–Hochberg false discovery rate control procedure to estimate the number of changed data streams adaptively. Our approach is illustrated in two concrete examples: one is a homogeneous case when all data streams are independent and identically distributed with the same known pre-change and post-change distributions. The other is when all data are normally distributed, and the mean shifts are unknown and can be positive or negative. Theoretically, we show that when the pre-change and post-change distributions are completely specified, our proposed method can estimate the number of changed data streams for both the pre-change and post-change status. Moreover, we perform simulations and two case studies to show its detection efficiency.

Controlling for false discoveries subsequently to large scale one-way ANOVA testing in proteomics: Practical considerations.

In discovery proteomics, as well as many other "omic" approaches, the possibility to test for the differential abundance of hundreds (or of thousands) of features simultaneously is appealing, despite requiring specific statistical safeguards, among which controlling for the false discovery rate (FDR) has become standard. Moreover, when more than two biological conditions or group treatments are considered, it has become customary to rely on the one-way analysis of variance (ANOVA) framework, where a first global differential abundance landscape provided by an omnibus test can be subsequently refined using various post-hoc tests (PHTs). However, the interactions between the FDR control procedures and the PHTs are complex, because both correspond to different types of multiple test corrections (MTCs). This article surveys various ways to orchestrate them in a data processing workflow and discusses their pros and cons.

Bridging the False Discovery Gap.

Controlling the false discovery rate (FDR) among discoveries from a tandem mass spectrometry proteomics experiment using target decoy competition (TDC) controls only the proportion of false discoveries in an average sense. Thus, for any particular analysis, even with a valid FDR control procedure, the proportion of false discoveries (the FDP) may be higher than the specified FDR threshold. We demonstrate this phenomenon using real data and describe two recently developed methods that help bridge the gap between controlling the expected or average rate of false discoveries and the empirical rate (FDP). The FDP Stepdown method controls the FDP at any desired confidence level, and the TDC Uniform Band provides a confidence, or upper prediction bound, on the FDP in TDC's list of discoveries.

Covariate-modulated large-scale multiple testing under dependence

Large-scale multiple testing, which calls for conducting tens of thousands of hypothesis testings simultaneously, has been applied in many scientific fields. Most conventional multiple testing procedures often focused on the control of false discovery rate (FDR) and largely ignored covariate information and the dependence structure among tests. A FDR control procedure, termed as Covariate-Modulated Local Index of Significance (cmLIS) procedure, which not only takes into account local correlations among tests but also accommodates the covariate information by leveraging a covariate-modulated hidden Markov model (HMM), has been proposed. In the oracle case where all parameters of the covariate-modulated HMM are known, the cmLIS procedure is shown to be valid and optimal in some sense. According to whether the number of mixed components in the non-null distribution is known, two Bayesian sampling algorithms are provided for parameter estimation. Extensive simulations are conducted to demonstrate the effectiveness of the cmLIS procedure over state-of-the-art multiple testing procedures. Finally, the cmLIS procedure is applied to an RNA sequencing data and a schizophrenia (SCZ) data.

Challenging Targets or Describing Mismatches? A Comment on Common Decoy Distribution by Madej et al.

In their recent article, Madej et al. (Madej, D.; Wu, L.; Lam, H.Common Decoy Distributions Simplify False Discovery Rate Estimation in Shotgun Proteomics. J. Proteome Res.2022, 21 (2), 339-348) proposed an original way to solve the recurrent issue of controlling for the false discovery rate (FDR) in peptide-spectrum-match (PSM) validation. Briefly, they proposed to derive a single precise distribution of decoy matches termed the Common Decoy Distribution (CDD) and to use it to control for FDR during a target-only search. Conceptually, this approach is appealing as it takes the best of two worlds, i.e., decoy-based approaches (which leverage a large-scale collection of empirical mismatches) and decoy-free approaches (which are not subject to the randomness of decoy generation while sparing an additional database search). Interestingly, CDD also corresponds to a middle-of-the-road approach in statistics with respect to the two main families of FDR control procedures: Although historically based on estimating the false-positive distribution, FDR control has recently been demonstrated to be possible thanks to competition between the original variables (in proteomics, target sequences) and their fictional counterparts (in proteomics, decoys). Discriminating between these two theoretical trends is of prime importance for computational proteomics. In addition to highlighting why proteomics was a source of inspiration for theoretical biostatistics, it provides practical insights into the improvements that can be made to FDR control methods used in proteomics, including CDD.

2dFDR: a new approach to confounder adjustment substantially increases detection power in omics association studies

One challenge facing omics association studies is the loss of statistical power when adjusting for confounders and multiple testing. The traditional statistical procedure involves fitting a confounder-adjusted regression model for each omics feature, followed by multiple testing correction. Here we show that the traditional procedure is not optimal and present a new approach, 2dFDR, a two-dimensional false discovery rate control procedure, for powerful confounder adjustment in multiple testing. Through extensive evaluation, we demonstrate that 2dFDR is more powerful than the traditional procedure, and in the presence of strong confounding and weak signals, the power improvement could be more than 100%.

Isolating changed panels and estimating common change point after sequential detection with FDR control

Quick detection of common changes is critical in sequential monitoring of multi-stream data where a common change is referred as a change that only occurs in a portion of panels. After a common change is detected by using a SR-SUM procedure, we compare the CUSUM and EWMA methods in terms of FDR and FNR after a common change is detected. The BH-type procedure is used for both CUSUM processes and EWMA processes at the detection time to isolate the changed panels with the control of FDR (False Discovery Rate). The numerical results show that the FNR for the CUSUM method is smaller than that for the EWMA method. Also the CUSUM method has the advantage that the common change point is conveniently estimated after isolation. To reduce the FNR, the tightened FDR control procedure is considered by using the estimated number of changed panels. Further, the estimated number of changed panels can also be used to identify whether the alarm is true or false.

Nonparametric false discovery rate control for identifying simultaneous signals

It is frequently of interest to identify simultaneous signals, defined as features that exhibit statistical significance across each of several independent experiments. For example, genes that are consistently differentially expressed across experiments in different animal species can reveal evolutionarily conserved biological mechanisms. However, in some problems the test statistics corresponding to these features can have complicated or unknown null distributions. This paper proposes a novel nonparametric false discovery rate control procedure that can identify simultaneous signals even without knowing these null distributions. The method is shown, theoretically and in simulations, to asymptotically control the false discovery rate. It was also used to identify genes that were both differentially expressed and proximal to differentially accessible chromatin in the brains of mice exposed to a conspecific intruder. The proposed method is available in the R package github.com/sdzhao/ssa.

Tensor Graphical Model: Non-Convex Optimization and Statistical Inference.

We consider the estimation and inference of graphical models that characterize the dependency structure of high-dimensional tensor-valued data. To facilitate the estimation of the precision matrix corresponding to each way of the tensor, we assume the data follow a tensor normal distribution whose covariance has a Kronecker product structure. A critical challenge in the estimation and inference of this model is the fact that its penalized maximum likelihood estimation involves minimizing a non-convex objective function. To address it, this paper makes two contributions: (i) In spite of the non-convexity of this estimation problem, we prove that an alternating minimization algorithm, which iteratively estimates each sparse precision matrix while fixing the others, attains an estimator with an optimal statistical rate of convergence. (ii) We propose a de-biased statistical inference procedure for testing hypotheses on the true support of the sparse precision matrices, and employ it for testing a growing number of hypothesis with false discovery rate (FDR) control. The asymptotic normality of our test statistic and the consistency of FDR control procedure are established. Our theoretical results are backed up by thorough numerical studies and our real applications on neuroimaging studies of Autism spectrum disorder and users' advertising click analysis bring new scientific findings and business insights. The proposed methods are encoded into a publicly available R package Tlasso.

On-line control of false discovery rates for multiple datastreams

Although some false discovery rate control procedures have been proposed in the continual surveillance of high dimensional datastreams, most of them ignore the sequential feature over the time domain and dependence information among the stream observations. This inspires us to exploit the sequential feature by restricting the ongoing streams at each time point to be a dynamic set, which is determined by previous complex controlling procedures. Based on the exponentially weighted moving average (EWMA) scheme, we develop a dynamic multiple testing procedure for high dimensional datastreams with the control of false discovery rates (FDR). The FDR is shown to be controlled pointwise under the condition that the average of correlations of the stream observations decreases to zero at a polynomial rate. Numerical results illustrate that the proposed method is able to deliver satisfactory control performance.

False discovery rate control incorporating phylogenetic tree increases detection power in microbiome-wide multiple testing.

Next generation sequencing technologies have enabled the study of the human microbiome through direct sequencing of microbial DNA, resulting in an enormous amount of microbiome sequencing data. One unique characteristic of microbiome data is the phylogenetic tree that relates all the bacterial species. Closely related bacterial species have a tendency to exhibit a similar relationship with the environment or disease. Thus, incorporating the phylogenetic tree information can potentially improve the detection power for microbiome-wide association studies, where hundreds or thousands of tests are conducted simultaneously to identify bacterial species associated with a phenotype of interest. Despite much progress in multiple testing procedures such as false discovery rate (FDR) control, methods that take into account the phylogenetic tree are largely limited. We propose a new FDR control procedure that incorporates the prior structure information and apply it to microbiome data. The proposed procedure is based on a hierarchical model, where a structure-based prior distribution is designed to utilize the phylogenetic tree. By borrowing information from neighboring bacterial species, we are able to improve the statistical power of detecting associated bacterial species while controlling the FDR at desired levels. When the phylogenetic tree is mis-specified or non-informative, our procedure achieves a similar power as traditional procedures that do not take into account the tree structure. We demonstrate the performance of our method through extensive simulations and real microbiome datasets. We identified far more alcohol-drinking associated bacterial species than traditional methods. R package StructFDR is available from CRAN. chen.jun2@mayo.edu. Supplementary data are available at Bioinformatics online.

MOST: most-similar ligand based approach to target prediction

BackgroundMany computational approaches have been used for target prediction, including machine learning, reverse docking, bioactivity spectra analysis, and chemical similarity searching. Recent studies have suggested that chemical similarity searching may be driven by the most-similar ligand. However, the extent of bioactivity of most-similar ligands has been oversimplified or even neglected in these studies, and this has impaired the prediction power.ResultsHere we propose the MOst-Similar ligand-based Target inference approach, namely MOST, which uses fingerprint similarity and explicit bioactivity of the most-similar ligands to predict targets of the query compound. Performance of MOST was evaluated by using combinations of different fingerprint schemes, machine learning methods, and bioactivity representations. In sevenfold cross-validation with a benchmark Ki dataset from CHEMBL release 19 containing 61,937 bioactivity data of 173 human targets, MOST achieved high average prediction accuracy (0.95 for pKi ≥ 5, and 0.87 for pKi ≥ 6). Morgan fingerprint was shown to be slightly better than FP2. Logistic Regression and Random Forest methods performed better than Naïve Bayes. In a temporal validation, the Ki dataset from CHEMBL19 were used to train models and predict the bioactivity of newly deposited ligands in CHEMBL20. MOST also performed well with high accuracy (0.90 for pKi ≥ 5, and 0.76 for pKi ≥ 6), when Logistic Regression and Morgan fingerprint were employed. Furthermore, the p values associated with explicit bioactivity were found be a robust index for removing false positive predictions. Implicit bioactivity did not offer this capability. Finally, p values generated with Logistic Regression, Morgan fingerprint and explicit activity were integrated with a false discovery rate (FDR) control procedure to reduce false positives in multiple-target prediction scenario, and the success of this strategy it was demonstrated with a case of fluanisone. In the case of aloe-emodin’s laxative effect, MOST predicted that acetylcholinesterase was the mechanism-of-action target; in vivo studies validated this prediction.ConclusionsUsing the MOST approach can result in highly accurate and robust target prediction. Integrated with a FDR control procedure, MOST provides a reliable framework for multiple-target inference. It has prospective applications in drug repurposing and mechanism-of-action target prediction.

A powerful FDR control procedure for multiple hypotheses

A powerful test procedure is proposed for multiple hypotheses for the false discovery rate (FDR) control. The proposed procedure is a weighted p-value procedure which explores false null hypotheses information. It is theoretically shown to control the FDR and be more powerful than the widely used plug-in BH procedure. When there are unknown parameters estimated from the data, the asymptotic properties of the proposed procedure are discussed. The extensive simulation studies further verify the theoretical results. A real data is analyzed to illustrate the proposed method.

Evaluation of statistical methods for safety signal detection: a simulation study.

Proactive evaluation of drug safety with systematic screening and detection is critical to protect patients' safety and important in regulatory approval of new drug indications and postmarketing communications and label renewals. In recent years, quite a few statistical methodologies have been developed to better evaluate drug safety through the life cycle of the product development. The statistical methods for flagging safety signals have been developed in two major areas - one for data collected from spontaneous reporting system, mostly in the postmarketing area, and the other for data from clinical trials. To our knowledge, the methods developed for one area have not been applied to the other one so far. In this article, we propose to utilize all such methods for flagging safety signals in both areas regardless of which specific area they were originally developed for. Therefore, we selected eight typical methods, that is, proportional reporting ratios, reporting odds ratios, the maximum likelihood ratio test, Bayesian confidence propagation neural network method, chi-square test for rates comparison, Benjamini and Hochberg procedure, new double false discovery rate control procedure, and Bayesian hierarchical mixture model for systematic comparison through simulations. The Benjamini and Hochberg procedure and new double false discovery rate control procedure perform best overall in terms of sensitivity and false discovery rate. The likelihood ratio test also performs well when the sample sizes are large.

An Optimal Test with Maximum Average Power While Controlling FDR with Application to RNA‐Seq Data

The recent RNA-seq technology is an attractive method to study gene expression. One of the most important goals in RNA-seq data analysis is to detect genes differentially expressed across treatments. Although several statistical methods have been published, there are no theoretical justifications for whether these methods are optimal or how to search for the optimal test. Furthermore, most proposed tests are designed for testing whether the mean expression levels are exactly the same or not across treatments, whereas sometimes, biologists are interested in detecting genes with expression changes larger than a certain threshold. Another issue with current methods is that the false discovery rate (FDR) control is not well studied. In this manuscript, we propose a test to address all the above issues. Under model assumptions, we derive an optimal test that achieves the maximum of average power among those that control FDR at the same level. We also provide an approximated version, the approximated most average powerful (AMAP) test, for practical implementation. The proposed method allows for testing null hypotheses that are much more general than the ones most previous studies have considered, and it leads to a natural way of controlling the FDR. Through simulation studies, we show that our test has a higher power than other methods, including the widely-used edgeR, DESeq, and baySeq methods, as well as better FDR control than two other FDR control procedures commonly used in practice. For demonstration, we also apply the proposed method to a real RNA-seq dataset obtained from maize.

Restoring coverage to the Bayesian false discovery rate control procedure

Principal among knowledge discovery tasks is recognition of insightful patterns or features from data that can inform otherwise challenging decisions. For the costly future decisions, there is little room for error. Features must provide substantial evidence to be robust for classification and dependable for important decisions. Here we seek statistical evidence for feature selection, that feature signals are of sufficient magnitude and frequency to be generalizable for classification. The Bayesian false discovery rate (bFDR) error control procedure is powerfully suited for this task. In realistic situations often encountered in practice, the bFDR procedure is biased, yielding a greater than desired FDR. In other less typical cases, the FDR is less than desired. We investigate the sources of bias in the bFDR procedure, and predict the direction of bias. A new algorithm has been developed to recover the bias in the bFDR control procedure. In simulation and real data mining examples, the new bFDR control algorithm shows promise. The strengths and limitations of the new approach are presented with examples and discussed.

The effect of FOXA2rs1209523 on glucose-related phenotypes and risk of type 2 diabetes in Danish individuals

BackgroundVariations within the FOXA family have been studied for a putative contribution to the risk of type 2 diabetes (T2D), and recently the minor T-allele of FOXA2 rs1209523 was reported to associate with decreased fasting plasma glucose levels in a study using a weighted false discovery rate control procedure to enhance the statistical power of genome wide association studies in detecting associations between low-frequency variants and a given trait.Thus, the primary aim of this study was to investigate whether the minor T-allele of rs1205923 in FOXA2 associated with 1) decreased fasting plasma glucose and 2) a lower risk of developing T2D. Secondly, we investigated whether rs1205923 in FOXA2 associated with other glucose-related phenotypes.MethodsThe variant was genotyped in Danish individuals from four different study populations using KASPar® PCR SNP genotyping system. We examined for associations of the FOXA2 genotype with fasting plasma glucose and estimates of insulin release and insulin sensitivity following an oral glucose tolerance test in 6,162 Danish individuals from the population-based Inter99 study while association with T2D risk was assessed in 10,196 Danish individuals including four different study populations.ResultsThe FOXA2 rs1209523 was not associated with fasting plasma glucose (effect size (β) = -0.03 mmol/l (95%CI: -0.07; 0.01), p = 0.2) in glucose-tolerant individuals from the general Danish population. Furthermore, when employing a case-control setting the variant showed no association with T2D (odds ratio (OR) = 0.82 (95%CI: 0.62-1.07), p = 0.1) among Danish individuals. However, when we performed the analysis in a subset of 6,022 non-obese individuals (BMI < 30 kg/m2) an association with T2D was observed (OR = 0.68 (95%CI: 0.49-0.94), p = 0.02). Also, several indices of insulin release and β-cell function were associated with the minor T-allele of FOXA2 rs1209523 in non-obese individuals.ConclusionsWe failed to replicate association of the minor T-allele of FOXA2 rs1209523 with fasting plasma glucose in a population based sample of glucose tolerant individuals. More extensive studies are needed in order to fully elucidate the potential role of FOXA2 in glucose homeostasis.

Adaptive and Dynamic Adaptive Procedures for False Discovery Rate Control and Estimation

SummaryMany methods for estimation or control of the false discovery rate (FDR) can be improved by incorporating information about π0, the proportion of all tested null hypotheses that are true. Estimates of π0 are often based on the number of p-values that exceed a threshold λ. We first give a finite sample proof for conservative point estimation of the FDR when the λ-parameter is fixed. Then we establish a condition under which a dynamic adaptive procedure, whose λ-parameter is determined by data, will lead to conservative π0- and FDR estimators. We also present asymptotic results on simultaneous conservative FDR estimation and control for a class of dynamic adaptive procedures. Simulation results show that a novel dynamic adaptive procedure achieves more power through smaller estimation errors for π0 under independence and mild dependence conditions. We conclude by discussing the connection between estimation and control of the FDR and show that several recently developed FDR control procedures can be cast in a unifying framework where the strength of the procedures can be easily evaluated.

Mass univariate analysis of event‐related brain potentials/fields II: Simulation studies

Mass univariate analysis is a relatively new approach for the study of ERPs/ERFs. It consists of many statistical tests and one of several powerful corrections for multiple comparisons. Multiple comparison corrections differ in their power and permissiveness. Moreover, some methods are not guaranteed to work or may be overly sensitive to uninteresting deviations from the null hypothesis. Here we report the results of simulations assessing the accuracy, permissiveness, and power of six popular multiple comparison corrections (permutation-based control of the familywise error rate [FWER], weak control of FWER via cluster-based permutation tests, permutation-based control of the generalized FWER, and three false discovery rate control procedures) using realistic ERP data. In addition, we look at the sensitivity of permutation tests to differences in population variance. These results will help researchers apply and interpret these procedures.

A Weighted False Discovery Rate Control Procedure Reveals Alleles at FOXA2 that Influence Fasting Glucose Levels

Association signals in GWAS are usually prioritized solely by p values. Here, we attempt to improve the power of GWAS by using a weighted false discovery rate control procedure to detect associations of low-frequency variants with effect sizes similar to or even larger than those of common variants. We used the Affymetrix Genome-Wide Human SNP Array 6.0 to test for association with fasting glucose levels in the Atherosclerosis Risk in Communities Study (ARIC) population. In addition to finding several previously identified sequence variations, we identified a low-frequency variant (rs1209523; minor allele frequency = 0.043) near FOXA2 that was associated with fasting glucose levels in European Americans (EAs) (n = 7428, p value = 1.3 x 10(-5)). The association between rs1209523 and glucose levels was also significant in African Americans (AAs) (n = 2029, p value = 6.7 x 10(-3)) of the ARIC and was confirmed by replication in both EAs and AAs of the Dallas Heart Study (n = 963 and 1571, respectively; p values = 5.3 x 10(-3) and 5.8 x 10(-4), respectively) and in EAs of the Cooper Center Longitudinal Study (n = 2862; p value = 1.6 x 10(-2)). A meta-analysis of these five populations yielded an estimated effect size of -1.31 mg/dl per minor allele (p value = 2.2 x 10(-11)). This study reveals that there is a cache of less-frequent variants in GWAS arrays that can be identified via analytical approaches accounting for allele frequencies.