False Discovery Rate Control Research Articles

MaxDIA enables library-based and library-free data-independent acquisition proteomics

MaxDIA is a software platform for analyzing data-independent acquisition (DIA) proteomics data within the MaxQuant software environment. Using spectral libraries, MaxDIA achieves deep proteome coverage with substantially better coefficients of variation in protein quantification than other software. MaxDIA is equipped with accurate false discovery rate (FDR) estimates on both library-to-DIA match and protein levels, including when using whole-proteome predicted spectral libraries. This is the foundation of discovery DIA—hypothesis-free analysis of DIA samples without library and with reliable FDR control. MaxDIA performs three- or four-dimensional feature detection of fragment data, and scoring of matches is augmented by machine learning on the features of an identification. MaxDIA’s bootstrap DIA workflow performs multiple rounds of matching with increasing quality of recalibration and stringency of matching to the library. Combining MaxDIA with two new technologies—BoxCar acquisition and trapped ion mobility spectrometry—both lead to deep and accurate proteome quantification.

Accurately Assigning Peptides to Spectra When Only a Subset of Peptides Are Relevant.

The standard proteomics database search strategy involves searching spectra against a peptide database and estimating the false discovery rate (FDR) of the resulting set of peptide-spectrum matches. One assumption of this protocol is that all the peptides in the database are relevant to the hypothesis being investigated. However, in settings where researchers are interested in a subset of peptides, alternative search and FDR control strategies are needed. Recently, two methods were proposed to address this problem: subset-search and all-sub. We show that both methods fail to control the FDR. For subset-search, this failure is due to the presence of "neighbor" peptides, which are defined as irrelevant peptides with a similar precursor mass and fragmentation spectrum as a relevant peptide. Not considering neighbors compromises the FDR estimate because a spectrum generated by an irrelevant peptide can incorrectly match well to a relevant peptide. Therefore, we have developed a new method, "subset-neighbor search" (SNS), that accounts for neighbor peptides. We show evidence that SNS controls the FDR when neighbors are present and that SNS outperforms group-FDR, the only other method that appears to control the FDR relative to a subset of relevant peptides.

GGM Knockoff Filter: False Discovery Rate Control for Gaussian Graphical Models

AbstractWe propose a new method to learn the structure of a Gaussian graphical model with finite sample false discovery rate control. Our method builds on the knockoff framework of Barber and Candès for linear models. We extend their approach to the graphical model setting by using a local (node-based) and a global (graph-based) step: we construct knockoffs and feature statistics for each node locally, and then solve a global optimization problem to determine a threshold for each node. We then estimate the neighbourhood of each node, by comparing its feature statistics to its threshold, resulting in our graph estimate. Our proposed method is very flexible, in the sense that there is freedom in the choice of knockoffs, feature statistics and the way in which the final graph estimate is obtained. For any given data set, it is not clear a priori what choices of these hyperparameters are optimal. We therefore use a sample-splitting-recycling procedure that first uses half of the samples to select the hyperparameters, and then learns the graph using all samples, in such a way that the finite sample FDR control still holds. We compare our method to several competitors in simulations and on a real data set.

Controlling False Discovery Rate Using Gaussian Mirrors

Simultaneously, finding multiple influential variables and controlling the false discovery rate (FDR) for linear regression models is a fundamental problem. We here propose the Gaussian Mirror (GM) method, which creates for each predictor variable a pair of mirror variables by adding and subtracting a randomly generated Gaussian perturbation, and proceeds with a certain regression method, such as the ordinary least-square or the Lasso (the mirror variables can also be created after selection). The mirror variables naturally lead to test statistics effective for controlling the FDR. Under a mild assumption on the dependence among the covariates, we show that the FDR can be controlled at any designated level asymptotically. We also demonstrate through extensive numerical studies that the GM method is more powerful than many existing methods for selecting relevant variables subject to FDR control, especially for cases when the covariates are highly correlated and the influential variables are not overly sparse.

Effect of diet and nonesterified fatty acid levels on global transcriptomic profiles in circulating peripheral blood mononuclear cells in early lactation dairy cows

After calving, lipid mobilization caused by increased nutrient demands for lactation leads to elevated circulating concentrations of nonesterified fatty acids (NEFA). Excessive NEFA levels have previously been identified as a major risk factor for postpartum immunosuppression. The aim of this study was to investigate changes in global transcriptomic gene expression of peripheral blood mononuclear cells (PBMC) in dairy cows offered different early lactation diets (high concentrate, n = 7; medium, n = 8; or low, n = 9) and with differing circulating levels of NEFA. Cows were classified as having NEFA concentrations of either <500 µM (low, n = 6), 500 to 750 µM (medium, n = 8) or >750 µM (high, n = 10) at 14 d in milk. Plasma urea concentrations were greater for cows on the high concentrate diet but β-hydroxybutyrate and glucose concentrations did not differ significantly between either dietary treatments or NEFA groups. Cows with high NEFA weighed more at drying off and suffered greater body condition score loss after calving. The PBMC were isolated at 14 d in milk, and RNA was extracted for RNA sequencing. Differential gene expression was analyzed with DESeq2 with q-value for false discovery rate control followed by Gene Ontology Enrichment. Although there were no differentially expressed genes associated with lactation diet, 304 differentially expressed genes were identified between cows with high and low circulating NEFA, with 118 upregulated and 186 downregulated. Gene Ontology enrichment analysis demonstrated that biological adhesion and immune system process were foremost among various PBMC functions which were altered relating to body defenses and immunity. High NEFA concentrations were associated with inhibited cellular adhesion function by downregulating 20 out of 26 genes (by up to 17-fold) related to this process. Medium NEFA concentrations altered a similar set of functions as high NEFA, but with smaller enrichment scores. Localization and immune system process were most significant, with biological adhesion ranking only eleventh. Our results demonstrated that increased circulating NEFA concentrations, but not diet, were associated with immune system processes in PBMC in early lactation cows. Leukocyte cell-to-cell adhesion was inhibited when the NEFA concentration exceeded 750 µM, which would reduce the efficiency of diapedesis and so contribute to decreased body defense mechanisms and predispose animals to infection.

SatuRn:Scalable analysis of differential transcript usagefor bulk and single-cell RNA-sequencing applications.

Alternative splicing produces multiple functional transcripts from a single gene. Dysregulation of splicing is known to be associated with disease and as a hallmark of cancer. Existing tools for differential transcript usage (DTU) analysis either lack in performance, cannot account for complex experimental designs or do not scale to massive scRNA-seq data. We introduce satuRn, a fast and flexible quasi-binomial generalized linear modelling framework that is on par with the best performing DTU methods from the bulk RNA-seq realm, while providing good false discovery rate control, addressing complex experimental designs and scaling to scRNA-seqapplications.

PseudotimeDE: inference of differential gene expression along cell pseudotime with well-calibrated p-values from single-cell RNA sequencing data

To investigate molecular mechanisms underlying cell state changes, a crucial analysis is to identify differentially expressed (DE) genes along the pseudotime inferred from single-cell RNA-sequencing data. However, existing methods do not account for pseudotime inference uncertainty, and they have either ill-posed p-values or restrictive models. Here we propose PseudotimeDE, a DE gene identification method that adapts to various pseudotime inference methods, accounts for pseudotime inference uncertainty, and outputs well-calibrated p-values. Comprehensive simulations and real-data applications verify that PseudotimeDE outperforms existing methods in false discovery rate control and power.

Robust and Efficient Parametric Spectral Density Estimation for High-Throughput Data

Modern scientific instruments readily record various dynamical phenomena at high frequency and for extended durations. Spanning timescales across several orders of magnitude, such “high-throughput” (HTP) data are routinely analyzed with parametric models in the frequency domain. However, the large size of HTP datasets can render maximum likelihood estimation prohibitively expensive. Moreover, HTP recording devices are operated by extensive electronic circuitry, producing periodic noise to which parameter estimates are highly sensitive. This article proposes to address these issues with a two-stage approach. Preliminary parameter estimates are first obtained by a periodogram variance-stabilizing procedure, for which data compression greatly reduces computational costs with minimal impact to statistical efficiency. Next, a novel test with false discovery rate control eliminates most periodic outliers, to which the second-stage estimator becomes more robust. Extensive simulations and experimental results indicate that for a widely used model in HTP data analysis, a substantial reduction in mean squared error can be expected by applying our methodology.

Variable selection with false discovery rate control in deep neural networks

Deep neural networks are famous for their high prediction accuracy, but they are also known for their black-box nature and poor interpretability. We consider the problem of variable selection, that is, selecting the input variables that have significant predictive power on the output, in deep neural networks. Most existing variable selection methods for neural networks are only applicable to shallow networks or are computationally infeasible on large datasets; moreover, they lack a control on the quality of selected variables. Here we propose a backward elimination procedure called SurvNet, which is based on a new measure of variable importance that applies to a wide variety of networks. More importantly, SurvNet is able to estimate and control the false discovery rate of selected variables empirically. Further, SurvNet adaptively determines how many variables to eliminate at each step in order to maximize the selection efficiency. The validity and efficiency of SurvNet are shown on various simulated and real datasets, and its performance is compared with other methods. Especially, a systematic comparison with knockoff-based methods shows that although they have more rigorous false discovery rate control on data with strong variable correlation, SurvNet usually has higher power. Identifying salient input features can be a challenge in neural networks. The authors developed a variable selection procedure with false discovery rate control that works on classification or regression problems, one or multiple output neurons, and deep or shallow neural networks.

Adapting to one- and two-way classified structures of hypotheses while controlling the false discovery rate

There is ample research on false discovery rate (FDR) control for testing hypotheses classified according to one criterion. However, scenarios of hypotheses partitioned via two different criteria are often encountered in practice. Such two-way classification encodes more structural information in the associated multiple testing of the hypotheses than its one-way or un-classified counterparts. Unfortunately, there seems to be very little research tailored for multiple testing under that classification setting. This article proposes weighted versions of the Benjamini–Hochberg (BH) method, both in their oracle and data-adaptive forms, efficiently capturing one- or two-way classified structure of hypotheses through appropriately chosen weights. The proposed methods control FDR non-asymptotically in their oracle forms under positive regression dependence on subset (PRDS) of null p-values and in their data-adaptive forms for independent p-values. The one-way data-adaptive methods are asymptotically conservative under weak dependence. Simulations demonstrate these methods’ superior power performances over some contemporary procedures and provide evidence of their non-asymptotic conservativeness under certain dependence scenarios. The proposed two-way adaptive procedure is effectively applied to a data set from microbial abundance study.

Aggregating Knockoffs for False Discovery Rate Control with an Application to Gut Microbiome Data.

Recent discoveries suggest that our gut microbiome plays an important role in our health and wellbeing. However, the gut microbiome data are intricate; for example, the microbial diversity in the gut makes the data high-dimensional. While there are dedicated high-dimensional methods, such as the lasso estimator, they always come with the risk of false discoveries. Knockoffs are a recent approach to control the number of false discoveries. In this paper, we show that knockoffs can be aggregated to increase power while retaining sharp control over the false discoveries. We support our method both in theory and simulations, and we show that it can lead to new discoveries on microbiome data from the American Gut Project. In particular, our results indicate that several phyla that have been overlooked so far are associated with obesity.

Estimating minimum effect with outlier selection

We introduce one-sided versions of Huber’s contamination model, in which corrupted samples tend to take larger values than uncorrupted ones. Two intertwined problems are addressed: estimation of the mean of the uncorrupted samples (minimum effect) and selection of the corrupted samples (outliers). Regarding estimation of the minimum effect, we derive the minimax risks and introduce estimators that are adaptive with respect to the unknown number of contaminations. The optimal convergence rates differ from the ones in the classical Huber contamination model. This fact uncovers the effect of the one-sided structural assumption of the contaminations. As for the problem of selecting the outliers, we formulate the problem in a multiple testing framework for which the location and scaling of the null hypotheses are unknown. We rigorously prove that estimating the null hypothesis while maintaining a theoretical guarantee on the amount of the falsely selected outliers is possible, both through false discovery rate (FDR) and through post hoc bounds. As a by-product, we address a long-standing open issue on FDR control under equi-correlation, which reinforces the interest of removing dependency in such a setting.

False discovery rate for functional data

Since Benjamini and Hochberg introduced false discovery rate (FDR) in their seminal paper, this has become a very popular approach to the multiple comparisons problem. An increasingly popular topic within functional data analysis is local inference, i.e. the continuous statistical testing of a null hypothesis along the domain. The principal issue in this topic is the infinite amount of tested hypotheses, which can be seen as an extreme case of the multiple comparisons problem. In this paper, we define and discuss the notion of FDR in a very general functional data setting. Moreover, a continuous version of the Benjamini–Hochberg procedure is introduced along with a definition of adjusted p value function. Some general conditions are stated, under which the functional Benjamini–Hochberg procedure provides control of the functional FDR. Two different simulation studies are presented; the first study has a one-dimensional domain and a comparison with another state-of-the-art method, and the second study has a planar two-dimensional domain. Finally, the proposed method is applied to satellite measurements of Earth temperature. In detail, we aim at identifying the regions of the planet where temperature has significantly increased in the last decades. After adjustment, large areas are still significant.

Comprehensive Comparative Analysis of Local False Discovery Rate Control Methods.

Due to the advance in technology, the type of data is getting more complicated and large-scale. To analyze such complex data, more advanced technique is required. In case of omics data from two different groups, it is interesting to find significant biomarkers between two groups while controlling error rate such as false discovery rate (FDR). Over the last few decades, a lot of methods that control local false discovery rate have been developed, ranging from one-dimensional to k-dimensional FDR procedure. For comparison study, we select three of them, which have unique and significant properties: Efron’s approach, Ploner’s approach, and Kim’s approach in chronological order. The first approach is one-dimensional approach while the other two are two-dimensional ones. Furthermore, we consider two more variants of Ploner’s approach. We compare the performance of those methods on both simulated and real data.

Hypothesis testing for high-dimensional multivariate regression with false discovery rate control

Motivated by applications in microarray data analysis, we consider the problem of estimating the row sparsity of the coefficient matrix in multivariate regression. The problem is formulated as a row-wise multiple testing problem. A new test statistic is constructed for each row of the coefficient matrix, and a row-wise multiple testing procedure is proposed. Our approach takes advantage of the row sparsity of the coefficient matrix to gain high power and handles the correlation structure of the error vector. Asymptotic distribution of the test statistic and asymptotic false discovery control of the row-wise multiple testing procedure are provided. Numerical results are also presented to evaluate the performance of the proposed method in both simulation studies and real data analysis.

Online control of the familywise error rate.

Biological research often involves testing a growing number of null hypotheses as new data are accumulated over time. We study the problem of online control of the familywise error rate, that is testing an a priori unbounded sequence of hypotheses (p-values) one by one over time without knowing the future, such that with high probability there are no false discoveries in the entire sequence. This paper unifies algorithmic concepts developed for offline (single batch) familywise error rate control and online false discovery rate control to develop novel online familywise error rate control methods. Though many offline familywise error rate methods (e.g., Bonferroni, fallback procedures and Sidak's method) can trivially be extended to the online setting, our main contribution is the design of new, powerful, adaptive online algorithms that control the familywise error rate when the p-values are independent or locally dependent in time. Our numerical experiments demonstrate substantial gains in power, that are also formally proved in an idealized Gaussian sequence model. A promising application to the International Mouse Phenotyping Consortium is described.

Circulating MicroRNAs in Relation to Esophageal Adenocarcinoma Diagnosis and Survival.

Tissue miRNA can discriminate between esophageal adenocarcinoma (EA) and normal epithelium. However, no studies have examined a comprehensive panel of circulating miRNAs in relation to EA diagnosis and survival. We used all 62 EA cases from the US Multi-Center case-control study with available serum matched 1:1 to controls. Cases were followed for vital status. MiRNAs (n = 2064) were assessed using the HTG EdgeSeq miRNA Whole Transcriptome Assay. Differential expression analysis of miRNAs in relation to case-control status was conducted. In cases, Cox regression models were fit to estimate hazard ratios (HRs) and 95% confidence intervals (CIs) for all-cause mortality. P values were adjusted using the Benjamini-Hochberg (BH) procedure for false discovery rate control. Predictive performance was assessed using cross-validation. Sixty-eight distinct miRNAs were significantly upregulated between cases and controls (e.g., miR-1255b-2-3p fold change = 1.74, BH-adjusted P = 0.01). Assessing the predictive performance of these significantly upregulated miRNAs yielded 60% sensitivity, 65% specificity, and 0.62 AUC. miR-4253 and miR-1238-5p were associated with risk of mortality after EA diagnosis (HR = 4.85, 95% CI: 2.30-10.23, BH-adjusted P = 0.04 and HR = 3.81, 95% CI: 2.02-7.19, BH-adjusted P = 0.04, respectively). While they require replication, these findings suggest that circulating miRNAs may be associated with EA diagnosis and survival.

School outcomes of adolescents with cerebral palsy in Sweden.

AimTo study school outcomes of adolescents with cerebral palsy (CP) compared with a matched comparison group from the general population, and to observe to what extent sociodemographic and disability‐specific factors are associated with school outcomes.MethodThis was a register study of persons with CP in Sweden, born between 1990 and 1999, with a matched comparison group. Logistic regressions were used to estimate the associations between CP and disability‐specific factors and school outcomes (receiving final grades, grade scores, fulfilling the requirements for progressing to secondary school/university, and attending secondary school).ResultsChildren with CP had substantially lower school achievement compared with a general population sample. Much of the difference can be attributed to intellectual disability; however, CP remained strongly negatively associated with school outcomes. Ability to communicate in an effective manner explained most of the variation in children with CP, whereas motor function played a smaller role.InterpretationThe results suggest that school achievements might be improved if the communication barrier could be reduced, for example by ascertaining access to appropriate communication devices and by educators being aware that communicative difficulties do not necessarily imply intellectual disability. This might enhance the school experience and create an environment where children with CP can reach their full potential.What this study adds There are reduced educational outcomes associated with cerebral palsy.Negative outcomes are mainly driven by reduced communicative and intellectual ability.Gross motor function is associated with outcomes, but plays a smaller part.

Heteroscedasticity-Adjusted Ranking and Thresholding for Large-Scale Multiple Testing

Standardization has been a widely adopted practice in multiple testing, for it takes into account the variability in sampling and makes the test statistics comparable across different study units. However, despite conventional wisdom to the contrary, we show that there can be a significant loss in information from basing hypothesis tests on standardized statistics rather than the full data. We develop a new class of heteroscedasticity-adjusted ranking and thresholding (HART) rules that aim to improve existing methods by simultaneously exploiting commonalities and adjusting heterogeneities among the study units. The main idea of HART is to bypass standardization by directly incorporating both the summary statistic and its variance into the testing procedure. A key message is that the variance structure of the alternative distribution, which is subsumed under standardized statistics, is highly informative and can be exploited to achieve higher power. The proposed HART procedure is shown to be asymptotically valid and optimal for false discovery rate (FDR) control. Our simulation results demonstrate that HART achieves substantial power gain over existing methods at the same FDR level. We illustrate the implementation through a microarray analysis of myeloma.

Optimal Control of False Discovery Criteria in the Two-Group Model

AbstractThe highly influential two-group model in testing a large number of statistical hypotheses assumes that the test statistics are drawn independently from a mixture of a high probability null distribution and a low probability alternative. Optimal control of the marginal false discovery rate (mFDR), in the sense that it provides maximal power (expected true discoveries) subject to mFDR control, is known to be achieved by thresholding the local false discovery rate (locFDR), the probability of the hypothesis being null given the set of test statistics, with a fixed threshold. We address the challenge of controlling optimally the popular false discovery rate (FDR) or positive FDR (pFDR) in the general two-group model, which also allows for dependence between the test statistics. These criteria are less conservative than the mFDR criterion, so they make more rejections in expectation. We derive their optimal multiple testing (OMT) policies, which turn out to be thresholding the locFDR with a threshold that is a function of the entire set of statistics. We develop an efficient algorithm for finding these policies, and use it for problems with thousands of hypotheses. We illustrate these procedures on gene expression studies.