False Discovery Rate Control Research Articles

Leveraging cell type specific regulatory regions to detect SNPs associated with tissue factor pathway inhibitor plasma levels.

Tissue factor pathway inhibitor (TFPI) regulates the formation of intravascular blood clots, which manifest clinically as ischemic heart disease, ischemic stroke, and venous thromboembolism (VTE). TFPI plasma levels are heritable, but the genetics underlying TFPI plasma level variability are poorly understood. Herein we report the first genome-wide association scan (GWAS) of TFPI plasma levels, conducted in 251 individuals from five extended French-Canadian Families ascertained on VTE. To improve discovery, we also applied a hypothesis-driven (HD) GWAS approach that prioritized single nucleotide polymorphisms (SNPs) in (1) hemostasis pathway genes, and (2) vascular endothelial cell (EC) regulatory regions, which are among the highest expressers of TFPI. Our GWAS identified 131 SNPs with suggestive evidence of association (P-value < 5 × 10-8 ), but no SNPs reached the genome-wide threshold for statistical significance. Hemostasis pathway genes were not enriched for TFPI plasma level associated SNPs (global hypothesis test P-value = 0.147), but EC regulatory regions contained more TFPI plasma level associated SNPs than expected by chance (global hypothesis test P-value = 0.046). We therefore stratified our genome-wide SNPs, prioritizing those in EC regulatory regions via stratified false discovery rate (sFDR) control, and reranked the SNPs by q-value. The minimum q-value was 0.27, and the top-ranked SNPs did not show association evidence in the MARTHA replication sample of 1,033 unrelated VTE cases. Although this study did not result in new loci for TFPI, our work lays out a strategy to utilize epigenomic data in prioritization schemes for future GWAS studies.

Epsilon-Q: An Automated Analyzer Interface for Mass Spectral Library Search and Label-Free Protein Quantification.

Mass spectrometry (MS) is a widely used proteome analysis tool for biomedical science. In an MS-based bottom-up proteomic approach to protein identification, sequence database (DB) searching has been routinely used because of its simplicity and convenience. However, searching a sequence DB with multiple variable modification options can increase processing time, false-positive errors in large and complicated MS data sets. Spectral library searching is an alternative solution, avoiding the limitations of sequence DB searching and allowing the detection of more peptides with high sensitivity. Unfortunately, this technique has less proteome coverage, resulting in limitations in the detection of novel and whole peptide sequences in biological samples. To solve these problems, we previously developed the "Combo-Spec Search" method, which uses manually multiple references and simulated spectral library searching to analyze whole proteomes in a biological sample. In this study, we have developed a new analytical interface tool called "Epsilon-Q" to enhance the functions of both the Combo-Spec Search method and label-free protein quantification. Epsilon-Q performs automatically multiple spectral library searching, class-specific false-discovery rate control, and result integration. It has a user-friendly graphical interface and demonstrates good performance in identifying and quantifying proteins by supporting standard MS data formats and spectrum-to-spectrum matching powered by SpectraST. Furthermore, when the Epsilon-Q interface is combined with the Combo-Spec search method, called the Epsilon-Q system, it shows a synergistic function by outperforming other sequence DB search engines for identifying and quantifying low-abundance proteins in biological samples. The Epsilon-Q system can be a versatile tool for comparative proteome analysis based on multiple spectral libraries and label-free quantification.

Accumulation Tests for FDR Control in Ordered Hypothesis Testing

ABSTRACT Multiple testing problems arising in modern scientific applications can involve simultaneously testing thousands or even millions of hypotheses, with relatively few true signals. In this article, we consider the multiple testing problem where prior information is available (for instance, from an earlier study under different experimental conditions), that can allow us to test the hypotheses as a ranked list to increase the number of discoveries. Given an ordered list of n hypotheses, the aim is to select a data-dependent cutoff k and declare the first k hypotheses to be statistically significant while bounding the false discovery rate (FDR). Generalizing several existing methods, we develop a family of “accumulation tests” to choose a cutoff k that adapts to the amount of signal at the top of the ranked list. We introduce a new method in this family, the HingeExp method, which offers higher power to detect true signals compared to existing techniques. Our theoretical results prove that these methods control a modified FDR on finite samples, and characterize the power of the methods in the family. We apply the tests to simulated data, including a high-dimensional model selection problem for linear regression. We also compare accumulation tests to existing methods for multiple testing on a real data problem of identifying differential gene expression over a dosage gradient. Supplementary materials for this article are available online.

Testing and support recovery of multiple high-dimensional covariance matrices with false discovery rate control

Motivated by applications in genomics, we study in this paper four interrelated high-dimensional hypothesis testing problems on dependence structures among multiple populations. A new test statistic is constructed for testing the global hypothesis that multiple covariance matrices are equal, and its limiting null distribution is established. Correction methods are introduced to improve the accuracy of the test for finite samples. It is shown that the proposed tests are powerful against sparse alternatives and enjoy certain optimality properties. We then propose a multiple testing procedure for simultaneously testing the equality of the entries of the covariance matrices across multiple populations. The proposed method is shown to control the false discovery rate. A simulation study demonstrates that the proposed tests maintain the desired error rates under the null and have good power under the alternative. The methods are also applied to a Novartis multi-tissue analysis. In addition, testing and support recovery of submatrices of multiple covariance matrices are studied.

Are neutral loss and internal product ions useful for top-down protein identification?

Neutral loss and internal product ions have been found to be significant in both peptide and protein tandem mass spectra and they have been proposed to be included in database search and for protein identification. In addition to common canonical b/y ions in collision-based dissociation or c/z ions in electron-based dissociation, inclusion of neutral loss and internal product ions would certainly make better use of tandem mass spectra data; however, their ultimate utility for protein identification with false discovery rate control remains unclear. Here we report our proteome-level utility benchmarking of neutral loss and internal product ions with tandem mass spectra of intact E. coli proteome. Utility of internal product ions was further evaluated at the protein level using selected tandem mass spectra of individual E. coli proteins. We found that both neutral loss and internal products ions do not have direct utility for protein identification when they were used for scoring of P Score; but they do have indirect utility for provision of more canonical b/y ions when they are included in the database search and overlapping ions between different ion types are resolved. Biological significanceTandem mass spectrometry has evolved to be a state-of-the-art method for characterization of protein primary structures (including amino acid sequence, post-translational modifications (PTMs) as well as their site location), where full study and utilization tandem mass spectra and product ions are indispensable. This primary structure information is essential for higher order structure and eventual function study of proteins.

Can mindfulness-based interventions improve the quality of life of patients with moderate/severe alopecia areata? A prospective pilot study

Can mindfulness-based interventions improve the quality of life of patients with moderate/severe alopecia areata? A prospective pilot study

MOST: most-similar ligand based approach to target prediction

BackgroundMany computational approaches have been used for target prediction, including machine learning, reverse docking, bioactivity spectra analysis, and chemical similarity searching. Recent studies have suggested that chemical similarity searching may be driven by the most-similar ligand. However, the extent of bioactivity of most-similar ligands has been oversimplified or even neglected in these studies, and this has impaired the prediction power.ResultsHere we propose the MOst-Similar ligand-based Target inference approach, namely MOST, which uses fingerprint similarity and explicit bioactivity of the most-similar ligands to predict targets of the query compound. Performance of MOST was evaluated by using combinations of different fingerprint schemes, machine learning methods, and bioactivity representations. In sevenfold cross-validation with a benchmark Ki dataset from CHEMBL release 19 containing 61,937 bioactivity data of 173 human targets, MOST achieved high average prediction accuracy (0.95 for pKi ≥ 5, and 0.87 for pKi ≥ 6). Morgan fingerprint was shown to be slightly better than FP2. Logistic Regression and Random Forest methods performed better than Naïve Bayes. In a temporal validation, the Ki dataset from CHEMBL19 were used to train models and predict the bioactivity of newly deposited ligands in CHEMBL20. MOST also performed well with high accuracy (0.90 for pKi ≥ 5, and 0.76 for pKi ≥ 6), when Logistic Regression and Morgan fingerprint were employed. Furthermore, the p values associated with explicit bioactivity were found be a robust index for removing false positive predictions. Implicit bioactivity did not offer this capability. Finally, p values generated with Logistic Regression, Morgan fingerprint and explicit activity were integrated with a false discovery rate (FDR) control procedure to reduce false positives in multiple-target prediction scenario, and the success of this strategy it was demonstrated with a case of fluanisone. In the case of aloe-emodin’s laxative effect, MOST predicted that acetylcholinesterase was the mechanism-of-action target; in vivo studies validated this prediction.ConclusionsUsing the MOST approach can result in highly accurate and robust target prediction. Integrated with a FDR control procedure, MOST provides a reliable framework for multiple-target inference. It has prospective applications in drug repurposing and mechanism-of-action target prediction.

Global Testing and Large-Scale Multiple Testing for High-Dimensional Covariance Structures

Driven by a wide range of contemporary applications, statistical inference for covariance structures has been an active area of current research in high-dimensional statistics. This review provides a selective survey of some recent developments in hypothesis testing for high-dimensional covariance structures, including global testing for the overall pattern of the covariance structures and simultaneous testing of a large collection of hypotheses on the local covariance structures with false discovery proportion and false discovery rate control. Both one-sample and two-sample settings are considered. The specific testing problems discussed include global testing for the covariance, correlation, and precision matrices, and multiple testing for the correlations, Gaussian graphical models, and differential networks.

Controlling the marginal false discovery rate in inferences from a soil dataset with α‐investment

SummaryLarge datasets on soil provide a temptation to search for relations between variables and then to model and make inferences about them with statistical methods more properly used to test preplanned hypotheses on data from designed experiments or sample surveys. The control of family‐wise error rate (FWER) is one way to improve the robustness of inferences from tests of multiple hypotheses. In its simplest form, hypothesis testing with FWER control lacks statistical power. The α‐investment approach to controlling the marginal false discovery rate is one method proposed to improve statistical power. In this paper I outline the α‐investment approach and then demonstrate it in the analysis of a dataset on the rate of CO2 emission from incubated intact cores of soil from a transect over Cretaceous rocks in eastern England. Hypotheses are advanced after considering the literature and examining relations among the available soil variables that might be proposed as explanatory factors for the variation of CO2 emissions. They are then tested in sequence with α‐investment, such that the rejection of null hypotheses increases the power to reject later ones, while controlling the overall marginal false discovery rate at a specified value. This paper illustrates the use of α‐investment to test a multiple set of hypotheses on a soil dataset; statistical power is improved by ordering the sequence of hypotheses on the basis of process knowledge. The approach could be useful in other areas of soil science where covariates must be selected for predictive statistical models, notably in the development of pedotransfer functions and in digital soil mapping.Highlights α‐investment controls marginal false discovery rate in statistical inference. Hypotheses were advanced about soil factors that affect CO2 emission from soil. These hypotheses were tested in sequence with control of marginal false discovery rate. Soil properties, land use and parent material were significant predictors.

A two-stage hidden Markov model design for biomarker detection, with application to microbiome research.

It has been recognized that for appropriately ordered data, hidden Markov models (HMM) with local false discovery rate (FDR) control can increase the power to detect significant associations. For many high-throughput technologies, the cost still limits their application. Two-stage designs are attractive, in which a set of interesting features or biomarkers is identified in a first stage, and then followed up in a second stage. However, to our knowledge no two-stage FDR control with HMMs has been developed. In this paper, we study an efficient HMM-FDR based two-stage design, using a simple integrated analysis procedure across the stages. Numeric studies show its excellent performance when compared to available methods. A power analysis method is also proposed. We use examples from microbiome data to illustrate the methods.

Bon-EV: an improved multiple testing procedure for controlling false discovery rates

BackgroundStability of multiple testing procedures, defined as the standard deviation of total number of discoveries, can be used as an indicator of variability of multiple testing procedures. Improving stability of multiple testing procedures can help to increase the consistency of findings from replicated experiments. Benjamini-Hochberg’s and Storey’s q-value procedures are two commonly used multiple testing procedures for controlling false discoveries in genomic studies. Storey’s q-value procedure has higher power and lower stability than Benjamini-Hochberg’s procedure. To improve upon the stability of Storey’s q-value procedure and maintain its high power in genomic data analysis, we propose a new multiple testing procedure, named Bon-EV, to control false discovery rate (FDR) based on Bonferroni’s approach.ResultsSimulation studies show that our proposed Bon-EV procedure can maintain the high power of the Storey’s q-value procedure and also result in better FDR control and higher stability than Storey’s q-value procedure for samples of large size(30 in each group) and medium size (15 in each group) for either independent, somewhat correlated, or highly correlated test statistics. When sample size is small (5 in each group), our proposed Bon-EV procedure has performance between the Benjamini-Hochberg procedure and the Storey’s q-value procedure. Examples using RNA-Seq data show that the Bon-EV procedure has higher stability than the Storey’s q-value procedure while maintaining equivalent power, and higher power than the Benjamini-Hochberg’s procedure.ConclusionsFor medium or large sample sizes, the Bon-EV procedure has improved FDR control and stability compared with the Storey’s q-value procedure and improved power compared with the Benjamini-Hochberg procedure. The Bon-EV multiple testing procedure is available as the BonEV package in R for download at https://CRAN.R-project.org/package=BonEV.

Controlling the Rate of GWAS False Discoveries.

With the rise of both the number and the complexity of traits of interest, control of the false discovery rate (FDR) in genetic association studies has become an increasingly appealing and accepted target for multiple comparison adjustment. While a number of robust FDR-controlling strategies exist, the nature of this error rate is intimately tied to the precise way in which discoveries are counted, and the performance of FDR-controlling procedures is satisfactory only if there is a one-to-one correspondence between what scientists describe as unique discoveries and the number of rejected hypotheses. The presence of linkage disequilibrium between markers in genome-wide association studies (GWAS) often leads researchers to consider the signal associated to multiple neighboring SNPs as indicating the existence of a single genomic locus with possible influence on the phenotype. This a posteriori aggregation of rejected hypotheses results in inflation of the relevant FDR. We propose a novel approach to FDR control that is based on prescreening to identify the level of resolution of distinct hypotheses. We show how FDR-controlling strategies can be adapted to account for this initial selection both with theoretical results and simulations that mimic the dependence structure to be expected in GWAS. We demonstrate that our approach is versatile and useful when the data are analyzed using both tests based on single markers and multiple regression. We provide an R package that allows practitioners to apply our procedure on standard GWAS format data, and illustrate its performance on lipid traits in the North Finland Birth Cohort 66 cohort study.

Oracle P-values and variable screening

The concept of P-value was proposed by Fisher to measure inconsistency of data with a specified null hypothesis, and it plays a central role in statistical inference. For classical linear regression analysis, it is a standard procedure to calculate P-values for regression coefficients based on least squares estimator (LSE) to determine their significance. However, for high dimensional data when the number of predictors exceeds the sample size, ordinary least squares are no longer proper and there is not a valid definition for P-values based on LSE. It is also challenging to define sensible P-values for other high dimensional regression methods such as penalization and resampling methods. In this paper, we introduce a new concept called oracle P-value to generalize traditional P-values based on LSE to high dimensional sparse regression models. Then we propose several estimation procedures to approximate oracle P-values for real data analysis. We show that the oracle P-value framework is useful for developing new and powerful tools to enhance high dimensional data analysis, including variable ranking, variable selection, and screening procedures with false discovery rate (FDR) control. Numerical examples are then presented to demonstrate performance of the proposed methods.

The control of the false discovery rate in fixed sequence multiple testing

Controlling false discovery rate (FDR) is a powerful approach to multiple testing. In many applications, the tested hypotheses have an inherent hierarchical structure. In this paper, we focus on the fixed sequence structure where the testing order of the hypotheses has been strictly specified in advance. We are motivated to study such a structure, since it is the most basic of hierarchical structure, yet it is often seen in real applications such as statistical process control and streaming data analysis. We first consider a conventional fixed sequence method that stops testing once an acceptance occurs, and develop such a method controlling FDR under both arbitrary and negative dependencies. The method under arbitrary dependency is shown to be unimprovable without losing control of FDR and, unlike existing FDR methods; it cannot be improved even by restricting to the usual positive regression dependence on subset (PRDS) condition. To account for any potential mistakes in the ordering of the tests, we extend the conventional fixed sequence method to one that allows more but a given number of acceptances. Simulation studies show that the proposed procedures can be powerful alternatives to existing FDR controlling procedures. The proposed procedures are illustrated through a real data set from a microarray experiment.

Statistical Evaluation of Labeled Comparative Profiling Proteomics Experiments Using Permutation Test.

Comparative profiling proteomics experiments are important tools in biological research. In such experiments, tens to hundreds of thousands of peptides are measured simultaneously, with the goal of inferring protein abundance levels. Statistical evaluation of these datasets are required to determine proteins that are differentially abundant between the test samples. Previously we have reported the non-normal distribution of SILAC datasets, and demonstrated the permutation test to be a superior method for the statistical evaluation of non-normal peptide ratios. This chapter outlines the steps and the R scripts that can be used for performing permutation analysis with false discovery rate control via the Benjamini-Yekutieli method.

Hypothesis testing of matrix graph model with application to brain connectivity analysis.

Brain connectivity analysis is now at the foreground of neuroscience research. A connectivity network is characterized by a graph, where nodes represent neural elements such as neurons and brain regions, and links represent statistical dependence that is often encoded in terms of partial correlation. Such a graph is inferred from the matrix-valued neuroimaging data such as electroencephalography and functional magnetic resonance imaging. There have been a good number of successful proposals for sparse precision matrix estimation under normal or matrix normal distribution; however, this family of solutions does not offer a direct statistical significance quantification for the estimated links. In this article, we adopt a matrix normal distribution framework and formulate the brain connectivity analysis as a precision matrix hypothesis testing problem. Based on the separable spatial-temporal dependence structure, we develop oracle and data-driven procedures to test both the global hypothesis that all spatial locations are conditionally independent, and simultaneous tests for identifying conditional dependent spatial locations with false discovery rate control. Our theoretical results show that the data-driven procedures perform asymptotically as well as the oracle procedures and enjoy certain optimality properties. The empirical finite-sample performance of the proposed tests is studied via intensive simulations, and the new tests are applied on a real electroencephalography data analysis.

Thep-filter: Multilayer False Discovery Rate Control for Grouped Hypotheses

SummaryIn many practical applications of multiple testing, there are natural ways to partition the hypotheses into groups by using the structural, spatial or temporal relatedness of the hypotheses, and this prior knowledge is not used in the classical Benjamini–Hochberg procedure for controlling the false discovery rate (FDR). When one can define (possibly several) such partitions, it may be desirable to control the group FDR simultaneously for all partitions (as special cases, the ‘finest’ partition divides the n hypotheses into n groups of one hypothesis each, and this corresponds to controlling the usual notion of FDR, whereas the ‘coarsest’ partition puts all n hypotheses into a single group, and this corresponds to testing the global null hypothesis). We introduce the p-filter, which takes as input a list of n p-values and M ⩾ 1 partitions of hypotheses, and produces as output a list of n or fewer discoveries such that the group FDR is provably simultaneously controlled for all partitions. Importantly, since the partitions are arbitrary, our procedure can also handle multiple partitions which are non-hierarchical. The p-filter generalizes two classical procedures—when M = 1, choosing the finest partition into n singletons, we exactly recover the Benjamini–Hochberg procedure, whereas, choosing instead the coarsest partition with a single group of size n, we exactly recover the Simes test for the global null hypothesis. We verify our findings with simulations that show how this technique can not only lead to the aforementioned multilayer FDR control but also lead to improved precision of rejected hypotheses. We present some illustrative results from an application to a neuroscience problem with functional magnetic resonance imaging data, where hypotheses are explicitly grouped according to predefined regions of interest in the brain, thus allowing the scientist to employ field-specific prior knowledge explicitly and flexibly.

The MaxQuant computational platform for mass spectrometry-based shotgun proteomics.

MaxQuant is one of the most frequently used platforms for mass-spectrometry (MS)-based proteomics data analysis. Since its first release in 2008, it has grown substantially in functionality and can be used in conjunction with more MS platforms. Here we present an updated protocol covering the most important basic computational workflows, including those designed for quantitative label-free proteomics, MS1-level labeling and isobaric labeling techniques. This protocol presents a complete description of the parameters used in MaxQuant, as well as of the configuration options of its integrated search engine, Andromeda. This protocol update describes an adaptation of an existing protocol that substantially modifies the technique. Important concepts of shotgun proteomics and their implementation in MaxQuant are briefly reviewed, including different quantification strategies and the control of false-discovery rates (FDRs), as well as the analysis of post-translational modifications (PTMs). The MaxQuant output tables, which contain information about quantification of proteins and PTMs, are explained in detail. Furthermore, we provide a short version of the workflow that is applicable to data sets with simple and standard experimental designs. The MaxQuant algorithms are efficiently parallelized on multiple processors and scale well from desktop computers to servers with many cores. The software is written in C# and is freely available at http://www.maxquant.org.

Cramér-type moderate deviations for Studentized two-sample $U$-statistics with applications

Two-sample $U$-statistics are widely used in a broad range of applications, including those in the fields of biostatistics and econometrics. In this paper, we establish sharp Cramér-type moderate deviation theorems for Studentized two-sample $U$-statistics in a general framework, including the two-sample $t$-statistic and Studentized Mann–Whitney test statistic as prototypical examples. In particular, a refined moderate deviation theorem with second-order accuracy is established for the two-sample $t$-statistic. These results extend the applicability of the existing statistical methodologies from the one-sample $t$-statistic to more general nonlinear statistics. Applications to two-sample large-scale multiple testing problems with false discovery rate control and the regularized bootstrap method are also discussed.

Inflammatory mediator polymorphisms associate with initial periodontitis in adolescents.

Several studies have addressed cytokine gene polymorphisms and their possible associations with periodontitis. We examined the association between salivary anti‐ and pro‐inflammatory mediator polymorphisms and initial periodontitis in Finnish adolescents, taking into account the effect of smoking. Salivary samples of 93 clinically examined adolescents from Eastern Finland were analyzed. Their oral health and smoking habits were recorded. Periodontal probing depth (PPD), and bleeding on probing (BOP) at four sites per tooth, root calculus (RC), and visible plaque index (VPI) were recorded from the index teeth. Salivary MMP‐8 median values were assessed. The sites with ≥4 mm PD were categorized as follows: PPD1 = one or more ≥4 mm pocket, PPD2 = two or more ≥4 mm pockets, and PPD3 = three or more ≥4 mm pockets. Genomic DNA was extracted from 300 μl of the saliva samples by genomic QIAamp® DNA Blood Mini Kit and genotyped for polymorphisms. Genetic variants for genotyping were selected from the following genes of interest: S100A8, FCGR2A, FCGR2B, IL10, MMP8, MMP3, MMP13, VDR, TLR4, MMP2, MPO, ELANE, IL1A, IL1B, IL1RN, CD28, MMP9, DDX39B, NFKBIL1, LTA, TNF, SOD2, IL6, TLR4, TIMP1, and SYN1. After false discovery rate control (FDR), polymorphisms in MMP3 (rs679620, rs520540, rs639752), CD28 (rs3116496), and VDR (rs2228570) associated (FDR q < 0.05) with deepened periodontal pockets. Smoking did not affect the results. Genetic polymorphisms of pro‐inflammatory mediators MMP3, CD28, and VDR seem to link to initial periodontitis.