Adaptive False Discovery Rate Research Articles

New results for adaptive false discovery rate control with p-value weighting

New results for adaptive false discovery rate control with p-value weighting

Covariate Adaptive False Discovery Rate Control With Applications to Omics-Wide Multiple Testing

Conventional multiple testing procedures often assume hypotheses for different features are exchangeable. However, in many scientific applications, additional covariate information regarding the patterns of signals and nulls are available. In this article, we introduce an FDR control procedure in large-scale inference problem that can incorporate covariate information. We develop a fast algorithm to implement the proposed procedure and prove its asymptotic validity even when the underlying likelihood ratio model is misspecified and the p-values are weakly dependent (e.g., strong mixing). Extensive simulations are conducted to study the finite sample performance of the proposed method and we demonstrate that the new approach improves over the state-of-the-art approaches by being flexible, robust, powerful, and computationally efficient. We finally apply the method to several omics datasets arising from genomics studies with the aim to identify omics features associated with some clinical and biological phenotypes. We show that the method is overall the most powerful among competing methods, especially when the signal is sparse. The proposed covariate adaptive multiple testing procedure is implemented in the R package CAMT. Supplementary materials for this article are available online.

Shifts in the Holstein dairy cow milk fat globule membrane proteome that occur during the first week of lactation are affected by parity

BackgroundThe milk fat globule membrane (MFGM) proteomes of colostrum and transition milk are rich sources of proteins that are likely important for neonatal calf health. In addition, characterization of these proteomes could also yield valuable information regarding mammary gland physiology of the early postpartum lactating cow. The objectives of this research were to characterize the MFGM proteomes of colostrum and transition milk through sample collections at four timepoints postpartum, including the first milking (M1, colostrum), second milking (M2, transition milk), fourth milking (M4, transition milk), and fourteenth milking (M14, mature milk), and compare these proteomes between multiparous (MP; n = 10) and primiparous (PP; n = 10) Holstein dairy cows. Isolated MFGM proteins were labeled using Tandem Mass tagging and analyzed using liquid chromatography-tandem mass spectrometry (LC-MS/MS). Protein identification was completed using MASCOT and Sequest in Proteome Discoverer 2.2. The scaled abundance values were analyzed using PROC MIXED in SAS to determine the effects of milking (MIL), parity (PAR), and MIL × PAR. The adaptive false-discovery rate (FDR)-adjusted P values were determined using PROC MULTTEST. Protein characterization and bioinformatic analysis were completed using a combination of PANTHER, Blast, and Uniprot.ResultsA total of 104 common proteins were identified in each of the MFGM samples. Statistical analysis revealed that 70.2% of identified proteins were affected by MIL. Of these, 78.1% were lower in M14 compared with M1, including immune-related proteins lactotransferrin, lactadherin and hemopexin. Parity affected 44.2% of proteins. Of the proteins affected by PAR, 84.8% were higher in MP cows compared with PP cows, including apolipoprotein E and histones 2A, 2B, 3, and 4 b. Butyrophilin subfamily 1 member 1A and annexin 5 were higher in samples from PP cows. Milking × parity affected 32.7% of identified proteins, including lactotransferrin, gelsolin, vitamin D binding protein, and S100 proteins.ConclusionsThis research supports previous findings that the Holstein MFGM proteome changes rapidly during the first week of lactation. In addition, this research identifies the impact of parity on the colostrum and transition milk MFGM proteomes, which may be important for milk-fed calf health or for the identification of protein biomarkers for mammary functionality.

Influence of multiple hypothesis testing on reproducibility in neuroimaging research: A simulation study and Python-based software

BackgroundReproducibility of research findings has been recently questioned in many fields of science, including psychology and neurosciences. One factor influencing reproducibility is the simultaneous testing of multiple hypotheses, which entails false positive findings unless the analyzed p-values are carefully corrected. While this multiple testing problem is well known and studied, it continues to be both a theoretical and practical problem. New methodHere we assess reproducibility in simulated experiments in the context of multiple testing. We consider methods that control either the family-wise error rate (FWER) or false discovery rate (FDR), including techniques based on random field theory (RFT), cluster-mass based permutation testing, and adaptive FDR. Several classical methods are also considered. The performance of these methods is investigated under two different models. ResultsWe found that permutation testing is the most powerful method among the considered approaches to multiple testing, and that grouping hypotheses based on prior knowledge can improve power. We also found that emphasizing primary and follow-up studies equally produced most reproducible outcomes. Comparison with existing method(s)We have extended the use of two-group and separate-classes models for analyzing reproducibility and provide a new open-source software “MultiPy” for multiple hypothesis testing. ConclusionsOur simulations suggest that performing strict corrections for multiple testing is not sufficient to improve reproducibility of neuroimaging experiments. The methods are freely available as a Python toolkit “MultiPy” and we aim this study to help in improving statistical data analysis practices and to assist in conducting power and reproducibility analyses for new experiments.

Estimating the number of equal components for two high-dimensional mean vectors

In this article, we propose a new method for estimating the number of equal components m 0 of two m-dimensional population means when m is large. The proposed method can be used to estimate the number of equally expressed or differentially expressed genes in DNA microarray studies. It can also be applied in the step of estimating m 0 in adaptive false discovery rate controlling procedures. Simulation results show that the bias of the moment estimator is very small for both normal and non normal data. It has higher precision than existing methods in most cases. It has more evident advantage under non normal data.

Adaptive controls of FWER and FDR under block dependence

Often in multiple testing, the hypotheses appear in non-overlapping blocks with the associated p-values exhibiting dependence within but not between blocks. We consider adapting the Benjamini–Hochberg method for controlling the false discovery rate (FDR) and the Bonferroni method for controlling the familywise error rate (FWER) to such dependence structure without losing their ultimate controls over the FDR and FWER, respectively, in a non-asymptotic setting. We present variants of conventional adaptive Benjamini–Hochberg and Bonferroni methods with proofs of their respective controls over the FDR and FWER. Numerical evidence is presented to show that these new adaptive methods can capture the present dependence structure more effectively than the corresponding conventional adaptive methods. This paper offers a solution to the open problem of constructing adaptive FDR and FWER controlling methods under dependence in a non-asymptotic setting and providing real improvements over the corresponding non-adaptive ones.

Characterization of the colostrum and transition milk proteomes from primiparous and multiparous Holstein dairy cows

Colostrum plays a vital role in the nutrition, development, and immunity of a newborn calf. This study aimed to characterize the protein profile of colostrum and to identify changes in the colostrum proteome across parity during the transition to mature milk. Colostrum and transition milk samples were collected at milkings 1, 2, 4, and 14 after calving from multiparous (n = 10) and primiparous cows (n = 10). Samples were skimmed, fractionated, and enriched before analysis for low-abundance proteins by liquid chromatography-tandem mass spectroscopy (LC-MS/MS). Changes in protein abundances were analyzed using PROC MIXED in SAS (SAS Institute Inc., Cary, NC) with determination of the adaptive false discovery rate adjustment using a MULTTEST procedure to identify effects of parity (P), milking number (MN), and their interaction (MN×P). We identified 86 proteins through LC-MS/MS, including 3 low-abundance proteins that were affected by P, 78 that were affected by MN, and 36 affected by MN×P. Prominent ontological groupings of proteins affected by MN included defense or immunity proteins, such as immunoglobulins. Proteins involved in the plasminogen activating cascade and more broadly, blood coagulation, were affected by MN×P. The results of this study add to increasing knowledge of the colostrum and transition milk proteomes, and this is the first study to find evidence of different abundances of these proteins when examined across P, MN, and MN×P. These findings aid in the identification of potential milk protein biomarkers for mammary health during the early postpartum period.

A direct approach to estimating false discovery rates conditional on covariates.

Modern scientific studies from many diverse areas of research abound with multiple hypothesis testing concerns. The false discovery rate (FDR) is one of the most commonly used approaches for measuring and controlling error rates when performing multiple tests. Adaptive FDRs rely on an estimate of the proportion of null hypotheses among all the hypotheses being tested. This proportion is typically estimated once for each collection of hypotheses. Here, we propose a regression framework to estimate the proportion of null hypotheses conditional on observed covariates. This may then be used as a multiplication factor with the Benjamini–Hochberg adjusted p-values, leading to a plug-in FDR estimator. We apply our method to a genome-wise association meta-analysis for body mass index. In our framework, we are able to use the sample sizes for the individual genomic loci and the minor allele frequencies as covariates. We further evaluate our approach via a number of simulation scenarios. We provide an implementation of this novel method for estimating the proportion of null hypotheses in a regression framework as part of the Bioconductor package swfdr.

Multiple testing with discrete data: Proportion of true null hypotheses and two adaptive FDR procedures.

We consider multiple testing with false discovery rate (FDR) control when p values have discrete and heterogeneous null distributions. We propose a new estimator of the proportion of true null hypotheses and demonstrate that it is less upwardly biased than Storey's estimator and two other estimators. The new estimator induces two adaptive procedures, that is, an adaptive Benjamini-Hochberg (BH) procedure and an adaptive Benjamini-Hochberg-Heyse (BHH) procedure. We prove that the adaptive BH (aBH) procedure is conservative nonasymptotically. Through simulation studies, we show that these procedures are usually more powerful than their nonadaptive counterparts and that the adaptive BHH procedure is usually more powerful than the aBH procedure and a procedure based on randomized p-value. The adaptive procedures are applied to a study of HIV vaccine efficacy, where they identify more differentially polymorphic positions than the BH procedure at the same FDR level.

Adaptive False Discovery Rate Control for Heterogeneous Data

Efforts to develop more efficient multiple hypothesis testing procedures for false discovery rate (FDR) control have focused on incorporating an estimate of the proportion of true null hypotheses (such procedures are called adaptive) or exploiting heterogeneity across tests via some optimal weighting scheme. This paper combines these approaches using a weighted adaptive multiple decision function (WAMDF) framework. Optimal weights for a flexible random effects model are derived and a WAMDF that controls the FDR for arbitrary weighting schemes when test statistics are independent under the null hypotheses is given. Asymptotic and numerical assessment reveals that, under weak dependence, the proposed WAMDFs provide more efficient FDR control even if optimal weights are misspecified. The robustness and flexibility of the proposed methodology facilitates the development of more efficient, yet practical, FDR procedures for heterogeneous data. To illustrate, two different weighted adaptive FDR methods for heterogeneous sample sizes are developed and applied to data.

Oracle and adaptive false discovery rate controlling methods for one‐sided testing: theory and application in treatment effect evaluation

Economists are often interested in identifying effective policies or treatments together with subpopulations of individuals who respond positively (or with a sign that is expected) to these treatment interventions. In this paper, we propose an optimal false discovery rate controlling method that is especially useful for such one‐sided testing problems. The proposed procedure is optimal in the sense of minimizing the false non‐discovery rate while controlling the false discovery rate at a pre‐specified level; it uses a deconvolution method based on non‐parametric maximum likelihood estimation, which allows for a broader class of treatment effect distributions than existing methods do. The proposed test demonstrates good small‐sample performance in Monte Carlo simulations and it is applied to study the effect of attending a more selective high school in Romania. The application reveals strong evidence of treatment effect heterogeneity, in that students who marginally gain access to higher‐ranked schools are more likely to benefit if the higher‐ranked school has a relatively high admission score cut‐off – or, in other words, is more selective.

Metabolic and affective consequences of fatherhood in male California mice

Physiological and affective condition can be modulated by the social environment and parental state in mammals. However, in species in which males assist with rearing offspring, the metabolic and affective effects of pair bonding and fatherhood on males have rarely been explored. In this study we tested the hypothesis that fathers, like mothers, experience energetic costs as well as behavioral and affective changes (e.g., depression, anxiety) associated with parenthood. We tested this hypothesis in the monogamous, biparental California mouse (Peromyscus californicus). Food intake, blood glucose and lipid levels, blood insulin and leptin levels, body composition, pain sensitivity, and depression-like behavior were compared in males from three reproductive groups: virgin males (VM, housed with another male), non-breeding males (NB, housed with a tubally ligated female), and breeding males (BM, housed with a female and their first litter). We found statistically significant (P<0.007, when modified for Adaptive False Discovery Rate) or nominally significant (0.007<P<0.05) differences among reproductive groups in relative testis mass, circulating glucose, triglyceride, and insulin concentrations, pain sensitivity, and anxiety-like behaviors. A priori contrasts indicated that VM produced significantly more fecal pellets than BM in the tail-suspension test, had significantly higher glucose levels than NB, and had significantly lower average testis masses than did NB and BM. A priori contrasts also indicated that VM had a nominally longer latency to the pain response than NB and that VM had nominally higher insulin levels than did NB. For breeding males, litter size (one to three pups) was a nominally significant positive predictor of body mass, food consumption, fat mass, and plasma leptin concentration. These results indicate that cohabitation with a female and/or fatherhood influences several metabolic, morphological, and affective measures in male California mice. Overall, the changes we observed in breeding males were minor, but stronger effects might occur in long-term breeding males and/or under more challenging environmental conditions.

Can mindfulness-based interventions improve the quality of life of patients with moderate/severe alopecia areata? A prospective pilot study

Can mindfulness-based interventions improve the quality of life of patients with moderate/severe alopecia areata? A prospective pilot study

Adaptive False Discovery Rate for Wavelet Denoising of Pavement Continuous Deflection Measurements

AbstractThe traffic speed deflectometer (TSD) is a device used to evaluate the pavement’s structural condition. Measurements obtained from the TSD are affected by noise, which can make it hard to interpret test results. The main objective of this paper is to develop a denoising methodology to use with TSD measurements and improve pavement structural evaluation. The denoising methodology comprises a computational algorithm to identify significant features in a high-dimensional vector of observations containing white Gaussian noise. The algorithm minimizes the classification error of features in the wavelet transform domain by adaptively selecting the level at which to control the false discovery rate. When tested in a simulation study, the results of the proposed algorithm compared favorably with other state-of-the-art methods. The proposed methodology was then successfully used with TSD measurements to identify possible weak joints in a jointed concrete pavement overlaid with an asphalt layer and to calcu...

Asymptotic results on adaptive false discovery rate controlling procedures based on kernel estimators

The False Discovery Rate (FDR) is a commonly used type I error rate in multiple testing problems. It is defined as the expected False Discovery Proportion (FDP), that is, the expected fraction of f...

Paradoxical results of adaptive false discovery rate procedures in neuroimaging studies

Adaptive false discovery rate (FDR) procedures, which offer greater power than the original FDR procedure of Benjamini and Hochberg, are often applied to statistical maps of the brain. When a large proportion of the null hypotheses are false, as in the case of widespread effects such as cortical thinning throughout much of the brain, adaptive FDR methods can surprisingly reject more null hypotheses than not accounting for multiple testing at all-i.e., using uncorrected p-values. A straightforward mathematical argument is presented to explain why this can occur with the q-value method of Storey and colleagues, and a simulation study shows that it can also occur, to a lesser extent, with a two-stage FDR procedure due to Benjamini and colleagues. We demonstrate the phenomenon with reference to a published data set documenting cortical thinning in attention deficit/hyperactivity disorder. The paper concludes with recommendations for how to proceed when adaptive FDR results of this kind are encountered in practice.

Abstract 1279: Proteomics signature of long-term versus short-term survival in glioblastoma

Abstract Glioblastoma (GB) is the most common malignant primary brain tumor in adults and disproportionately contributes to cancer mortality and morbidity. It has been repeatedly hypothesized that GB patients with varying response to treatment and hence prognosis have differential gene or protein expression profiles. To evaluate proteomic differences between long term and short term survivors with glioblastoma, an unbiased, label-free, unfractionated LC-MS/MS methodology was used on snap-frozen tumor samples from 10 patients with short term survival and 6 patients with long term survival from University Hospitals Case Medical Center. Short term survival was defined as 18 months after diagnosis. Significant protein intensity differences between long and short term survivors were tested using a mixed model. For each protein, the survival group was set as a fixed effect; while the peptide was set as a random effect which allowed for the within-protein correlation of the peptides to be accounted for in the model. In total 1490 proteins were detected in the glioblastoma sample set. One hundred eighty-three (183) proteins were found to be significant between survival groups using an adaptive False Discovery Rate p-value &amp;lt;0.05. Isocitrate Dehydrogenase 1 (IDH1), a mitochondrial energy-production enzyme, which has been implicated in differential response to temozolomide, was among these significant proteins. The top canonical pathways using these top 183 proteins, as generated by Ingenuity Pathway Analysis (IPA), were oxidative phosphorylation and mitochondrial dysfunction. These two pathways were at least twice as statistically significant as compared to any other discovered pathways. Examination of the individual proteins revealed a significant down regulation of oxidative phosphorylation, including subunits of cytochrome-oxidase and the BC1 complex in short term survivors. In addition, matched formalin-fixed paraffin embedded (FFPE) tumor tissue was used in order to verify the protein targets. Using 2 slides of 6 microns each, sufficient protein over 1 microgram of protein was extracted in order to perform an unbiased label-free LC-MS/MS analysis and 930 total proteins were detected. One hundred twenty-six (126) of the 183 significant proteins were successfully identified in the matched FFPE samples; thus FFPE can be used to verify many of the protein targets predicted to be changing. Further validation of the proteomics signature is ongoing in an expanded, independent cohort of glioblastoma patients with targeted validation. Progress in treatment for glioblastoma patients has been slowed by the absence of biomarkers to define treatment response and/or stratify patient sub-groups and help guide physician treatment. Proteomics analysis provides the most direct readout of the functional state of the tumor and thus promises improved functional annotation of patient status. Citation Format: {Authors}. {Abstract title} [abstract]. In: Proceedings of the 103rd Annual Meeting of the American Association for Cancer Research; 2012 Mar 31-Apr 4; Chicago, IL. Philadelphia (PA): AACR; Cancer Res 2012;72(8 Suppl):Abstract nr 1279. doi:1538-7445.AM2012-1279

Comparisons of estimators of the number of true null hypotheses and adaptive FDR procedures in multiplicity testing

Many exploratory studies such as microarray experiments require the simultaneous comparison of hundreds or thousands of genes. It is common to see that most genes in many microarray experiments are not expected to be differentially expressed. Under such a setting, a procedure that is designed to control the false discovery rate (FDR) is aimed at identifying as many potential differentially expressed genes as possible. The usual FDR controlling procedure is constructed based on the number of hypotheses. However, it can become very conservative when some of the alternative hypotheses are expected to be true. The power of a controlling procedure can be improved if the number of true null hypotheses (m 0) instead of the number of hypotheses is incorporated in the procedure [Y. Benjamini and Y. Hochberg, On the adaptive control of the false discovery rate in multiple testing with independent statistics, J. Edu. Behav. Statist. 25(2000), pp. 60–83]. Nevertheless, m 0 is unknown, and has to be estimated. The objective of this article is to evaluate some existing estimators of m 0 and discuss the feasibility of these estimators in incorporating into FDR controlling procedures under various experimental settings. The results of simulations can help the investigator to choose an appropriate procedure to meet the requirement of the study.

Determining the optimal level of smoothing in cortical thickness analysis: A hierarchical approach based on sequential statistical thresholding

The extent of smoothing applied to cortical thickness maps critically influences sensitivity, anatomical precision and resolution of statistical change detection. Theoretically, it could be optimized by increasing the trade-off between vertex-wise sensitivity and specificity across several levels of smoothing. But to date neither parametric nor nonparametric methods are able to control the error at the vertex level if the null hypothesis is rejected after smoothing of cortical thickness maps. To overcome these drawbacks, we applied sequential statistical thresholding based on a simple hierarchical model. This methodology aims at controlling erroneous detections; firstly at the level of clusters, over smoothed statistical maps; and secondly at the vertex level, over unsmoothed statistical maps, by applying an adaptive false discovery rate (FDR) procedure to clusters previously detected. The superior performance of the proposed methodology over other conventional procedures was demonstrated in simulation studies. As expected, only the hierarchical method yielded a predictable false discovery proportion near the predefined FDR q-value for any smoothing level at the same time as being as sensitive as the others at the optimal setting. It was therefore the only method able to approximate the optimal size of spatial smoothing when the true change was assumed unknown. The hierarchical method was further validated in a cross-sectional study comparing moderate Alzheimer's disease (AD) patients with healthy elderly subjects. Results suggest that the extent of cortical thinning reported in previous AD studies might be artificially inflated by the choice of inadequate smoothing. In these cases, interpretation should be based on the location of local maxima of suprathreshold regions rather than on the spatial extent of the detected signal in the statistical parametric map.

Adaptive False Discovery Rate Control under Independence and Dependence

In the context of multiple hypothesis testing, the proportion 0 of true null hypotheses in the pool of hypotheses to test often plays a crucial role, although it is generally unknown a priori. A te...