Abstract
BackgroundStability of multiple testing procedures, defined as the standard deviation of total number of discoveries, can be used as an indicator of variability of multiple testing procedures. Improving stability of multiple testing procedures can help to increase the consistency of findings from replicated experiments. Benjamini-Hochberg’s and Storey’s q-value procedures are two commonly used multiple testing procedures for controlling false discoveries in genomic studies. Storey’s q-value procedure has higher power and lower stability than Benjamini-Hochberg’s procedure. To improve upon the stability of Storey’s q-value procedure and maintain its high power in genomic data analysis, we propose a new multiple testing procedure, named Bon-EV, to control false discovery rate (FDR) based on Bonferroni’s approach.ResultsSimulation studies show that our proposed Bon-EV procedure can maintain the high power of the Storey’s q-value procedure and also result in better FDR control and higher stability than Storey’s q-value procedure for samples of large size(30 in each group) and medium size (15 in each group) for either independent, somewhat correlated, or highly correlated test statistics. When sample size is small (5 in each group), our proposed Bon-EV procedure has performance between the Benjamini-Hochberg procedure and the Storey’s q-value procedure. Examples using RNA-Seq data show that the Bon-EV procedure has higher stability than the Storey’s q-value procedure while maintaining equivalent power, and higher power than the Benjamini-Hochberg’s procedure.ConclusionsFor medium or large sample sizes, the Bon-EV procedure has improved FDR control and stability compared with the Storey’s q-value procedure and improved power compared with the Benjamini-Hochberg procedure. The Bon-EV multiple testing procedure is available as the BonEV package in R for download at https://CRAN.R-project.org/package=BonEV.
Highlights
Stability of multiple testing procedures, defined as the standard deviation of total number of discoveries, can be used as an indicator of variability of multiple testing procedures
The Bon-EV multiple testing procedure will be attractive to genomic data analysts as it maintains the high power of Storey’s q-value procedure, and offers better false discovery rate (FDR) control and higher stability, especially for small to medium sample size studies that need high stability, high power and good FDR control to maximize the odds of true discoveries
Power is defined as the proportion of true rejections among total non-true null hypotheses, and stability is defined as the standard deviation (SD) of total number of rejections [11]
Summary
Stability of multiple testing procedures, defined as the standard deviation of total number of discoveries, can be used as an indicator of variability of multiple testing procedures. To improve upon the stability of Storey’s q-value procedure and maintain its high power in genomic data analysis, we propose a new multiple testing procedure, named Bon-EV, to control false discovery rate (FDR) based on Bonferroni’s approach. We examine the stability (defined as standard deviation of the total number of rejected hypotheses) of both Benjamini-Hochberg’s FDR controlling procedure and Storey’s q-value procedure for generating adjusted p-values to select significant genes or biomarkers in microarray and NGS data analysis. We propose our own multiple testing procedure (named Bon-EV) based on Bonferroni’s EV controlling procedure, that has equivalent power, higher stability, and better FDR control than the Storey’s q-value procedure with at least mediumsized samples in microarray and NGS data analysis. The Bon-EV multiple testing procedure will be attractive to genomic data analysts as it maintains the high power of Storey’s q-value procedure, and offers better FDR control and higher stability, especially for small to medium sample size studies that need high stability, high power and good FDR control to maximize the odds of true discoveries
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
