True Discoveries Research Articles

Rigorous Statistical Methods for Rigorous Microbiome Science.

High-throughput sequencing has facilitated discovery in microbiome science, but distinguishing true discoveries from spurious signals can be challenging. The Statistical Diversity Lab develops rigorous statistical methods and statistical software for the analysis of microbiome and biodiversity data. Developing statistical methods that produce valid P values requires thoughtful modeling and careful validation, but careful statistical analysis reduces the risk of false discoveries and increases scientific understanding.

Diamonds in the Kamchatka Peninsula (Tolbachik and Avacha Volcanoes): Natural Origin or Contamination?

Abstract —Lavas of the Kamchatka volcanoes store cubic-octahedral diamonds identical in morphology and structure to synthetic crystals, and their natural origin is doubted. Judging by published data, the diamonds discovered in the Tolbachik lavas are similar to synthetic diamonds made by different producers, and the analyzed samples rather result from contamination with synthetic material. Ophiolite-hosted diamonds reported from Europe, China, Mongolia, and Polar Urals look like the Tolbachik diamonds and are of the same type. The similarity between crystals coming from geologically dissimilar objects indicates that contamination may occur in those cases as well. Thus, diamonds found in unusual hosts or geologic settings require careful checking. These findings have to be reproduced repeatedly in other in situ samples and approved by independent experts; with all respect to the priority of the first finders, the sampling sites should be open to many researchers, especially the respective specialists. The inevitable disproval of false diamond findings is discouraging and discredits the true discoveries. Possible contamination with synthetic or natural material from cutting tools has to be excluded in all newly found diamonds before claiming their natural origin.

Recommendation to use exact P-values in biomarker discovery research in place of approximate P-values

BackgroundBiomarker candidates are often ranked using P-values. Standard P-value calculations use normal or logit-normal approximations, which may not be correct for small P-values and small sample sizes common in discovery research. MethodsWe compared exact P-values, correct by definition, with logit-normal approximations in a simulated study of 40 cases and 160 controls. The key measure of biomarker performance was sensitivity at 90% specificity. Data for 3000 uninformative false markers and 30 informative true markers were generated randomly. We also analyzed real data for 2371 plasma protein markers measured in 121 breast cancer cases and 121 controls. ResultsIn our simulation, using the same discovery criterion, exact P-values led to discovery of 24 true and 82 false biomarkers, while logit-normal approximate P-values yielded 20 true and 106 false biomarkers. The estimated true discovery rate was substantially off for approximate P-values: logit-normal estimated 42 but found 20. The exact method estimated 22, very close to 24, which was the actual number of true discoveries. Although these results are based on one specific simulation, qualitatively similar results were obtained from 10 random repetitions. With real data, ranking candidate biomarkers by exact P-values, versus approximate P-values, resulted in a very different ordering of these markers. ConclusionsExact P-values, which correspond to permutation tests with non-parametric rank statistics such as empirical ROC statistics, are preferred over approximate P-values. Approximate P-values can lead to inappropriate biomarker selection rules and incorrect conclusions. ImpactExact P-values in place of approximate P-values in discovery research may improve the yield of biomarkers that validate clinically.

A framework for the estimation of the proportion of true discoveries in single nucleotide variant detection studies for human data.

Any single nucleotide variant detection study could benefit from a fast and cheap method of measuring the quality of variant call list. It is advantageous to be able to see how the call list quality is affected by different variant filtering thresholds and other adjustments to the study parameters. Here we look into a possibility of estimating the proportion of true positives in a single nucleotide variant call list for human data. Using whole-exome and whole-genome gold standard data sets for training, we focus on building a generic model that only relies on information available from any variant caller. We assess and compare the performance of different candidate models based on their practical accuracy. We find that the generic model delivers decent accuracy most of the time. Further, we conclude that its performance could be improved substantially by leveraging the variant quality metrics that are specific to each variant calling tool.

Benefits of open and high-powered research outweigh costs.

Several researchers recently outlined unacknowledged costs of open science practices, arguing these costs may outweigh benefits and stifle discovery of novel findings. We scrutinize these researchers' (a) statistical concern that heightened stringency with respect to false-positives will increase false-negatives and (b) metascientific concern that larger samples and executing direct replications engender opportunity costs that will decrease the rate of making novel discoveries. We argue their statistical concern is unwarranted given open science proponents recommend such practices to reduce the inflated Type I error rate from .35 down to .05 and simultaneously call for high-powered research to reduce the inflated Type II error rate. Regarding their metaconcern, we demonstrate that incurring some costs is required to increase the rate (and frequency) of making true discoveries because distinguishing true from false hypotheses requires a low Type I error rate, high statistical power, and independent direct replications. We also examine pragmatic concerns raised regarding adopting open science practices for relationship science (preregistration, open materials, open data, direct replications, sample size); while acknowledging these concerns, we argue they are overstated given available solutions. We conclude benefits of open science practices outweigh costs for both individual researchers and the collective field in the long run, but that short term costs may exist for researchers because of the currently dysfunctional academic incentive structure. Our analysis implies our field's incentive structure needs to change whereby better alignment exists between researcher's career interests and the field's cumulative progress. We delineate recent proposals aimed at such incentive structure realignment. (PsycINFO Database Record

Combining Multiple Hypothesis Testing with Machine Learning Increases the Statistical Power of Genome-wide Association Studies.

The standard approach to the analysis of genome-wide association studies (GWAS) is based on testing each position in the genome individually for statistical significance of its association with the phenotype under investigation. To improve the analysis of GWAS, we propose a combination of machine learning and statistical testing that takes correlation structures within the set of SNPs under investigation in a mathematically well-controlled manner into account. The novel two-step algorithm, COMBI, first trains a support vector machine to determine a subset of candidate SNPs and then performs hypothesis tests for these SNPs together with an adequate threshold correction. Applying COMBI to data from a WTCCC study (2007) and measuring performance as replication by independent GWAS published within the 2008–2015 period, we show that our method outperforms ordinary raw p-value thresholding as well as other state-of-the-art methods. COMBI presents higher power and precision than the examined alternatives while yielding fewer false (i.e. non-replicated) and more true (i.e. replicated) discoveries when its results are validated on later GWAS studies. More than 80% of the discoveries made by COMBI upon WTCCC data have been validated by independent studies. Implementations of the COMBI method are available as a part of the GWASpi toolbox 2.0.

Crossmodal reorganization of face-voice interactions in Cochlear implanted deaf patients

A recent trend in functional magnetic resonance imaging is to test for association of clinical disorders with every possible connection between selected brain parcels. We investigated the impact of the resolution of functional brain parcels, ranging from large-scale networks to local regions, on a mass univariate general linear model (GLM) of connectomes. For each resolution taken independently, the Benjamini–Hochberg procedure controlled the false-discovery rate (FDR) at nominal level on realistic simulations. However, the FDR for tests pooled across all resolutions could be inflated compared to the FDR within resolution. This inflation was severe in the presence of no or weak effects, but became negligible for strong effects. We thus developed an omnibus test to establish the overall presence of true discoveries across all resolutions. Although not a guarantee to control the FDR across resolutions, the omnibus test may be used for descriptive analysis of the impact of resolution on a GLM analysis, in complement to a primary analysis at a predefined single resolution. On three real datasets with significant omnibus test (schizophrenia, congenital blindness, motor practice), markedly higher rate of discovery were obtained at low resolutions, below 50, in line with simulations showing increase in sensitivity at such resolutions. This increase in discovery rate came at the cost of a lower ability to localize effects, as low resolution parcels merged many different brain regions together. However, with 30 or more parcels, the statistical effect maps were biologically plausible and very consistent across resolutions. These results show that resolution is a key parameter for GLM-connectome analysis with FDR control, and that a functional brain parcellation with 30 to 50 parcels may lead to an accurate summary of full connectome effects with good sensitivity in many situations.

Benefits of Open and High-Powered Research Outweigh Costs

Several researchers recently outlined unacknowledged costs of open science practices, arguing these costs may outweigh benefits and stifle discovery of novel findings. We scrutinize these researchers' (1) statistical concern that heightened stringency with respect to false-positives will increase false-negatives and (2) meta-scientific concern that larger samples and executing direct replications engender opportunity costs that will decrease the rate of making novel discoveries. We argue their statistical concern is unwarranted given open science proponents recommend such practices to reduce the inflated Type I error rate from .35 down to .05 and simultaneously call for high-powered research to reduce the inflated Type II error rate. Regarding their meta-concern, we demonstrate that incurring some costs is required to increase the rate (and frequency) of making true discoveries because distinguishing true from false hypotheses requires a low Type I error rate, high statistical power, and independent direct replications. We also examine pragmatic concerns raised regarding adopting open science practices for relationship science (pre-registration, open materials, open data, direct replications, sample size); while acknowledging these concerns, we argue they are overstated given available solutions. We conclude benefits of open science practices outweigh costs for both individual researchers and the collective field in the long run, but that short term costs may exist for researchers because of the currently dysfunctional academic incentive structure. Our analysis implies our field's incentive structure needs to change whereby better alignment exists between researcher's career interests and the field's cumulative progress. We delineate recent proposals aimed at such incentive structure re-alignment.

Innovatus interregnum: waiting for a paradigm shift

Purpose– The purpose of this paper is to establish the case that innovation in the theory and practice of educational administration/leadership is very unlikely to occur within the existingdoxaof our times. By innovation is meant a novel conceptual or practical change in the field of practice. Bydoxais meant the unquestioned rules of the game and the linkage between the agencies and organs of government and foundations supporting research in the field. An approach toward thinking outside of the prevailingdoxais presented and explained as one possible antidote to the current dominant model.Design/methodology/approach– The paper is a conceptual/logical analysis of the reasons why the current paradigm dominant in the study and the practice of educational administration/leadership is inadequate. The paradigm has not predicted anything currently unknown or understood yet its continued dominance in the field will not lead to any new discoveries or innovation but only continued verification of what is already known.Findings– The major findings are that the boundaries of behavioral empiricism and social science methods impose an orthodoxy of approach in examining matters of administrative and leadership practice. Subsequently, it not only limits but also prohibits any new breakthroughs in understanding or predicting novel thinking about administration and leadership in educational institutions. Breaking out of this conceptual and theoretical box will be difficult as it is embraced by an interlocking apparatus of agencies and institutions and enshrined in most research journals in the field.Research limitations/implications– It is unlikely that true new discoveries in understanding educational leadership will occur without a restoration of the full range of human emotions and motivations which inspire and sustain leaders. New visions of leadership are required which will lead to what Lakatos has called a progressive research program in which prediction is enhanced and novel aspects of leadership emerge. These are not likely to occur given the tradition of inquiry currently in use. To use Lakatos’ term, the current research program is de-generative or regressive and lags behind the actual practice of school leadership. Thus, the authors perpetuate the theory-practice gap.Practical implications– The continued employment of social science protocols anchored in behavioral empiricism and the scientific method are unlikely to lead to any new breakthroughs in the practice of educational administration/leadership. The lens of behavioral empiricism prohibits a complete understanding of the practice of leadership where that practice becomes “subjective” and/or essentially artistic in nature. Practice, therefore, is anchored only in what is considered “rational” and the non-rational aspects marginalized or eliminated.Social implications– Researchers working in the dominant social science perspectives using hard behavioral empirical traditions embodied in the usual perspective regarding the scientific method will continue to miss or marginalize the emotional and intuitive side of leadership, aspects which are hard to quantify and assess. Leaders not only act but they feel as well. Without emotion in leadership it is extremely hard to build trust in an organization. The moral responsibilities of leaders are also anchored in emotion and values held by the leader. These elements continue to be understated or marginalized in check list approaches to preparation and licensure.Originality/value– The originality of the paper synthesizes the parallel perspectives of William Foster, Karl Popper, Paul Feyerabend, Michel Foucault, Pierre Bourdieu, and Imre Lakatos as it pertains to explaining why the current theory of knowledge is not likely to lead to any new breakthroughs in the practice of educational administration/leadership. One different approach to thinking of leadership as connoisseurship is presented as a potential perspective from the arts as a way of viewing leadership as a form of performance in which emotion and intuition are recognized aspects of practice.

Impact of the resolution of brain parcels on connectome-wide association studies in fMRI

A recent trend in functional magnetic resonance imaging is to test for association of clinical disorders with every possible connection between selected brain parcels. We investigated the impact of the resolution of functional brain parcels, ranging from large-scale networks to local regions, on a mass univariate general linear model (GLM) of connectomes. For each resolution taken independently, the Benjamini–Hochberg procedure controlled the false-discovery rate (FDR) at nominal level on realistic simulations. However, the FDR for tests pooled across all resolutions could be inflated compared to the FDR within resolution. This inflation was severe in the presence of no or weak effects, but became negligible for strong effects. We thus developed an omnibus test to establish the overall presence of true discoveries across all resolutions. Although not a guarantee to control the FDR across resolutions, the omnibus test may be used for descriptive analysis of the impact of resolution on a GLM analysis, in complement to a primary analysis at a predefined single resolution. On three real datasets with significant omnibus test (schizophrenia, congenital blindness, motor practice), markedly higher rate of discovery were obtained at low resolutions, below 50, in line with simulations showing increase in sensitivity at such resolutions. This increase in discovery rate came at the cost of a lower ability to localize effects, as low resolution parcels merged many different brain regions together. However, with 30 or more parcels, the statistical effect maps were biologically plausible and very consistent across resolutions. These results show that resolution is a key parameter for GLM-connectome analysis with FDR control, and that a functional brain parcellation with 30 to 50 parcels may lead to an accurate summary of full connectome effects with good sensitivity in many situations.

Power and stability comparisons of multiple testing procedures with false discovery rate control

High-throughput data analyses are widely used for examining differential gene expression, identifying single nucleotide polymorphisms, and detecting methylation loci. False discovery rate (FDR) has been considered a proper type I error rate to control for discovery-based high-throughput data analysis. Various multiple testing procedures have been proposed to control the FDR. The power and stability properties of some commonly used multiple testing procedures have not been extensively investigated yet, however. Simulation studies were conducted to compare power and stability properties of five widely used multiple testing procedures at different proportions of true discoveries for various sample sizes for both independent and dependent test statistics. Storey's two linear step-up procedures showed the best performance among all tested procedures considering FDR control, power, and variance of true discoveries. Leukaemia and ovarian cancer microarray studies were used to illustrate the power and stability characteristics of these five multiple testing procedures with FDR control.

Literarische Entdeckungsreisen: Vorfahren—Nachfahrten—Revisionen ed. by Hansjörg Bay and Wolfgang Struck (review)

Reviewed by: Literarische Entdeckungsreisen: Vorfahren—Nachfahrten—Revisionen ed. by Hansjörg Bay and Wolfgang Struck Yvonne Franke Literarische Entdeckungsreisen: Vorfahren—Nachfahrten—Revisionen. Edited by Hansjörg Bay and Wolfgang Struck. Cologne: Böhlau, 2012. Pp. 376. Paper € 49.90. ISBN 978-3412207649. The expeditions to the North and South Poles a century ago may have ended the investigation of unknown territory. Yet, the legendary histories and stories of the heyday of European expansion still continue. In recent years, these stories have been the focus of numerous texts in German language, including literature and film. The authors who contributed to Hansjörg Bay and Wolfgang Struck's edited volume Literarische Entdeckungsreisen: Vorfahren—Nachfahrten—Revisionen investigate this reformulated interest in narratives about the past and discuss new literary and filmic adaptations from aesthetic, sociopolitical, and imaginative points of view. The volume's contributions look at the tension between fact and fiction, generic conventions, colonial appropriations, postcolonial revisions, and their consequences for poetics and literary theory. The nineteen articles are organized in four thematic sections. The first section, "Über die letzte Grenze," focuses on cartography and the topographic figure of border drawings and crossings that are always preceded by myths and imaginings. Thus, the exploration of the allegedly "weiße Flecken" inevitably has always been a "Relektüre" and belie any claim of true discoveries. Accordingly, Bettine Menke's contribution "Grenzüberschreitungen (in) der Schrift, Exterritorialität der Pole" shows how literary journeys are strongly intertwined with physical explorations. Literary polar expeditions ("nachfahrendes Schreiben") precede actual expeditions ("Nachfahrten," 75). Because the latter have already been "vorgeschrieben" (53) through literary investigations, each text follows in the footsteps of its literary precursors, which bespeaks to the [End Page 686] "Un-/Erreichbarkeit" of both the poles and the texts (77). Wolfgang Struck's article, "Ingenjör Andrées luftfärd oder Die melancholischen Entdeckungen des Films," revisits the "Cold Case" (29) of three lost Swedish North Pole scientists from 1897. Struck demonstrates how Jan Troell's feature film Ingenjör Andrées luftfärd (1982) revisits this case by employing the "Poetologie des Scheiterns" (37) that brings forth a contradistinction from the original photographic documents and questions the alleged completeness of cartography. The articles on August Petermann's "Theorie vom eisfreien Polarmeer" and a close reading of Christoph Ransmayr's novel Der fliegende Berg and its transgression of territorial and generic borders complete this section. Hansjörg Bay's essay "Literarische Landnahme? Um-Schreibung, Partizipation und Wiederholung in aktuellen Relektüren historischer 'Entdeckungsreisen'" opens the second section of the volume, "Vor-Schriften, Nachfahren und Relektüren." At first, Bay focuses on how the genre of travel writing contributed directly to heroism and colonialism by way of following specific conventions marked by ideological stand points. The article continues with his investigation of contemporary "Relektüren" of (hi)stories of both arctic and tropical expeditions that have been thriving since the 1990s: Sten Nadolny's Die Entdeckung der Langsamkeit, Christoph Ransmayr's Die Schrecken des Eises und der Finsternis, Christof Hamann's Der Usambara, Ilija Trojanow's Der Weltensammler, and Thomas Stangl's Der einzige Ort. While Bay considers the North Pole texts from the 1980s as important predecessors for the current literary trend, he gives particular credit to the latter three, in which the colonial entanglements are conveyed through literature. Nevertheless, as he concludes, these recent texts participate in "eine[r] Art kolonialer Wiederholungszwang" (130), which adheres to the conventions of the genre. Bay convincingly demonstrates the usefulness of the ambiguous concept of "Relektüren" that lies at the heart of the entire anthology. The remaining articles in this part deal with nineteenth-century narrative strategies in rewritings of Le Vaillant, Christof Hamann's Usambara, Urs Widmer's Im Kongo, and Yambo Ouologuem's Le Devoir de Violence. John Zilcosky's article "Unheimliche Begegnungen: Abenteuerliteratur, Psychoanalyse, Moderne" opens the anthology's third section on "Tropologien" that shifts the focus to tropical places and lets the geographical and rhetorical realms of the term interact with each other. Zilcosky analyzes examples of popular travel writing (Norbert Jacques' Heisses Land. Eine Reise nach Brasilien), seminal psychological and philosophical theories (Ernst Jentsch, Martin Heidegger and Sigmund Freud), and modern literature...

Selective Inference on Multiple Families of Hypotheses

SummaryIn many complex multiple-testing problems the hypotheses are divided into families. Given the data, families with evidence for true discoveries are selected, and hypotheses within them are tested. Neither controlling the error rate in each family separately nor controlling the error rate over all hypotheses together can assure some level of confidence about the filtration of errors within the selected families. We formulate this concern about selective inference in its generality, for a very wide class of error rates and for any selection criterion, and present an adjustment of the testing level inside the selected families that retains control of the expected average error over the selected families.

Two optimization strategies of multi-stage design in clinical proteomic studies

We evaluated statistical approaches to facilitate and improve multi-stage designs for clinical proteomic studies which plan to transit from laboratory discovery to clinical utility. To find the design with the greatest expected number of true discoveries under constraints on cost and false discovery, the operating characteristics of the multi-stage study were optimized as a function of sample sizes and nominal type-I error rates at each stage. A nested simulated annealing algorithm was used to find the best solution in the bounded spaces constructed by multiple design parameters. This approach is demonstrated to be feasible and lead to efficient designs. The use of biological grouping information in the study design was also investigated using synthetic datasets based on a cardiac proteomic study, and an actual dataset from a clinical immunology proteomic study. When different protein patterns presented, performance improved when the grouping was informative, with little loss in performance when the grouping was uninformative.

Alterations in Cerebrospinal Fluid Proteins in a Presymptomatic Primary Glioma Model

BackgroundUnderstanding the early relationship between brain tumor cells and their environment could lead to more sensitive biomarkers and new therapeutic strategies. We have been using a rodent model of neurocarcinogenesis in which all animals develop brain tumors by six months of age to establish two early landmarks in glioma development: the appearance of a nestin+ cell at thirty days of age and the appearance of cellular hyperplasia between 60 and 120 days of age. We now report an assessment of the CSF proteome to determine the changes in protein composition that occur during this period.Materials and MethodsNestin+ cell clusters and microtumors were assessed in 63 ethylnitrosourea-exposed rats on 30, 60, and 90 days of age. CSF was obtained from the cisterna magna from 101 exposed and control rats at 30, 60, and 90 days and then analyzed using mass spectrometry. Differentially expressed peaks were isolated and identified.ResultsNestin+ cells were noted in all ethylnitrosourea-exposed rats assessed pathologically. Small microtumors were noted in 0%, 18%, and 67% of 30-, 60-, and 90-day old rats, respectively (p<0.05, Chi square). False Discovery Rate analysis of peak intensities showed that the number of true discoveries with p<0.05 increased markedly with increasing age. Isolation and identification of highly differentially detected proteins at 90 days of age revealed increases in albumin and a fragment of α1 macroglobulin and alterations in glutathionylated transthyretin.ConclusionsThe presence of increased albumin, fragments of cerebrospinal fluid proteins, and glutathione breakdown in temporal association with the development of cellular hyperplasia, suggests that, similar to many other systemic cancers, inflammation and oxidative stress is playing an important early role in the host’s response to brain tumor development and may be involved in affecting the early growth of brain tumor.

The failure of protein cancer biomarkers to reach the clinic: why, and what can be done to address the problem?

There is a plethora of published cancer biomarkers but the reality is that very few, if any, new circulating cancer biomarkers have entered the clinic in the last 30 years. I here try to explain this apparent oxymoron by classifying circulating cancer biomarkers into three categories: fraudulent reports (rare); true discoveries of biomarkers, that then fail to meet the demands of the clinic; and false discoveries, which represent artifactual biomarkers. I further provide examples of combinations of some known cancer biomarkers that can perform well in niche clinical applications, despite individually being not useful.

NOIseq: a RNA-seq differential expression method robust for sequencing depth biases

http://bioinfo.cipf.es/aconesaIntroductionNext Generation Sequencing (NGS) technologies have brought a revolution to research in genome and genome regulation. One of the most breaking applications of NGS is in transcriptome analysis. RNA-seq has revealed exciting new data on gene models, alternative splicing and extra-genic expression. Also RNA-seq permits the quantification of gene expression across a large dynamic range and with more reproducibility than microarrays. Several methods for the assessment of differential expression from count data have been proposed but biases associated to transcript length and transcript frequency distributions have been reported. It is still not clear how much sequencing reads should be generated in a RNA-seq experiment to obtain reliable results and what’s exactly being detected. In general we observed that many RNA-seq datasets have not reached saturation for detection of expressed genes and that the relative proportion of different transcript biotypes changes with increasing sequencing depth. In this work we investigate the effect that library size has on the assessment of differential expression on different aspects of the selected genes. We show that current statistical methods suffer from a strong dependency of their significant calls on the number of mapped reads considered and proposed a novel differential expression methodology – NOISeq1- that is robust to the amount of reads.ResultsNOISeq is a non-parametric approach for the differential expression analysis of RNseq-data. NOISeq creates a null or noise distribution of count changes by comparing the number of reads of each gene in samples within the same condition. This reference distribution is then used to assess whether the change in count number between two conditions for a given gene is likely to be part of the noise or represents a true differential expression. Two variants of the method are implemented: NOISeq-real uses replicates, when available, to compute the noise distribution and, NOISeq-sim simulates them in absence of replication. We compared our method with edgeR2, DESeq3, baySeq4 and Fisher Exact Test (FET) using three different experimental datasets. Results are presented for MAQC experiment where the transcriptome of brain and Universal Human Reference (HUR) samples were sequenced at about 45 million Solexa reads each.We first determined that although protein-coding gene is the most abundant transcript type within differential expression calls for all methodologies, other RNA types, such as processed-transcript, pseudogenes and lincRNAs are readily detected. NOISeq dected comparatively more protein-coding genes than other methods that called significant a considerable number of non-coding and small RNA transcripts. Additionally, all comparing methods except FET greatly increased the number of detected (non-coding) genes as sequencing depth raised while NOISeq showed a constant pattern. Also these other methods tend to select shorter genes and smaller fold change differences with the increasing amounts of reads. In general, parametric approaches selected much more genes than NOISeq, specially at high sequencing depth rates. When analyzing the functional content of these genes by functional enrichment analysis, we observed that the pool of genes detected both by NOISeq and the parametric methods where highly enriched in functional categories, while genes selected only by parametric methods did not. To check whether this differences were indicative of different false calls between methods, we used the RT-PCR data available at the MAQC project that contains 330 true positive and 83 true negative differentially expressed genes. Performance plots indicate that edgeR, DESeq, baySeq strongly increased the number of false calls with sequencing depth, while NOISeq was constant and low. On the contrary true discoveries were slightly better for these methods, presumably consequence of their large number of selected genes. FET showed in low false and true discovery rates, due to its general lower detection power.ConclusionsWe showed that most current RNA-seq statistical analysis methods fail to control the number of false discoveries as the size of the sequenced library increases. These false positive are mainly short, non- coding genes and/or genes with small fold changes. NOISeq, but adopting an empirical approach to model the null distribution of differential expression captures better the shape of noise in RNA-seq data, resulting in a sequencing-depth robust method for differential expression analysis.

False discovery rate for scanning statistics

The false discovery rate is a criterion for controlling Type I error in simultaneous testing of multiple hypotheses. For scanning statistics, due to local dependence, clusters of neighbouring hypotheses are likely to be rejected together. In such situations, it is more intuitive and informative to group neighbouring rejections together and count them as a single discovery, with the false discovery rate defined as the proportion of clusters that are falsely declared among all declared clusters. Assuming that the number of false discoveries, under this broader definition of a discovery, is approximately Poisson and independent of the number of true discoveries, we examine approaches for estimating and controlling the false discovery rate, and provide examples from biological applications. Copyright 2011, Oxford University Press.

Estimating the number of true discoveries in genome‐wide association studies

Recent genome-wide association studies have reported the discoveries of genetic variants of small to moderate effects. However, most studies of complex diseases face a great challenge because the number of significant variants is less than what is required to explain the disease heritability. A new approach is needed to recognize all potential discoveries in the data. In this paper, we present a practical model-free procedure to estimate the number of true discoveries as a function of the number of top-ranking SNPs together with the confidence bounds. This approach allows a practical methodology of general utility and produces relevant statistical quantities with simple interpretation.

Application of the Lasso to Expression Quantitative Trait Loci Mapping

Univariate methods have frequently been used to discover Quantitative Trait Loci for gene expression measurements, often with much success. However, correlations caused by Linkage Disequilibrium as well as chance correlations, which are functions of the large number of markers typically used in such studies, mean that causative regions can often cause multiple signals. Traditional investigations into the number of QTL for a given phenotype, such as visual inspection of likelihood plots, are not feasible when considering thousands of phenotypes. Stepwise methods have been suggested to counter this, but these are known to produce unstable models and there are difficulties in deriving significance estimates. The Lasso is a shrinkage method which has often been employed to discover true signals when the number of variables exceeds the number of observations. We propose a test statistic based on the threshold at which variables enter the Lasso model, prove analytic properties of this statistic which demonstrate parallels with univariate methods and demonstrate its utility in proposing candidate QTL. We show that this method controls for LD structure, and the estimates of statistical significance produced have superior properties when compared to those derived by stepwise methods. We study the performance of our method using simulation studies. These simulations find that the ratio of true discoveries to false positives is often superior for our method compared to univariate and stepwise approaches. Finally, we apply the derived method to data from a previous eQTL mapping experiment to investigate the nature of genetic regulation in this population.