Statistical Hypothesis Testing Framework Research Articles

A new framework for exploratory network mediator analysis in omics data.

Omics methods are widely used in basic biology and translational medicine research. More and more omics data are collected to explain the impact of certain risk factors on clinical outcomes. To explain the mechanism of the risk factors, a core question is how to find the genes/proteins/metabolites that mediate their effects on the clinical outcome. Mediation analysis is a modeling framework to study the relationship between risk factors and pathological outcomes, via mediator variables. However, high-dimensional omics data are far more challenging than traditional data: (1) From tens of thousands of genes, can we overcome the curse of dimensionality to reliably select a set of mediators? (2) How do we ensure that the selected mediators are functionally consistent? (3) Many biological mechanisms contain nonlinear effects. How do we include nonlinear effects in the high-dimensional mediation analysis? (4) How do we consider multiple risk factors at the same time? To meet these challenges, we propose a new exploratory mediation analysis framework, medNet, which focuses on finding mediators through predictive modeling. We propose new definitions for predictive exposure, predictive mediator, and predictive network mediator, using a statistical hypothesis testing framework to identify predictive exposures and mediators. Additionally, two heuristic search algorithms are proposed to identify network mediators, essentially subnetworks in the genome-scale biological network that mediate the effects of single or multiple exposures. We applied medNet on a breast cancer data set and a metabolomics data set combined with food intake questionnaire data. It identified functionally consistent network mediators for the exposures' impact on the outcome, facilitating data interpretation.

Phylogenomic Testing of Root Hypotheses.

The determination of the last common ancestor (LCA) of a group of species plays a vital role in evolutionary theory. Traditionally, an LCA is inferred by the rooting of a fully resolved species tree. From a theoretical perspective, however, inference of the LCA amounts to the reconstruction of just one branch - the root branch - of the true species tree, and should therefore be a much easier task than the full resolution of the species tree. Discarding the reliance on a hypothesised species tree and its rooting leads us to re-evaluate what phylogenetic signal is directly relevant to LCA inference, and to recast the task as that of sampling the total evidence from all gene families at the genomic scope. Here we reformulate LCA and root inference in the framework of statistical hypothesis testing and outline an analytical procedure to formally test competing a-priori LCA hypotheses and to infer confidence sets for the earliest speciation events in the history of a group of species. Applying our methods to two demonstrative datasets we show that our inference of the opisthokonta LCA is well in agreement with the common knowledge. Inference of the proteobacteria LCA shows that it is most closely related to modern Epsilonproteobacteria, raising the possibility that it may have been characterized by a chemolithoautotrophic and anaerobic life-style. Our inference is based on data comprising between 43% (opisthokonta) and 86% (proteobacteria) of all gene families. Approaching LCA inference within a statistical framework renders the phylogenomic inference powerful and robust.

Rough-Fuzzy CPD: a gradual change point detection algorithm

Changepoint detection is the problem of finding abrupt or gradual changes in time series data when the distribution of the time series changes significantly. There are many sophisticated statistical algorithms for solving changepoint detection problem, although there is not much work devoted towards gradual changepoints as compared to abrupt ones. Here we present a new approach to solve the changepoint detection problem using the fuzzy rough set theory which is able to detect such gradual changepoints. An expression for the rough-fuzzy estimate of changepoints is derived along with its mathematical properties concerning fast computation. In a statistical hypothesis testing framework, the asymptotic distribution of the proposed statistic on both single and multiple changepoints is derived under the null hypothesis enabling multiple changepoint detection. Extensive simulation studies have been performed to investigate how simple crude statistical measures of disparity can be subjected to improve their efficiency in the estimation of gradual changepoints. Also, the said rough-fuzzy estimate is robust to signal-to-noise ratio, a high degree of fuzziness in true changepoints, and also to hyperparameter values. Simulation studies reveal that the proposed method beats other methods of gradual changepoint detection (including MJPD, HSMUCE, fuzzy methods like FCP, FCMLCP etc) and also popular crisp methods like Binary Segmentation, PELT, and BOCD in detecting gradual changepoints. The applicability of the estimate is demonstrated using multiple real-life datasets including Covid-19. We have developed the python package roufcp for broader dissemination of the methods.

A standardized and statistically defensible framework for quantitative morphological analyses in taxonomic studies.

Although body size correction and inferential statistics have been used in morphological studies for many decades, their applications are far from being ubiquitous. We performed a meta-analysis to quantify the extent of taxonomic papers that performed body size correction and implemented a statistical hypothesis testing framework during the analysis of morphological data. Our results indicate that in most papers, neither of these analyses were performed but instead, cursory comparisons of descriptive statistics were presented. With the development of numerous freely available and powerful statistical programs such as R, we find it prudent to outline a standardized and statistically defensible framework to enhance the workflow of morphological analyses in taxonomic studies. This 5-step approach can be applied to meristic and mensural data across a wide range of taxonomic groups. We include an easy-to-use companion R script to facilitate the implementation of this workflow. Our proposed framework is not rooted in phylogenetic or evolutionary theory and hence, should not be used in place of explicit species delimitation techniques. Nevertheless, it can be incorporated into a more robust integrative taxonomic framework and is particularly useful for identifying diagnostic characters for species diagnoses.

Hypothesis-based Acceptance Sampling for Modules F and F1 of the European Measuring Instruments Directive

Millions of measuring instruments are verified each year before being placed on the markets worldwide. In the EU, such initial conformity assessments are regulated by the Measuring Instruments Directive (MID). The MID modules F and F1 on product verification allow for statistical acceptance sampling, whereby only random subsets of instruments need to be inspected. This article re-interprets the acceptance sampling conditions formulated by the MID. The new interpretation is contrasted with the one advanced in WELMEC guide 8.10, and three advantages have become apparent. First, an economic advantage of the new interpretation is a producers’ risk bounded from above, such that measuring instruments with sufficient quality are accepted with a guaranteed probability of no less than 95%. Second, a conceptual advantage is that the new MID interpretation fits into the well known, formal framework of statistical hypothesis testing. Thirdly, the new interpretation applies unambiguously to finite-sized lots, even very small ones. We conclude that the new interpretation is to be preferred and suggest re-formulating the statistical sampling conditions in the MID. Re-interpreting the MID conditions implies that currently available sampling plans are either not admissible or not optimal. We derive a new acceptance sampling scheme and recommend its application. Supplementary materials for this article are available online.

Sepal: identifying transcript profiles with spatial patterns by diffusion-based modeling.

MotivationCollection of spatial signals in large numbers has become a routine task in multiple omics-fields, but parsing of these rich datasets still pose certain challenges. In whole or near-full transcriptome spatial techniques, spurious expression profiles are intermixed with those exhibiting an organized structure. To distinguish profiles with spatial patterns from the background noise, a metric that enables quantification of spatial structure is desirable. Current methods designed for similar purposes tend to be built around a framework of statistical hypothesis testing, hence we were compelled to explore a fundamentally different strategy.ResultsWe propose an unexplored approach to analyze spatial transcriptomics data, simulating diffusion of individual transcripts to extract genes with spatial patterns. The method performed as expected when presented with synthetic data. When applied to real data, it identified genes with distinct spatial profiles, involved in key biological processes or characteristic for certain cell types. Compared to existing methods, ours seemed to be less informed by the genes’ expression levels and showed better time performance when run with multiple cores.Availabilityand implementationOpen-source Python package with a command line interface (CLI), freely available at https://github.com/almaan/sepal under an MIT licence. A mirror of the GitHub repository can be found at Zenodo, doi: 10.5281/zenodo.4573237.Supplementary informationSupplementary data are available at Bioinformatics online.

Decision trees in epidemiological research

BackgroundIn many studies, it is of interest to identify population subgroups that are relatively homogeneous with respect to an outcome. The nature of these subgroups can provide insight into effect mechanisms and suggest targets for tailored interventions. However, identifying relevant subgroups can be challenging with standard statistical methods.Main textWe review the literature on decision trees, a family of techniques for partitioning the population, on the basis of covariates, into distinct subgroups who share similar values of an outcome variable. We compare two decision tree methods, the popular Classification and Regression tree (CART) technique and the newer Conditional Inference tree (CTree) technique, assessing their performance in a simulation study and using data from the Box Lunch Study, a randomized controlled trial of a portion size intervention. Both CART and CTree identify homogeneous population subgroups and offer improved prediction accuracy relative to regression-based approaches when subgroups are truly present in the data. An important distinction between CART and CTree is that the latter uses a formal statistical hypothesis testing framework in building decision trees, which simplifies the process of identifying and interpreting the final tree model. We also introduce a novel way to visualize the subgroups defined by decision trees. Our novel graphical visualization provides a more scientifically meaningful characterization of the subgroups identified by decision trees.ConclusionsDecision trees are a useful tool for identifying homogeneous subgroups defined by combinations of individual characteristics. While all decision tree techniques generate subgroups, we advocate the use of the newer CTree technique due to its simplicity and ease of interpretation.

Redhyte: a self-diagnosing, self-correcting, and helpful hypothesis analysis platform

ABSTRACTAn interactive platform for ‘Rapid exploration of data and hypothesis testing’, named Redhyte, is described in this article. Redhyte provides a more efficient and encompassing hypothesis testing procedure than the conventional statistical hypothesis testing framework, by integrating the latter with data-mining techniques. Redhyte is self-diagnosing (it tries to detect whether the user is doing a valid statistical test), self-correcting (it tries to propose and make corrections to the user’s statistical test), and helpful (it searches for promising or interesting hypotheses related to the initial user-specified hypothesis). Hypothesis mining in Redhyte consists of the following steps: context mining, mined-hypothesis formulation, mined-hypothesis scoring on interestingness, and statistical adjustments. And Redhyte supports multiple hypothesis-mining metrics (e.g. several forms of difference lift) that are useful for capturing and evaluating specific aspects of interestingness (e.g. changes in trends and manner of shrinkage). Redhyte is implemented as an R shiny web application and can be found online at https://tohweizhong.shinyapps.io/redhyte, and the source codes can be found at https://github.com/tohweizhong/redhyte.

Not Just a Sum? Identifying Different Types of Interplay between Constituents in Combined Interventions.

MotivationExperiments in which the effect of combined manipulations is compared with the effects of their pure constituents have received a great deal of attention. Examples include the study of combination therapies and the comparison of double and single knockout model organisms. Often the effect of the combined manipulation is not a mere addition of the effects of its constituents, with quite different forms of interplay between the constituents being possible. Yet, a well-formalized taxonomy of possible forms of interplay is lacking, let alone a statistical methodology to test for their presence in empirical data.ResultsStarting from a taxonomy of a broad range of forms of interplay between constituents of a combined manipulation, we propose a sound statistical hypothesis testing framework to test for the presence of each particular form of interplay. We illustrate the framework with analyses of public gene expression data on the combined treatment of dendritic cells with curdlan and GM-CSF and show that these lead to valuable insights into the mode of action of the constituent treatments and their combination.Availability and ImplementationR code implementing the statistical testing procedure for microarray gene expression data is available as supplementary material. The data are available from the Gene Expression Omnibus with accession number GSE32986.

Improved gene tree error correction in the presence of horizontal gene transfer.

Motivation: The accurate inference of gene trees is a necessary step in many evolutionary studies. Although the problem of accurate gene tree inference has received considerable attention, most existing methods are only applicable to gene families unaffected by horizontal gene transfer. As a result, the accurate inference of gene trees affected by horizontal gene transfer remains a largely unaddressed problem.Results: In this study, we introduce a new and highly effective method for gene tree error correction in the presence of horizontal gene transfer. Our method efficiently models horizontal gene transfers, gene duplications and losses, and uses a statistical hypothesis testing framework [Shimodaira–Hasegawa (SH) test] to balance sequence likelihood with topological information from a known species tree. Using a thorough simulation study, we show that existing phylogenetic methods yield inaccurate gene trees when applied to horizontally transferred gene families and that our method dramatically improves gene tree accuracy. We apply our method to a dataset of 11 cyanobacterial species and demonstrate the large impact of gene tree accuracy on downstream evolutionary analyses.Availability and implementation: An implementation of our method is available at http://compbio.mit.edu/treefix-dtl/Contact: mukul@engr.uconn.edu or manoli@mit.eduSupplementary information: Supplementary data are available at Bioinformatics online.

A Sensor Fault Diagnosis Scheme for a DC/DC Converter used in Hybrid Electric Vehicles

With the extensive use of power converters in modern hybrid vehicles, rises the need for efficient methods of condition monitoring and fault diagnosis to ensure the reliability of the automotive electrical power system. This paper addresses detection and isolation of sensor faults in a DC/DC power converter system interfacing the main energy storage unit and the AC drive in a hybrid electric vehicle. A residual-based fault diagnosis scheme that timely detects and localizes sensor faults in the examined system is designed. Residual signals are derived from comparison between estimates generated by a bank of extended Kalman filters and real measurements obtained from a hardware prototype of the power converter. The generalized likelihood ratio test is utilized in a statistical hypothesis testing framework to evaluate the residuals and detect the occurrence of sensor faults. The receiver operating characteristic curve is then employed to optimize the value of the detection threshold and the sliding window width of the statistical test. The proposed algorithm is tested on data obtained from simulated and real measurements with faults injected on the converter sensors. The results obtained promise maximum probability of correct detection and minimum false alarm rate.

Regional parent flood frequency distributions in Europe – Part 1: Is the GEV model suitable as a pan-European parent?

Abstract. This study addresses the question of the existence of a parent flood frequency distribution on a European scale. A new database of L-moment ratios of flood annual maximum series (AMS) from 4105 catchments was compiled by joining 13 national data sets. Simple exploration of the database presents the generalized extreme value (GEV) distribution as a potential pan-European flood frequency distribution, being the three-parameter statistical model that with the closest resemblance to the estimated average of the sample L-moment ratios. Additional Monte Carlo simulations show that the variability in terms of sample skewness and kurtosis present in the data is larger than in a hypothetical scenario where all the samples were drawn from a GEV model. Overall, the generalized extreme value distribution fails to represent the kurtosis dispersion, especially for the longer sample lengths and medium to high skewness values, and therefore may be rejected in a statistical hypothesis testing framework as a single pan-European parent distribution for annual flood maxima. The results presented in this paper suggest that one single statistical model may not be able to fit the entire variety of flood processes present at a European scale, and presents an opportunity to further investigate the catchment and climatic factors controlling European flood regimes and their effects on the underlying flood frequency distributions.

Statistical significance of episodes with general partial orders

Frequent episode discovery is one of the methods used for temporal pattern discovery in sequential data. An episode is a partially ordered set of nodes with each node associated with an event type. For more than a decade, algorithms existed for episode discovery only when the associated partial order is total (serial episode) or trivial (parallel episode). Recently, the literature has seen algorithms for discovering episodes with general partial orders. In frequent pattern mining, the threshold beyond which a pattern is inferred to be interesting is typically user-defined and arbitrary. One way of addressing this issue in the pattern mining literature has been based on the framework of statistical hypothesis testing. This paper presents a method of assessing statistical significance of episode patterns with general partial orders. A method is proposed to calculate thresholds, on the non-overlapped frequency, beyond which an episode pattern would be inferred to be statistically significant. The method is first explained for the case of injective episodes with general partial orders. An injective episode is one where event-types are not allowed to repeat. Later it is pointed out how the method can be extended to the class of all episodes. The significance threshold calculations for general partial order episodes proposed here also generalize the existing significance results for serial episodes. Through simulations studies, the usefulness of these statistical thresholds in pruning uninteresting patterns is illustrated.

Seismic Fragility Evaluation with Incomplete Structural Appraisal Data: An Iterative Statistical Approach

This paper presents an iterative statistical approach to evaluating seismic structural safety using incomplete appraisal data. Despite the continuous improvement to traditional structural assessment procedures and the recent progress in structural health monitoring methodologies, practically acquired structural appraisal data may often be incomplete. The occurrence of the appraisal data missingness could be ascribed to the malfunction of data acquisition systems, the abnormality during data transfer, and the inaccessibility of critical quantities, among other reasons. The study begins with a quantitative investigation into the sensitivity of the seismic fragility evaluation with respect to the structural appraisal data missingness through the defined additional information loss and probability of noninformativeness. Subsequently, a remedy for the missingness of the structural appraisal data, instead of a precaution against it, is formulated by employing the expectation-maximization (EM) algorithm. With synthetic or real seismic ground accelerations involved, the efficacy of the EM algorithm embedded remedy is demonstrated by examples of typical linear or nonlinear hysteretic systems in the framework of statistical hypothesis testing. Resorting to the bootstrap technique, the influence of the related correlations and missingness probability is also examined.

Direction detector for distributed targets in unknown noise and interference

Adaptive detection of distributed radar targets in homogeneous Gaussian noise plus subspace interference is addressed. It is assumed that the actual steering vectors lie along a fixed and unknown direction of a preassigned and known subspace, while interfering signals are supposed to belong to an unknown subspace, with directions possibly varying from one resolution cell to another. The resulting detection problem is formulated in the framework of statistical hypothesis testing and solved using an ad hoc algorithm strongly related to the generalised likelihood ratio test. A performance analysis, carried out also in comparison to natural benchmarks, is presented.

On reliable discovery of molecular signatures

BackgroundMolecular signatures are sets of genes, proteins, genetic variants or other variables that can be used as markers for a particular phenotype. Reliable signature discovery methods could yield valuable insight into cell biology and mechanisms of human disease. However, it is currently not clear how to control error rates such as the false discovery rate (FDR) in signature discovery. Moreover, signatures for cancer gene expression have been shown to be unstable, that is, difficult to replicate in independent studies, casting doubts on their reliability.ResultsWe demonstrate that with modern prediction methods, signatures that yield accurate predictions may still have a high FDR. Further, we show that even signatures with low FDR may fail to replicate in independent studies due to limited statistical power. Thus, neither stability nor predictive accuracy are relevant when FDR control is the primary goal. We therefore develop a general statistical hypothesis testing framework that for the first time provides FDR control for signature discovery. Our method is demonstrated to be correct in simulation studies. When applied to five cancer data sets, the method was able to discover molecular signatures with 5% FDR in three cases, while two data sets yielded no significant findings.ConclusionOur approach enables reliable discovery of molecular signatures from genome-wide data with current sample sizes. The statistical framework developed herein is potentially applicable to a wide range of prediction problems in bioinformatics.

A deterministic model of admixture and genetic introgression: The case of Neanderthal and Cro-Magnon

There is an ongoing debate in the field of human evolution about the possible contribution of Neanderthals to the modern human gene pool. To study how the Neanderthal private alleles may have spread over the genes of Homo sapiens, we propose a deterministic model based on recursive equations and ordinary differential equations. If the Neanderthal population was large compared to the Homo sapiens population at the beginning of the contact period, we show that genetic introgression should have been fast and complete meaning that most of the Neanderthal private alleles should be found in the modern human gene pool in case of ancient admixture. In order to test/reject ancient admixture from genome-wide data, we incorporate the model of genetic introgression into a statistical hypothesis-testing framework. We show that the power to reject ancient admixture increases as the ratio, at the time of putative admixture, of the population size of Homo sapiens over that of Neanderthal decreases. We find that the power to reject ancient admixture might be particularly low if the population size of Homo sapiens was comparable to the Neanderthal population size.

General solution and learning method for binary classification with performance constraints

In this paper, the problem of binary classification is studied with one or two performance constraints. When the constraints cannot be satisfied, the initial problem has no solution and an alternative problem is solved by introducing a rejection option. The optimal solution for such problems in the framework of statistical hypothesis testing is shown to be based on likelihood ratio with one or two thresholds depending on whether it is necessary to introduce a rejection option or not. These problems are then addressed when classes are only defined by labelled samples. To illustrate the resolution of cases with and without rejection option, the problem of Neyman–Pearson and the one of minimizing reject probability subject to a constraint on error probability are studied. Solutions based on SVMs and on a kernel based classifier are experimentally compared and discussed.

GENETICS AND RECENT HUMAN EVOLUTION

Starting with "mitochondrial Eve" in 1987, genetics has played an increasingly important role in studies of the last two million years of human evolution. It initially appeared that genetic data resolved the basic models of recent human evolution in favor of the "out-of-Africa replacement" hypothesis in which anatomically modern humans evolved in Africa about 150,000 years ago, started to spread throughout the world about 100,000 years ago, and subsequently drove to complete genetic extinction (replacement) all other human populations in Eurasia. Unfortunately, many of the genetic studies on recent human evolution have suffered from scientific flaws, including misrepresenting the models of recent human evolution, focusing upon hypothesis compatibility rather than hypothesis testing, committing the ecological fallacy, and failing to consider a broader array of alternative hypotheses. Once these flaws are corrected, there is actually little genetic support for the out-of-Africa replacement hypothesis. Indeed, when genetic data are used in a hypothesis-testing framework, the out-of-Africa replacement hypothesis is strongly rejected. The model of recent human evolution that emerges from a statistical hypothesis-testing framework does not correspond to any of the traditional models of human evolution, but it is compatible with fossil and archaeological data. These studies also reveal that any one gene or DNA region captures only a small part of human evolutionary history, so multilocus studies are essential. As more and more loci became available, genetics will undoubtedly offer additional insights and resolutions of human evolution.

Modelling the probability distribution of the number of DNA double-strand breaks due to sporadic alkylation of nucleotide bases

Metabolites and certain chemical agents (for example methyl methanesulfonate) can induce nucleotide bases on chromosomal strands to become alkylated. These alkylated sites have the potential to become single-strand chromosomal breaks, a form of DNA damage, if they are exposed to a sufficient temperature in vitro. It has been proposed that a single-strand break (SSB) sufficiently close to another SSB on the opposite chromosomal strand will form a double-strand break (DSB). DNA repair mechanisms are less able to repair DSBs compared to SSBs. Because of the complex three-dimensional structure of DNA, some chromosomal regions are more susceptible to alkylation than others. A question of interest is therefore whether these alkylated bases are randomly distributed or tend to be clustered. Pulsed-field gel electrophoresis allows the number of DNA fragments (and hence the number of DSBs) to be observed directly. The randomness of alkylation events can therefore be tested using the standard statistical hypothesis-testing framework. Under the null hypothesis, that the SSBs are randomly distributed on each of the strands, we can calculate the probability of observing a number of DSBs at least as large as that observed and hence the associated p-value. Previously, the probability distribution of the number of DSBs has been determined by Monte Carlo simulations; when considering the whole genome this can be very time consuming. In this paper, we theoretically derive an approximation to the distribution enabling appropriate probabilities to be calculated quickly. Based on previous findings we assume that the number of breaks on each strand is small compared to the number of nucleotide bases. We show that our method can give the correct probability distribution when alkylation events are relatively rare, discuss how rare these events have to be and suggest potential extensions to the model when a greater proportion of bases are alkylated.