Hypothesis Testing Perspective Research Articles

A hypothesis-testing perspective on the G-normal distribution theory

The G-normal distribution was introduced by Peng (2007) as the limiting distribution in the central limit theorem for sublinear expectation spaces. Equivalently, it can be interpreted as the solution to a stochastic control problem where we have a sequence of random variables, whose variances can be chosen based on all past information. In this note we study the tail behavior of the G-normal distribution through analyzing a nonlinear heat equation. Asymptotic results are provided so that the tail “probabilities” can be easily evaluated with high accuracy. This study also has a significant impact on the hypothesis testing theory for heteroscedastic data; we show that even if the data are generated under the null hypothesis, it is possible to cheat and attain statistical significance by sequentially manipulating the error variances of the observations.

Investigating Statistical Privacy Frameworks from the Perspective of Hypothesis Testing

Abstract Over the last decade, differential privacy (DP) has emerged as the gold standard of a rigorous and provable privacy framework. However, there are very few practical guidelines on how to apply differential privacy in practice, and a key challenge is how to set an appropriate value for the privacy parameter ɛ. In this work, we employ a statistical tool called hypothesis testing for discovering useful and interpretable guidelines for the state-of-the-art privacy-preserving frameworks. We formalize and implement hypothesis testing in terms of an adversary’s capability to infer mutually exclusive sensitive information about the input data (such as whether an individual has participated or not) from the output of the privacy-preserving mechanism. We quantify the success of the hypothesis testing using the precision- recall-relation, which provides an interpretable and natural guideline for practitioners and researchers on selecting ɛ. Our key results include a quantitative analysis of how hypothesis testing can guide the choice of the privacy parameter ɛ in an interpretable manner for a differentially private mechanism and its variants. Importantly, our findings show that an adversary’s auxiliary information - in the form of prior distribution of the database and correlation across records and time - indeed influences the proper choice of ɛ. Finally, we also show how the perspective of hypothesis testing can provide useful insights on the relationships among a broad range of privacy frameworks including differential privacy, Pufferfish privacy, Blowfish privacy, dependent differential privacy, inferential privacy, membership privacy and mutual-information based differential privacy.

A statistical normalization method and differential expression analysis for RNA-seq data between different species

BackgroundHigh-throughput techniques bring novel tools and also statistical challenges to genomic research. Identifying genes with differential expression between different species is an effective way to discover evolutionarily conserved transcriptional responses. To remove systematic variation between different species for a fair comparison, normalization serves as a crucial pre-processing step that adjusts for the varying sample sequencing depths and other confounding technical effects.ResultsIn this paper, we propose a scale based normalization (SCBN) method by taking into account the available knowledge of conserved orthologous genes and by using the hypothesis testing framework. Considering the different gene lengths and unmapped genes between different species, we formulate the problem from the perspective of hypothesis testing and search for the optimal scaling factor that minimizes the deviation between the empirical and nominal type I errors.ConclusionsSimulation studies show that the proposed method performs significantly better than the existing competitor in a wide range of settings. An RNA-seq dataset of different species is also analyzed and it coincides with the conclusion that the proposed method outperforms the existing method. For practical applications, we have also developed an R package named “SCBN”, which is freely available at http://www.bioconductor.org/packages/devel/bioc/html/SCBN.html.

A New Framework for Detection and Identification of Network Parameter Errors

Normalized Lagrange multiplier test has been shown to be very effective for network parameter error identification, but its validation has so far been solely based on extensive simulations. This paper presents a new framework by which: 1) the normalized Lagrange multiplier test is re-formulated from the perspective of hypothesis testing, enabling proper handling of missing bad parameter cases; 2) formal proofs are given for the combined utilization of normalized Lagrange multiplier test and normalized residual test for simultaneous handling of measurement and parameter errors; and 3) the concepts of detectability and identifiability for measurement errors are extended to parameter errors, and a systematic approach for identifying critical parameters and critical k-tuples is provided.

Discussion on “A New Framework for Detection and Identification of Network Parameter Errors”

The authors present an interesting method on Normalized Lagrange Multiplier test for network parameter errors identification. The authors state that validation has so far been solely based on extensive simulations. They also state that the paper presents a new framework by which: (1) the normalized Lagrange multiplier test is re-formulated from the perspective of hypothesis testing, enabling proper handling of missing bad parameter cases; (2) formal proofs are given for the combined utilization of normalized Lagrange multiplier test and normalized residual test for simultaneous handling of measurement and parameter errors; and (3) the concepts of detectability and identifiability for measurement errors are extended to parameter errors, and a systematic approach for identifying critical parameters and critical k-tuples is provided. However, in the paper section II, they present in the problem formulation: \begin{equation*} z=h\left ({x,p_{e}}\right )+e \tag{1}\end{equation*} where $z$ is the measurement vector, $x$ is the state vector, $e$ is the measurement error vector, and $h$ is the nonlinear function linking $x$ and $p_{e}$ to $z$ . Also, \begin{equation*} p_{e}=p-p_{t} \end{equation*} $p_{t}$ being the true parameter value, not known, and $p$ the parameter available from the data.

A Hypothesis Testing Based Method for Normalization and Differential Expression Analysis of RNA-Seq Data.

Next-generation sequencing technologies have made RNA sequencing (RNA-seq) a popular choice for measuring gene expression level. To reduce the noise of gene expression measures and compare them between several conditions or samples, normalization is an essential step to adjust for varying sample sequencing depths and other unwanted technical effects. In this paper, we develop a novel global scaling normalization method by employing the available knowledge of housekeeping genes. We formulate the problem from the hypothesis testing perspective and find an optimal scaling factor that minimizes the deviation between the empirical and the nominal type I error. Applying our approach to various simulation studies and real examples, we demonstrate that it is more accurate and robust than the state-of-the-art alternatives in detecting differentially expression genes.

Nonparametric Assessment of Contamination in Multivariate Data Using Generalized Quantile Sets and FDR

Large, multivariate datasets from high-throughput instrumentation have become ubiquitous in the sciences. Frequently, it is of interest to characterize the measurements in these datasets by the extent to which they represent ‘nominal’ versus ‘contaminated’ instances. However, often the nature of even the nominal patterns in the data is unknown and potentially quite complex, making their explicit parametric modeling a daunting task. In this article, we introduce a nonparametric method for the simultaneous annotation of multivariate data (called MN-SCAnn), by which one may produce an annotated ranking of the observations, indicating the relative extent to which each may or may not be considered nominal, while making minimal assumptions on the nature of the nominal distribution. In our framework each observation is linked to a corresponding generalized quantile set and, implicitly adopting a hypothesis testing perspective, each set is associated with a test, which in turn is accompanied by a certain false discovery rate. The combination of generalized quantile set methods with false discovery rate principles, in the context of contaminated data, is new, and estimation of the key underlying quantities requires that a number of issues be addressed. We illustrate MN-SCAnn through examples in two contexts: the preprocessing of flow cytometry data in bioinformatics, and the detection of anomalous traffic patterns in Internet measurement studies. This article has supplementary material online.

Unrealistically Optimistic Consumers: A Selective Hypothesis Testing Account for Optimism in Predictions of Future Behavior

We propose that when predicting future behavior, consumers selectively (but unwittingly) test the hypothesis that they will behave ideally. This selective hypothesis testing perspective on unrealistic optimism suggests that estimates of future behavior should be similar to those made by individuals who assume that conditions will be ideal. Moreover, consumers who initially provide estimates assuming that conditions will be ideal should recognize that the world is not ideal and so should test a more realistic hypothesis. In line with these predictions, we find that ideal-world estimates (e.g., In an ideal world, how often will you exercise next week?) do not differ from standard estimates (e.g., How often will you exercise next week?). We also find that individuals who initially estimate their behavior in an ideal world subsequently make more realistic predictions.

A Selective Hypothesis Testing Perspective on Price-Quality Inference and Inference-Based Choice

Consumers often rely heavily on price as a predictor of quality and typically overestimate the strength of this relation. Furthermore, the inferences of quality they make on the basis of price can influence their actual purchase decisions. Selective hypothesis testing appears to underlie the effects of information load and format on price–quality inferences. Results of 5 experiments converge on the conclusion that quality inferences are more heavily influenced by price when individuals have a high need for cognitive closure, when the amount of information presented is high (vs. low), and when the information presented is rank ordered in terms of quality rather than presented randomly. Furthermore, because consumers are willing to purchase more expensive brands when they perceive a high price–quality correlation, these variables can also influence their purchase decisions.

Host-based intrusion detection using dynamic and static behavioral models

Intrusion detection has emerged as an important approach to network security. In this paper, we adopt an anomaly detection approach by detecting possible intrusions based on program or user profiles built from normal usage data. In particular, program profiles based on Unix system calls and user profiles based on Unix shell commands are modeled using two different types of behavioral models for data mining. The dynamic modeling approach is based on hidden Markov models (HMM) and the principle of maximum likelihood, while the static modeling approach is based on event occurrence frequency distributions and the principle of minimum cross entropy. The novelty detection approach is adopted to estimate the model parameters using normal training data only, as opposed to the classification approach which has to use both normal and intrusion data for training. To determine whether or not a certain behavior is similar enough to the normal model and hence should be classified as normal, we use a scheme that can be justified from the perspective of hypothesis testing. Our experimental results show that the dynamic modeling approach is better than the static modeling approach for the system call datasets, while the dynamic modeling approach is worse for the shell command datasets. Moreover, the static modeling approach is similar in performance to instance-based learning reported previously by others for the same shell command database but with much higher computational and storage requirements than our method.