Alternative Test Statistics Research Articles

Tests of linear hypotheses using indirect information

AbstractIn multigroup data settings with small within‐group sample sizes, standard ‐tests of group‐specific linear hypotheses can have low power, particularly if the within‐group sample sizes are not large relative to the number of explanatory variables. To remedy this situation, in this article we derive alternative test statistics based on information sharing across groups. Each group‐specific test has potentially much larger power than the standard ‐test, while still exactly maintaining a target type I error rate if the null hypothesis for the group is true. The proposed test for a given group uses a statistic that has optimal marginal power under a prior distribution derived from the data of the other groups. This statistic approaches the usual ‐statistic as the prior distribution becomes more diffuse, but approaches a limiting “cone” test statistic as the prior distribution becomes extremely concentrated. We compare the power and ‐values of the cone test to that of the ‐test in some high‐dimensional asymptotic scenarios. An analysis of educational outcome data is provided, demonstrating empirically that the proposed test is more powerful than the ‐test.

An Entropy-Based Approach for Nonparametrically Testing Simple Probability Distribution Hypotheses

In this paper, we introduce a flexible and widely applicable nonparametric entropy-based testing procedure that can be used to assess the validity of simple hypotheses about a specific parametric population distribution. The testing methodology relies on the characteristic function of the population probability distribution being tested and is attractive in that, regardless of the null hypothesis being tested, it provides a unified framework for conducting such tests. The testing procedure is also computationally tractable and relatively straightforward to implement. In contrast to some alternative test statistics, the proposed entropy test is free from user-specified kernel and bandwidth choices, idiosyncratic and complex regularity conditions, and/or choices of evaluation grids. Several simulation exercises were performed to document the empirical performance of our proposed test, including a regression example that is illustrative of how, in some contexts, the approach can be applied to composite hypothesis-testing situations via data transformations. Overall, the testing procedure exhibits notable promise, exhibiting appreciable increasing power as sample size increases for a number of alternative distributions when contrasted with hypothesized null distributions. Possible general extensions of the approach to composite hypothesis-testing contexts, and directions for future work are also discussed.

Multi-arm multi-stage clinical trials for time-to-event outcomes

ABSTRACT This paper investigates the use of a general multi-arm multi-stage (MAMS) approach for time-to-event outcomes that would streamline simultaneous comparison of a large number of promising therapies in clinical trials, thus significantly reducing the time and the number of patients needed to evaluate the treatment. Controlling type I error in this setting is different than regular clinical trials as this approach incorporates both multiple comparison between arms and multiple stages. Historically, pairwise (PWER) and familywise (FWER) type I error rates have been primarily used to regulate the type I error in such designs. This paper will focus on constructing the efficacy and futility boundaries for a MAMS clinical trial in two different scenarios. In the first, it is assumed that the same outcome is used throughout the clinical trial for both intermediate and final assessments. In this scenario, we propose using the generalized Dunnett procedure that controls FWER. In the latter scenario, where intermediate and final outcomes are different in nature, we propose modifications to the existing method that originally concentrated on controlling PWER and extend the method to include FWER in the design. We also explore the performance of the proposed MAMS design in a setting where the proportional hazard assumption is violated in the presence of a delayed treatment effect and demonstrate the loss of power because of that. An alternative test statistic that can help circumvent this problem to maintain the desired power is also suggested.

Consensus-Based Distributed Quickest Detection of Attacks With Unknown Parameters

Sequential attack detection in a distributed sensor network is considered, where each sensor successively produces one-bit quantized samples of a desired deterministic scalar parameter corrupted by additive noise. The unknown parameters in the pre-attack and post-attack models, namely the desired parameter to be estimated and the injected malicious data at the attacked sensors pose a significant challenge for designing a computationally efficient scheme for each sensor to detect the occurrence of attacks by only using local communication with neighboring sensors. The generalized Cumulative Sum (GCUSUM) algorithm is considered, which replaces the unknown parameters with their maximum likelihood estimates in the CUSUM test statistic. For the problem under consideration, a sufficient condition is provided under which the expected false alarm period of the GCUSUM can be guaranteed to be larger than any given value. Next, we consider the distributed implementation of the GCUSUM. We first propose an alternative test statistic which is asymptotically equivalent to that of GCUSUM. Then based on the proposed alternative test statistic and running consensus algorithms, we propose a distributed approximate GCUSUM algorithm which significantly reduce the prohibitively high computational complexity of the centralized GCUSUM. Numerical results show that the proposed distributed approximate GCUSUM algorithm can provide a performance that is comparable to the centralized GCUSUM.

Comparison of Rank Transformation Test Statistics with Its Nonparametric Counterpart Using Real-Life Data

Over the years, non-parametric test statistics have been the only solution to solve data that do not follow a normal distribution. However, giving statistical interpretation used to be a great challenge to some researchers. Hence, to overcome these hurdles, another test statistics was proposed called Rank transformation test statistics so as to close the gap between parametric and non-parametric test statistics. The purpose of this study is to compare the conclusion statement of Rank transformation test statistics with its equivalent non parametric test statistics in both one and two samples problems using real-life data. In this study, (2018/2019) Post Unified Tertiary Matriculation Examinations (UTME) results of prospective students of Ladoke Akintola University of Technology (LAUTECH) Ogbomoso across all faculties of the institution were used for the analysis. The data were subjected to nonparametric test statistics which include; Asymptotic Wilcoxon sign test and Wilcoxon sum Rank (both Asymptotic and Distribution) using Statistical Packages for Social Sciences (SPSS). In the same vein, R-statistical programming codes were written for Rank Transformation test statistics. Their P-values were extracted and compared with each other with respect to the pre-selected alpha level (α) = 0.05. Results in both cases revealed that there is a significant difference in the median of the scores across all faculties since their type I error rate are less than the preselected alpha level 0.05. Therefore, Rank transformation test statistics is recommended as alternative test statistics to non-parametric test in both one sample and two-sample problems.

On testing the hypothesis of population stability for credit risk scorecards

Scorecards are models used in credit risk modelling. These models segments a population into various so-called \risk based on the risk characteristics of the individual clients. Once a scorecard has been developed, the credit provider typically prefers to keep this model in use for an extended period. As a result, it is important to test whether or not the model still ts the population. To this end, the hypothesis of population stability is tested; this hypothesis speci es that the current proportions of the population in the various risk buckets are the same as was the case at the point in time at which the scorecard was developed. In practice, this assumption is usually tested using a measure known as the population stability index (which corresponds to the asymmetric Kullback-Leibler discrepancy between discrete distributions) together with a well-known rule of thumb. This paper considers the statistical motivation for the use of the population stability index. Numerical examples are provided in order to demonstrate the e ect of the rule of thumb as well as other critical values. Although previous numerical studies relating to this statistic are available, the sample sizes are not realistic for the South African credit market. The paper demonstrates that the population stability index has little statisticalmerit as either a goodness-of- t statistic to test the hypothesis of population stability or as an intuitive discrepancy measure. As a result, a novel methodology for testing the mentioned hypothesis is proposed. This methodology includes a restatement of the hypothesis to specify a range of \acceptable deviations from the speci ed model. An alternative test statistic is also employed as discrepancy measure; this measure has the advantage of having a simple heuristic interpretation in the context of credit risk modelling. Key words: Goodness-of- t testing, hypothesis testing, population stability, risk analysis.

A non-parametric statistic for testing conditional heteroscedasticity for unobserved component models

When prediction intervals are constructed using unobserved component models (UCM), problems can arise due to the possible existence of components that may or may not be conditionally heteroscedastic. Accurate coverage depends on correctly identifying the source of the heteroscedasticity. Different proposals for testing heteroscedasticity have been applied to UCM; however, in most cases, these procedures are unable to identify the heteroscedastic component correctly. The main issue is that test statistics are affected by the presence of serial correlation, causing the distribution of the statistic under conditional homoscedasticity to remain unknown. We propose a nonparametric statistic for testing heteroscedasticity based on the well-known Wilcoxon's rank statistic. We study the asymptotic validation of the statistic and examine bootstrap procedures for approximating its finite sample distribution. Simulation results show an improvement in the size of the homoscedasticity tests and a power that is clearly comparable with the best alternative in the literature. We also apply the test on real inflation data. Looking for the presence of a conditionally heteroscedastic effect on the error terms, we arrive at conclusions that almost all cases are different than those given by the alternative test statistics presented in the literature.

Improving the power of gene set enrichment analyses

BackgroundSet enrichment methods are commonly used to analyze high-dimensional molecular data and gain biological insight into molecular or clinical phenotypes. One important category of analysis methods employs an enrichment score, which is created from ranked univariate correlations between phenotype and each molecular attribute. Estimates of the significance of the associations are determined via a null distribution generated from phenotype permutation. We investigate some statistical properties of this method and demonstrate how alternative assessments of enrichment can be used to increase the statistical power of such analyses to detect associations between phenotype and biological processes and pathways.ResultsFor this category of set enrichment analysis, the null distribution is largely independent of the number of samples with available molecular data. Hence, providing the sample cohort is not too small, we show that increased statistical power to identify associations between biological processes and phenotype can be achieved by splitting the cohort into two halves and using the average of the enrichment scores evaluated for each half as an alternative test statistic. Further, we demonstrate that this principle can be extended by averaging over multiple random splits of the cohort into halves. This enables the calculation of an enrichment statistic and associated p value of arbitrary precision, independent of the exact random splits used.ConclusionsIt is possible to increase the statistical power of gene set enrichment analyses that employ enrichment scores created from running sums of univariate phenotype-attribute correlations and phenotype-permutation generated null distributions. This increase can be achieved by using alternative test statistics that average enrichment scores calculated for splits of the dataset. Apart from the special case of a close balance between up- and down-regulated genes within a gene set, statistical power can be improved, or at least maintained, by this method down to small sample sizes, where accurate assessment of univariate phenotype-gene correlations becomes unfeasible.

Visualizing Tests for Equality of Covariance Matrices

ABSTRACTThis article explores a variety of topics related to the question of testing the equality of covariance matrices in multivariate linear models, particularly in the MANOVA setting. Further, a plot of the components of Box’s M test is proposed that shows how groups differ in covariance and also suggests other visualizations and alternative test statistics. These methods are implemented and freely available in the heplots and candisc packages for R. Examples from the article and some further extensions are available in the online supplementary materials.

Statistical Tests for Detecting Granger Causality

Detection of a causal relationship between two or more sets of data is an important problem across various scientific disciplines. The Granger causality index and its derivatives are important metrics developed and used for this purpose. However, the test statistics based on these metrics ignore the effect of practical measurement impairments such as subsampling, additive noise, and finite sample effects. In this paper, we model the problem of detecting a causal relationship between two time series as a binary hypothesis test with the null and alternate hypotheses corresponding to the absence and presence of a causal relationship, respectively. We derive the distribution of the test statistic under the two hypotheses and show that measurement impairments can lead to suppression of a causal relationship between the signals, as well as false detection of a causal relationship, where there is none. We also use the derived results to propose two alternative test statistics for causality detection. These detectors are analytically tractable, which allows us to design the detection threshold and determine the number of samples required to achieve a given missed detection and false alarm rate. Finally, we validate the derived results using extensive Monte Carlo simulations as well as experiments based on real-world data, and illustrate the dependence of detection performance of the conventional and proposed causality detectors on parameters such as the additive noise variance and the strength of the causal relationship.

A randomization-based perspective on analysis of variance: a test statistic robust to treatment effect heterogeneity

Summary Fisher randomization tests for Neyman’s null hypothesis of no average treatment effect are considered in a finite-population setting associated with completely randomized experiments involving more than two treatments. The consequences of using the $F$ statistic to conduct such a test are examined, and we argue that under treatment effect heterogeneity, use of the $F$ statistic in the Fisher randomization test can severely inflate the Type I error under Neyman’s null hypothesis. We propose to use an alternative test statistic, derive its asymptotic distributions under Fisher’s and Neyman’s null hypotheses, and demonstrate its advantages through simulations.

Spectrum sensing for OFDM signals using pilot induced cyclostationarity in the presence of cyclic frequency offset

In this paper, the problem of spectrum sensing of OFDM signals for cognitive radios is considered. It is proposed to detect the cyclostationary features introduced in an OFDM signal due to inter-pilot correlation. The performance of the proposed detector is derived and verified in case of AWGN channels. It is observed that the performance of cyclostationary detectors relies on the knowledge of the exact value of the cyclic frequency of the signal of interest. However, an offset in the cyclic frequency may arise due to several reasons. Therefore, for the proposed detector to perform reliably, there is a need to estimate the cyclic frequency offset. The Cramer–Rao bound for the cyclic frequency offset (CFO) estimator is derived, and based on it, two algorithms to estimate and compensate for the CFO are proposed. Simulation results are then used to study the performance of the proposed detection technique under Rayleigh fading both in the presence and the absence of CFO. The performance of the proposed system model is also studied under fast fading, and an alternative test statistic is proposed.

Maximum Likelihood Estimation of Structural Equation Models for Continuous Data: Standard Errors and Goodness of Fit

Classical accounts of maximum likelihood (ML) estimation of structural equation models for continuous outcomes involve normality assumptions: standard errors (SEs) are obtained using the expected information matrix and the goodness of fit of the model is tested using the likelihood ratio (LR) statistic. Satorra and Bentler (1994) introduced SEs and mean adjustments or mean and variance adjustments to the LR statistic (involving also the expected information matrix) that are robust to nonnormality. However, in recent years, SEs obtained using the observed information matrix and alternative test statistics have become available. We investigate what choice of SE and test statistic yields better results using an extensive simulation study. We found that robust SEs computed using the expected information matrix coupled with a mean- and variance-adjusted LR test statistic (i.e., MLMV) is the optimal choice, even with normally distributed data, as it yielded the best combination of accurate SEs and Type I errors.

Cognitive radio: a study of the role of KL distance function in energy detection scheme

ABSTRACTCognitive radio has been identified as a frontline technology in wireless communication domain where unlicenced spectrum users can utilise unused licensed frequency bands without affecting the licensed users’ transmission process. Spectrum hole detection is the primary requirement to fulfil this objective. Usually energy detector is used for its easy realisation. In this method, a test statistic is formulated by Neyman–Pearson lemma and compared with a threshold to make a decision regarding the vacancy of a frequency band. This article shows an alternative test statistic formulation by Kullback–Leibler distance function and it is established that the proposed approach gives much better result in low signal to noise ratio zone with comparatively lesser number of samples. The performances are adjudicated by false alarm probability and detection probability. Numerical results are provided to validate the proposition.

Tests alternative to higher criticism for high-dimensional means under sparsity and column-wise dependence

We consider two alternative tests to the Higher Criticism test of Donoho and Jin [Ann. Statist. 32 (2004) 962-994] for high-dimensional means under the sparsity of the nonzero means for sub-Gaussian distributed data with unknown column-wise dependence. The two alternative test statistics are constructed by first thresholding $L_1$ and $L_2$ statistics based on the sample means, respectively, followed by maximizing over a range of thresholding levels to make the tests adaptive to the unknown signal strength and sparsity. The two alternative tests can attain the same detection boundary of the Higher Criticism test in [Ann. Statist. 32 (2004) 962-994] which was established for uncorrelated Gaussian data. It is demonstrated that the maximal $L_2$-thresholding test is at least as powerful as the maximal $L_1$-thresholding test, and both the maximal $L_2$ and $L_1$-thresholding tests are at least as powerful as the Higher Criticism test.

Robust Within Groups Anova: Dealing With Missing Values

The paper considers the problem of testing the hypothesis that J 2 dependent groups have equal population measures of location when using a robust estimator and there are missing values. For J = 2, methods have been studied based on trimmed means. But the methods are not readily extended to the case J > 2. Here, two alternative test statistics were considered, one of which performed poorly in some situations. The one method that performed well in simulations is based on a very simple test statistic with the null distribution approximated via a basic bootstrap technique. The method uses all of the avail- able data to estimate each of the marginal (population) trimmed means. Other robust measures of location were considered, for which imputation methods have been derived, but in simulations the actual Type I error probability was estimated to be substantially less than the nominal level, even when there are no missing values.

Testing statistical hypotheses based on the density power divergence

The family of density power divergences is an useful class which generates robust parameter estimates with high efficiency. None of these divergences require any non-parametric density estimate to carry out the inference procedure. However, these divergences have so far not been used effectively in robust testing of hypotheses. In this paper, we develop tests of hypotheses based on this family of divergences. The asymptotic variances of the estimators are generally different from the inverse of the Fisher information matrix, so that the usual drop-in-divergence type statistics do not lead to standard Chi-square limits. It is shown that the alternative test statistics proposed herein have asymptotic limits which are described by linear combinations of Chi-square statistics. Extensive simulation results are presented to substantiate the theory developed.

Combined test procedures in the meta-analysis of controlled clinical trials.

Concerning the actual significance level, we investigate the commonly used combined test procedures of meta-analysis, which contain the choice of the model, in which the analysis is carried out, and the commonly used tests for treatment effect in the fixed and random effects model, and some new combined test procedures, which use an alternative test statistic in the random effects model or the t-distribution as test distribution of the commonly used test statistics. A simulation study indicates that the new combined test procedures are better with respect to a prescribed significance level.

Markov switching and exchange rate predictability

Abstract We first show that the recent success of modern macroeconomic models in forecasting nominal exchange rates, evaluated using the Clark and West (2006) inference procedure, is partly due to the presence of the constant term (drift), in addition to the economic fundamentals. We then model the drift term using the two-state Markov switching stochastic segmented trend model and present evidence of both short-run (one month) and long-run (up to one year) predictability for monthly exchange rates over the post-Bretton Woods period. This is an important result, as the recent literature has typically failed to find evidence of consistent multi-horizon predictability. The model strongly outperforms the random walk for 9 out of 12 exchange rate series at short horizons; for 7 of the 12 exchange rates, we find evidence of a long-run predictability that declines as the forecast horizon increases. Our results remain robust to alternative test statistics and forecast windows.

A flexible genome‐wide bootstrap method that accounts for rankingand threshold‐selection bias in GWAS interpretation and replication study design

The phenomenon known as the winner's curse is a form of selection bias that affects estimates of genetic association. In genome-wide association studies (GWAS) the bias is exacerbated by the use of stringent selection thresholds and ranking over hundreds of thousands of single nucleotide polymorphisms (SNPs). We develop an improved multi-locus bootstrap point estimate and confidence interval, which accounts for both ranking- and threshold-selection bias in the presence of genome-wide SNP linkage disequilibrium structure. The bootstrap method easily adapts to various study designs and alternative test statistics as well as complex SNP selection criteria. The latter is demonstrated by our application to the Wellcome Trust Case Control Consortium findings, in which the selection criterion was the minimum of the p-values for the additive and genotypic genetic effect models. In contrast, existing likelihood-based bias-reduced estimators account for the selection criterion applied to an SNP as if it were the only one tested, and so are more simple computationally, but do not address ranking across SNPs. Our simulation studies show that the bootstrap bias-reduced estimates are usually closer to the true genetic effect than the likelihood estimates and are less variable with a narrower confidence interval. Replication study sample size requirements computed from the bootstrap bias-reduced estimates are adequate 75-90 per cent of the time compared to 53-60 per cent of the time for the likelihood method. The bootstrap methods are implemented in a user-friendly package able to provide point and interval estimation for both binary and quantitative phenotypes in large-scale GWAS.