Optimal Test Statistic Research Articles

Sequential Locally Optimum Test (SLOT): A Sequential Detection Scheme Based on Locally Optimum Test Statistic

Based on the characteristics of the thresholds of two detection schemes employing locally optimum test statistics, a sequential detection design procedure is proposed and analyzed. The proposed sequential test, called the sequential locally optimum test (SLOT), inherently provides finite stopping time (terminates with probability one within the finite horizon), and thereby avoids undesirable forced termination. The performance of the SLOT is compared with that of the fixed sample-size test, sequential probability ratio test (SPRT), truncated SPRT, and 2-SPRT. It is observed that the SLOT requires smaller average sample numbers than other schemes at most values of the normalized signal amplitude while maintaining the error performance close to the SPRT.

Reproducibility-Optimized Test Statistic for Ranking Genes in Microarray Studies

A principal goal of microarray studies is to identify the genes showing differential expression under distinct conditions. In such studies, the selection of an optimal test statistic is a crucial challenge, which depends on the type and amount of data under analysis. While previous studies on simulated or spike-in datasets do not provide practical guidance on how to choose the best method for a given real dataset, we introduce an enhanced reproducibility-optimization procedure, which enables the selection of a suitable gene- anking statistic directly from the data. In comparison with existing ranking methods, the reproducibilityoptimized statistic shows good performance consistently under various simulated conditions and on Affymetrix spike-in dataset. Further, the feasibility of the novel statistic is confirmed in a practical research setting using data from an in-house cDNA microarray study of asthma-related gene expression changes. These results suggest that the procedure facilitates the selection of an appropriate test statistic for a given dataset without relying on a priori assumptions, which may bias the findings and their interpretation. Moreover, the general reproducibilityoptimization procedure is not limited to detecting differential expression only but could be extended to a wide range of other applications as well.

Two‐Stage Group Sequential Robust Tests in Family‐Based Association Studies: Controlling Type I Error

In family-based association studies, an optimal test statistic with asymptotic normal distribution is available when the underlying genetic model is known (e.g., recessive, additive, multiplicative, or dominant). In practice, however, genetic models for many complex diseases are usually unknown. Using a single test statistic optimal for one genetic model may lose substantial power when the model is mis-specified. When a family of genetic models is scientifically plausible, the maximum of several tests, each optimal for a specific genetic model, is robust against the model mis-specification. This robust test is preferred over a single optimal test. Recently, cost-effective group sequential approaches have been introduced to genetic studies. The group sequential approach allows interim analyses and has been applied to many test statistics, but not to the maximum statistic. When the group sequential method is applied, type I error should be controlled. We propose and compare several approaches of controlling type I error rates when group sequential analysis is conducted with the maximum test for family-based candidate-gene association studies. For a two-stage group sequential robust procedure with a single interim analysis, two critical values for the maximum tests are provided based on a given alpha spending function to control the desired overall type I error.

Determining Image Origin and Integrity Using Sensor Noise

In this paper, we provide a unified framework for identifying the source digital camera from its images and for revealing digitally altered images using photo-response nonuniformity noise (PRNU), which is a unique stochastic fingerprint of imaging sensors. The PRNU is obtained using a maximum-likelihood estimator derived from a simplified model of the sensor output. Both digital forensics tasks are then achieved by detecting the presence of sensor PRNU in specific regions of the image under investigation. The detection is formulated as a hypothesis testing problem. The statistical distribution of the optimal test statistics is obtained using a predictor of the test statistics on small image blocks. The predictor enables more accurate and meaningful estimation of probabilities of false rejection of a correct camera and missed detection of a tampered region. We also include a benchmark implementation of this framework and detailed experimental validation. The robustness of the proposed forensic methods is tested on common image processing, such as JPEG compression, gamma correction, resizing, and denoising.

Statistical detection and imaging of objects hidden in turbid media using ballistic photons

We exploit recent advances in active high-resolution imaging through scattering media with ballistic photons. We derive the fundamental limits on the accuracy of the estimated parameters of a mathematical model that describes such an imaging scenario and compare the performance of ballistic and conventional imaging systems. This model is later used to derive optimal single-pixel statistical tests for detecting objects hidden in turbid media. To improve the detection rate of the aforementioned single-pixel detectors, we develop a multiscale algorithm based on the generalized likelihood ratio test framework. Moreover, considering the effect of diffraction, we derive a lower bound on the achievable spatial resolution of the proposed imaging systems. Furthermore, we present the first experimental ballistic scanner that directly takes advantage of novel adaptive sampling and reconstruction techniques.

Optimal decision network with distributed representation

On the basis of detailed analysis of reaction times and neurophysiological data from tasks involving choice, it has been proposed that the brain implements an optimal statistical test during simple perceptual decisions. It has been shown recently how this optimal test can be implemented in biologically plausible models of decision networks, but this analysis was restricted to very simplified localist models which include abstract units describing activity of whole cell assemblies rather than individual neurons. This paper derives the optimal parameters in a model of a decision network including individual neurons, in which the alternatives are represented by distributed patterns of neuronal activity. It is also shown how the optimal weights in the decision network can be learnt via iterative rules using information accessible for individual synapses. Simulations demonstrate that the network with the optimal synaptic weights achieves better performance and matches fundamental behavioural regularities observed in choice tasks (Hick's law and the relationship between the error rate and the time for decision) better than a network with synaptic weights set according to a standard Hebb rule.

Non-Bayesian Detection and Detectability of Anomalies From a Few Noisy Tomographic Projections

The detection of an anomaly from a few noisy tomographic projections is addressed from the statistical point of view. An unknown scene is composed of a background, considered as a deterministic nuisance parameter, with a possibly hidden anomaly. Because the full pixel-by-pixel reconstruction is impossible, a parametric non-Bayesian approach is proposed to fill up the gap in the missing data. An optimal statistical test which eliminates the background and detects the anomaly is designed. The potential advantage of such an approach is its capacity to detect an anomaly/target hidden in background designed by an adversary to mask the anomaly. A key issue in the non-Bayesian anomaly detection, i.e., the problem of anomaly detectability, is stated and solved in this paper. In the case of a bivariate polynomial background defined on an unknown rectangular support, the size of detectable anomaly reaches its maximum defined by the number of elementary cells of X-ray detector and degree of the polynomial function

The Basal Ganglia and Cortex Implement Optimal Decision Making Between Alternative Actions

Neurophysiological studies have identified a number of brain regions critically involved in solving the problem of action selection or decision making. In the case of highly practiced tasks, these regions include cortical areas hypothesized to integrate evidence supporting alternative actions and the basal ganglia, hypothesized to act as a central switch in gating behavioral requests. However, despite our relatively detailed knowledge of basal ganglia biology and its connectivity with the cortex and numerical simulation studies demonstrating selective function, no formal theoretical framework exists that supplies an algorithmic description of these circuits. This article shows how many aspects of the anatomy and physiology of the circuit involving the cortex and basal ganglia are exactly those required to implement the computation defined by an asymptotically optimal statistical test for decision making: the multihypothesis sequential probability ratio test (MSPRT). The resulting model of basal ganglia provides a new framework for understanding the computation in the basal ganglia during decision making in highly practiced tasks. The predictions of the theory concerning the properties of particular neuronal populations are validated in existing experimental data. Further, we show that this neurobiologically grounded implementation of MSPRT outperforms other candidates for neural decision making, that it is structurally and parametrically robust, and that it can accommodate cortical mechanisms for decision making in a way that complements those in basal ganglia.

On the optimality of equal gain combining for energy detection of unknown signals

In this letter, the optimally of equal gain combining (EGC) for energy detection of unknown signals in a system with receive-diversity is studied. It is shown that EGC results in an optimal test statistic for infinitesimally small signal-to-noise ratios (SNRs); however, it does not yield the optimal test for all SNR values according to the Neyman-Pearson criterion. In other words, EGC does not induce a uniformly most powerful (UMP) test for unknown signal detection

An asymptotically optimal detector for Gaussianity testing

The aim of this paper is to provide a novel approach to the classical problem of testing the Gaussianity of a stationary process where the data can be assumed to be independent and identically distributed. Some results of Large Deviations theory are used to select a set of asymptotically optimal test statistics so that their combination results in a suitable detection procedure. The Johnson system of distributions has been used as the basis for the design of the detector. The fact that the good asymptotical properties can be extended to finite samples with small sizes provides the motivation for the rest of the work. In this respect, the performance of the proposed procedure is also compared with that of some known Gaussianity tests. The possibility to directly calculate a reliable measure of support such as the Bayesian posterior probabilities, as opposed to P-values, is an additional advantage. Furthermore, it will be shown how the proposed procedure can provide some extra information concerning the type of deviation from Gaussianity, which is present in the data.

A Statistical Detection of an Anomaly from a Few Noisy Tomographic Projections

The problem of detecting an anomaly/target from a very limited number of noisy tomographic projections is addressed from the statistical point of view. The imaged object is composed of an environment, considered as a nuisance parameter, with a possibly hidden anomaly/target. The GLR test is used to solve the problem. When the projection linearly depends on the nuisance parameters, the GLR test coincides with an optimal statistical invariant test.

9. Model Fit in Structural Equation Models with Censored, Ordinal, and Dichotomous Variables: Testing Vanishing Tetrads

Although the methodology for handling ordinal and dichotomous observed variables in structural equation models (SEMs) is developing rapidly, several important issues are unresolved. One of these is the optimal test statistic to apply as a test of overall model fit. We propose a new “vanishing tetrad” test statistic for such models. We build on Bollen's (1990) simultaneous test statistic for testing multiple vanishing tetrads and on Bollen and Ting's (1993) confirmatory tetrad analysis (CTA) for hypothesis testing of model structures. These and other works on vanishing tetrads assume continuous observed variables and do not consider observed categorical variables. In this paper we present a method to test models when some or all of the observed variables are collapsed or categorical versions of underlying continuous variables. The test statistic that we provide is an alternative “overall fit” statistic for SEMs with censored, ordinal, or dichotomous observed variables. Furthermore, the vanishing tetrad test sometimes permits us to compare the fit of some models that are not nested in the traditional likelihood ratio test. We illustrate the new test statistic with examples and a small simulation experiment comparing it with two other tests of model fit for SEMs with ordinal or dichotomous endogenous variables.

Optimal tests for carcinogenicity in a model with fatal and incidental tumours

In a recent paper Heimann and Neuhaus (Biometrika 88 (2001) 435) studied the combined test of Peto et al. (Long Term and Short Term Screening Assays for Carcinogens: A Critical Appraisal, IARC Monograph on the evaluation of the carcinogenic risk of chemicals to humans, Annex to supplement 2, WHO, Geneva, pp. 311–426) for the two sample testing problem. This test is an ad hoc test adding the log rank test statistic for the random censorship model and the Mantel Haenszel test statistic for the interval censored data model. Heimann and Neuhaus (Biometrika 88 (2001) 435) also introduced an appropriate statistical model combining randomly censored and interval censored data, to handle lethal and incidental tumours jointly. In the present paper we will derive asymptotically optimal tests for this model in a systematic way. The optimal test statistic is decomposed into two orthogonal parts, representing the fatal and incidental components. This will allow to use well known results from the literature for the fatal part, to estimate unknown quantities for the incidental part, and last but not least to compare with Peto's combined test. Moreover, we introduce conditional permutation versions of our tests which are finite sample distribution free under the null hypothesis with equal censoring and are asymptotically equivalent to their unconditional counterparts even under locally unequal censoring. Crucial for these results is a conditional limit theorem for the test statistics under local alternatives which follows from results of Strasser and Weber (Math. Methods Statist. 2 (1999) 220).

Time–frequency-based detection in impulsive noise environments using α-stable noise models

We develop near-optimal test statistics for the detection of arbitrary non-stationary second-order random signals in impulsive noise, modelled using a bivariate, isotropic α-stable distribution. The test statistics are derived by approximating the noise model using a mixture of Gaussians, trained using an expectation-maximisation algorithm. We consider the extension to the case when the signal to be detected is subjected to an unknown time–frequency or time-scale shift, and show that approximations to locally optimal test statistics can be implemented using bilinear time–frequency or time-scale representations. We demonstrate that the performance of the locally optimal linear receiver is poor in even mildly impulsive noise; the alternative detection statistics proposed in this paper offer considerably enhanced performance.

A Bootstrap Procedure for Adaptive Selection of the Test Statistic in Flexible Two-Stage DesignsThis paper is based on a presentation given at the Workshop “Frontiers in Adaptive Designs”, 17.–18. 5. 2001, Vienna/Austria.

Adaptive two-stage designs allow a data-driven change of design characteristics during the ongoing trial. One of the available options is an adaptive choice of the test statistic for the second stage of the trial based on the results of the interim analysis. Since there is often only a vague knowledge of the distribution shape of the primary endpoint in the planning phase of a study, a change of the test statistic may then be considered if the data indicate that the assumptions underlying the initial choice of the test are not correct. Collings and Hamilton proposed a bootstrap method for the estimation of the power of the two-sample Wilcoxon test for shift alternatives. We use this approach for the selection of the test statistic. By means of a simulation study, we show that the gain in terms of power may be considerable when the initial assumption about the underlying distribution was wrong, whereas the loss is relatively small when in the first instance the optimal test statistic was chosen. The results also hold true for comparison with a one-stage design. Application of the method is illustrated by a clinical trial example.

A saddlepoint approximation for testing exponentiality against some increasing failure rate alternatives

In this article we discuss uniformly most powerful unbiased tests for testing exponentiality against a specific class of two-parameter exponential models with increasing failure rate. We show that the optimal test statistic for this problem admits an alternative representation in terms of a spacings statistic. Using the conditional saddlepoint approximation proposed by Gatto and Jammalamadaka (J. Amer. Statist. Assoc. 94 (1999) 533), we provide highly accurate approximations for the significance values. The test procedure is illustrated with two practical examples from reliability and survival analysis. We also obtain the asymptotic distribution of the test statistic under a sequence of converging alternatives, which allows for the computation of asymptotic relative efficiency.

An optimal statistical testing policy for software reliability demonstration of safety-critical systems

When software reliability demonstration of safety-critical systems by statistical testing is treated as a Test, Analyse and Fix (TAAF) process, an optimal testing policy can be found, which maximises the probability of success of the whole process, over a pre-determined period of time. The optimisation problem is formulated, solved by stochastic dynamic programming, and demonstrated by two numerical examples.

DS/SS code acquisition with joint detection of multiple correct cells using locally optimum test statistics in Rayleigh fading channels

A new PN code acquisition scheme is proposed based on the joint detection of correct cells ( H 1 cells) at which the code phase offset between the locally generated and received PN codes is within a desired small offset. Most of the acquisition schemes which have been proposed assume that there exists one and only one H 1 cell over the uncertainty region. However, in practical systems, there exist multiple H 1 cells. In this paper, using the locally optimum test statistic, the joint decision rule of two H 1 cells is derived for nonselective Rayleigh fading channels. Then, with the proposed acquisition scheme based on the joint decision rule, a closed-form expression of the mean acquisition time is obtained with the state diagram, and the probabilities of detection, missing, and false alarm are derived. The mean acquisition time performance of the proposed acquisition scheme is compared with that of a conventional acquisition scheme in which only an H 1 cell is used for detection at each step. The numerical results show that the proposed scheme not only has better performance but also is more robust to the residual code phase offset variation than the conventional scheme.

Asymptotically Optimal Tests and Optimal Designs for Testing the Mean in Regression Models with Applications to Change-Point Problems

Let a linear regression model be given with an experimental region [a, b] → R and regression functions f1, ..., fd+1 : [a, b] → R. In practice it is an important question whether a certain regression function fd+1, say, does or does not belong to the model. Therefore, we investigate the test problem H0 : "fd+1 does not belong to the model" against K : "fd+1 belong to the model" based on the least-squares residuals of the observations made at design points of the experimental region [a, b]. By a new functional central limit theorem given in Bischoff (1998, Ann. Statist. 26, 1398–1410), we are able to determine optimal tests in an asymptotic way. Moreover, we introduce the problem of experimental design for the optimal test statistics. Further, we compare the asymptotically optimal test with the likelihood ratio test (F-test) under the assumption that the error is normally distributed. Finally, we consider real change-point problems as examples and investigate by simulations the behavior of the asymptotic test for finite sample sizes. We determine optimal designs for these examples.

Selection of an Adaptive Test Statistic for Use with Multiple Comparison Analyses of Neuroimaging Data

Statistical analysis of neuroimages is commonly approached with intergroup comparisons made by repeated application of univariate or multivariate tests performed on the set of the regions of interest sampled in the acquired images. The use of such large numbers of tests requires application of techniques for correction for multiple comparisons. Standard multiple comparison adjustments (such as the Bonferroni) may be overly conservative when data are correlated and/or not normally distributed. Resampling-based step-down procedures that successfully account for unknown correlation structures in the data have recently been introduced. We combined resampling step-down procedures with the Minimum Variance Adaptive method, which allows selection of an optimal test statistic from a predefined class of statistics for the data under analysis. As shown in simulation studies and analysis of autoradiographic data, the combined technique exhibits a significant increase in statistical power, even for small sample sizes (n = 8, 9, 10).