Fixed Sample Size Test Research Articles

Nonparametric Sequential Clustering of Data Streams with Composite Distributions

We study a sequential nonparametric clustering problem to group a finite set of S data streams into K clusters. The data streams are real-valued i.i.d data sequences generated from unknown continuous distributions. The distributions themselves are organized into clusters according to their proximity to each other based on a certain distance metric. The sequential tests are universal in the sense that they are independent of the underlying configuration of the distribution clusters, and the distributions themselves, as long as the maximum intra-cluster distance is smaller than the minimum inter-cluster distance. We propose sequential nonparametric clustering tests for two cases: (1) K known and (2) K unknown. In both cases, we show that the proposed sequential nonparametric clustering tests stop in finite time almost surely and are universally exponentially consistent. Further, we also bound the asymptotic growth rate of the expected stopping time as probability of error goes to zero. Our results generalize earlier work on sequential nonparametric anomaly detection to the more general sequential nonparametric clustering problem. This generalization also provides a new test for the special case of anomaly detection where the anomalous data streams can follow distinct probability distributions. We also devise a modification of the proposed sequential nonparametric clustering tests that can result in significant computational savings with negligible performance degradation. Simulations show that all our proposed sequential clustering tests outperform the corresponding fixed sample size tests in terms of the expected number of samples for a given probability of error. The simulation results also demonstrate the advantage of our proposed clustering tests in anomaly detection problems with distinct anomalies.

A computational approach to the Kiefer-Weiss problem for sampling from a Bernoulli population

We present a computational approach to the solution of the Kiefer-Weiss problem. Algorithms for construction of the optimal sampling plans and evaluation of their performance are proposed. In the particular case of Bernoulli observations, the proposed algorithms are implemented in the form of R program code. Using the developed computer program, we numerically compare the optimal tests with the respective sequential probability ratio test (SPRT) and the fixed sample size test for a wide range of hypothesized values and type I and type II errors. The results are compared with those of D. Freeman and L. Weiss (Journal of the American Statistical Association, 59, 1964). The R source code for the algorithms of construction of optimal sampling plans and evaluation of their characteristics is available at https://github.com/tosinabase/Kiefer-Weiss.

Two Strategies in Transient Change Detection

The paper addresses the transient change detection (TCD) problem, assuming that the duration of change is finite. The TCD criterion minimizes the worst-case probability of missed detection among all tests with a prescribed worst-case probability of false alarm. We study the fixed sample size (FSS) test as a solution to the TCD problem. First, the operating characteristics of the FSS test have been established for arbitrary pre- and post-change distributions. Next, a numerical method of the sample (block) size optimization has been considered for three particular log-likelihood ratio distributions, i.e., Gaussian, <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\chi ^{2}$</tex-math></inline-formula> and exponential. Moreover, simple asymptotic equations for the optimal operating characteristics and block size have been proposed in the Gaussian case. Numerical results are provided to confirm the theoretical findings for the above-mentioned distributions. The accuracy and sharpness of the asymptotic analytical equation is analyzed in the Gaussian case. Finally, the FSS test is compared to the finite moving average (FMA) test obtained by optimizing the CUSUM-type test with respect to the TCD optimality criterion for the above-mentioned distributions. The application of the FSS and FMA tests to the radio-navigation integrity monitoring is also considered.

Sequential Nonparametric Detection of Anomalous Data Streams

We study a nonparametric search problem to detect $L$ anomalous streams from a finite set of $ S$ data streams. The $L$ anomalous streams are real-valued independent and identically distributed (i.i.d.) sequences drawn from the distribution $ q$ , while the remaining $S-L$ data streams are i.i.d. sequences drawn from the distribution $ p$ . The distributions $ p$ and $ q$ are assumed to be arbitrary and unknown , but distinct. We consider two cases: one where $L = 1$ , and the other where $0 \leq L \leq A$ . In both cases, we propose universal distribution-free sequential tests that are consistent. For the first case, we also: (1) show that the test is universally exponentially consistent and stops in finite time almost surely, and (2) bound the limiting growth rate of the expected stopping time as the probability of error decreases to zero. Simulations show that the performance of the proposed test is better than that of the fixed sample size test.

Quickest Detection for Changes in Maximal kNN Coherence of Random Matrices

This paper addresses the problem of quickest detection of a change in the maximal coherence between columns of an $n\times p$ random matrix based on a sequence of matrix observations having a single unknown change point. The random matrix is assumed to have identically distributed rows and the maximal coherence is defined as the largest of the $p \atopwithdelims ()2$ correlation coefficients associated with any row. Likewise the $k$ nearest neighbor (kNN) coherence is defined as the $k$ -th largest of these correlation coefficients. The forms of the pre- and postchange distributions of the observed matrices are assumed to belong to the family of elliptically contoured densities with sparse dispersion matrices but are otherwise unknown. A nonparametric stopping rule is proposed that is based on the maximal kNN sample coherence between columns of each observed random matrix. This is a summary statistic that is related to a test of existence of a hub vertex in a sample correlation graph having degree at least $k$ . Performance bounds on the delay and false alarm performance of the proposed stopping rule are obtained in the purely high-dimensional regime where $p\rightarrow \infty$ and $n$ is fixed. When the prechange dispersion matrix is diagonal it is shown that, among all functions of the proposed summary statistic, the proposed stopping rule is asymptotically optimal under a minimax quickest change detection model as the stopping threshold approaches infinity. The theory developed also applies to sequential hypothesis testing and fixed sample size tests.

A sequential hypothesis testing procedure for the process capability index Cpk

AbstractIn this study, we propose a sequential method for hypothesis testing on the Cpk process capability index. We compare the properties of the sequential test with the performances of nonsequential tests by performing an extensive simulation study. The results show that the proposed sequential test makes it possible to save a large amount of sample size as compared with the fixed sample size tests while maintaining the desired α‐level and power.

Multi-Objective Clustering Optimization for Multi-Channel Cooperative Spectrum Sensing in Heterogeneous Green CRNs

In this paper, we address energy efficient (EE) cooperative spectrum sensing policies for large scale heterogeneous cognitive radio networks(CRNs) which consist of multiple primary channels and large number of secondary users (SUs) with heterogeneous sensing and reporting channel qualities. We approach this issue from macro and micro perspectives. Macro perspective groups SUs into clusters with the objectives: 1) total energy consumption minimization; 2) total throughput maximization; and 3) inter-cluster energy and throughput fairness. We adopt and demonstrate how to solve these using the non-dominated sorting genetic algorithm-II. The micro perspective, on the other hand, operates as a sub-procedure on cluster formations decided by the macro perspective. For the micro perspectives, we first propose a procedure to select the cluster head (CH) which yields: 1) the best CH which gives the minimum total multi-hop error rate and 2) the optimal routing paths from SUs to the CH. Exploiting Poisson-Binomial distribution, a novel and generalized ${K}$ -out-of- ${N}$ voting rule is developed for heterogeneous CRNs to allow SUs to have different local detection performances. Then, a convex optimization framework is established to minimize the intra-cluster energy cost by jointly obtaining the optimal sensing durations and thresholds of feature detectors for the proposed voting rule. Likewise, instead of a common fixed sample size test, we developed a weighted sample size test for quantized soft decision fusion to obtain a more EE regime under heterogeneity. We have shown that the combination of proposed CH selection and cooperation schemes gives a superior performance in terms of energy efficiency and robustness against reporting error wall.

Hypothesis testing for Markov chain Monte Carlo

Testing between hypotheses, when independent sampling is possible, is a well developed subject. In this paper, we propose hypothesis tests that are applicable when the samples are obtained using Markov chain Monte Carlo. These tests are useful when one is interested in deciding whether the expected value of a certain quantity is above or below a given threshold. We show non-asymptotic error bounds and bounds on the expected number of samples for three types of tests, a fixed sample size test, a sequential test with indifference region, and a sequential test without indifference region. Our tests can lead to significant savings in sample size. We illustrate our results on an example of Bayesian parameter inference involving an ODE model of a biochemical pathway.

To Listen or Not: Distributed Detection With Asynchronous Transmissions

This letter examines a variation of the canonical distributed detection system: a sensor may overhear other sensors’ transmissions and thus may choose to refine its output in the hope of achieving a better detection performance. We show that while this is indeed possible for the fixed sample size test, asymptotically (in the number of samples) there is no performance gain, as measured by the Kullback-Leibler distance achievable at the fusion center, provided that the observations are conditionally independent. For conditionally dependent observations, however, we demonstrate that asymptotic detection performance may indeed be improved when overhearing is utilized.

Robust Service Time Measurement Using Comparison Sequential Test

The sequential comparison test is a tool for evaluation of the operational innovation in information technology service delivery processes. Due to the strong variability of these processes, the evaluation is done in comparison with the parallel running servers taken as reference. We consider the streams of service-completion events. When the time between events (TBE) is exponentially distributed, the binomial sequential probability ratio test (SPRT) can be used for evaluation. The effect of deviations from the exponential distribution on the characteristics of the test is analysed. We suggest a novel criterion that allows analysing robustness of the test. We show that the main factor influencing these characteristics is the coefficients of variation (CV) of the TBEs. Thus just by using CV of the TBEs, we may conclude whether the test is robust or not. We also suggest approach of handling the case when test for pair of single TBEs is not robust (case of CV > 1). Transition from a single server to a group of servers and from a single stream to a superposed stream of events improves robustness, since superposition of event streams brings the TBEs’ distribution closer to the exponential. Superposition makes it possible to deal with the problem for CV > 1. The analytical dependency of the fixed sample size test (FSST) robustness vs. CV permits simple estimation of robustness of the test in question. The advantage of the test is shown vs. the FSST, and illustrated on a real-life case.

Interactive Distributed Detection: Architecture and Performance Analysis

This paper studies the impact of interactive fusion on detection performance in tandem fusion networks with conditionally independent observations. Within the Neyman-Pearson framework, two distinct regimes are considered: the fixed sample size test and the large sample test. For the former, it is established that interactive distributed detection may strictly outperform the one-way tandem fusion structure. However, for the large sample regime, it is shown that interactive fusion has no improvement on the asymptotic performance characterized by the Kullback-Leibler (KL) distance compared with the simple one-way tandem fusion. The results are then extended to interactive fusion systems where the fusion center and the sensor may undergo multiple steps of memoryless interactions or that involve multiple peripheral sensors, as well as to interactive fusion with soft sensor outputs.

Sequential Detection Using an Ordered Statistics Method

Most of the development of distributed detection is mainly based on a fixed number of samples under test specification. However, in reality, is more suitable for sequential tests is used, because it can take advantage of relatively small sample size and reach with a fixed sample size test the same detection performance. Heuristic-based approach, this paper proposes a new sort of samples distributed based detection scheme. Finally, can be illustrated by the simulation results that the proposed method compared to traditional SPRT can further reduce the number of samples needed for the final decision.

A sequential triangular test of a correlation coefficient’s null-hypothesis: $$0 &lt; \rho \le \rho _{0}$$ 0 &lt; ρ ≤ ρ 0

A sequential triangular test of the null-hypothesis $$\hbox {H}_{0}{:} 0<\rho \le \rho _{0}$$ is derived, given a two-dimensional vector of normal random variables (x, y). The test is based on an approximate normally distributed test statistic by Fisher’s transformation of the sample correlation coefficient. We show via simulation that for certain requirements of precision (type-I-, type-II-risk, and a practical relevant effect $$\delta =\rho _1 -\rho _0$$ ) the average sample size of the sequential triangular test is smaller than the sample size of the pertinent fixed sample size test.

Cooperative detection of primary user emulation attacks based on channel-tap power in mobile cognitive radio networks

This work proposes a new method for identifying a primary user emulation attack (PUEA) at low signal–to–noise ratios (SNRs) in mobile cognitive radio (CR) networks. Channel–tap power is utilised as a radio–frequency (RF) fingerprint to directly detect users via physical (PHY) layer. To improve the detection performance in fading channels, the cooperative detection schemes using the fixed sample size test (FSST) and the sequential probability ratio test (SPRT) are devised. From simulation results, for a mobile CR speed of 70 km/h, SNR = –5 dB and false alarm probability of 0.03, the FSST using 10 cooperative nodes can achieve the detection probability of 0.99, which is increased by 1.94 times that of the single Neyman–Pearson detector. When the SPRT is compared to the FSST, the SPRT only needs less than half of the detection time of the FSST to provide the same detection performance for all SNRs considered.

Fuzzy Approach for Group Sequential Test

Buckley’s approach (Buckley (2004), (2005), (2006)) uses sets of confidence intervals by taking into consideration both of the uncertainty and impreciseness of concepts that produce triangular shaped fuzzy numbers for the estimator. This approach produces fuzzy test statistics and fuzzy critical values in hypothesis testing. In addition, the sample size is fixed for this test. When data comes sequentially, however, it is not suitable to study with a fixed sample size test. In such cases, sequential and group sequential tests are recommended. Unlike a sequential test, a group of sequential test provides substantial savings in sample and enables us to make decisions as early as possible. This intends paper to combine the benefits of group sequential test and Buckley's approach usingα-cuts. It attempts to show that usingα-cuts can be used within the group sequential tests. To illustrate the test more explicitly a numerical example is also given.

Splitting Tree Algorithm for Decentralized Detection in Sensor Networks

In this paper, we propose a novel collision resolution scheme for random access in wireless sensor networks. If a collision incurs during fusion process, splitting algorithm is applied to resolve the collision dynamically and recursively, based on past channel states alone or channel states and sensing information together. The novelty of our splitting algorithm is two-fold: 1. we perform splitting based on the informativeness of sensor data, ensuring that more informative data will be collected first; 2. we optimize splitting intervals based on local summaries of sensors collected at each fusion step. As shown in our simulation results, the proposed schemes achieve significant channel and power efficiency gain, compared with fixed sample size test, traditional sequential probability ratio test, and Pseudo-Bayesian based contention protocol.

Group-Ordered SPRT for Decentralized Detection

The problem of decentralized detection in a large wireless sensor network is considered. An adaptive decentralized detection scheme, group-ordered sequential probability ratio test (GO-SPRT), is proposed. This scheme groups sensors according to the informativeness of their data. Fusion center collects sensor data sequentially, starting from the most informative data and terminates the process when the target performance is reached. Wald's approximations are shown to be applicable even though the problem setting deviates from that of the traditional sequential probability ratio test (SPRT). To analyze the efficiency of GO-SPRT, the asymptotic equivalence between the average sample number of GO-SPRT, which is a function of a multinomial random variable, and a function of a normal random variable, is established. Closed-form approximations for the average sample number are then obtained. Compared with fixed sample size test and traditional SPRT, the proposed scheme achieves significant savings in the cost of data fusion.

Sequential analysis methodology for a Poisson GLMM with applications to multicenter randomized clinical trials

Sequential analyses in clinical trials have ethical and economic advantages over fixed sample size methods. The sequential probability ratio test (SPRT) is a hypothesis testing procedure which evaluates data as it is collected. The original SPRT was developed by Wald for one-parameter families of distributions and later extended by Bartlett to handle the case of nuisance parameters. However, Bartlett's SPRT requires independent and identically distributed observations. In this paper we show that Bartlett's SPRT can be applied to generalized linear model (GLM) contexts. Then we propose an SPRT analysis methodology for a Poisson generalized linear mixed model (GLMM) that is suitable for our application to the design of a multicenter randomized clinical trial that compares two preventive treatments for surgical site infections. We validate the methodology with a simulation study that includes a comparison to Neyman–Pearson and Bayesian fixed sample size test designs and the Wald SPRT.

Contention-Based Transmission for Decentralized Detection

In this work, we propose a contention-based protocol for general decentralized detection problem in the context of wireless sensor networks. In this scheme, fusion task is implemented in a multi-stage fashion: sensors are first grouped according to the informativeness of their data; fusion center then polls the sensor sets sequentially in the order of their informativeness until a target performance is reached. Within one stage, all polled sensors compete for a common channel medium where exists near-far effect, Raleigh fading, and shadowing. To determine the optimal transmission probability, we propose a novel Bayesian update algorithm utilizing both sensing information and channel feedback. The proposed dynamic protocol is applied to signal detection in Gaussian noise. As shown by our simulations, incorporating sensing information greatly improves efficiency over a generic Bayesian update scheme relying only on channel feedback. Our results also show that exploiting capture effect can significantly improve communication and energy efficiency. Comparison with fixed sample size test and sequential probability ratio test shows that the proposed scheme achieves significant efficiency gain over existing fusion strategies.

Maximum inflation of the type 1 error rate when sample size and allocation rate are adapted in a pre‐planned interim look

We calculate the maximum type 1 error rate of the pre-planned conventional fixed sample size test for comparing the means of independent normal distributions (with common known variance) which can be yielded when sample size and allocation rate to the treatment arms can be modified in an interim analysis. Thereby it is assumed that the experimenter fully exploits knowledge of the unblinded interim estimates of the treatment effects in order to maximize the conditional type 1 error rate. The 'worst-case' strategies require knowledge of the unknown common treatment effect under the null hypothesis. Although this is a rather hypothetical scenario it may be approached in practice when using a standard control treatment for which precise estimates are available from historical data. The maximum inflation of the type 1 error rate is substantially larger than derived by Proschan and Hunsberger (Biometrics 1995; 51:1315-1324) for design modifications applying balanced samples before and after the interim analysis. Corresponding upper limits for the maximum type 1 error rate are calculated for a number of situations arising from practical considerations (e.g. restricting the maximum sample size, not allowing sample size to decrease, allowing only increase in the sample size in the experimental treatment). The application is discussed for a motivating example.