Sequential Sampling by Individuals and Groups: An Experimental Study

SUMMARY The negative binomial family of distributions is indexed by two parameters, m, the mean, and k, where 1/k is a measure of aggregation. In constructing sequential probability ratio tests (SPRTs) concerning m, it has been found necessary to assume a common value of k for the null and alternative hypotheses. A method is shown of assessing the performance of SPRTs without making this assumption, using a model of the dependence of k on m. Truncation of the tests is also allowed for. A test can be adjusted to have given error probabilities under the assumptions of varying k and truncation. The power and average sample number of a test are calculated. The method is illustrated by performing sequential hypothesis tests concerning infestation by grass-grub larvae. The accuracy of Wald's approximations for constructing tests having given error probabilities is assessed and found to be low. Wald (1947) describes a method for finding sequential tests of hypotheses which are known as sequential probability ratio tests (SPRTs) and which have certain optimality properties. The negative binomial distribution can be defined in terms of two parameters, m, the mean, and k where 1/k is a measure of aggregation or clumping. In applying Wald's method to constructing tests of hypotheses concerning the mean of a negative binomial distribution, it has been necessary to assume a value of k common to the null and alternative values of m (Oakland, 1950). Wetherill (1975), in describing a sequential sampling scheme for negative binomial data, comments on the assumption of a fixed value for k and states that Variations in [k] will alter the properties of the SPRT, and the effect of such variations does not appear to have been investigated. In addition there are two general sources of inaccuracy in calculations for sequential tests. First, Wald's formulas for constructing a test with given Type I and Type II error probabilities are only approximations. Second, the sample size in the SPRTs constructed by Wald's method is unbounded whereas in practice the tests are truncated. In this paper a method is shown whereby tests constructed by Wald's method for the negative binomial distribution, assuming a constant value of k, can be assessed after truncation and without assuming a constant value of k, with error probabilities being calculated. The test is then adjusted so that it has the desired error probabilities to a high degree of accuracy under these assumptions. The method is applied to the sequential sampling of grass grub (Costelytra zealandica). The dependence of k on m is explored by calculating maximum likelihood estimates, mn and k, of m and k, for a number of data sets. A model of this dependence is built by regressing k on mh. Then, in calculating error probabilities and average sample numbers for the sequential hypothesis test, k is assumed to have a distribution whose mean varies with m. A discrete approximation to this distribution is used in the calculations. The power and average sample number are calculated and compared with those calculated under an assumption of constant k.

Previous article Next article A Sequential Procedure for Testing Many Composite HypothesesI. V. PavlovI. V. Pavlovhttps://doi.org/10.1137/1132017PDFBibTexSections ToolsAdd to favoritesExport CitationTrack CitationsEmail SectionsAbout[1] Abraham Wald, Sequential Analysis, John Wiley & Sons Inc., New York, 1947xii+212 8,593h 0029.15805 Google Scholar[2] Herbert Robbins and , David Siegmund, A class of stopping rules for testing parametric hypotheses, Proceedings of the Sixth Berkeley Symposium on Mathematical Statistics and Probability (Univ. California, Berkeley, Calif., 1970/1971), Vol. IV: Biology and health, Univ. California Press, Berkeley, Calif., 1972, 37–41 53:6924 Google Scholar[3] H. Robbins and , D. Siegmund, The expected sample size of some tests of power one, Ann. Statist., 2 (1974), 415–436 56:7055 0318.62069 CrossrefGoogle Scholar[4] Gideon Schwarz, Asymptotic shapes of Bayes sequential testing regions, Ann. Math. Statist., 33 (1962), 224–236 25:682 0158.36704 CrossrefGoogle Scholar[5] Herman Chernoff, Sequential design of experiments, Ann. Math. Statist., 30 (1959), 755–770 21:7586 0092.36103 CrossrefGoogle Scholar[6] Arthur E. Albert, The sequential design of experiments for infinitely many states of nature, Ann. Math. Statist., 32 (1961), 774–799 23:A3019 0109.12401 CrossrefGoogle Scholar[7] Gary Lorden, Likelihood ratio tests for sequential k-decision problems, Ann. Math. Statist., 43 (1972), 1412–1427 49:8242 0262.62045 CrossrefGoogle Scholar[8] Shelemyahu Zacks, The theory of statistical inference, John Wiley & Sons Inc., New York, 1971xiii+609 54:8934a Google Scholar[9] A. N. Shiryaev, Statistical sequential analysis, American Mathematical Society, Providence, R.I., 1973iv+174 50:3482 Google Scholar[10] I. V. Pavlov, Optimal Sequential Decision Rules, Moskov. Cos. Univ., Moscow, 1985, (In Russian.) Google Scholar[11] Wassily Hoeffding, Lower bounds for the expected sample size and the average risk of a sequential procedure, Ann. Math. Statist., 31 (1960), 352–368 22:11499 0098.32705 CrossrefGoogle Scholar Previous article Next article FiguresRelatedReferencesCited ByDetails Sequential probability ratio test for Multiple-Objective Ranking and Selection2017 Winter Simulation Conference (WSC) | 1 Dec 2017 Cross Ref Decentralized Sequential Composite Hypothesis Test Based on One-Bit CommunicationIEEE Transactions on Information Theory, Vol. 63, No. 6 | 1 Jun 2017 Cross Ref Asymptotic Optimality of Combined Double Sequential Weighted Probability Ratio Test for Three Composite HypothesesMathematical Problems in Engineering, Vol. 2015 | 1 Jan 2015 Cross Ref Optimal Sequential Tests for Testing Two Composite and Multiple Simple HypothesesSequential Analysis, Vol. 30, No. 4 | 1 Oct 2011 Cross Ref Multistage Tests of Multiple HypothesesCommunications in Statistics - Theory and Methods, Vol. 39, No. 8-9 | 21 Apr 2010 Cross Ref Minimax Sequential Tests for Many Composite Hypotheses. IIB. E. Brodsky and B. S. DarkhovskyTheory of Probability & Its Applications, Vol. 53, No. 1 | 27 February 2009AbstractPDF (177 KB)Minimax Methods for Multihypothesis Sequential Testing and Change-Point Detection ProblemsSequential Analysis, Vol. 27, No. 2 | 13 May 2008 Cross Ref Minimax Sequential Tests for Many Composite Hypotheses. IB. E. Brodsky and B. S. DarkhovskyTheory of Probability & Its Applications, Vol. 52, No. 4 | 19 November 2008AbstractPDF (193 KB)Discussion on “Likelihood Ratio Identities and Their Applications to Sequential Analysis” by Tze L. LaiSequential Analysis, Vol. 23, No. 4 | 31 Dec 2004 Cross Ref On the Invariance of SomeMartingale Inequalities forthe Processes of More General KindI. V. PavlovTheory of Probability & Its Applications, Vol. 41, No. 2 | 25 July 2006AbstractPDF (470 KB) Volume 32, Issue 1| 1988Theory of Probability & Its Applications1-197 History Submitted:28 September 1984Published online:17 July 2006 InformationCopyright © Society for Industrial and Applied MathematicsPDF Download Article & Publication DataArticle DOI:10.1137/1132017Article page range:pp. 138-142ISSN (print):0040-585XISSN (online):1095-7219Publisher:Society for Industrial and Applied Mathematics

In a sequential hypothesis test, the analyst checks at multiple steps during data collection whether sufficient evidence has accrued to make a decision about the tested hypotheses. As soon as sufficient information has been obtained, data collection is terminated. Here, we compare two sequential hypothesis testing procedures that have recently been proposed for use in psychological research: Sequential Probability Ratio Test (SPRT; Psychological Methods, 25(2), 206–226, 2020) and the Sequential Bayes Factor Test (SBFT; Psychological Methods, 22(2), 322–339, 2017). We show that although the two methods have different philosophical roots, they share many similarities and can even be mathematically regarded as two instances of an overarching hypothesis testing framework. We demonstrate that the two methods use the same mechanisms for evidence monitoring and error control, and that differences in efficiency between the methods depend on the exact specification of the statistical models involved, as well as on the population truth. Our simulations indicate that when deciding on a sequential design within a unified sequential testing framework, researchers need to balance the needs of test efficiency, robustness against model misspecification, and appropriate uncertainty quantification. We provide guidance for navigating these design decisions based on individual preferences and simulation-based design analyses.

We revisit the problem of sequential testing composite hypotheses, considering multiple hypotheses and very general non-i.i.d. stochastic models. Two sequential tests are studied: the multihypothesis generalized sequential likelihood ratio test and the multihypothesis adaptive sequential likelihood ratio test with one-stage delayed estimators. While the latter loses information compared to the former, it has an advantage in designing thresholds to guarantee given upper bounds for probabilities of errors, which is practically impossible for the generalized likelihood ratio type tests. It is shown that both tests have asymptotic optimality properties minimizing the expected sample size or even more generally higher moments of the stopping time as probabilities of errors vanish. Two examples that illustrate the general theory are presented.

Sequential sampling is a fast efficient tool for many sampling problems. Sequential sampling may be used (1) to obtain precise estimate(s) of the parameters), or (2) to test hypotheses concerning the parameters. Sequential estimation is used when the purpose of sampling is to obtain precise parameter estimates. Several sequential estimation procedures are discussed in Chapter 4. The focus of this chapter is sequential hypothesis testing. This approach is appropriate when we are interested in determining whether the population density is above or below a stated threshold. As in sequential estimation, sequential hypothesis testing requires taking observations sequentially until some stopping criterion is satisfied. The observations are taken at random over the sampling area. Generally, the accumulated total of the observations relative to the number of observations taken determines when sampling is stopped. The sequential hypothesis testing we consider requires some prior knowledge of the population distribution. This permits most computations to be completed in advance of sampling and to be stored in handheld calculators, laptop computers, or printed on cards or sheets. Wald’s sequential probability ratio test was the earliest sequential test and is described first. Lorden’s 2-SPRT is a more recent development that has some exciting possibilities for tests of hypotheses concerning population density and is discussed in the latter parts of this chapter.

This work considers the cooperative sequential hypothesis testing problem in a distributed network with quantized communication channels. The sensors observe independent sequences of samples and in the meantime, exchange their local information in the form of quantized statistics at every sampling interval. The communication links are represented as an undirected graph. In this distributed setup, every sensor performs its own sequential test based on the local samples and the messages from the neighbour sensors. Our goal is to devise the distributed sequential test that comprises the quantization scheme, the message-exchange protocol and the test procedure such that every sensor in the network fully exploits the network diversity and achieves the (asymptotically) optimal performance in terms of the stopping time. In particular, two distributed sequential tests are proposed based on different quantization schemes and a quantized message-exchange protocol that satisfies certain conditions. The first quantization scheme uniformly quantizes the local statistic at each sensor and at every sampling interval; the second one hinges on a modified level-triggered quantization technique, and resembles the Lebesgue sampling of the running local statistic. Our analyses show that the uniform quantization based distributed sequential test yields sub-optimal performance, while the one based on level-triggered quantization achieves the order-2 asymptotically optimal performance at every sensor for any fixed quantization step-size. Furthermore, we generalize the proposed sequential tests to the cluster-based network. Numerical results are provided to corroborate our analyses and demonstrate the effectiveness of the proposed sequential tests.

Conditional frequentist tests of a precise hypothesis versus a composite alternative have recently been developed, and have been shown to be equivalent to conventional Bayes tests in the very strong sense that the reported frequentist error probabilities equal the posterior probabilities of the hypotheses. These results are herein extended to sequential testing, and yield fully frequentist sequential tests that are considerably easier to use than are conventional sequential tests. Among the interesting properties of these new tests is the lack of dependence of the reported error probabilities on the stopping rule, seeming to lend frequentist support to the stopping rule principle. Keywords:Bayes factor; Composite hypothesis; Conditional test; Error probability; Likelihood ratio; Sequential probability ratio test; Sequential test.

We consider the classical sequential binary hypothesis testing problem in which there are two hypotheses governed respectively by distributions $P_{0}$ and $P_{1}$ and we would like to decide which hypothesis is true using a sequential test. It is known from the work of Wald and Wolfowitz that as the expectation of the length of the test grows, the optimal type-I and type-II error exponents approach the relative entropies $D(P_{1}\|P_{0})$ and $D(P_{0}\|P_{1})$ . We refine this result by considering the optimal backoff—or second-order asymptotics—from the corner point of the achievable exponent region $(D(P_{1}\|P_{0}),D(P_{0}\|P_{1}))$ under two different constraints on the length of the test (or the sample size). First, we consider a probabilistic constraint in which the probability that the length of test exceeds a prescribed integer $n$ is less than a certain threshold $0 . Second, the expectation of the sample size is bounded by $n$ . In both cases, and under mild conditions, the second-order asymptotics is characterized exactly. Numerical examples are provided to illustrate our results.

We consider the classical sequential binary hypothesis testing problem in which there are two hypotheses governed respectively by distributions $P_0$ and $P_1$ and we would like to decide which hypothesis is true using a sequential test. It is known from the work of Wald and Wolfowitz that as the expectation of the length of the test grows, the optimal type-I and type-II error exponents approach the relative entropies $D(P_1\|P_0)$ and $D(P_0\|P_1)$. We refine this result by considering the optimal backoff---or second-order asymptotics---from the corner point of the achievable exponent region $(D(P_1\|P_0),D(P_0\|P_1))$ under two different constraints on the length of the test (or the sample size). First, we consider a probabilistic constraint in which the probability that the length of test exceeds a prescribed integer $n$ is less than a certain threshold $0<\varepsilon <1$. Second, the expectation of the sample size is bounded by $n$. In both cases, and under mild conditions, the second-order asymptotics is characterized exactly. Numerical examples are provided to illustrate our results.

Under mild Markov assumptions, sufficient conditions for strict minimax optimality of sequential tests for multiple hypotheses under distributional uncertainty are derived. First, the design of optimal sequential tests for simple hypotheses is revisited, and it is shown that the partial derivatives of the corresponding cost function are closely related to the performance metrics of the underlying sequential test. Second, an implicit characterization of the least favorable distributions for a given testing policy is stated. By combining the results on optimal sequential tests and least favorable distributions, sufficient conditions for a sequential test to be minimax optimal under general distributional uncertainties are obtained. The cost function of the minimax optimal test is further identified as a generalized $f$-dissimilarity and the least favorable distributions as those that are most similar with respect to this dissimilarity. Numerical examples for minimax optimal sequential tests under different uncertainties illustrate the theoretical results.

In this paper Pitman's method of constructing and comparing tests based on statistics which are asymptotically normal under the null hypothesis and the local alternatives is extended to sequential tests of statistical hypotheses. The asymptotic normality assumption in Pitman's theory is replaced in its sequential analogue by the weak convergence of normalized processes formed from these statistics under the null hypothesis and the local alternatives. Uniform invariance principles are developed for a large class of statistics, and as an immediate corollary of these results, the desired weak convergence assumption is shown to hold. Furthermore uniform large deviation theorems are obtained for the test statistics and these results guarantee that the sequential tests under consideration have finite expected sample sizes under the null hypothesis and the local alternatives. As an illustration of the general method, the two-sample location problem is studied in detail, and the asymptotic relative efficiencies of the sequential Wilcoxon test, the sequential van der Waerden test and the sequential normal scores test relative to the two-sample sequential $t$-test are easily obtained since one of our key results (Theorem 1) implies that the asymptotic relative efficiencies of these sequential tests coincide with the corresponding Pitman efficiencies of their nonsequential analogues.

A pest management decision to initiate a control treatment depends upon an accurate estimate of mean pest density. Presence-absence sampling plans significantly reduce sampling efforts to make treatment decisions by using the proportion of infested leaves to estimate mean pest density in lieu of counting individual pests. The use of sequential hypothesis testing procedures can significantly reduce the number of samples required to make a treatment decision. Here we construct a mean-proportion relationship for Oligonychus perseae Tuttle, Baker, and Abatiello, a mite pest of avocados, from empirical data, and develop a sequential presence-absence sampling plan using Bartlett's sequential test procedure. Bartlett's test can accommodate pest population models that contain nuisance parameters that are not of primary interest. However, it requires that population measurements be independent, which may not be realistic because of spatial correlation of pest densities across trees within an orchard. We propose to mitigate the effect of spatial correlation in a sequential sampling procedure by using a tree-selection rule (i.e., maximin) that sequentially selects each newly sampled tree to be maximally spaced from all other previously sampled trees. Our proposed presence-absence sampling methodology applies Bartlett's test to a hypothesis test developed using an empirical mean-proportion relationship coupled with a spatial, statistical model of pest populations, with spatial correlation mitigated via the aforementioned tree-selection rule. We demonstrate the effectiveness of our proposed methodology over a range of parameter estimates appropriate for densities of O. perseae that would be observed in avocado orchards in California.

We consider the problem of sensor selection for time-optimal detection of a hypothesis. We consider a group of sensors transmitting their observations to a fusion center. The fusion center considers the output of only one randomly chosen sensor at the time, and performs a sequential hypothesis test. We study a sequential multiple hypothesis test with randomized sensor selection strategy. We incorporate the random processing times of the sensors to determine the asymptotic performance characteristics of this test. For three distinct performance metrics, we show that, for a generic set of sensors and binary hypothesis, the time-optimal policy requires the fusion center to consider at most two sensors. We also show that for the case of multiple hypothesis, the time-optimal policy needs at most as many sensors to be observed as the number of underlying hypotheses.

The problem of sequential test for many simple hypotheses on parameters of time series with trend is considered. Two approaches, including M-ary sequential probability ratio test and matrix sequential probability ratio test are used for constructing the sequential test. The sufficient conditions of finite terminations of the test and the existence of finite moments of their stopping times are given. The upper bounds for the average numbers of observations are obtained. With the thresholds chosen suitably, these tests can belong to some specified classes of statistical tests. Numerical examples are presented.

Discussion of the Paper by Jennison and Turnbull