Horvitz-Thompson Estimator Research Articles

Inference for Two-Stage Sampling Designs

SummaryTwo-stage sampling designs are commonly used for household and health surveys. To produce reliable estimators with associated confidence intervals, some basic statistical properties like consistency and asymptotic normality of the Horvitz–Thompson estimator are desirable, along with the consistency of associated variance estimators. These properties have been mainly studied for single-stage sampling designs. In this work, we prove the consistency of the Horvitz–Thompson estimator and of associated variance estimators for a general class of two-stage sampling designs, under mild assumptions. We also study two-stage sampling with a large entropy sampling design at the first stage and prove that the Horvitz–Thompson estimator is asymptotically normally distributed through a coupling argument. When the first-stage sampling fraction is negligible, simplified variance estimators which do not require estimating the variance within the primary sampling units are proposed and shown to be consistent. An application to a panel for urban policy, which is the initial motivation for this work, is also presented.

Sampling a two dimensional matrix

A new sampling design for populations whose units can be arranged as an N×M matrix is proposed. The sample must satisfy some constraints: row and column sample sizes are set in advance. The proposed sampling method gives the same selection probability to all the sample matrices that satisfy the constraints. Three algorithms to select a sample uniformly in the feasible set are presented: an exact algorithm based on the multivariate hypergeometric distribution, an MCMC algorithm, and the cube method. Their performances are evaluated using Monte Carlo simulations. The designs for sampling elements in a given row or a given column are investigated and the single inclusion and joint selection probabilities under the proposed design are evaluated. Several variance estimators are proposed for the Horvitz–Thompson estimator of the population mean of the survey variable y and their performances are compared in a Monte Carlo study. A numerical example dealing with a creel survey of fishermen found at 9 sites over 36 days is presented.

Horvitz-Thompson estimator based on theauxiliary variable

Abstract In this paper, the Horvitz and Thompson (1952) estimator will be modified; so that, the modified estimators will use the availability of the auxiliary variable. Furthermore, the modified estimators are extended to be used in stratified sampling designs. Empirical studies are given for comparison purposes.

Causal Inference in Matching Markets: Simulable Mechanisms

We formalize an econometric model for two-sided matching mechanisms in a school choice context, where exogenous variation is generated by using lotteries as a tie-breaking mechanism. Our model accommodates a wide range of matching algorithms studied in the theoretical market design literature. We propose a Horvitz–Thompson estimator for the average treatment effect that is exactly unbiased, compatible with multiple treatments, and compatible with heterogeneous treatment effects. We present theoretical properties of the estimator and inference procedures. Our work clarifies the econometric model used in Abdulkadiroglu et al. (2017) and provides a robustness check on their results.

Consistency of the Horvitz–Thompson estimator under general sampling and experimental designs

We extend current concentration results for the Horvitz–Thompson estimator in finite population settings. The estimator is demonstrated to converge in quadratic mean to its target under weaker and more general conditions than previously known. Specifically, we do not require that the variables of interest nor the normalized inclusion probabilities are bounded. Rates of convergence are provided.

Generalized Regression Estimators with High-Dimensional Covariates.

Data from a large number of covariates with known population totals are frequently observed in survey studies. These auxiliary variables contain valuable information that can be incorporated into estimation of the population total of a survey variable to improve the estimation precision. We consider the generalized regression estimator formulated under the model-assisted framework in which a regression model is utilized to make use of the available covariates while the estimator still has basic design-based properties. The generalized regression estimator has been shown to improve the efficiency of the design-based Horvitz-Thompson estimator when the number of covariates is fixed. In this study, we investigate the performance of the generalized regression estimator when the number of covariates p is allowed to diverge as the sample size n increases. We examine two approaches where the model parameter is estimated using the weighted least squares method when p < n and the LASSO method when the model parameter is sparse. We show that under an assisted model and certain conditions on the joint distribution of the covariates as well as the divergence rates of n and p, the generalized regression estimator is asymptotically more efficient than the Horvitz-Thompson estimator, and is robust against model misspecification. We also study the consistency of variance estimation for the generalized regression estimator. Our theoretical results are corroborated by simulation studies and an example.

Large-scale two-phase estimation of wood production by poplar plantations exploiting Sentinel-2 data as auxiliary information

Growing demand for wood products, combined with efforts to conserve natural forests, have supported a steady increase in the global extent of planted forests. Here, a two-phase sampling strategy for large-scale assessment of the total area and the total wood volume of fast-growing forest tree crops within agricultural land is presented. The first phase is performed using tessellation stratified sampling on high-resolution remotely sensed imagery and is sufficient for estimating the total area of plantations by means of a Monte Carlo integration estimator. The second phase is performed using stratified sampling of the plantations selected in the first phase and is aimed at estimating total wood volume by means of an approximation of the first-phase Horvitz-Thompson estimator. Vegetation indices from Sentinel-2 are exploited as freely available auxiliary information in a linear regression estimator to improve the design-based precision of the estimator based on the sole sample data. Estimators of the totals and of the design-based variances of total estimators are presented. A simulation study is developed in order to check the design-based performance of the two alternative estimators under several artificial distributions supposed for poplar plantations (random, clustered, spatially trended). An application in Northern Italy is also reported. The regression estimator turns out to be invariably better than that based on the sole sample information. Possible integrations of the proposed sampling scheme with conventional national forest inventories adopting tessellation stratified sampling in the first phase are discussed.

Non-parametric Estimator for a Finite Population Total Based on Edgeworth Expansion

In survey sampling, the main objective is to make inference about the entire population parameters using the sample statistics. In this study, a nonparametric estimator of finite population total is proposed and the coverage probabilities using the Edgeworth expansion explored. Three properties; unbiasedness, efficiency and the confidence interval of the proposed estimator are studied. There is a lot of literature on study of two properties; unbiasedness and efficiency of the finite population total. This study therefore has more focus on confidence interval and coverage probability. The amount of bias and MSE are studied partially analytically, followed by an empirical study on the two properties and the confidence interval of the proposed estimator. Based on the empirical study with simulations in R, the proposed estimator resulted into smaller bias and MSE compared to the nonparametric estimator due to [6], the design-based Horvitz-Thompson estimator and the model-based ratio estimator. Further, the proposed estimator is tighter compared to the other three considered in this study and has higher converging coverage probabilities.

Heterogeneous Treatment and Spillover Effects Under Clustered Network Interference

The bulk of causal inference studies rules out the presence of interference between units. However, in many real-world settings units are interconnected by social, physical or virtual ties and the effect of a treatment can spill from one unit to other connected individuals in the network. In these settings, interference should be taken into account to avoid biased estimates of the treatment effect, but it can also be leveraged to save resources and provide the intervention to a lower percentage of the population where the treatment is more effective and where the effect can spill over to other susceptible individuals. In fact, different people might respond differently not only to the treatment received but also to the treatment received by their network contacts. Understanding the heterogeneity of treatment and spillover effects can help policy-makers in the scale-up phase of the intervention, it can guide the design of targeting strategies with the ultimate goal of making the interventions more cost-effective, and it might even allow generalizing the level of treatment spillover effects in other populations. In this paper, we develop a machine learning method that makes use of tree-based algorithms and an Horvitz-Thompson estimator to assess the heterogeneity of treatment and spillover effects with respect to individual, neighborhood and network characteristics in the context of clustered network interference. We illustrate how the proposed binary tree methodology performs in a Monte Carlo simulation study. Additionally, we provide an application on a randomized experiment aimed at assessing the heterogeneous effects of information sessions on the uptake of a new weather insurance policy in rural China.

Exploring Data-Reflection Technique in Nonparametric Regression Estimation of Finite Population Total: An Empirical Study

In survey sampling statisticians often make estimation of population parameters. This can be done using a number of the available approaches which include design-based, model-based, model-assisted or randomization-assisted model based approach. In this paper regression estimation under model based approach has been studied. In regression estimation, researchers can opt to use parametric or nonparametric estimation technique. Because of the challenges that one can encounter as a result of model misspecification in the parametric type of regression, the nonparametric regression has become popular especially in the recent past. This paper explores this type of regression estimation. Kernel estimation usually forms an integral part in this type of regression. There are a number of functions available for such a use. The goal of this study is to compare the performance of the different nonparametric regression estimators (the finite population total estimator due Dorfman (1992), the proposed finite population total estimator that incorporates reflection technique in modifying the kernel smoother), the ratio estimator and the design-based Horvitz-Thompson estimator. To achieve this, data was simulated using a number of commonly used models. From this data the assessment of the estimators mentioned above has been done using the conditional biases. Confidence intervals have also been constructed with a view to determining the better estimator of those studied. The findings indicate that proposed estimator of finite population total that is nonparametric and uses data reflection technique is better in the context of the analysis done.

Investigating the Performance of Horvitz-Thompson and Generalized Regression Estimators in Small Domains with Application to Income and Expenditure of Households in Makurdi-Nigeria

In this study, efficiencies of Horvitz-Thompson (HT) and Generalized Regression (GREG) estimatorsare investigated in small domains namely Domain 1, 2 and 3. Efficiency criteria include the standarderror of the estimates (SE) and coefficient of variation (CV) Computed for the study population ofsize 1052 households consisting of domain size of 885, 151 and 16 and domain samples sizes of 167,29 and 3 respectively for Domains 1, 2 and 3. The estimates produced by HT estimator GREGestimator performs better than HT estimator with domain

Design-based consistency of the Horvitz–Thompson estimator under spatial sampling with applications to environmental surveys

Spatial populations are usually located on a continuous support. They can be surfaces representing the values of the survey variable at any location, finite collections of units with the corresponding values of the survey variable, or finite collections of areal units partitioning the support, where the survey variable is the total amount of an attribute within. We derive conditions on the design sequence ensuring consistency of the Horvitz–Thompson estimator of spatial population totals, supposing minimal requirements on the survey variable. A simulation study is performed to check theoretical results. Consistency and its implications in real surveys are discussed with focus on environmental surveys.

A computationally efficient method for selecting a split questionnaire design

Split questionnaire design (SQD) is a relatively new survey tool to reduce response burden and increase quality of responses. Among a set of possible SQD choices, a design is considered the best if it leads to the least amount of information loss quantified by the Kullback-Leibler divergence (KLD) distance. The calculation of this distance requires computation of the distribution function for the observed data after integrating out all the missing variables in a particular SQD. For a typical survey questionnaire with a large number of categorical variables, this computation can become practically infeasible. Motivated by the Horvitz-Thompson estimator, we propose to approximate the distribution function of the observed data in much reduced computation time and lose little information when comparing different choices of SQDs. We construct a thorough simulation study to test if the proposed approximation method can correctly identify the best SQD under several simulation scenarios created to cover different distribution shapes of continuous variables, and different correlation structures in the variables. Finally, the proposed approach is applied to the 2012 Pet Demographic Survey data. Both of the simulation studies and the empirical study demonstrate that the proposed method is computationally efficient and can accurately select the best SQD design.

P.065 The dual-hit model of schizophrenia-like behavior in the rat: effects of antipsychotics on locomotor hyperactivity and social recognition deficits

Mark variograms are widely adopted in forest ecosystem studies for analyzing spatial interaction among trees. Inference on mark variograms can be performed under a model-dependent perspective (ergodic variograms) or under a deterministic perspective (non-ergodic variograms). A simple and workable definition of non-ergodic mark variogram is introduced based on distances between pairs of trees distinguished by distance classes. Design-based estimators of mark variogram values are proposed as the ratio of two Horvitz–Thompson estimators using replicated plots randomly located on the study area and making use of the inclusion probabilities of the pairs of trees. Jackknife estimators of their variances are considered. A simulation study is performed on a real forest stand to check the statistical performance of the proposed strategy.

Normal and Bootstrap Confidence Intervals in Bitterlich Sampling

The Bitterlich Sampling (horizontal point sampling) is a common method in forest inventories. By this method, the Horvitz-Thompson estimator is used in a number of independent sampling points for the estimation of overall tree volume in a forest area/stand. In this paper, confidence intervals are constructed and evaluated using the normal approach and two bootstrap methods; the percentile method (Cα) and the bias-corrected and accelerated method (BCα). The simulation results show that the normal confidence interval has better coverage of true value at sample size 10. At sample sizes 20 and 30, it seems that there are no substantial differences in coverage between confidence intervals, although it could be noted a small superiority of BCα method. At sample size 40, the coverage of the three confidence intervals is higher than the nominal coverage (95%).

Finite Population Model-Assisted Estimation Using Combined Parametric and Nonparametric Regression Smoothers

This paper considers estimating finite population totals from complex sample surveys in the presence of auxiliary information. Model-assisted estimators which assume a working regression model relating the study variable with the auxiliary data are common in this context. Both parametric and nonparametric working models have been utilized individually in constructing several model-assisted estimators. Model-assisted estimators with parametric working models are known to be efficient when the assumed working model is correctly specified, while using nonparametric smoothers gives more robust estimates but requires relatively large sample sizes. In this paper, we consider the situation where the researcher has an idea of which parametric model can describe the relationship between the study variable and the auxiliary data, but this model may not be adequate in some areas of the data range. Using combined parametric and nonparametric regression smoothers for the working model, we introduce a new class of model-assisted estimators for finite population totals. The proposed estimators are shown to have the desirable asymptotic properties of traditional model-assisted estimators of population totals. The finite sample performance of the new estimators is studied via Monte Carlo simulations from both artificial and real populations. The empirical results suggest that our proposed estimators perform well relative to other model-based and model-assisted estimators as well as the customary Horvitz–Thompson estimator under different levels of misspecification in the working model. We also discuss the problem of variance estimation for the proposed estimators.

A COMPARISON OF NONPARAMETRIC ESTIMATORS FOR LENGTH DISTRIBUTION IN LINE SEGMENT PROCESSES

We study nonparametric estimation of the length distribution for stationary line segment processes in the d-dimensional Euclidean space. Several methods have been proposed in the literature. We review different approaches (Horvitz-Thompson type estimator, reduced-sample estimator, Kaplan-Meier estimator, nonparametric maximum likelihood estimator, stochastic restoration estimation) and compare the finite sample behaviour by means of a simulation study for stationary line segment processes in 2D and 3D. Several data generating processes (Poisson point process, Matérn cluster process and Matérn hard-core process II) are considered with both independent and dependent segments. Our finite sample comparison reveals that the nonparametric likelihood estimator provides the most preferable method which works reasonably also if its assumptions are not satisfied.

The one-inflated positive Poisson mixture model for use in population size estimation.

The one-inflated positive Poisson mixture model (OIPPMM) is presented, for use as the truncated count model in Horvitz-Thompson estimation of an unknown population size. The OIPPMM offers a way to address two important features of some capture-recapture data: one-inflation and unobserved heterogeneity. The OIPPMM provides markedly different results than some other popular estimators, and these other estimators can appear to be quite biased, or utterly fail due to the boundary problem, when the OIPPMM is the true data-generating process. In addition, the OIPPMM provides a solution to the boundary problem, by labelling any mixture components on the boundary instead asone-inflation.

APPLICATION OF THE STRATEGY COMBINING MONETARY UNIT SAMPLING AND THE HORVITZ– THOMPSON ESTIMATOR OF ERROR AMOUNT IN AUDITING – RESULTS OF A SIMULATION STUDY

Abstract Auditors need information on the performance of different statistical methods when applied to audit populations. The aim of the study was to examine the reliability and efficiency of a strategy combining systematic Monetary Unit Sampling and confidence intervals for the total error based on the Horvitz-Thompson estimator with normality assumption. This strategy is a possible alternative for testing audit populations with high error rates. Using real and simulated data sets, for the majority of populations, the interval coverage rate was lower than the assumed confidence level. In most cases confidence intervals were too wide to be of practical use to auditors. Confidence intervals tended to become wider as the observed error rate increased. Tests disclosed the distribution of the Horvitz-Thompson estimator was not normal. A detailed analysis of the distributions of the error amount in the examined real audit populations is also given.

The effect of spatial structure of forests on the precision and costs of plot-level forest resource estimation

BackgroundWe investigated how the precision and costs of forest resource estimates for sample plots of different type and size depend on the spatial structure of forests and jointly studied the effects of tree density and size distribution. Statistically thinking, the trees in a forest can be regarded as a point pattern. Based on the spatial properties of the point pattern, we classified the forests into clustered, random, and regular. We used empirical data from 396 mapped forest plots from Finland. The variance of the unbiased Horvitz-Thompson estimator and expected costs of the basal area and tree density estimation were calculated for 99 different sample plots of different type and size in each of the 396 forest plots. Further, we considered the estimation of the change between two time points for a subset of the data.ResultsThe precision and expected cost depended on the tree size distribution and spatial pattern of trees. While large sample plots are advisable for clustered forests or the monitoring of young forests with small trees, we see potential for measuring smaller sample plots in regular forests. The choice of sample plot was more important in clustered forests, where also the variability of the expected costs was higher.ConclusionsIf the spatial structure of forests could be predicted accurately and precisely prior to field measurements, for instance from remote sensing data, the precision of forest inventories could potentially be improved or costs decreased by allowing the sample plot size and type to vary from one forest stand to another. When using a compromise sample plot over a large region and a long inventory rotation, optimizing the sample plot for one time point ignores possible changes in forest structures caused by changes in forest management practices.