Unequal Probability Sampling Research Articles

Fitting a distribution to censored contamination data using Markov Chain Monte Carlo methods and samples selected with unequal probabilities.

The fitting of statistical distributions to chemical and microbial contamination data is a common application in risk assessment. These distributions are used to make inferences regarding even the most pedestrian of statistics, such as the population mean. The reason for the heavy reliance on a fitted distribution is the presence of left-, right-, and interval-censored observations in the data sets, with censored observations being the result of nondetects in an assay, the use of screening tests, and other practical limitations. Considerable effort has been expended to develop statistical distributions and fitting techniques for a wide variety of applications. Of the various fitting methods, Markov Chain Monte Carlo methods are common. An underlying assumption for many of the proposed Markov Chain Monte Carlo methods is that the data represent independent and identically distributed (iid) observations from an assumed distribution. This condition is satisfied when samples are collected using a simple random sampling design. Unfortunately, samples of food commodities are generally not collected in accordance with a strict probability design. Nevertheless, pseudosystematic sampling efforts (e.g., collection of a sample hourly or weekly) from a single location in the farm-to-table continuum are reasonable approximations of a simple random sample. The assumption that the data represent an iid sample from a single distribution is more difficult to defend if samples are collected at multiple locations in the farm-to-table continuum or risk-based sampling methods are employed to preferentially select samples that are more likely to be contaminated. This paper develops a weighted bootstrap estimation framework that is appropriate for fitting a distribution to microbiological samples that are collected with unequal probabilities of selection. An example based on microbial data, derived by the Most Probable Number technique, demonstrates the method and highlights the magnitude of biases in an estimator that ignores the effects of an unequal probability sample design.

Estimating population mean with missing data in unequal probability sampling

Nonresponse problem is a serious obstacle to the validity of estimates in a survey. The estimates become biased due to the missing values in data. The problem is how to deal with missing values, once they have been deemed impossible to recover. One way of exploring a possible lack of representativity in missing data is to estimate the response probabilities which are usually done by logistic regression model. However, the drawback of the logit model is that this requires values of the explanatory variables of the model to be known for all nonrespondents. Bethlehem (2012) showed that the response probabilities can be estimated by some weighting adjustment technique without having the individual data of the nonrespondents. Here we consider the doubtful nature of nonresponse regarding possible existence of relationship with any of the covariates. Moreover, instead of simple random sampling, we consider general unequal probability sampling scheme for selecting respondents. This paper presents the modification of Bethlehem (2012) proposal for unequal probability sampling to obtain the unbiased estimators for population total/average of a variable of interest and variance estimator and compares them with the usual estimators through numerical simulations.

Estimation of the proportion of a sensitive attribute based on a two-stage randomized response model with stratified unequal probability sampling

To estimate the proportion of a sensitive attribute of the population that is composed of the number of different sized clusters, we suggest a two-stage randomized response model with unequal probability sampling by using Abdelfatah et al.’s procedure [Braz. J. Probab. Stat. 27 (2013) 608–617]. We compute the estimate of the sensitive parameter, its variance, and the variance estimator for both pps sampling and two-stage equal probability sampling. We extend our model to the case of stratified unequal probability sampling and compute them. Finally, we compare the efficiency of the two estimators, one obtained by unequal probability sampling and the other by stratified unequal probability sampling.

Applying the Nonrandomized Diagonal Model to Estimate a Sensitive Distribution in Complex Sample Surveys

In surveys with a sensitive characteristic X, such as income or tax evasion, direct questioning causes answer refusal and untruthful answers. To increase the respondents’ cooperation, nonrandomized response survey techniques are currently emerging. In this field of research, the diagonal model (DM) survey technique was recently published to gather data on multicategorical X. However, the estimation of the distribution of X from DM data is so far only derived for simple random sampling with replacement. To overcome this limitation, we develop in this article DM estimates for complex sampling designs that are often applied in practice, including stratified, cluster, multistage, and unequal probability sampling. Here, we apply quadratic programming to obtain admissible estimates in the unit simplex for the probability masses of X. Bootstrap variance estimates for the admissible estimators are described, and a method to investigate the connection between estimation efficiency in complex sample surveys and the...

Estimation of Population Mean Under Unequal Probability Sampling with Unknown Selection Probabilities

In many practical problems of biology, medicine, and social sciences, inference about population mean has to be made froma sample generated with unknown selection probabilities that may vary between the units in the population. The most common approach to analysis of such samples is to use the ordinary sample mean as an estimator of population mean, under the assumption that the sample is “representative” of the population, although the concept of representative sampling is not given a formal mathematical definition. The main objective of this paper is to identify a necessary and sufficient condition for unbiasedness of the ordinary sample mean under unequal probability sampling. It is shown that in the process of sampling with unequal selection probabilities with a fixed sample size, expectation of the ordinary sample mean is exactly equal to population mean plus the covariance of the unit-level outcome with the unit-level relative selection probability, where the relative selection probability is defined as the ratio of the unit’s selection probability to the mean selection probability in the population. Hence, ordinary sample mean is unbiased if and only if the unit-level outcome is uncorrelated with the unit-level relative selection probability. Samples generated under this condition may be considered “representative” of the population for the purpose of inference about population mean. It is also shown that under representative sampling, asymptotically conservative confidence intervals for population mean can be constructed based on the ordinary sample variance estimator as long as there are no positive correlations among the sample observations (negative correlations are allowed) and conditions for asymptotic normality of the sample mean are satisfied.

Variance Estimation and Asymptotic Confidence Bands for the Mean Estimator of Sampled Functional Data with High Entropy Unequal Probability Sampling Designs

ABSTRACTFor fixed size sampling designs with high entropy, it is well known that the variance of the Horvitz–Thompson estimator can be approximated by the Hájek formula. The interest of this asymptotic variance approximation is that it only involves the first order inclusion probabilities of the statistical units. We extend this variance formula when the variable under study is functional, and we prove, under general conditions on the regularity of the individual trajectories and the sampling design, that we can get a uniformly convergent estimator of the variance function of the Horvitz–Thompson estimator of the mean function. Rates of convergence to the true variance function are given for the rejective sampling. We deduce, under conditions on the entropy of the sampling design, that it is possible to build confidence bands whose coverage is asymptotically the desired one via simulation of Gaussian processes with variance function given by the Hájek formula. Finally, the accuracy of the proposed variance estimator is evaluated on samples of electricity consumption data measured every half an hour over a period of 1 week.

Replication variance estimation in unequal probability sampling without replacement: One‐stage and two‐stage

AbstractReplication‐based variance estimation methods including the bootstrap, balanced repeated replication, and the Jackknife have been studied extensively. They have been applicable primarily to stratified multistage sampling designs in which the clusters within strata are sampled with replacement or the first‐stage sampling fraction is negligible with a notable exception of a two‐stage cluster sampling with equal probability and without replacement in Rao & Wu (1988). It is common practice, however, that the first‐stage sampling fraction may not be negligible, resulting in overestimation. To alleviate this practical issue, we derive the balanced repeated replication methods and the bootstrap methods for one‐ and two‐stage stratified unequal probability sampling, where the sampling fractions are not negligible. The asymptotic property of the proposed methods is studied. In addition, the methodologies are applied to a simulated population with characteristics of a real sample survey. The Canadian Journal of Statistics 41: 696‐716; 2013 © 2013 Statistical Society of Canada

Complex national sampling design for long-term monitoring of protected dry grasslands in Switzerland

We describe a probabilistic sampling design of circular permanent plots for the long-term monitoring of protected dry grasslands in Switzerland. The population under study is defined by the perimeter of a national inventory. The monitoring focus is on the species composition of the protected grassland vegetation and derived conservation values. Efficient trend estimations are required for the whole country and for some predefined target groups (six biogeographical regions and eleven vegetation types). The target groups are equally important regardless of their size. Consequently, intensified sampling of the less frequent groups is essential for sample efficiency. The prior information needed to draw a targeted sample is obtained from the sampling frame and external databases. The logistics and generalized delineation of the target population may pose further problems. Thus, investments in fieldwork and travel time should be well balanced by selecting a cluster sample. Second, any access problems in the field and non-target units in the sample should be compensated for by selecting reserve plots as they otherwise may considerably reduce the effective sample size. Finally, the design has to be flexible as the sampling frame may change over time and sampling intensity might have to be adjusted to redefined budgets or requirements. Likewise, the variables and biological items of interest may change. To fulfil all these constraints and to optimally use the available prior information, we propose a multi-stage self-weighted unequal probability sampling design. The design uses modern techniques such as: balanced sampling, spreading, stratified balancing, calibration, unequal probability sampling and power allocation. This sampling design meets the numerous requirements of this study and provides a very efficient estimator.

A new replicate variance estimator for unequal probability sampling without replacement

AbstractWe propose a new replicate variance estimator suitable for differentiable functions of estimated totals. The proposed variance estimator is defined for any unequal‐probability without‐replacement sampling design, it naturally includes finite population corrections and it allows two‐stage sampling. We show its design‐consistency and its close relationship with linearization variance estimators. When estimating a total, the proposed estimator reduces to the Horvitz–Thompson variance estimator. Simulations suggest that the proposed variance estimator is more stable than its replicate competitors. The Canadian Journal of Statistics 41: 508–524; 2013 © 2013 Statistical Society of Canada

Fitting distributions to microbial contamination data collected with an unequal probability sampling design

The fitting of statistical distributions to microbial sampling data is a common application in quantitative microbiology and risk assessment applications. An underlying assumption of most fitting techniques is that data are collected with simple random sampling, which is often times not the case. This study develops a weighted maximum likelihood estimation framework that is appropriate for microbiological samples that are collected with unequal probabilities of selection. A weighted maximum likelihood estimation framework is proposed for microbiological samples that are collected with unequal probabilities of selection. Two examples, based on the collection of food samples during processing, are provided to demonstrate the method and highlight the magnitude of biases in the maximum likelihood estimator when data are inappropriately treated as a simple random sample. Failure to properly weight samples to account for how data are collected can introduce substantial biases into inferences drawn from the data. The proposed methodology will reduce or eliminate an important source of bias in inferences drawn from the analysis of microbial data. This will also make comparisons between studies and the combination of results from different studies more reliable, which is important for risk assessment applications.

Adaptive survey designs for sampling rare and clustered populations

Designing an efficient large-area survey is a challenge, especially in environmental science when many populations are rare and clustered. Adaptive and unequal probability sampling designs are appealing when populations are rare and clustered because survey effort can be targeted to subareas of high interest. For example, higher density subareas are usually of more interest than lower density areas. Adaptive and unequal probability sampling offer flexibility for designing a long-term survey because they can accommodate changes in survey objectives, changes in underlying environmental habitat, and changes in species-habitat models. There are many different adaptive sampling designs including adaptive cluster sampling, two-phase stratified sampling, two-stage sequential sampling, and complete allocation stratified sampling. Sample efficiency of these designs can be very high compared with simple random sampling. Large gains in efficiency can be made when survey effort is targeted to the subareas of the study site where there are clusters of individuals from the underlying population. These survey methods work by partitioning the study area in some way, into strata, or primary sample units, or in the case of adaptive cluster sampling, into networks. Survey effort is then adaptively allocated to the strata or primary unit where there is some indication of higher species counts. Having smaller, and more numerous, strata improves efficiency because it allows more effective targeting of the adaptive, second-phase survey effort.

META-ANALYSIS OF CHOICE SET GENERATION EFFECTS ON ROUTE CHOICE MODEL ESTIMATES AND PREDICTIONS

Large scale applications of behaviorally realistic transport models pose several challenges to transport modelers on both the demand and the supply sides. On the supply side, path-based solutions to the user assignment equilibrium problem help modelers in enhancing the route choice behavior modeling, but require them to generate choice sets by selecting a path generation technique and its parameters according to personal judgments. This paper proposes a methodology and an experimental setting to provide general indications about objective judgments for an effective route choice set generation. Initially, path generation techniques are implemented within a synthetic network to generate possible subjective choice sets considered by travelers. Next, ‘true model estimates’ and ‘postulated predicted routes’ are assumed from the simulation of a route choice model. Then, objective choice sets are applied for model estimation and results are compared to the ‘true model estimates’. Last, predictions from the simulation of models estimated with objective choice sets are compared to the ‘postulated predicted routes’. A meta-analytical approach allows synthesizing the effect of judgments for the implementation of path generation techniques, since a large number of models generate a large amount of results that are otherwise difficult to summarize and to process. Meta-analysis estimates suggest that transport modelers should implement stochastic path generation techniques with average variance of its distribution parameters and correction for unequal sampling probabilities of the alternative routes in order to obtain satisfactory results in terms of coverage of ‘postulated chosen routes’, reproduction of ‘true model estimates’ and prediction of ‘postulated predicted routes’.

Weighting in the regression analysis of survey data with a cross‐national application

AbstractA class of survey weighting methods provides consistent estimation of regression coefficients under unequal probability sampling. The minimization of the variance of the estimated coefficients within this class is considered. A series of approximations leads to a simple modification of the usual design weight. One type of application where unequal probabilities of selection arise is in cross‐national comparative surveys. In this case, our argument suggests the use of a certain kind of within‐country weight. We investigate this idea in an application to data from the European Social Survey, where we fit a logistic regression model with vote in an election as the dependent variable and with various variables of political science interest included as explanatory variables. We show that the use of the modified weights leads to a considerable reduction in standard errors compared to design weighting. The Canadian Journal of Statistics 40: 697–711; 2012 © 2012 Statistical Society of Canada

Abstract P010: Dried Capillary Whole Blood Spot-Based Hemoglobin A 1c , Fasting Glucose, and Diabetes Prevalence in a Nationally Representative Population of Young U.S. Adults: Add Health, Wave IV

Little is known about the biomarker-based prevalence of diabetes among U.S. adults aged 24-32 years, an age group historically characterized by low cardiovascular disease risk. We addressed the paucity of information within this age group among 15,701 participants at Wave IV of the National Longitudinal Study of Adolescent Health (Add Health, 2008), a study including nationally representative oversamples of racial / ethnic groups underrepresented by the National Health and Nutrition Examination Survey (NHANES). Capillary whole blood was collected via finger prick onto Whatman 903® Protein Saver cards by trained and certified field interviewers, desiccated, then shipped to central laboratories for assay and archival. Sensitivity of the glucose assay was 22 mg/dl. Assayed values in the lowest half percentile of the distribution were re-assayed. Re-assayed and original values were averaged. The within- and between-assay coefficients of variation (CVs) were 4.4% and 4.8%. For HbA 1c , the corresponding sensitivity, within- and between-assay CVs were 3%, 2.2%, and 2.4%. In paired serum and blood spots, glucose concentrations (mg/dl) were strongly associated (n = 83; Pearson r = 0.97). Associations were equally strong for HbA 1c (%) in paired whole blood and blood spots (n = 80; Pearson r = 0.99). In a race/ethnicity- and sex-stratified random sample of 100 Add Health participants among whom capillary whole blood was collected twice, one to two weeks apart, reliability of random (fasting ≥ 8 hr or non-fasting) glucose and HbA 1c was estimated as an intra-class correlation coefficient and 95% confidence interval, ICC (95% CI): 0.39 (0.21, 0.58) and 0.97 (0.96-0.98). Add Health participants were more likely than similarly aged NHANES (2007-2008) participants to be native-born, insured, college educated, and overweight or obese. After weighting for unequal sampling probabilities and clustering, mean (standard deviation) HbA 1c and fasting glucose were higher in Add Health than NHANES: 5.6% (0.8%) and 107 (35) mg/dl vs. 5.2% (0.5%) and 97 (14) mg/dl. The weighted prevalence (95% CI) of HbA 1c ≥ 6.5% and fasting glucose ≥ 126 mg/dl also were higher in Add Health than NHANES: 3.6% (2.9-4.3) and 10.3% (8.7%-12.2%) vs. 1.7% (0.9%-3.2%) and 2.1% (0.8%-5.5%). Corresponding odds ratios (95% CIs) were: 2.1 (1.1-3.9) and 5.2 (2.1-13.3). Adjustment for sociodemographic, clinical and behavioral risk factors attenuated the associations: 1.5 (0.8-3.1) and 4.2 (1.7-10.4). However, the addition of self-reported history of diabetes and use of anti-diabetics had relatively little effect on them. Carefully standardized, in-home collection of whole blood spots can yield valid and reliable estimates of glucose and HbA 1c . Their interpretation in context of the prevalent obesity and hypertension at Add Health Wave IV reinforces suggestions that young, U.S. adults face a historically high risk of cardiovascular disease.

On the inclusion probabilities in some unequal probability sampling plans without replacement

Comparison results are obtained for the inclusion probabilities in some unequal probability sampling plans without replacement. For either successive sampling or H\'{a}jek's rejective sampling, the larger the sample size, the more uniform the inclusion probabilities in the sense of majorization. In particular, the inclusion probabilities are more uniform than the drawing probabilities. For the same sample size, and given the same set of drawing probabilities, the inclusion probabilities are more uniform for rejective sampling than for successive sampling. This last result confirms a conjecture of H\'{a}jek (Sampling from a Finite Population (1981) Dekker). Results are also presented in terms of the Kullback--Leibler divergence, showing that the inclusion probabilities for successive sampling are more proportional to the drawing probabilities.

Size constrained unequal probability sampling with a non-integer sum of inclusion probabilities

More than 50 methods have been developed to draw unequal probability samples with fixed sample size. All these methods require the sum of the inclusion probabilities to be an integer number. There are cases, however, where the sum of desired inclusion probabilities is not an integer. Then, classical algorithms for drawing samples cannot be directly applied. We present two methods to overcome the problem of sample selection with unequal inclusion probabilities when their sum is not an integer and the sample size cannot be fixed. The first one consists in splitting the inclusion probability vector. The second method is based on extending the population with a phantom unit. For both methods the sample size is almost fixed, and equal to the integer part of the sum of the inclusion probabilities or this integer plus one.

A Gaussian copula approach for the analysis of secondary phenotypes in case-control genetic association studies

In many case-control genetic association studies, a set of correlated secondary phenotypes that may share common genetic factors with disease status are collected. Examination of these secondary phenotypes can yield valuable insights about the disease etiology and supplement the main studies. However, due to unequal sampling probabilities between cases and controls, standard regression analysis that assesses the effect of SNPs (single nucleotide polymorphisms) on secondary phenotypes using cases only, controls only, or combined samples of cases and controls can yield inflated type I error rates when the test SNP is associated with the disease. To solve this issue, we propose a Gaussian copula-based approach that efficiently models the dependence between disease status and secondary phenotypes. Through simulations, we show that our method yields correct type I error rates for the analysis of secondary phenotypes under a wide range of situations. To illustrate the effectiveness of our method in the analysis of real data, we applied our method to a genome-wide association study on high-density lipoprotein cholesterol (HDL-C), where "cases" are defined as individuals with extremely high HDL-C level and "controls" are defined as those with low HDL-C level. We treated 4 quantitative traits with varying degrees of correlation with HDL-C as secondary phenotypes and tested for association with SNPs in LIPG, a gene that is well known to be associated with HDL-C. We show that when the correlation between the primary and secondary phenotypes is >0.2, the P values from case-control combined unadjusted analysis are much more significant than methods that aim to correct for ascertainment bias. Our results suggest that to avoid false-positive associations, it is important to appropriately model secondary phenotypes in case-control genetic association studies.

Spatially correlated Poisson sampling

A new method for sampling from a finite population that is spread in one, two or more dimensions is presented. Weights are used to create strong negative correlations between the inclusion indicators of nearby units. The method can be used to produce unequal probability samples that are well spread over the population in every dimension, without any spatial stratification. Since the method is very general there are numerous possible applications, especially in sampling of natural resources where spatially balanced sampling has proven to be efficient. Two examples show that the method gives better estimates than other commonly used designs.

A Direct Bootstrap Method for Complex Sampling Designs From a Finite Population

In complex designs, classical bootstrap methods result in a biased variance estimator when the sampling design is not taken into account. Resampled units are usually rescaled or weighted in order to achieve unbiasedness in the linear case. In the present article, we propose novel resampling methods that may be directly applied to variance estimation. These methods consist of selecting subsamples under a completely different sampling scheme from that which generated the original sample, which is composed of several sampling designs. In particular, a portion of the subsampled units is selected without replacement, while another is selected with replacement, thereby adjusting for the finite population setting. We show that these bootstrap estimators directly and precisely reproduce unbiased estimators of the variance in the linear case in a time-efficient manner, and eliminate the need for classical adjustment methods such as rescaling, correction factors, or artificial populations. Moreover, we show via simulation studies that our method is at least as efficient as those currently existing, which call for additional adjustment. This methodology can be applied to classical sampling designs, including simple random sampling with and without replacement, Poisson sampling, and unequal probability sampling with and without replacement.

An Extension of Sampford's Method for Unequal Probability Sampling

Abstract. Sampford's unequal probability sampling method is extended to the case that the inclusion probabilities do not sum to an integer. In this case, the sampling outcome is left open for exactly one randomly chosen unit and that unit gets a new inclusion probability. Three applications are presented. Two of them challenge traditional sampling routines. The simple Pareto sampling design, which was introduced by Rosén in 1997, is also extended. The extended Pareto design is shown to be close to the extended Sampford design.