Presence Of Auxiliary Information Research Articles

Weighted Strategy for Estimation of Finite Population Mean in Presence of Auxiliary Information

ABSTRACT In sample designs, it is commonly recognized that using auxiliary information significantly increases an estimator’s precision. This manuscript introduces an weighted strategy for computing the finite population mean using auxiliary information in sample surveys. The equations for the mean squared error (MSE) of the proposed estimator are derived under large sampling approximation. A graphical representation of efficiency comparison is provided to show the ranges where the proposed estimator performs more efficiently. Additionally, a numerical analysis is conducted to evaluate the performance of the suggested estimator using actual datasets.

Optimized inferences of finite population mean using robust parameters in systematic sampling.

In this article, we have proposed a generalized estimator for mean estimation by combining the ratio and regression methods of estimation in the presence of auxiliary information using systematic sampling. We incorporated some robust parameters of the auxiliary variable to obtain precise estimates of the proposed estimator. The mathematical expressions for bias and mean square error of proposed the estimator are derived under large sample approximation. Many other generalized ratio and product-type estimators are obtained from the proposed estimator using different choices of scalar constants. Some special cases are also discussed in which the proposed generalized estimator reduces to the usual mean, classical ratio, product, and regression type estimators. Mathematical conditions are obtained for which the proposed estimator will perform more precisely than the challenging estimators mentioned in this article. The efficiency of the proposed estimator is evaluated using four populations. Results showed that the proposed estimator is efficient and useful for survey sampling in comparison to the other existing estimators.

Geostatistical semi-supervised learning for spatial prediction

Geoscientists are increasingly tasked with spatially predicting a target variable in the presence of auxiliary information using supervised machine learning algorithms. Typically, the target variable is observed at a few sampling locations due to the relatively time-consuming and costly process of obtaining measurements. In contrast, auxiliary variables are often exhaustively observed within the region under study through the increasing development of remote sensing platforms and sensor networks. Supervised machine learning methods do not fully leverage this large amount of auxiliary spatial data. Indeed, in these methods, the training dataset includes only labeled data locations (where both target and auxiliary variables were measured). At the same time, unlabeled data locations (where auxiliary variables were measured but not the target variable) are not considered during the model training phase. Consequently, only a limited amount of auxiliary spatial data is utilized during the model training stage. As an alternative to supervised learning, semi-supervised learning, which learns from labeled as well as unlabeled data, can be used to address this problem. However, conventional semi-supervised learning techniques do not account for the specificities of spatial data. This paper introduces a spatial semi-supervised learning framework where geostatistics and machine learning are combined to harness a large amount of unlabeled spatial data in combination with typically a smaller set of labeled spatial data. The main idea consists of leveraging the target variable’s spatial autocorrelation to generate pseudo labels at unlabeled data points that are geographically close to labeled data points. This is achieved through geostatistical conditional simulation, where an ensemble of pseudo labels is generated to account for the uncertainty in the pseudo labeling process. The observed labels are augmented by this ensemble of pseudo labels to create an ensemble of pseudo training datasets. A supervised machine learning model is then trained on each pseudo training dataset, followed by an aggregation of trained models. The proposed geostatistical semi-supervised learning method is applied to synthetic and real-world spatial datasets. Its predictive performance is compared with some classical supervised and semi-supervised machine learning methods. It appears that it can effectively leverage a large amount of unlabeled spatial data to improve the target variable’s spatial prediction.

Empirical Likelihood Ratio Tests for Homogeneity of Multiple Populations in the Presence of Auxiliary Information

The empirical likelihood ratio test (ELRT) statistic is constructed for testing the homogeneity of several nonparametric populations in the presence of some auxiliary information. It is shown—under some regularity conditions and under the null hypothesis that all distribution functions of the populations are equal—that the asymptotic distribution of the ELRT is a chi-squared distribution. The proposed ELRT could be more powerful than the Kruskal–Wallis test, as extra information can be efficiently employed by ELRT. The advantage of ELRT over T&P (2006) is that researchers do not need to select approximately normal statistics for inter-group comparisons, and ELRT is more suitable for the multi-population consistency test with a small sample size.

Bayesian Deep Learning for Spatial Interpolation in the Presence of Auxiliary Information

Earth scientists increasingly deal with ‘big data’. For spatial interpolation tasks, variants of kriging have long been regarded as the established geostatistical methods. However, kriging and its variants (such as regression kriging, in which auxiliary variables or derivatives of these are included as covariates) are relatively restrictive models and lack capabilities provided by deep neural networks. Principal among these is feature learning: the ability to learn filters to recognise task-relevant patterns in gridded data such as images. Here, we demonstrate the power of feature learning in a geostatistical context by showing how deep neural networks can automatically learn the complex high-order patterns by which point-sampled target variables relate to gridded auxiliary variables (such as those provided by remote sensing) and in doing so produce detailed maps. In order to cater for the needs of decision makers who require well-calibrated probabilities, we also demonstrate how both aleatoric and epistemic uncertainty can be quantified in our deep learning approach via a Bayesian approximation known as Monte Carlo dropout. In our example, we produce a national-scale probabilistic geochemical map from point-sampled observations with auxiliary data provided by a terrain elevation grid. By combining location information with automatically learned terrain derivatives, our deep learning approach achieves an excellent coefficient of determination (R^{2} = 0.74) and near-perfect probabilistic calibration on held-out test data. Our results indicate the suitability of Bayesian deep learning and its feature-learning capabilities for large-scale geostatistical applications where uncertainty matters.Graphic

Small area quantile estimation based on distribution function using linear mixed models

In economic studies researchers are often interested in the estimation of the distribution function or certain functions of the distribution function such as quantiles. This work focuses on the estimation quantiles as inverses of the estimates of the distribution function in the presence of auxiliary information that is correlated with the study variable. In the paper a plug-in estimator of the distribution function is proposed which is used to obtain quantiles in the population and in the small areas. Performance of the proposed method is compared with other estimators of the distribution function and quantiles using the simulation study. The obtained results show that the proposed method usually has smaller relative biases and relative RMSE comparing to other methods of obtaining quantiles based on inverting the distribution function.

NGO online disclosures index in the presence of auxiliary information.

This study highlights the need for analysis of online disclosure practices followed by non-governmental organizations; furthermore, it justifies the crucial role of potential correlates of online disclosure practices followed by non-governmental organizations. We propose a novel index for analyzing the extent of online disclosure of non-governmental organizations (NGO). Using the information stored in an auxiliary variable, we propose a new estimator for gauging the average value of the proposed index. Our approach relies on the use of two factors: imperfect ranked-set sampling procedure to link the auxiliary variable with the study variable, and an NGO disclosure index under simple random sampling that uses information only about the study variable. Relative efficiency of the proposed index is compared with the conventional estimator for the population average under the imperfect ranked-set sampling scheme. Mathematical conditions required for retaining the efficiency of the proposed index, in comparison to the imperfect ranked set sampling estimator, are derived. Numerical scrutiny of the relative efficiency, in response to the input variables, indicates; if the variance of the NGO disclosure index is less than the variance of the estimator under imperfect ranked set sampling, then the proposed index is universally efficient compared to the estimator under imperfect ranked set sampling. If the condition on variances is unmet, even then the proposed estimator remains efficient if majority of the NGO share online data on the auxiliary variable. This work can facilitate nonprofit regulation in the countries where most of the non-governmental organizations maintain their websites.

On a Class of Almost Unbiased Ratio Type Estimators

In sample surveys ratio estimator has found extensive applications to obtain more precise estimators of the population ratio, population mean, and population total of the study variable in the presence of auxiliary information, when the study variable is positively correlated with the auxiliary variable. The theory underlying the ratio method of estimation is same whether we estimate the population ratio or population mean/population total, excepting the fact that in the latter case we assume the advance knowledge of the population mean or total of the auxiliary variable in question. In this paper we use the term ratio estimator for both the purposes. However, in spite of its simplicity the ratio estimator is accompanied by an unwelcome bias, although the bias decreases with increase in sample size and is negligible for large sample sizes. In small samples the bias may be substantial so as to downgrade its utility by affecting the reliability of the estimate. As pointed out by L.A. Goodman, H.O. Hartley, J. Am. Stat. Assoc. 53 (1958), 491–508, in sample surveys where we draw very small samples from a large number of strata in stratified random sampling with the ratio method of estimation in each stratum, the combined bias from all the strata may assume serious proportions, affecting the reliability of the estimate. This calls for devising techniques either at estimation stage or in the sampling scheme at the selection stage to reduce the bias or completely eliminating it to make it usable in practice. This has motivated many research workers like E.M.L. Beale, Ind. Organ. 31 (1962), 27–28 and M. Tin, J. Am. Stat. Assoc. 60 (1965), 294–307 among others to construct estimators at the estimation stage removing the bias of O(1/n), where n is the sample size, and thus reducing the bias to O (1/n2). Such estimators are termed as Almost Unbiased ratio-type estimators found in literature. In this paper we have proposed a class of almost ratio type estimators following the techniques of E.M.L. Beale, Ind. Organ. 31 (1962), 27–28 and M. Tin, J. Am. Stat. Assoc. 60 (1965), 294–307 and made comparison with regard to bias and efficiency.

Finite Population Model-Assisted Estimation Using Combined Parametric and Nonparametric Regression Smoothers

This paper considers estimating finite population totals from complex sample surveys in the presence of auxiliary information. Model-assisted estimators which assume a working regression model relating the study variable with the auxiliary data are common in this context. Both parametric and nonparametric working models have been utilized individually in constructing several model-assisted estimators. Model-assisted estimators with parametric working models are known to be efficient when the assumed working model is correctly specified, while using nonparametric smoothers gives more robust estimates but requires relatively large sample sizes. In this paper, we consider the situation where the researcher has an idea of which parametric model can describe the relationship between the study variable and the auxiliary data, but this model may not be adequate in some areas of the data range. Using combined parametric and nonparametric regression smoothers for the working model, we introduce a new class of model-assisted estimators for finite population totals. The proposed estimators are shown to have the desirable asymptotic properties of traditional model-assisted estimators of population totals. The finite sample performance of the new estimators is studied via Monte Carlo simulations from both artificial and real populations. The empirical results suggest that our proposed estimators perform well relative to other model-based and model-assisted estimators as well as the customary Horvitz–Thompson estimator under different levels of misspecification in the working model. We also discuss the problem of variance estimation for the proposed estimators.

Generalized variance estimators using randomized device in the presence of auxiliary information

In this study, the problem of estimation of variance for sensitive variable is considered and suggested generalized variance estimator for sensitive variable using randomized device. A simulation study is conducted to evaluate the performance of the proposed estimator.

Enhanced estimation of population mean in the presence of auxiliary information

In this article, we propose a general family of exponential-type estimators for enhanced estimation of population mean in simple random sampling. These estimators are based on the available parameters of the auxiliary variable such as coefficient of skewness, coefficient of kurtosis, standard deviation and coefficient of variation etc. Expressions for bias, mean squared error and minimum mean squared error of the proposed family are derived up to first degree of approximation. Five natural populations are considered to assess the performance of the proposed estimators. Numerical findings confirm that the proposed estimators dominate over the existing estimators such as sample mean, ratio, regression, Singh et al. (2008, 2009), Upadhyaya et al. (2011), Yadav and Kadilar (2013) and Kadilar (2016) in terms of mean squared error.

On auxiliary information-based control charts for autocorrelated processes with application in manufacturing industry

Multivariate autoregressive (MAR) models are an attractive choice for applications in the processes related to finance, medical, and industry. For the monitoring of such processes, control chart is the most important and widely used tool of statistical process control tool kit. Moreover, the presence of auxiliary information helps in better estimation of different process parameters. The literature on use of auxiliary variables in control charts assumes independence of observations. In practice, we may come across processes dealing with autocorrelated outcomes. In such situations, a control chart usually produces high false alarms and exhibits slow detection of shifts when the process is out-of-control. This study intends to suggest some auxiliary information-based Shewhart charts for autocorrelated univariate and bivariate AR(1) processes. The proposed structures take into account the autocorrelation structure and offer more effective designs of control charts for efficient process monitoring. The performance measures used in this study are based on run length measures such as average run length, extra quadratic loss, relative average run length and performance comparison index. A detailed performance analysis is carried out to sort out the best performing charts. In addition, we have considered an application from a manufacturing process to demonstrate the implementation of the proposed charting structures in real scenario.

Approximation of Finite Population Totals Using Lagrange Polynomial

Approximation of finite population totals in the presence of auxiliary information is considered. A polynomial based on Lagrange polynomial is proposed. Like the local polynomial regression, Horvitz Thompson and ratio estimators, this approximation technique is based on annual population total in order to fit in the best approximating polynomial within a given period of time (years) in this study. This proposed technique has shown to be unbiased under a linear polynomial. The use of real data indicated that the polynomial is efficient and can approximate properly even when the data is unevenly spaced.

Sampling and empirical risk minimization

ABSTRACTIn certain situations that shall be undoubtedly more and more common in the Big Data era, the datasets available are so massive that computing statistics over the full samples is hardly feasible, if not unfeasible. A natural approach in this context consists in using survey schemes and substituting the ‘full data’ statistics with their counterparts based on the resulting random samples, of manageable size. It is the main purpose of this paper to investigate the impact of survey sampling on statistical learning methods based on empirical risk minimization through the standard binary classification problem, considered here as a ‘case in point’. Precisely, we prove that, in presence of auxiliary information, appropriate use of optimally coupled Poisson survey plans may not affect much the learning rates, while possibly reducing significantly the number of terms that must be averaged to compute the empirical risk functional with overwhelming probability. These striking results are next shown to extend to more general sampling schemes by means of a coupling technique, originally introduced by Hajek [Asymptotic theory of rejective sampling with varying probabilities from a finite population. Ann Math Stat. 1964;35(4):1491–1523].

Empirical likelihood-based inference in Poisson autoregressive model with conditional moment restrictions

In this paper, we employ the empirical likelihood method to estimate the unknown parameters in Poisson autoregressive model in the presence of auxiliary information. It is shown that our approach proposed, compared to the maximum likelihood estimator, the least squares estimator, and the weighted least squares estimator, yields more efficient estimators. Some simulation studies are also conducted to investigate the finite sample performance of the proposed method.

Efficient Algorithms for Privately Releasing Marginals via Convex Relaxations

Differential privacy is a definition giving a strong privacy guarantee even in the presence of auxiliary information. In this work, we pursue the application of geometric techniques for achieving differential privacy, a highly promising line of work initiated by Hardt and Talwar (Proceedings of the 42nd ACM Symposium on Theory of Computing, STOC'10, pp 705---714. ACM Press, New York, 2010). We apply these techniques to the problem of marginal release. Here, a database refers to a collection of the data of $$n$$n individuals, each characterized by $$d$$d binary attributes. A $$k$$k-way marginal query is specified by a subset $$S$$S of $$k$$k attributes, together with a $$|S|$$|S|-dimensional binary vector $$\beta $$β specifying their values. The true answer to this query is a count of the number of people in the database whose attribute vector restricted to $$S$$S agrees with $$\beta $$β. Information theoretically, the error complexity of marginal queries--how wrong do the answers have to be in order to preserve differential privacy--is well understood: the per-query additive error is known to be at least $$\varOmega (\min \{ \sqrt{n},d^{k/2}\})$$Ω(min{n,dk/2}) and at most $$\tilde{O}(\sqrt{n}d^{{\lceil k/2\rceil /4}})$$O~(nd?k/2?/4). However, no polynomial time algorithm with error complexity as low as the information-theoretic upper bound is known for small $$n$$n. We present a polynomial time algorithm that matches the best known information-theoretic bounds when $$k=2$$k=2; more generally, by reducing to the case $$k=2$$k=2, for any distribution on marginal queries, our algorithm achieves average error at most $$\tilde{O}(\sqrt{n}d^{{\lceil k/2\rceil /4}})$$O~(nd?k/2?/4), an improvement over previous work when $$k$$k is small and when error $$o(n)$$o(n) is desirable. Using private boosting, we are also able to give nearly matching worst-case error bounds. Our algorithms are based on the geometric techniques of Nikolov et al. (Proceedings of the 45th Annual ACM Symposium on Theory of Computing, STOC'13, pp 351---360. ACM Press, New York, 2013), wherein a vector of sufficiently noisy answers is projected onto a particular convex body. We reduce the projection step, which is expensive, to a simple geometric question: given (a succinct representation of) a convex body $$K$$K, find a containing convex body $$L$$L that one can efficiently optimize over, while keeping the Gaussian width of $$L$$L small. This reduction is achieved by a careful use of the Frank---Wolfe algorithm.

Web Data Extraction and Alignment Tools: A survey

Data extraction from the web pages is the process of analyzing and retrieving relevant data out of the data sources (usually unstructured or poorly structure) in a specific pattern for further processing, involves addition of metadata and data integration details for further process in the data workflow. This survey describes overview of the different web data extraction and data alignment techniques. Extraction techniques are DeLa, DEPTA, ViPER, and ViNT. Data alignment techniques are Pairwise QRR alignment, Holistic alignment, Nested structure processing. Query Result pages are generated by using Web database based on Users Query. The data from these query result pages should be automatically extracted which is very important for many applications, such as data integration, which are needed to cooperate with multiple web databases. New method is proposed for data extraction t that combines both tag and value similarity. It automatically extracts data from query result pages by first identifying and segmenting the query result records (QRRs) in the query result pages and then aligning the segmented QRRs into a table. In which the data values from the same attribute are put into the same column. Data region identification method identify the noncontiguous QRRs that have the same parents according to their tag similarities. Specifically, we propose new techniques to handle the case when the QRRs are not contiguous, which may be due to presence of auxiliary information, such as a comment, recommendation or advertisement, and for handling any nested structure that may exist in the QRRs.

Estimation of Bowley's Coefficient of Skewness in the Presence of Auxiliary Information

In this paper, we suggest regression-type estimators for estimating the Bowley's coefficient of skewness using auxiliary information. To the first degree of approximation, the bias and mean-squared error expressions of the regression-type estimators are obtained, and the regions under which these estimators are more efficient than the conventional estimator are also determined. Further, a general class of estimators of the Bowley's coefficient of skewness is defined along with its properties. A class of estimators based on estimated optimum values is also defined. It is shown to the first degree of approximations that the variance of the class of estimators based on estimated optimum values is the same as that of the minimum variance of the proposed class of estimators. A simulation study is carried out to demonstrate the performance of the proposed difference estimator over the usual estimator.

Estimation of Finite Population Variance Using Scrambled Responses in the Presence of Auxiliary Information

In this article, a new estimator for estimating the finite population variance of a sensitive variable based on scrambled responses collected using a randomization device is introduced. The estimator is then improved by using known auxiliary information. The estimators due to Das and Tripathi (1978: Sankhya) and Isaki (1983: JASA) are shown to be special cases of the proposed estimator. Numerical simulations are performed to study the magnitude of the gain in efficiency when using the estimator with auxiliary information with respect to the estimator based only on the scrambled responses. An idea to extend the present work from SRSWOR design to more complex design is also given.

Robust Bayesian inference in finite population sampling with auxiliary information under balanced loss function

In this paper, we develop Bayesian inference of the finite population mean with the assumption of posterior linearity rather than normality of the superpopulation in the presence of auxiliary information under the balanced loss function. We compare the performance of the optimal Bayes estimator under the balanced loss function with ones of the classical ratio estimator and the usual Bayes estimator in terms of the posterior expected losses, risks and Bayes risks.