Sampling Distribution Of Estimator Research Articles

Post-model-selection inference in linear regression models: An integrated review

The research on statistical inference after data-driven model selection can be traced as far back as Koopmans (1949). The intensive research on modern model selection methods for high-dimensional data over the past three decades revived the interest in statistical inference after model selection. In recent years, there has been a surge of articles on statistical inference after model selection and now a rather vast literature exists on this topic. Our manuscript aims at presenting a holistic review of post-model-selection inference in linear regression models, while also incorporating perspectives from high-dimensional inference in these models. We first give a simulated example motivating the necessity for valid statistical inference after model selection. We then provide theoretical insights explaining the phenomena observed in the example. This is done through a literature survey on the post-selection sampling distribution of regression parameter estimators and properties of coverage probabilities of naïve confidence intervals. Categorized according to two types of estimation targets, namely the population- and projection-based regression coefficients, we present a review of recent uncertainty assessment methods. We also discuss possible pros and cons for the confidence intervals constructed by different methods.

Receiver operating characteristic (ROC) curves: equivalences, beta model, and minimum distance estimation

Receiver operating characteristic (ROC) curves are used ubiquitously to evaluate scores, features, covariates or markers as potential predictors in binary problems. We characterize ROC curves from a probabilistic perspective and establish an equivalence between ROC curves and cumulative distribution functions (CDFs). These results support a subtle shift of paradigms in the statistical modelling of ROC curves, which we view as curve fitting. We propose the flexible two-parameter beta family for fitting CDFs to empirical ROC curves and derive the large sample distribution of minimum distance estimators in general parametric settings. In a range of empirical examples the beta family fits better than the classical binormal model, particularly under the vital constraint of the fitted curve being concave.

Optimal bandwidth estimators of kernel density functionals for contaminated data

In this study, we provide simulation-based exploration and characterization of the two most crucial kernel density functionals that play a central role in kernel density estimation, considering the probability density functions that are members of the location-scale family. Kernel density functional estimates are known to rely on the choice of preliminary bandwidth. Normal-scale estimators are commonly used to obtain preliminary bandwidth estimates, with the assumption that the data come from normal distribution. Here, we present an alternative approach, called the Cauchy-scale estimators, to obtain preliminary bandwidth estimates. In this approach, data are assumed to come from a Cauchy distribution. Furthermore, analysis results related to the sampling distribution of bandwidth estimators based on the normal- and Cauchy-scale approaches are presented. As a case study, we provide a comprehensive characterization of different contamination levels with a simulation study constructed for the random samples from normal distributions with various parameters and various contamination levels. The proposed preliminary bandwidth selection shows lower variance in both mixture and contaminated data in our simulations. Besides, functional bandwidth presents results similar to the simulation results in the applications we made on the real data set.

BOOTSTRAP INFERENCE FOR MULTIPLE CHANGE-POINTS IN TIME SERIES

In this paper, we propose two bootstrap procedures, namely parametric and block bootstrap, to approximate the finite sample distribution of change-point estimators for piecewise stationary time series. The bootstrap procedures are then used to develop a generalized likelihood ratio scan method (GLRSM) for multiple change-point inference in piecewise stationary time series, which estimates the number and locations of change-points and provides a confidence interval for each change-point. The computational complexity of using GLRSM for multiple change-point detection is as low as $O(n(\log n)^{3})$ for a series of length n. Extensive simulation studies are provided to demonstrate the effectiveness of the proposed methodology under different scenarios. Applications to financial time series are also illustrated.

An adjustable inspection scheme for lot sentencing based on one-sided capability indices

Acceptance sampling is an extensively used strategy for determining if the quality of a submitted product lots meets the required standards. Several sampling plans were developed based on different strategies to meet this aim and to increase their overall efficiency and flexibility. The repetitive group sampling plan (RGSP) is demonstrated to be an efficient sampling scheme but has its demerits in that the number of times sampled depends on the sampling results and theoretically could be infinite. To overcome this drawback, an adjustable inspection scheme based on one-sided capability indices was developed by setting a limit on total sampling times to the conventional RGSP. The operating characteristic (OC) function and average sample number of the proposed plan are derived based on the exact sampling distribution of unbiased estimators. To determine the plan parameters, an optimization model with non-linear constraints related to quality and risk requirements is formulated, and those under various commonly used settings are tabulated for practical use. The plan's performance is examined, discussed, and compared with alternative conventional plans. Finally, an example is provided to illustrate the applicability of the proposed plan.

Evaluation of Test Statistics for Detection of Outliers and Shifts

Existence of outliers and structural breaks having mutually unknown nature, in time series data, offer challenges to data analysts in model identification, estimation and validation. Detection of these outliers has been an important area of research in time series since long. To analyze the impact of these structural breaks and outliers on model identification, estimation and their inferential analysis, we use two data generating processes: MA(1) and ARMA(1,1). The performance of the test statistics for detecting additive outlier(AO), innovative outlier(IO), level shift(LS) and transient change(TC) is investigated using simulation strategy through power of a test, empirical level of significance, empirical critical values, misspecification frequencies and sampling distribution of estimators for the two models. The empirical critical values are found higher than the theoretical cut-off points, empirical power of the test statistics is not satisfactory for small sample size, large cut-off points and large model coefficient. We have explored confusion between LS, AO, TC and IO at different critical values(c) by varying sample size. We have also collected empirical evidence from time series data for Pakistan using 3-stage iterative procedure to detect multiple outliers and structural breaks. We find that neglecting shocks lead to wrong identification, biased estimation and excess kurtosis.
 JEL Classification Codes: C15, C18, C63, C32, C87, C51, C52, C82
 AMS Classification Codes: 62, 65, 91, DI, 62-08, 62J20, 00A72, 91-08, 91-10, 91-11 62P20, 91B82, 91B84, 62M07, 62M09, 62M10, 62M15, 62M20

Sensitivity Analysis of an OLS Multiple Regression Inference with Respect to Possible Linear Endogeneity in the Explanatory Variables, for Both Modest and for Extremely Large Samples

This work describes a versatile and readily-deployable sensitivity analysis of an ordinary least squares (OLS) inference with respect to possible endogeneity in the explanatory variables of the usual k-variate linear multiple regression model. This sensitivity analysis is based on a derivation of the sampling distribution of the OLS parameter estimator, extended to the setting where some, or all, of the explanatory variables are endogenous. In exchange for restricting attention to possible endogeneity which is solely linear in nature—the most typical case—no additional model assumptions must be made, beyond the usual ones for a model with stochastic regressors. The sensitivity analysis quantifies the sensitivity of hypothesis test rejection p-values and/or estimated confidence intervals to such endogeneity, enabling an informed judgment as to whether any selected inference is “robust” versus “fragile.” The usefulness of this sensitivity analysis—as a “screen” for potential endogeneity issues—is illustrated with an example from the empirical growth literature. This example is extended to an extremely large sample, so as to illustrate how this sensitivity analysis can be applied to parameter confidence intervals in the context of massive datasets, as in “big data”.

Stationary distribution of the linkage disequilibrium coefficient [formula omitted

The linkage disequilibrium coefficient r2 is a measure of statistical dependence of the alleles possessed by an individual at different genetic loci. It is widely used in association studies to search for the locations of disease-causing genes on chromosomes. Most studies to date treat r2 as a fixed property of two loci in a finite population, and investigate the sampling distribution of estimators due to the statistical sampling of individuals from the population. Here, we instead consider the distribution of r2 itself under a process of genetic sampling through the generations. Using a classical two-locus model for genetic drift, mutation, and recombination, we investigate the probability density function of r2 at stationarity. This density function provides a tool for inference on evolutionary parameters such as mutation and recombination rates. We reconstruct the approximate stationary density of r2 by calculating a finite sequence of the distribution’s moments and applying the maximum entropy principle. Our approach is based on the diffusion approximation, under which we demonstrate that for certain models in population genetics, moments of the stationary distribution can be obtained without knowing the probability distribution itself. To illustrate our approach, we show how the stationary probability density of r2 can be used in a maximum likelihood framework to estimate mutation and recombination rates from sample data of r2.

Inference for Income Mobility Measures in the Presence of Spatial Dependence

Income mobility measures provide convenient and concise ways to reveal the dynamic nature of regional income distributions. Statistical inference about these measures is important especially when it comes to a comparison of two regional income systems. Although the analytical sampling distributions of relevant estimators and test statistics have been asymptotically derived, their properties in small sample settings and in the presence of contemporaneous spatial dependence within a regional income system are underexplored. We approach these issues via a series of Monte Carlo experiments that require the proposal of a novel data generating process capable of generating spatially dependent time series given a transition probability matrix and a specified level of spatial dependence. Results suggest that when sample size is small, the mobility estimator is biased while spatial dependence inflates its asymptotic variance, raising the Type I error rate for a one-sample test. For the two-sample test of the difference in mobility between two regional economic systems, the size tends to become increasingly upward biased with stronger spatial dependence in either income system, which indicates that conclusions about differences in mobility between two different regional systems need to be drawn with caution as the presence of spatial dependence can lead to false positives. In light of this, we suggest adjustments for the critical values of relevant test statistics.

A Monograph on Nonlinear Regression Models

This research article uses Matrix Calculus techniques to study least squares application of nonlinear regression model, sampling distributions of nonlinear least squares estimators of regression parametric vector and error variance and testing of general nonlinear hypothesis on parameters of nonlinear regression model. Arthipova Irina et.al [1], in this paper, discussed some examples of different nonlinear models and the application of OLS (Ordinary Least Squares). MA Tabati et.al (2), proposed a robust alternative technique to OLS nonlinear regression method which provide accurate parameter estimates when outliers and/or influential observations are present. Xu Zheng et.al [3] presented new parametric tests for heteroscedasticity in nonlinear and nonparametric models.

Statistical inference using generalized linear mixed models under informative cluster sampling

AbstractWhen a sample is obtained from a two‐stage cluster sampling scheme with unequal selection probabilities the sample distribution can differ from that of the population and the sampling design can be informative. In this case making valid inference under generalized linear mixed models can be quite challenging. We propose a novel approach for parameter estimation using an EM algorithm based on the approximate predictive distribution of the random effect. In the approximate predictive distribution instead of using the intractable sample likelihood function we use a normal approximation of the sampling distribution of the profile pseudo maximum likelihood estimator of the random effects in the level‐one model. Two limited simulation studies show that the proposed method using the normal approximation performs well for modest cluster sizes. The proposed method is applied to the real data arising from 2011 Private Education Expenditures Survey (PEES) in Korea. The Canadian Journal of Statistics 45: 479–497; 2017 © 2017 Statistical Society of Canada

Randomization Inference with Rainfall Data: Using Historical Weather Patterns for Variance Estimation

Many recent papers in political science and economics use rainfall as a strategy to facilitate causal inference. Rainfall shocks are as-if randomly assigned, but the assignment of rainfall by county is highly correlated across space. Since clustered assignment does not occur within well-defined boundaries, it is challenging to estimate the variance of the effect of rainfall on political outcomes. I propose using randomization inference with historical weather patterns from 73 years as potential randomizations. I replicate the influential work on rainfall and voter turnout in presidential elections in the United States by Gomez, Hansford, and Krause (2007) and compare the estimated average treatment effect (ATE) to a sampling distribution of estimates under the sharp null hypothesis of no effect. The alternate randomizations are random draws from national rainfall patterns on election and would-be election days, which preserve the clustering in treatment assignment and eliminate the need to simulate weather patterns or make assumptions about unit boundaries for clustering. I find that the effect of rainfall on turnout is subject to greater sampling variability than previously estimated using conventional standard errors.

Sufficient m-out-of-n (m/n) bootstrap

ABSTRACTTraditional resampling methods for estimating sampling distributions sometimes fail, and alternative approaches are then needed. For example, if the classical central limit theorem does not hold and the naïve bootstrap fails, the m/n bootstrap, based on smaller-sized resamples, may be used as an alternative. An alternative to the naïve bootstrap, the sufficient bootstrap, which uses only the distinct observations in a bootstrap sample, is another recently proposed bootstrap approach that has been suggested to reduce the computational burden associated with bootstrapping. It works as long as naïve bootstrap does. However, if the naïve bootstrap fails, so will the sufficient bootstrap. In this paper, we propose combining the sufficient bootstrap with the m/n bootstrap in order to both regain consistent estimation of sampling distributions and to reduce the computational burden of the bootstrap. We obtain necessary and sufficient conditions for asymptotic normality of the proposed method, and propose new values for the resample size m. We compare the proposed method with the naïve bootstrap, the sufficient bootstrap, and the m/n bootstrap by simulation.

On Bootstrap Confidence Intervals Associated with Nonparametric Regression Estimators for A Finite Population Total

The precision of an estimator is at times discussed regarding the variance. Usually, the exact value of the variance is unknown. The discussion relies on unknown populace quantities. When a researcher obtains the survey data, an estimate of the variance can, therefore, be calculated. When survey results are presented, it is good practice to provide variance estimates for the estimator used in the study. The estimator of the variance can further be used to construct confidence interval, assuming that the sampling distribution of estimator is approximately normal. This study proposes estimation of standard error and confidence interval for a nonparametric regression estimator for a finite population using bootstrapping method. The idea behind bootstrapping is to carry out computations on the collected data. Computation activity assists in estimating the disparity of statistics that are themselves computed from the same data. The variance of the Nadaraya-Watson estimator is derived, based on bootstrap procedure. This operation has led to the derivation of confidence interval associated with Nadaraya-Watson estimator of the population total. A simulation study has been carried out. The overall conclusion is that the confidence interval associated with Nadaraya-Watson estimator is tighter than all the other estimators (Horvitz-Thompson estimator, Local linear estimator, and Ratio estimator).

Estimator augmentation with applications in high-dimensional group inference

To make inference about a group of parameters on high-dimensional data, we develop the method of estimator augmentation for the block Lasso, which is defined via the block norm. By augmenting a block Lasso estimator $\hat{\beta}$ with the subgradient $S$ of the block norm evaluated at $\hat{\beta}$, we derive a closed-form density for the joint distribution of $(\hat{\beta},S)$ under a high-dimensional setting. This allows us to draw from an estimated sampling distribution of $\hat{\beta}$, or more generally any function of $(\hat{\beta},S)$, by Monte Carlo algorithms. We demonstrate the application of estimator augmentation in group inference with the group Lasso and a de-biased group Lasso constructed as a function of $(\hat{\beta},S)$. Our numerical results show that importance sampling via estimator augmentation can be orders of magnitude more efficient than parametric bootstrap in estimating tail probabilities for significance tests. This work also brings new insights into the geometry of the sample space and the solution uniqueness of the block Lasso.

DSGE pileups

The sampling distribution of estimators for DSGE structural parameters tends to be non-normal and/or pile up on the boundary of the theoretically admissible parameter space. This calls into question both the reliability of asymptotic approximations and the presumption of correct specification. This paper seeks to develop a conceptual framework for understanding how these phenomena arise, and to provide pragmatic methods for dealing with them in practice. The results are presented in three examples and a medium scale DSGE model.

Grid-spacing and the quality of abundance maps for species that show spatial autocorrelation and zero-inflation

The effect of grid-spacing on the quality of species abundance maps is explored for species that show zero-inflation and spatial autocorrelation. Using a zero-inflated Poisson mixture model multiple fields of the prevalence parameter π and the intensity parameter μ were simulated. A selected field was sampled by grid-sampling with 200, 400, 800, 1600, and 3200 m grid-spacing and used to predict at a fixed set of validation locations by simple kriging with an external drift. The external drift variables were silt, silt squared and altitude. The estimated sampling distribution of MSE against grid-spacing shows that beyond a spacing of 1600 m the mean of MSE increases at a much faster rate. Based on these findings the 1600 m grid which consists of 446 locations for our study area of 2400 km2 gives a compromise between sampling costs and prediction accuracy.

A New Multivariate Nonlinear Time Series Model for Portfolio Risk Measurement: The Threshold Copula‐Based TAR Approach

We propose a threshold copula‐based nonlinear time series model for evaluating quantitative risk measures for financial portfolios with a flexible structure to incorporate nonlinearities in both univariate (component) time series and their dependent structure. We incorporate different dependent structures of asset returns over different market regimes, which are manifested in their price levels. We estimate the model parameters by a two‐stage maximum likelihood method. Real financial data and appropriate statistical tests are used to illustrate the efficacy of the proposed model. Simulated results for sampling distribution of parameters estimates are given. Empirical results suggest that the proposed model leads to significant improvement of the accuracy of value‐at‐risk forecasts at the portfolio level.

The sampling design effect on partial least squares algorithm

The objective of this article is to analyze the effect of the probability sampling’s selection on the estimated results in Structural Equation Modeling (SEM) using the Partial Least Squares (PLS) algorithm.The idea leading this work is to estimate the satisfaction level of government service users in a large and dispersed population, for which a sample design with an equal selection probability is not a feasible option. This study is based on the analysis of the sampling distributions of estimators under different sampling designs.It is shown that the probability of selection of the units behind the sampling design affect the results of the PLS algorithm, both the scores of latent variables and the impacts between them.To the author's knowledge, this issue has not been addressed before in the literature.

Matching on the Estimated Propensity Score

Propensity score matching estimators (Rosenbaum and Rubin (1983)) are widely used in evaluation research to estimate average treatment effects. In this article, we derive the large sample distribution of propensity score matching estimators. Our derivations take into account that the propensity score is itself estimated in a first step, prior to matching. We prove that first step estimation of the propensity score affects the large sample distribution of propensity score matching estimators, and derive adjustments to the large sample variances of propensity score matching estimators of the average treatment effect (ATE) and the average treatment effect on the treated (ATET). The adjustment for the ATE estimator is negative (or zero in some special cases), implying that matching on the estimated propensity score is more efficient than matching on the true propensity score in large samples. However, for the ATET estimator, the sign of the adjustment term depends on the data generating process, and ignoring the estimation error in the propensity score may lead to confidence intervals that are either too large or too small.