Asymptotic Bias Research Articles

Asymptotic bias of $C_p$ type criterion for model selection in the GEE when the sample size and the cluster sizes are large

In this paper, we evaluate the asymptotic bias of $C_p$ type criterion for model selection in the GEE (generalized estimating equation) method when the sample and cluster sizes are large. We present the asymptotic properties of GEE estimator and the model selection criterion. Then, we present the order of the asymptotic bias of PMSEG (the prediction mean squared error in the GEE).

Large sample properties of partitioning-based series estimators

We present large sample results for partitioning-based least squares nonparametric regression, a popular method for approximating conditional expectation functions in statistics, econometrics and machine learning. First, we obtain a general characterization of their leading asymptotic bias. Second, we establish integrated mean squared error approximations for the point estimator and propose feasible tuning parameter selection. Third, we develop pointwise inference methods based on undersmoothing and robust bias correction. Fourth, employing different coupling approaches, we develop uniform distributional approximations for the undersmoothed and robust bias-corrected $t$-statistic processes and construct valid confidence bands. In the univariate case, our uniform distributional approximations require seemingly minimal rate restrictions and improve on approximation rates known in the literature. Finally, we apply our general results to three partitioning-based estimators: splines, wavelets and piecewise polynomials. The Supplemental Appendix includes several other general and example-specific technical and methodological results. A companion $\mathsf{R}$ package is provided.

Estimation of the Mann–Whitney effect in the two-sample problem under dependent censoring

The Mann–Whitney effect is a nonparametric measure for comparing the distribution between two groups, which can be estimated by right-censored data. However, the traditional estimator of the Mann–Whitney effect based on the Kaplan–Meier estimators can be inconsistent when the independent censoring assumption fails to hold. Investigation is made on the asymptotic bias of the traditional estimator of the Mann–Whitney effect when the independent censoring assumption is violated due to dependence between survival time and censoring time. A new estimator of the Mann–Whitney effect is proposed by applying the copula-graphic estimator to adjust for the effect of dependent censoring. The proposed estimator and test are consistent when the assumed copulas for the two groups are correct. Some consistency properties under misspecified copulas are also given. Simulations are conducted to verify the proposed method under possible misspecification on copulas. The method is illustrated by a real data set. We provide an R function “MW.test” to implement the proposed estimator and test.

Optimally Compressed Nonparametric Online Learning: Tradeoffs between memory and consistency

Batch training of machine learning models based on neural networks is well established, whereas, to date, streaming methods are largely based on linear models. To go beyond linear in the online setting, nonparametric methods are of interest due to their universality and ability to stably incorporate new information via convexity or Bayes's rule. Unfortunately, when applied online, nonparametric methods suffer a curse of dimensionality, which precludes their use: their complexity scales at least with the time index. We survey online compression tools that bring their memory under control and attain approximate convergence. The asymptotic bias depends on a compression parameter that trades off memory and accuracy. Applications to robotics, communications, economics, and power are discussed as well as extensions to multiagent systems.

An alternative two-step generalized method of moments estimator based on a reduced form model

I propose an alternative two-step generalized method of moment (GMM) estimator when an exactly-identified underlying parameter is available. The resulting estimator does not have the higher-order asymptotic bias arising from the choice of a preliminary weighting matrix.

A general approach to sensitivity analysis for Mendelian randomization.

Mendelian Randomization (MR) represents a class of instrumental variable methods using genetic variants. It has become popular in epidemiological studies to account for the unmeasured confounders when estimating the effect of exposure on outcome. The success of Mendelian Randomization depends on three critical assumptions, which are difficult to verify. Therefore, sensitivity analysis methods are needed for evaluating results and making plausible conclusions. We propose a general and easy to apply approach to conduct sensitivity analysis for Mendelian Randomization studies. Bound et al. (1995) derived a formula for the asymptotic bias of the instrumental variable estimator. Based on their work, we derive a new sensitivity analysis formula. The parameters in the formula include sensitivity parameters such as the correlation between instruments and unmeasured confounder, the direct effect of instruments on outcome and the strength of instruments. In our simulation studies, we examined our approach in various scenarios using either individual SNPs or unweighted allele score as instruments. By using a previously published dataset from researchers involving a bone mineral density study, we demonstrate that our proposed method is a useful tool for MR studies, and that investigators can combine their domain knowledge with our method to obtain bias-corrected results and make informed conclusions on the scientific plausibility of their findings.

Estimation of a Finite Population Mean under Random Nonresponse Using Kernel Weights

Nonresponse is a potential source of errors in sample surveys. It introduces bias and large variance in the estimation of finite population parameters. Regression models have been recognized as one of the techniques of reducing bias and variance due to random nonresponse using auxiliary data. In this study, it is assumed that random nonresponse occurs in the survey variable in the second stage of cluster sampling, assuming full auxiliary information is available throughout. Auxiliary information is used at the estimation stage via a regression model to address the problem of random nonresponse. In particular, auxiliary information is used via an improved Nadaraya–Watson kernel regression technique to compensate for random nonresponse. The asymptotic bias and mean squared error of the estimator proposed are derived. Besides, a simulation study conducted indicates that the proposed estimator has smaller values of the bias and smaller mean squared error values compared to existing estimators of a finite population mean. The proposed estimator is also shown to have tighter confidence interval lengths at 95% coverage rate. The results obtained in this study are useful for instance in choosing efficient estimators of a finite population mean in demographic sample surveys.

On the comparison of several classical estimators of the extreme value index

Due to the fact that for heavy tails the classical Hill estimator of a positive extreme value index is asymptotically biased, new and interesting alternative estimators have appeared in the literature. In this work we compare several classical estimators of the extreme value index based on moments of the upper order statistics. Since several alternative estimators have eventually a null asymptotic bias, for some heavy tailed models, the comparison is performed not only with the Hill and recent generalized means estimators but also with an asymptotically unbiased Hill estimator. The comparison study is performed asymptotically, under a third-order framework, and for finite samples, through a Monte Carlo simulation study.

Improved estimation in elliptical linear mixed measurement error models

In this paper, we propose a set of improved estimators for the fixed effect parameters in the linear mixed models when the covariates are measured with additive errors and prior information for the parameters is available. When it is suspected that the parameter vector may be the null-vector with some degree of uncertainty, we define improved estimators which including the preliminary test estimator, the Stein-type estimator and the postive-rule Stein-type estimator. It is assumed that the measurement error distributed according to the law belonging to the class of elliptically contoured distributions. The asymptotic properties of resulting estimators such as the asymptotic distributional quadratic biases and the asymptotic distributional quadratic risks are examined. A simulation study is also performed to illustrate the finite sample performance of the proposed procedures and finally Boston housing data set is analysed.

The robust focused information criterion for strong mixing stochastic processes with $$\mathscr {L}^{2}$$-differentiable parametric densities

Claeskens and Hjort (J Am Stat Assoc 98(464):900–916, 2003) constructed the focused information criterion (FIC) using maximum likelihood estimators to facilitate the contextual selection of probability models for independently distributed observations. We generalize these results to the case of stationary, strong mixing stochastic processes exhibiting outliers when the “true” finite dimensional distribution of the process lies in the contamination neighbourhood of the assumed parametric model for the process. Given the natural filtration of a stochastic process, we obtain the FIC using robust M-estimators having bounded conditional influence functions. We utilize Le Cam’s contiguity lemmas to tract the parametric form of the asymptotic bias of M-estimators under model misspecification induced by additive outliers. The local asymptotic normality is established assuming the finite dimensional parametric density of the process to be $$\mathscr {L}^{2}$$ -differentiable (differentiable in quadratic mean). As a result, our theory is also applicable for constructing FIC for moderately irregular models outside the exponential family such as Laplace and related densities. We apply our results to derive the robust FIC for simultaneous selection of the order and the innovation density of asymmetric Laplace autoregressive processes observed with outliers. We demonstrate our theory with the focused modeling of United States Dollar to Indian Rupee currency exchange rates exhibiting outliers post Indian demonetization.

The Improved Value-at-Risk for Heteroscedastic Processes and Their Coverage Probability

A risk measure commonly used in financial risk management, namely, Value-at-Risk (VaR), is studied. In particular, we find a VaR forecast for heteroscedastic processes such that its (conditional) coverage probability is close to the nominal. To do so, we pay attention to the effect of estimator variability such as asymptotic bias and mean square error. Numerical analysis is carried out to illustrate this calculation for the Autoregressive Conditional Heteroscedastic (ARCH) model, an observable volatility type model. In comparison, we find VaR for the latent volatility model i.e., the Stochastic Volatility Autoregressive (SVAR) model. It is found that the effect of estimator variability is significant to obtain VaR forecast with better coverage. In addition, we may only be able to assess unconditional coverage probability for VaR forecast of the SVAR model. This is due to the fact that the volatility process of the model is unobservable.

Penalized and ridge-type shrinkage estimators in Poisson regression model

The paper considers the problem of estimation of the regression coefficients in a Poisson regression model under multicollinearity situation. We propose non-penalty Stein-type shrinkage ridge estimation approach when it is conjectured that some prior information is available in the form of potential linear restrictions on the coefficients. We establish the asymptotic distributional biases and risks of the proposed estimators and investigate their relative performance with respect to the unrestricted ridge estimator. For comparison sake, we consider the two penalty estimators, namely, least absolute shrinkage and selection operator and Elastic-Net estimators and compare numerically their relative performance with the other listed estimators. Monte-Carlo simulation experiment is conducted to evaluate the performance of each estimator in terms of the simulated relative efficiency. The results show that the shrinkage ridge estimators perform better than the penalty estimators in certain parts of the parameter space. Finally, a real data example is illustrated to evaluate of the proposed methods.

Performance analysis and DOA estimation method over acoustic vector sensor array in the presence of polarity inconsistency

This paper investigates the direction of arrival (DOA) estimation performance in the presence of polarity inconsistency in uniform acoustic vector sensor (AVS) linear array. We analyze the influence of polarity bias on beampattern directivity of the AVS array. The analysis results show that the polarity bias leads to asymptotically biased estimation. Then, the analytical expression for the asymptotic bias based on classical beamforming is derived in the presence of polarity error. Moreover, to improve the DOA estimation performance in the presence of polarity inconsistency, a polarity calibration method is proposed. Numerical simulations reveal the effectiveness and superiority of the proposed calibration method when the polarity error satisfies with the uniform distribution and the normal distribution.

(Structural) VAR Models with Ignored Changes in Mean and Volatility

We discuss estimation of so-called long vector autoregressions for multivariate series exhibiting possibly time-varying mean and (co)variances. In applied work, such changes often escape undetected, and we ask how standard tools (least squares estimation, point forecasts, and estimated impulse responses) are affected when ignoring the changes altogether. Keeping the order of the autoregression fixed is known to lead to asymptotic bias in autoregressive parameter estimators in the presence of ignored changes in the mean. Yet we show that allowing the complexity of the model to increase with the sample size leads to consistent estimators of the AR coefficient matrices individually. The fitted long VAR models appear to have unit root behavior, in spite of the absence of any stochastic trend in the model, and may even mimic cointegration; but, in spite of the structural change in the data generating process, out-of-sample forecasts based on long VARs are consistent. These findings hold under constant as well as under time-varying covariances. In what concerns estimated impulse responses, their sampling behavior depends primarily on whether the residual covariance matrix is employed for identification of the structural shocks or not. While MA coefficient matrices obtained by inversion of the fitted long VAR model are consistent for the true coefficients under mild additional restrictions even under time-varying error (co)variances, the residual covariance matrix estimator converges to an average covariance matrix, such that localized estimators may be more suitable for a precise identification. Monte Carlo simulations and empirical illustration support our theoretical findings. Empirical relevance of the theory is illustrated through two illustrations: (i) international dynamics of inflation and (ii) uncertainty and economics activity.

Debiased magnitude-preserving ranking: Learning rate and bias characterization

Magnitude-preserving ranking (MPRank) under Tikhonov regularization framework has shown competitive performance on information retrial besides theoretical advantages for computation feasibility and statistical guarantees. In this paper, we further characterize the learning rate and asymptotic bias of MPRank, and then propose a new debiased ranking algorithm. In terms of the operator representation and approximation techniques, we establish their convergence rates and bias characterizations. These theoretical results demonstrate that the new model has smaller asymptotic bias than MPRank, and can achieve the satisfactory convergence rate under appropriate conditions. In addition, some empirical examples are provided to verify the effectiveness of our debiased strategy.

On bias reduction estimators of skew-normal and skew-t distributions

A particular concerns of researchers in statistical inference is bias in parameters estimation. Maximum likelihood estimators are often biased and for small sample size, the first order bias of them can be large and so it may influence the efficiency of the estimator. There are different methods for reduction of this bias. In this paper, we proposed a modified maximum likelihood estimator for the shape parameter of two popular skew distributions, namely skew-normal and skew-t, by offering a new method. We show that this estimator has lower asymptotic bias than the maximum likelihood estimator and is more efficient than those based on the existing methods.

Model Mis-Specification in Newsvendor Decisions: A Comparison of Frequentist Parametric, Bayesian Parametric and Nonparametric Approaches

We compare three different approaches studied by past literature on data-driven inventory optimization--- Frequentist Parametric (FP), Bayesian Parametric (BP) and Nonparametric--- for the newsvendor problem. For the Parametric approaches, we allow for mis-specification of the demand model. We prove, under mild regularity conditions, (i) asymptotic bias and variance formulas of FP and BP are equivalent, (ii) mis-specified Parametric approaches yield asymptotically biased decisions, unlike the correctly-specified Parametric approaches and the Nonparametric approach, and (iii) asymptotic variance of the mis-specified Parametric approaches converges to zero at rate $1/n$, in contrast to the $1/n^2$ rate for the correctly-specified Parametric approaches and the Nonparametric approach, where $n$ is the number of demand samples. We then show, for nine pairs of assumed versus true demand distribution pairs, (iv) asymptotic bias and variance formulas approximate finite-sample counterparts very well, (v) correctly-specified Parametric approaches dominate the Nonparametric approach in the asymptotic mean-squared error (AMSE) of the decision and the cost, and (vi) surprisingly, it is possible for mis-specified Parametric approaches to dominate the Nonparametric approach in the AMSE of the decision and the cost. We compare the approaches on a dataset from a large fresh food chain, and discuss the nuances of choosing the ``best'' approach.

IV Estimation of Spatial Dynamic Panels with Interactive Effects: Large Sample Theory and an Application on Bank Attitude Toward Risk

The present paper develops a new Instrumental Variables (IV) estimator for spatial, dynamic panel data models with interactive effects under large N and T asymptotics. For this class of models, the only approaches available in the literature are based on quasi-maximum likelihood estimation. The approach put forward in this paper is appealing from both a theoretical and a practical point of view for a number of reasons. Firstly, the proposed IV estimator is linear in the parameters of interest and it is computationally inexpensive. Secondly, the IV estimator is free from asymptotic bias. In contrast, existing QML estimators suffer from incidental parameter bias, depending on the magnitude of unknown parameters. Thirdly, the IV estimator retains the attractive feature of Method of Moments estimation in that it can accommodate endogenous regressors, so long as external exogenous instruments are available. The IV estimator is consistent and asymptotically normal as N, T go to infinity, with N/T^2 going to 0 and T/N^2 tending to 0. The proposed methodology is employed to study the determinants of risk attitude of banking institutions. The results of our analysis provide evidence that the more risk-sensitive capital regulation that was introduced by the Basel III framework in 2011 has succeeded in influencing banks’ behavior in a substantial manner.

Interactive Effects Panel Data Models with General Factors and Regressors

The presence of unobserved heterogeneity and its detrimental effect on inference has recently motivated the use of interactive effects panel data models. One of the workhorses of this literature is based on iterating between principal components estimation of the un- known factors and OLS estimation of the model parameters. We refer to this as the “PC” approach, the existing asymptotic theory for which is based on the requirement that all the factors and regressors are either stationary or unit root non-stationary. Deterministic factors are typically treated as known and are projected out prior to the application of PC. This is a problem in practice where there is typically great uncertainty over both the order of integration of the stochastic component of the data and the terms needed to capture the deterministic component. This paper relaxes the above mentioned assumptions by considering a model wherein both the factors and regressors are essentially unrestricted, up to mild regulatory conditions. An estimator based on iterating PC is proposed, which is shown to be not only asymptotically normal and oracle efficient, but under certain conditions also free of the otherwise so common asymptotic incidental parameters bias. Interestingly, the conditions required to achieve unbiasedness become weaker the stronger the trends in the factors, and if the trending is strong enough unbiasedness comes at no cost at all. Equally important as these theoretical properties is the ease with which the new approach can be applied. In particular, the approach does not require any knowledge of how many factors there are, or whether they are deterministic/stochastic. The order of integration of the factors is also treated as unknown, as is the order of integration of the regressors, which means that there is no need to pre-test for unit roots, or to decide on which deterministic terms to include in the model. The new estimation approach should therefore be attractive for practitioners.

Causal inference with noisy data: Bias analysis and estimation approaches to simultaneously addressing missingness and misclassification in binary outcomes.

Causal inference has been widely conducted in various fields and many methods have been proposed for different settings. However, for noisy data with both mismeasurements and missing observations, those methods often break down. In this paper, we consider a problem that binary outcomes are subject to both missingness and misclassification, when the interest is in estimation of the average treatment effects (ATE). We examine the asymptotic biases caused by ignoring missingness and/or misclassification and establish the intrinsic connections between missingness effects and misclassification effects on the estimation of ATE. We develop valid weighted estimation methods to simultaneously correct for missingness and misclassification effects. To provide protection against model misspecification, we further propose a doubly robust correction method which yields consistent estimators when either the treatment model or the outcome model is misspecified. Simulation studies are conducted to assess the performance of the proposed methods. An application to smoking cessation data is reported to illustrate the use of the proposed methods.