Asymptotic Mean Square Error Research Articles

Exogeneity tests and weak identification in IV regressions: Asymptotic theory and point estimation

This paper provides new insights on exogeneity tests in linear IV models and their use for estimation, when identification fails or may not be strong. We make two main contributions. First, we show that Durbin-Wu-Hausman (DWH) and Revankar-Hartley (RH) exogeneity tests have correct level asymptotically, even when the first-stage coefficient matrix (which controls identification) is rank-deficient. We provide necessary and sufficient conditions under which these tests are consistent. In particular, we show that test consistency can hold even when identification fails, provided at least one component of the structural parameter vector is identifiable. Second, we study point estimation after estimator (or model) selection, when the outcome of a DWH/RH test determines whether OLS or an IV method is employed in the second-stage. For this purpose, we use (non-local) concepts of asymptotic bias, asymptotic mean squared error (AMSE), and asymptotic relative efficiency (ARE), which remain applicable even when the estimators considered do not have moments (as can happen for 2SLS) or may be inconsistent. We study the asymptotic properties of OLS, 2SLS, and pretest estimators which select OLS or 2SLS based on the outcome of a DWH/RH test. We show that: (i) OLS typically dominates 2SLS estimator asymptotically for MSE across a broad spectrum of cases, including weak identification and moderate endogeneity; (ii) exogeneity-pretest estimators exhibit consistently good performance and asymptotically dominate both OLS and 2SLS. The proposed theoretical findings are documented by Monte Carlo simulations.

Designing shape-preserving descriptors for classifying signals with application to vibrations of large mechanical structures

The main contribution of this paper is introduction of descriptors based on the moments of the Bernstein–Durrmeyer polynomials. As it is demonstrated, their distinctive feature is their ability to preserve knowledge about the shapes of the signals such as monotonicity and convexity, without imposing any a priori constraints. These descriptors act in time-domain as shape-preserving knowledge extractors. For their empirical version, the asymptotic unbiasedness and MSE consistency are also proved without additional pre-filtering.Another significant contribution is the method of designing an autoencoder for selecting the number of descriptors suitable for a whole class of underlying signals. For this purpose, a generalized version of Akaike’s information criterion (AIC) is derived, and its approximate version is proposed and tested. It serves as the main nonlinear part of the autoencoder. Its linear part computes the descriptors, while the decoder part computes the AIC for all particular signals from a considered family.To demonstrate the usefulness of the proposed methodology, we consider the problem of classifying signals based on knowledge about their shapes gathered by the descriptors. First, a general scheme of selecting an appropriate classifier from a predefined list is proposed. Then, the proposed approach is applied to classifying vibrations of a large excavator in an open pit mine.

Multistep Forecast Averaging with Stochastic and Deterministic Trends

This paper presents a new approach to constructing multistep combination forecasts in a nonstationary framework with stochastic and deterministic trends. Existing forecast combination approaches in the stationary setup typically target the in-sample asymptotic mean squared error (AMSE), relying on its approximate equivalence with the asymptotic forecast risk (AFR). Such equivalence, however, breaks down in a nonstationary setup. This paper develops combination forecasts based on minimizing an accumulated prediction errors (APE) criterion that directly targets the AFR and remains valid whether the time series is stationary or not. We show that the performance of APE-weighted forecasts is close to that of the optimal, infeasible combination forecasts. Simulation experiments are used to demonstrate the finite sample efficacy of the proposed procedure relative to Mallows/Cross-Validation weighting that target the AMSE as well as underscore the importance of accounting for both persistence and lag order uncertainty. An application to forecasting US macroeconomic time series confirms the simulation findings and illustrates the benefits of employing the APE criterion for real as well as nominal variables at both short and long horizons. A practical implication of our analysis is that the degree of persistence can play an important role in the choice of combination weights.

Instrumental variable estimation with first-stage heterogeneity

We propose a simple data-driven procedure that exploits heterogeneity in the first-stage correlation between an instrument and an endogenous variable to improve the asymptotic mean squared error (MSE) of instrumental variable estimators. We show that the resulting gains in asymptotic MSE can be quite large in settings where there is substantial heterogeneity in the first-stage parameters. We also show that a naive procedure used in some applied work, which consists of selecting the composition of the sample based on the value of the first-stage t-statistic, may cause substantial over-rejection of a null hypothesis on a second-stage parameter. We apply the methods to study (1) the return to schooling using the minimum school leaving age as the exogenous instrument and (2) the effect of local economic conditions on voter turnout using energy supply shocks as the source of identification.

Simultaneous bandwidths determination for DK-HAC estimators and long-run variance estimation in nonparametric settings

We consider the derivation of data-dependent simultaneous bandwidths for double kernel heteroscedasticity and autocorrelation consistent (DK-HAC) estimators. In addition to the usual smoothing over lagged autocovariances for classical HAC estimators, the DK-HAC estimator also applies smoothing over the time direction. We obtain the optimal bandwidths that jointly minimize the global asymptotic MSE criterion and discuss the tradeoff between bias and variance with respect to smoothing over lagged autocovariances and over time. Unlike the MSE results of Andrews, we establish how nonstationarity affects the bias-variance tradeoff. We use the plug-in approach to construct data-dependent bandwidths for the DK-HAC estimators and compare them with the DK-HAC estimators from Casini that use data-dependent bandwidths obtained from a sequential MSE criterion. The former performs better in terms of size control, especially with stationary and close to stationary data. Finally, we consider long-run variance (LRV) estimation under the assumption that the series is a function of a nonparametric estimator rather than of a semiparametric estimator that enjoys the usual rate of convergence. Thus, we also establish the validity of consistent LRV estimation in nonparametric parameter estimation settings.

GMM Model Averaging Using Higher Order Approximations

Moment conditions model averaging (MA) estimators in the GMM framework are considered. Under finite sample considerations, MA estimators with optimal weights are proposed, in the sense that weights minimize the corresponding higher-order asymptotic mean squared error (AMSE). It is shown that the higher-order AMSE objective function has a closed-form expression, which makes this procedure applicable in practice. In addition, and as an alternative, different averaging schemes based on moment selection criteria are considered, in which weights for averaging across GMM estimates can be obtained by direct smoothing or by numerical minimization of a specific criterion. Asymptotic properties assuming correctly specified models are derived and the performance of the proposed averaging approaches is contrasted with existing model selection alternatives i) analytically, for a simple IV example, and ii) by means of Monte Carlo experiments in a nonlinear setting, showing that MA compares favourably in many relevant setups. The usefulness of MA methods is illustrated by studying the effect of institutions on economic performance.

Choice of smoothing parameter in multivariate copula-based tail coefficients

Multivariate tail coefficients are of interest when one is trying to summarize dependence of extremes. The difficulty is that these coefficients are defined as limits and thus to estimate them from data one needs to choose a kind of smoothing parameter kn. A standard approach in such problems is to choose the smoothing parameter as a minimum of an estimated asymptotic mean squared error (AMSE). In this paper several ways of estimating AMSE are suggested and investigated. We focus on estimating (Frahm’s) extremal dependence coefficient. In order to derive AMSE of this estimator we provide its asymptotic representation. Two methods (fully or partially nonparametric) of estimating AMSE are suggested. An extensive simulation study shows that both suggested methods usually slightly outperform the available alternatives with the fully nonparametric method being recommended. Finally the different ways of choosing kn are illustrated on a real data set.

Recent Advances in Robust Design for Accelerated Failure Time Models with Type I Censoring

Many fields including clinical and manufacturing areas usually perform life-testing experiments and accelerated failure time models (AFT) play an essential role in these investigations. In these models the covariate causes an accelerant effect on the course of the event through the term named acceleration factor (AF). Despite the influence of this factor on the model, recent studies state that the form of AF is weakly or partially known in most real applications. In these cases, the classical optimal design theory may produce low efficient designs since they are highly model dependent. This work explores planning and techniques that can provide the best robust designs for AFT models with type I censoring when the form of the AF is misspecified, which is an issue little explored in the literature. Main idea is focused on considering the AF to vary over a neighbourhood of perturbation functions and assuming the mean square error matrix as the basis for measuring the design quality. A key result of this research was obtaining the asymptotic MSE matrix for type I censoring under the assumption of known variance regardless the selected failure time distribution. In order to illustrate the applicability of previous result to a study case, analytical characterizations and numerical approaches were developed to construct optimal robust designs under different contaminating scenarios for a failure time following a log-logistic distribution.

Modified-mean-shift-based noisy label detection for hyperspectral image classification

Labeling mistakes occur in the many real training sets of hyperspectral image (HSI) classification, due to mistakes in the collection of labeled training data samples phase. Label noise is an essential problem in classification, with many possible negative consequences. In this paper, a new method is proposed to detect and delete noisy label samples and therefore remove the effect of errors in the classification process. For this purpose, first, a modified mean shift (MMS) method originated from minimum Bayesian risk is proposed and used to improve the separability of the mislabeled samples from training sets. Then denoised training samples are given to the classification algorithms such as SVM, KNN, MLR, and KOMP to classify the HSI data sets. Then, the effect of several types of loss functions is investigated for the proposed MMS method. Also, the analysis of bounded differences inequality, mean square error (MSE), asymptotic mean square error (AMSE), and asymptotic normality are obtained for our method. The performance of the mentioned compared methods greatly improves at high levels of noise, when the MMS method are combined to them. The obtained experimental results show that the proposed MMS method improved the performance of these classification methods for real HSI data sets.

Smooth distribution function estimation for lifetime distributions using Szasz–Mirakyan operators

In this paper, we introduce a new smooth estimator for continuous distribution functions on the positive real half-line using Szasz–Mirakyan operators, similar to Bernstein’s approximation theorem. We show that the proposed estimator outperforms the empirical distribution function in terms of asymptotic (integrated) mean-squared error and generally compares favorably with other competitors in theoretical comparisons. Also, we conduct the simulations to demonstrate the finite sample performance of the proposed estimator.

Multi-Layer Bilinear Generalized Approximate Message Passing

In this paper, we extend the bilinear generalized approximate message passing (BiG-AMP) approach, originally proposed for high-dimensional generalized bilinear regression, to the multi-layer case for the handling of cascaded problem such as matrix-factorization problem arising in relay communication among others. Assuming statistically independent matrix entries with known priors, the new algorithm called ML-BiGAMP could approximate the general sum-product loopy belief propagation (LBP) in the high-dimensional limit enjoying a substantial reduction in computational complexity. We demonstrate that, in large system limit, the asymptotic MSE performance of ML-BiGAMP could be fully characterized via a set of simple one-dimensional equations termed state evolution (SE). We establish that the asymptotic MSE predicted by ML-BiGAMP' SE matches perfectly the exact MMSE predicted by the replica method, which is well-known to be Bayes-optimal but infeasible in practice. This consistency indicates that the ML-BiGAMP may still retain the same Bayes-optimal performance as the MMSE estimator in high-dimensional applications, although ML-BiGAMP's computational burden is far lower. As an illustrative example of the general ML-BiGAMP, we provide a detector design that could estimate the channel fading and the data symbols jointly with high precision for the two-hop amplify-and-forward relay communication systems.

Asymptotic risk and phase transition of $l_{1}$-penalized robust estimator

Mean square error (MSE) of the estimator can be used to evaluate the performance of a regression model. In this paper, we derive the asymptotic MSE of $l_{1}$-penalized robust estimators in the limit of both sample size $n$ and dimension $p$ going to infinity with fixed ratio $n/p\rightarrow \delta $. We focus on the $l_{1}$-penalized least absolute deviation and $l_{1}$-penalized Huber’s regressions. Our analytic study shows the appearance of a sharp phase transition in the two-dimensional sparsity-undersampling phase space. We derive the explicit formula of the phase boundary. Remarkably, the phase boundary is identical to the phase transition curve of LASSO which is also identical to the previously known Donoho–Tanner phase transition for sparse recovery. Our derivation is based on the asymptotic analysis of the generalized approximation passing (GAMP) algorithm. We establish the asymptotic MSE of the $l_{1}$-penalized robust estimator by connecting it to the asymptotic MSE of the corresponding GAMP estimator. Our results provide some theoretical insight into the high-dimensional regression methods. Extensive computational experiments have been conducted to validate the correctness of our analytic results. We obtain fairly good agreement between theoretical prediction and numerical simulations on finite-size systems.

Model-robust designs for nonlinear quantile regression.

We construct robust designs for nonlinear quantile regression, in the presence of both a possibly misspecified nonlinear quantile function and heteroscedasticity of an unknown form. The asymptotic mean-squared error of the quantile estimate is evaluated and maximized over a neighbourhood of the fitted quantile regression model. This maximum depends on the scale function and on the design. We entertain two methods to find designs that minimize the maximum loss. The first is local - we minimize for given values of the parameters and the scale function, using a sequential approach, whereby each new design point minimizes the subsequent loss, given the current design. The second is adaptive - at each stage, the maximized loss is evaluated at quantile estimates of the parameters, and a kernel estimate of scale, and then the next design point is obtained as in the sequential method. In the context of a Michaelis-Menten response model for an estrogen/hormone study, and a variety of scale functions, we demonstrate that the adaptive approach performs as well, in large study sizes, as if the parameter values and scale function were known beforehand and the sequential method applied. When the sequential method uses an incorrectly specified scale function, the adaptive method yields an, often substantial, improvement. The performance of the adaptive designs for smaller study sizes is assessed and seen to still be very favourable, especially so since the prior information required to design sequentially is rarely available.

Covariance-Aided CSI Acquisition With Non-Orthogonal Pilots in Massive MIMO: A Large-System Performance Analysis

Massive multiple-input multiple-output (MIMO) systems use antenna arrays with a large number of antenna elements to serve many different users simultaneously. The large number of antennas in the system makes, however, the channel state information (CSI) acquisition strategy design critical and particularly challenging. Interestingly, in the context of massive MIMO systems, channels exhibit a large degree of spatial correlation which results in strongly rank-deficient spatial covariance matrices at the base station (BS). With the final objective of analyzing the benefits of covariance-aided uplink multi-user CSI acquisition in massive MIMO systems, here we compare the channel estimation mean-square error (MSE) for (i) conventional CSI acquisition, which does not assume any knowledge on the user spatial covariance matrices and uses orthogonal pilot sequences; and (ii) covariance-aided CSI acquisition, which exploits the individual covariance matrices for channel estimation and enables the use of non-orthogonal pilot sequences. We apply a large-system analysis to the latter case, for which new asymptotic MSE expressions are established under various assumptions on the distributions of the pilot sequences and on the covariance matrices. We link these expressions to those describing the estimation MSE of conventional CSI acquisition with orthogonal pilot sequences of some equivalent length. This analysis provides insights on how much training overhead can be reduced with respect to the conventional strategy when a covariance-aided approach is adopted.

Estimation of Wideband Dynamic mmWave and THz Channels for 5G Systems and Beyond

Millimeter wave (mmWave) wideband channels in a multiple-input multiple-output (MIMO) transmission are described by a sparse set of impulse responses in the angle-delay, or space-time (ST), domain. These characteristics will be even more prominent in the THz band used in future systems. We consider two approaches for channel estimation: compressed-sensing (CS), exploiting the sparsity in the angular/delay domain, and low-rank (LR), exploiting the algebraic structure of channel matrix. Both approaches share several commonalities, and this paper provides for the first time i) a comparison of the two approaches, and ii) new versions of CS and LR methods that significantly improve performance in terms of mean squared error (MSE), computational complexity, and latency. We derive the asymptotic MSE bound for any estimator of the ST-MIMO multipath channels with invariant angles/delays and time-varying fading, with unknown angle/delay diversity order: the bound also accounts for the degradation introduced by sub-optimal separable channel models. We will show that in the considered scenarios both CS and LR approaches attain the bound. Our performance assessment over ideal and 3 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">rd</sup> generation partnership project (3GPP) channel models, suitable for the fifth-generation (5G) and beyond of cellular networks, shows the trade-off obtained by the methods over various metrics: i) CS methods are converging faster than the LR methods, both attaining the asymptotic MSE bound; ii) the CS methods depend on the array manifold, while LR methods are independent of the array calibration; iii) CS solutions are more complex than LR solutions.

Using penalized likelihood to select parameters in a random coefficients multinomial logit model

This paper is about estimating a random coefficients logit model in which the distribution of each coefficient is characterized by finitely many parameters, some of which may be zero. The paper gives conditions under which, with probability approaching 1 as the sample size increases, penalized maximum likelihood (PML) estimation with the adaptive LASSO (AL) penalty distinguishes correctly between zero and non-zero parameters. The paper also gives conditions under which PML reduces the asymptotic mean-square estimation error of any continuously differentiable function of the model’s parameters. The paper describes a method for computing PML estimates and presents the results of Monte Carlo experiments that illustrate their performance. It also presents the results of PML estimation of a random coefficients logit model of choice among brands of butter and margarine in the British groceries market.

Model Mis-Specification in Newsvendor Decisions: A Comparison of Frequentist Parametric, Bayesian Parametric and Nonparametric Approaches

We compare three different approaches studied by past literature on data-driven inventory optimization--- Frequentist Parametric (FP), Bayesian Parametric (BP) and Nonparametric--- for the newsvendor problem. For the Parametric approaches, we allow for mis-specification of the demand model. We prove, under mild regularity conditions, (i) asymptotic bias and variance formulas of FP and BP are equivalent, (ii) mis-specified Parametric approaches yield asymptotically biased decisions, unlike the correctly-specified Parametric approaches and the Nonparametric approach, and (iii) asymptotic variance of the mis-specified Parametric approaches converges to zero at rate $1/n$, in contrast to the $1/n^2$ rate for the correctly-specified Parametric approaches and the Nonparametric approach, where $n$ is the number of demand samples. We then show, for nine pairs of assumed versus true demand distribution pairs, (iv) asymptotic bias and variance formulas approximate finite-sample counterparts very well, (v) correctly-specified Parametric approaches dominate the Nonparametric approach in the asymptotic mean-squared error (AMSE) of the decision and the cost, and (vi) surprisingly, it is possible for mis-specified Parametric approaches to dominate the Nonparametric approach in the AMSE of the decision and the cost. We compare the approaches on a dataset from a large fresh food chain, and discuss the nuances of choosing the ``best'' approach.

Computational Implementation and Asymptotic Statistical Performance Analysis of Range Frequency Autocorrelation Function for Radar High-Speed Target Detection

The range frequency autocorrelation function (RFAF) based algorithm is proposed for radar target detection and motion parameter estimation in our previous work. In the RFAF-based method, the symmetric autocorrelation function (SAF) is constructed with respect to the range frequency, and three dimensional (slow time, range frequency, and shift frequency) energy accumulation can be completed coherently. In this article, as a further study of the RFAF-based method, the fast implementation and detailed asymptotic statistical performance analyses of RFAF are studied. First, on the basis of the frequency circular convolution theorem, the fast implementation is proposed. Then, to evaluate the anti-noise performance and estimation accuracy, the output signal to noise ratio (SNR) and asymptotic mean square errors (AMSEs) of estimated parameters are derived in closed forms. Theoretical analyses and numerical simulation results reveal that the RFAF-based method outperforms several state-of-the-art algorithms in terms of anti-noise performance and parameter estimation accuracy, and its computational efficiency is high.

Local Polynomial Order in Regression Discontinuity Designs

The local linear estimator has become the standard in the regression discontinuity design literature, but we argue that it should not always dominate other local polynomial estimators in empirical studies. We show that the local linear estimator in the data generating processes (DGP’s) based on two well- known empirical examples does not always have the lowest (asymptotic) mean squared error (MSE). Therefore, we advocate for a more flexible view towards the choice of the polynomial order, p, and suggest two complementary approaches for picking p: comparing the MSE of alternative estimators from Monte Carlo simulations based on an approximating DGP, and comparing the estimated asymptotic MSE using actual data.Length: 47 pages

Cramér-Rao Bound for DOA Estimators Under the Partial Relaxation Framework: Derivation and Comparison

A class of computationally efficient DOA estimators under the Partial Relaxation (PR) framework has recently been proposed. Conceptually different from conventional DOA estimation methods in the literature, the estimators under the PR framework rely on the non-complete relaxation of the array manifold while performing a spectral-search in the field of view. This particular type of relaxation essentially implies a modified signal model with partial information loss due to the relaxation. The information loss and its impact on the DOA estimation performance have not yet been analytically quantified in the literature. In this paper, the information loss induced by the relaxation of the array manifold is investigated through the Cramér-Rao Bound (CRB). The closed-form expression of the CRB for DOA estimation under the PR model, on the one hand, provides insight on the information loss in the asymptotic region where the number of snapshots tends to infinity. On the other hand, the proposed CRB characterizes the lower bound for the DOA estimation performance of all PR estimators. We prove that, under the assumptions of Gaussian source signal and noise, the CRB of the PR signal model is lower-bounded by the conventional stochastic CRB. We also prove that the previously proposed Weighted Subspace Fitting estimator under the PR framework asymptotically achieves the CRB of the PR signal model. Furthermore, it is shown that the asymptotic mean-squared errors of all Weighted Subspace Fitting estimators under the PR framework for any positive definite weighting matrix are identical.