Convergence Rate Of Estimator Research Articles

On Stability Estimate of Optimal Transport Maps Using m-th Polynomial Convexity

In 1781, Gaspard Monge first proposed the practical problem of relocating building materials while minimizing workers’ effort. Mathematically, the problem can be reiterated as finding a mapping T0 that transforms a random variable (X) following probability measure (μ) into a random variable (Y) following probability measure (ν), with minimal cost. Afterward, it has been widely studied and applied in statistics, machine learning, and economics, which concern the study of “distance” between usually a pair of probability distributions. The focus of this paper is centered on investigating and generalizing stability estimates for optimal transport plans, particularly through the lens of strong polynomial convexity. Building on previous research using plug-in estimators to strengthen the convergence rate of discrete or semi-discrete estimators for optimal transport plans, this paper introduces a novel stability estimate leveraging L-Lipschitz continuity and a paradigmatic methodology based on polynomial convexity, the understanding of which remains limited.

Minimax estimation of functional principal components from noisy discretized functional data

AbstractFunctional Principal Component Analysis is a reference method for dimension reduction of curve data. Its theoretical properties are now well understood in the simplified case where the sample curves are fully observed without noise. However, functional data are noisy and necessarily observed on a finite discretization grid. Common practice consists in smoothing the data and then to compute the functional estimates, but the impact of this denoising step on the procedure's statistical performance are rarely considered. Here we prove new convergence rates for functional principal component estimators. We introduce a double asymptotic framework: one corresponding to the sampling size and a second to the size of the grid. We prove that estimates based on projection onto histograms show optimal rates in a minimax sense. Theoretical results are illustrated on simulated data and the method is applied to the visualization of genomic data.

The Kullback–Leibler Divergence and the Convergence Rate of Fast Covariance Matrix Estimators in Galaxy Clustering Analysis

We present a method to quantify the convergence rate of the fast estimators of the covariance matrices in the large-scale structure analysis. Our method is based on the Kullback–Leibler (KL) divergence, which describes the relative entropy of two probability distributions. As a case study, we analyze the delete-d jackknife estimator for the covariance matrix of the galaxy correlation function. We introduce the information factor or the normalized KL divergence with the help of a set of baseline covariance matrices to diagnose the information contained in the jackknife covariance matrix. Using a set of quick particle mesh mock catalogs designed for the Baryon Oscillation Spectroscopic Survey DR11 CMASS galaxy survey, we find that the jackknife resampling method succeeds in recovering the covariance matrix with 10 times fewer simulation mocks than that of the baseline method at small scales (s ≤ 40 h −1 Mpc). However, the ability to reduce the number of mock catalogs is degraded at larger scales due to the increasing bias on the jackknife covariance matrix. Note that the analysis in this paper can be applied to any fast estimator of the covariance matrix for galaxy clustering measurements.

Partial Derivatives Estimation of Multivariate Variance Function in Heteroscedastic Model via Wavelet Method

For derivative function estimation, conventional research only focuses on the derivative estimation of one-dimensional functions. This paper considers partial derivatives estimation of a multivariate variance function in a heteroscedastic model. A wavelet estimator of partial derivatives of a multivariate variance function is proposed. The convergence rates of a wavelet estimator under different estimation errors are discussed. It turns out that the strong convergence rate of the wavelet estimator is the same as the optimal uniform almost sure convergence rate of nonparametric function problems.

Adaptively robust high-dimensional matrix factor analysis under Huber loss function

The explosion of data volume and the expansion in data dimensionality have led to a critical challenge in analyzing high-dimensional matrix time series for big data-related applications. In this regard, factor models for matrix-valued high-dimensional time series provide a powerful tool, that reduces the dimensionality of the variables with low-rank structures. However, existing high-dimensional matrix factor models suffer from two limitations in complex scenarios. One is that it is difficult to make robust inferences for datasets with heavy-tailed distributions. The other is that existing models require additional parameters for fine-tuning to guarantee performance. We propose an adaptively robust high-dimensional matrix factor model based on a specified Huber loss function to tackle the challenges mentioned above. An efficient iterative algorithm is provided to consistently determine the additional parameters of our model for robust estimation. The robustness of the model estimation is greatly improved by incorporating the Huber loss. Furthermore, we theoretically investigate the proposed method and derive the convergence rates of the robust estimators to examine its performance. Simulations show that the proposed method outperforms previous models in the estimation of heavy-tailed data. A real-world data analysis on a financial portfolio dataset illustrates that the method can be used to extract useful knowledge from high-dimensional matrix time series.

Regression analysis of longitudinal data with mixed synchronous and asynchronous longitudinal covariates

In linear models, omitting a covariate that is orthogonal to covariates in the model does not result in biased coefficient estimation. This generally does not hold for longitudinal data, where additional assumptions are needed to get an unbiased coefficient estimation in addition to the orthogonality between omitted longitudinal covariates and longitudinal covariates in the model. We propose methods to mitigate the omitted variable bias under weaker assumptions. A two-step estimation procedure is proposed to infer the asynchronous longitudinal covariates when such covariates are observed. For mixed synchronous and asynchronous longitudinal covariates, we get a parametric convergence rate for the coefficient estimation of the synchronous longitudinal covariates by the two-step method. Extensive simulation studies provide numerical support for the theoretical findings. We illustrate the performance of our method on a dataset from the Alzheimer’s Disease Neuroimaging Initiative study.

Penalised estimation of partially linear additive zero-inflated Bernoulli regression models

We develop a practical and computationally efficient penalised estimation approach for partially linear additive models to zero-inflated binary outcome data. To facilitate estimation, B-splines are employed to approximate unknown nonparametric components. A two-stage iterative expectation-maximisation (EM) algorithm is proposed to calculate penalised spline estimates. The large-sample properties such as the uniform convergence and the optimal rate of convergence for functional estimators, and the asymptotic normality and efficiency for regression coefficient estimators are established. Further, two variance-covariance estimation approaches are proposed to provide reliable Wald-type inference for regression coefficients. We conducted an extensive Monte Carlo study to evaluate the numerical properties of the proposed penalised methodology and compare it to the competing spline method [Li and Lu. ‘Semiparametric Zero-Inflated Bernoulli Regression with Applications’, Journal of Applied Statistics, 49, 2845–2869]. The methodology is further illustrated by an egocentric network study.

Evidence of Strain Accumulation and Coupling Variation in the Himachal Region of NW Himalaya From Short Term Geodetic Measurements

AbstractHimachal region of Northwest Himalaya exhibits the widest structural re‐entrant in Kangra region and significant strain partitioning along the frontal and hinterland out‐of‐sequence faults. We report results of continuous GPS measurements from 10 new sites in the region and analyze them along with the previously published results to constrain the ongoing arc‐normal and arc‐parallel convergence rates at 16.5 ± 1.1 and 4–5 mm/yr respectively. Thus, the ongoing convergence is oblique by 15°–20°. The Main Himalayan Thrust (MHT) is strongly coupled up to ∼100 km from the Main Frontal Thrust but displays significant variation in coupling in the transition zone across the Kangra re‐entrant and the adjoining western salient. Joint analysis of the coupling variation, the geologically inferred MHT geometry variations and the local topographic anomaly pattern strongly suggest the possibility of a potentially active, strain accumulating segment of MBT along the southern margin of Dhauladhar ranges in Western Himachal region, which is also proposed to be influencing the long‐term topographic growth in the region. Although a general agreement is observed between the long‐term shortening rates along the active faults and the estimated geodetic convergence in this region, the ensuing discussion highlights their complex relationship in terms of temporal and spatial variability in the fault activity and elastic‐inelastic deformation. We use the fault orientation and the estimated convergence rate to geometrically constrain a mean dextral slip‐rate of 4.4–5.7 mm/yr along a recently discovered Khetpurali‐Taksal fault, which is proposed to partition the majority of ongoing arc‐parallel deformation along it.

Adaptive Model Predictive Control Scheme Based on Non-Minimal State Space Representation

The model predictive control (MPC) technique is widely employed in process industries as a control scheme. The quality of the model used greatly influences the performance of the MPC. In time-varying systems, the plant model plays a critical role in determining the controller’s performance, as the controller’s control action relies on an adaptive model. Therefore, updating the system parameters rapidly and symmetrically in time-varying systems becomes necessary. To address this need, in the proposed work, a non-minimal state space model of a time-varying system is utilized for parameter estimation, and these parameters are updated at every sampling instant using a multi-innovation recursive least squares (MIRLS) scheme, which enables the timely updates of system parameters. We have explored various extensions of the recursive least square (RLS) scheme, such as the multi-innovation recursive least squares (MIRLS) method. This extension aims to achieve a higher convergence rate for parameter estimation. Furthermore, we have focused on the parameter estimation of a non-minimal state space time-varying system, where the system parameters change at each time interval. Additionally, we have incorporated a time-varying objective function into the MPC formulations, which enables adaptability to change the system dynamics. To demonstrate the applicability of our proposed approach, we have conducted simulation experiments using a benchmark time-varying model. These experiments showcase the effectiveness and benefits of our proposed methodology in dealing with time-varying systems.

An Adaptive and Robust Deep Learning Framework for THz Ultra-Massive MIMO Channel Estimation

Terahertz ultra-massive MIMO (THz UM-MIMO) is envisioned as one of the key enablers of 6G wireless networks, for which channel estimation is highly challenging. Traditional analytical estimation methods are no longer effective, as the enlarged array aperture and the small wavelength result in a mixture of far-field and near-field paths, constituting a hybrid-field channel. Deep learning (DL)-based methods, despite the competitive performance, generally lack theoretical guarantees and scale poorly with the size of the array. In this paper, we propose a general DL framework for THz UM-MIMO channel estimation, which leverages existing iterative channel estimators and is with provable guarantees. Each iteration is implemented by a fixed point network (FPN), consisting of a closed-form linear estimator and a DL-based non-linear estimator. The proposed method perfectly matches the THz UM-MIMO channel estimation due to several unique advantages. First, the complexity is low and adaptive. It enjoys provable linear convergence with a low per-iteration cost and monotonically increasing accuracy, which enables an adaptive accuracy-complexity tradeoff. Second, it is robust to practical distribution shifts and can directly generalize to a variety of heavily out-of-distribution scenarios with almost no performance loss, which is suitable for the complicated THz channel conditions. For practical usage, the proposed framework is further extended to wideband THz UM-MIMO systems with beam squint effect. Theoretical analysis and extensive simulation results are provided to illustrate the advantages over the state-of-the-art methods in estimation accuracy, convergence rate, complexity, and robustness.

Neural Networks for Partially Linear Quantile Regression

Deep learning has enjoyed tremendous success in a variety of applications but its application to quantile regression remains scarce. A major advantage of the deep learning approach is its flexibility to model complex data in a more parsimonious way than nonparametric smoothing methods. However, while deep learning brought breakthroughs in prediction, it is not well suited for statistical inference due to its black box nature. In this article, we leverage the advantages of deep learning and apply it to quantile regression where the goal is to produce interpretable results and perform statistical inference. We achieve this by adopting a semiparametric approach based on the partially linear quantile regression model, where covariates of primary interest for statistical inference are modeled linearly and all other covariates are modeled nonparametrically by means of a deep neural network. In addition to the new methodology, we provide theoretical justification for the proposed model by establishing the root-n consistency and asymptotically normality of the parametric coefficient estimator and the minimax optimal convergence rate of the neural nonparametric function estimator. Across several simulated and real data examples, the proposed model empirically produces superior estimates and more accurate predictions than various alternative approaches.

A spline-assisted semiparametric approach to nonparametric measurement error models

A spline-assisted approach is proposed to handle the measurement error problem in treating the pollution and asthma data. It is well known that the minimax rate of convergence in nonparametric regression function estimation of a random variable measured with error is much slower than the rate in the error free case. A different problem is considered. It is shown that if one is willing to impose a relatively mild assumption in requiring that the error-prone variable has a compact support, then standard nonparametric results are obtainable for measurement error models. New and constructive methods to take full advantage of the compact support assumption via spline-assisted semiparametric methods are proposed. It is proven that the new estimator achieves the usual nonparametric rate as if there were no measurement error. Furthermore it is shown that similar spline approach can be used to assess the probability density function on a compact set, using observations with error, and the resulting estimator differs from the true density function only by a constant scale. In addition, it retains the nonparametric density convergence properties of the error free case. The performance of the new methods is demonstrated through simulations and the methods are implemented to analyze the relation between asthma and pollution.

Enhancing model estimation accuracy and convergence rate in hysteresis modeling of MFC actuators using modified differential evolution algorithm

This paper presents a study on improving the estimation accuracy and convergence rate of hysteresis modeling of MFC actuators using mutation enhanced differential evolution (MEDE) algorithm, a modified version of the differential evolution algorithm. The proposed MEDE algorithm uses three mutation strategies, i.e., best, rand, and pbest. To model the secondary path of a smart flexible beam with MFC actuators, a Hammerstein model that combines an asymmetric Bouc-Wen model with an ARX model connected in series is proposed. The fitness function values of the Hammerstein model are compared with evolutionary algorithms.

Uniform almost sure convergence rate of wavelet estimator for regression model with mixed noise

ABSTRACT This article considers a non parametric estimation problem in a regression model with mixed noise. A wavelet estimator is proposed by using projection of wavelet coefficients estimators. Uniform almost sure convergence rate of this wavelet estimator is derived under some mild conditions. It should be pointed out that the convergence rate of the wavelet estimator coincides with the optimal strong uniform convergence rate of non parametric estimations. Finally, simulation studies illustrate the good performances of the wavelet estimator.

Uniform and [formula omitted] convergences for nonparametric continuous time regressions with semiparametric applications

We obtain uniform and Lp convergence rates of kernel type nonparametric estimators for the instantaneous conditional mean and variance functions of continuous time regressions, where the regressor is assumed to be a general recurrent diffusion. Our asymptotics are developed under a general set-up, with a shrinking sampling interval and an increasing time span, and without the stationarity assumption. Based on our convergence results, we develop a semiparametric inferential procedure for continuous time predictive regressions. In particular, a robust semiparametric likelihood ratio test for linear predictability is proposed, with its limit distribution established. We also apply our convergence results to obtain the asymptotics of a semiparametric maximum likelihood estimator of the drift of recurrent diffusions. In our simulation study, we examine the finite sample performance of our robust test against several existing tests in the literature. An empirical illustration is presented to test the predictability of the excess returns of two major stock indices using the commonly used dividend–price ratio and earnings–price ratio as the predictor.

Tail Spectral Density Estimation and Its Uncertainty Quantification: Another Look at Tail Dependent Time Series Analysis

We consider the estimation and uncertainty quantification of the tail spectral density, which provide a foundation for tail spectral analysis of tail dependent time series. The tail spectral density has a particular focus on serial dependence in the tail, and can reveal dependence information that is otherwise not discoverable by the traditional spectral analysis. Understanding the convergence rate of tail spectral density estimators and finding rigorous ways to quantify their statistical uncertainty, however, still stand as a somewhat open problem. The current article aims to fill this gap by providing a novel asymptotic theory on quadratic forms of tail statistics in the double asymptotic setting, based on which we develop the consistency and the long desired central limit theorem for tail spectral density estimators. The results are then used to devise a clean and effective method for constructing confidence intervals to gauge the statistical uncertainty of tail spectral density estimators, and it can be turned into a visualization tool to aid practitioners in examining the tail dependence for their data of interest. Numerical examples including data applications are presented to illustrate the developed results.

A modified Salp Swarm Algorithm for parameter estimation of fractional-order chaotic systems

For the parameter estimation problem in research related to the fractional-order chaotic systems (FOCSs), a modified optimization algorithm based on Salp Swarm Algorithm (SSA) was developed in this paper. The proposed algorithm introduced several improvements on SSA: adding a grouping step, introducing “betrayal” behavior, and improving the update method of the followers. We applied multiple classical optimization algorithms to conduct the parameter estimation experiments on the fractional-order Lorenz chaotic system (Lorenz-FOCS) and the fractional-order Financial chaotic system (Financial-FOCS). In addition, we explored the impact of searching space on parameters estimation through experiments. The experimental results confirmed the feasibility of the modified Salp Swarm Algorithm (MSSA). The MSSA performed better than the SSA and other classical optimization algorithms in terms of the estimation accuracy and convergence rate.

Robust transfer learning of high-dimensional generalized linear model

This paper studies transfer learning of a high-dimensional generalized linear model with the target model as well as source data from different but possibly related models. Both known and unknown transferable domain settings are considered. On the one hand, an improved two-step transfer learning algorithm is proposed and the optimal rate of convergence for estimation is proved when the set of transferable domain is known. On the other hand, when the set of transferable domain is unknown, we propose a data-driven procedure for transfer learning, called Stepwise Selection algorithm, and investigate its finite-sample performance through simulations studies. Experimental results on six datasets demonstrate that the proposed method can perform better.

Estimation and Inference for High-Dimensional Generalized Linear Models with Knowledge Transfer

Transfer learning provides a powerful tool for incorporating data from related studies into a target study of interest. In epidemiology and medical studies, the classification of a target disease could borrow information across other related diseases and populations. In this work, we consider transfer learning for high-dimensional generalized linear models (GLMs). A novel algorithm, TransHDGLM, that integrates data from the target study and the source studies is proposed. Minimax rate of convergence for estimation is established and the proposed estimator is shown to be rate-optimal. Statistical inference for the target regression coefficients is also studied. Asymptotic normality for a debiased estimator is established, which can be used for constructing coordinate-wise confidence intervals of the regression coefficients. Numerical studies show significant improvement in estimation and inference accuracy over GLMs that only use the target data. The proposed methods are applied to a real data study concerning the classification of colorectal cancer using gut microbiomes, and are shown to enhance the classification accuracy in comparison to methods that only use the target data.

Estimation of the invariant density for discretely observed diffusion processes: impact of the sampling and of the asynchronicity

We aim at estimating in a non-parametric way the density π of the stationary distribution of a d-dimensional stochastic differential equation , for , from the discrete observations of a finite sample ,…, with . We propose a kernel density estimator and we study its convergence rates for the pointwise estimation of the invariant density under anisotropic Hölder smoothness constraints. First of all, we find some conditions on the discretization step that ensures it is possible to recover the same rates as if the continuous trajectory of the process was available. As proven in the recent work [Amorino C, Gloter A. Minimax rate of estimation for invariant densities associated to continuous stochastic differential equations over anisotropic Holder classes; 2021. arXiv preprint arXiv:2110.02774], such rates are optimal and new in the context of density estimator. Then we deal with the case where such a condition on the discretization step is not satisfied, which we refer to as the intermediate regime. In this new regime we identify the convergence rate for the estimation of the invariant density over anisotropic Hölder classes, which is the same convergence rate as for the estimation of a probability density belonging to an anisotropic Hölder class, associated to n iid random variables . After that we focus on the asynchronous case, in which each component can be observed at different time points. Even if the asynchronicity of the observations complexifies the computation of the variance of the estimator, we are able to find conditions ensuring that this variance is comparable to the one of the continuous case. We also exhibit that the non-synchronicity of the data introduces additional bias terms in the study of the estimator.