Convergence Rate Of Estimator Research Articles

Transfer Learning for High-Dimensional Linear Regression: Prediction, Estimation and Minimax Optimality.

This paper considers estimation and prediction of a high-dimensional linear regression in the setting of transfer learning where, in addition to observations from the target model, auxiliary samples from different but possibly related regression models are available. When the set of informative auxiliary studies is known, an estimator and a predictor are proposed and their optimality is established. The optimal rates of convergence for prediction and estimation are faster than the corresponding rates without using the auxiliary samples. This implies that knowledge from the informative auxiliary samples can be transferred to improve the learning performance of the target problem. When the set of informative auxiliary samples is unknown, we propose a data-driven procedure for transfer learning, called Trans-Lasso, and show its robustness to non-informative auxiliary samples and its efficiency in knowledge transfer. The proposed procedures are demonstrated in numerical studies and are applied to a dataset concerning the associations among gene expressions. It is shown that Trans-Lasso leads to improved performance in gene expression prediction in a target tissue by incorporating data from multiple different tissues as auxiliary samples.

Over-parametrized deep neural networks minimizing the empirical risk do not generalize well

Recently it was shown in several papers that backpropagation is able to find the global minimum of the empirical risk on the training data using over-parametrized deep neural networks. In this paper, a similar result is shown for deep neural networks with the sigmoidal squasher activation function in a regression setting, and a lower bound is presented which proves that these networks do not generalize well on a new data in the sense that networks which minimize the empirical risk do not achieve the optimal minimax rate of convergence for estimation of smooth regression functions.

Strong convergence rates of multiple change-point estimator for ρ-mixing sequence

We consider the detection of multiple change-points for ρ-mixing sequence. A CUSUM-type method is proposed to estimate the multiple mean change-points and the strong consistency of the multiple mean change-point estimation is proved when the number of change-point is known. Furthermore, we establish the strong convergence rate of this change-point estimation under mild conditions and simulations of multiple change-point detection are also presented.

A statistical learning assessment of Huber regression

As one of the triumphs and milestones of robust statistics, Huber regression plays an important role in robust inference and estimation. It has also been finding a great variety of applications in machine learning. In a parametric setup, it has been extensively studied. However, in the statistical learning context where a function is typically learned in a nonparametric way, there is still a lack of theoretical understanding of how Huber regression estimators learn the conditional mean function and why it works in the absence of light-tailed noise assumptions. To address these fundamental questions, this paper conducts an assessment of Huber regression from a statistical learning viewpoint. First, we show that the usual risk consistency property of Huber regression estimators, which is usually pursued in machine learning, cannot guarantee their learnability in mean regression. Second, we argue that Huber regression should be implemented in an adaptive way to perform mean regression, implying that one needs to tune the scale parameter in accordance with the sample size and the moment condition of the noise. Third, with an adaptive choice of the scale parameter, we demonstrate that Huber regression estimators can be asymptotic mean regression calibrated under (1+ε)-moment conditions (ε>0) on the conditional distribution. Last but not least, under the same moment conditions, we establish almost sure convergence rates for Huber regression estimators. Note that the (1+ε)-moment conditions accommodate the special case where the response variable possesses infinite variance and so the established convergence rates justify the robustness feature of Huber regression estimators. In the above senses, the present study provides a systematic statistical learning assessment of Huber regression estimators and justifies their merits in terms of robustness from a theoretical viewpoint.

Nonparametric Pointwise Estimation for a Regression Model with Multiplicative Noise

In this paper, we consider a general nonparametric regression estimation model with the feature of having multiplicative noise. We propose a linear estimator and nonlinear estimator by wavelet method. The convergence rates of those regression estimators under pointwise error over Besov spaces are proved. It turns out that the obtained convergence rates are consistent with the optimal convergence rate of pointwise nonparametric functional estimation.

Spline estimation of partially linear regression models for time series with correlated errors

Partially linear regression smoothing is a useful technique for modeling time series. Using polynomial splines and a weighted least squares method, this study investigates a class of partially linear regression models of time series with correlated errors. -consistency of parametric estimators and the convergence rate of the nonparametric estimator are derived under some suitable conditions. Simulations reveal that the proposed approach is more valid than that of ignoring correlated errors. Moreover, the importance of considering autoregressive errors is illustrated by making multi-step-ahead forecasts for Australian blow-fly data.

Application of Adaptive Robust CKF in SINS/GPS Initial Alignment with Large Azimuth Misalignment Angle

When the strapdown inertial navigation system does not perform coarse alignment, the misalignment angle is generally a large angle, and a nonlinear error model and a nonlinear filtering method are required. For large azimuth misalignment, the initial alignment technology with a large azimuth misalignment angle is researched in this paper. The initial alignment technology with a large azimuth misalignment angle is researched in this paper. First, the SINS/GPS nonlinear error model is established. Secondly, in the view of observation gross errors and inaccurate noise statistical characteristics, an adaptive robust CKF algorithm is proposed. Finally, according to the simulation analysis and experiment, the adaptive robust CKF algorithm can augment the stability and improve the filter estimation precision and convergence rate.

Ensemble Estimation of Generalized Mutual Information With Applications to Genomics

Mutual information is a measure of the dependence between random variables that has been used successfully in myriad applications in many fields. Generalized mutual information measures that go beyond classical Shannon mutual information have also received much interest in these applications. We derive the mean squared error convergence rates of kernel density-based plug-in estimators of general mutual information measures between two multidimensional random variables $\mathbf{X}$ and $\mathbf{Y}$ for two cases: 1) $\mathbf{X}$ and $\mathbf{Y}$ are continuous; 2) $\mathbf{X}$ and $\mathbf{Y}$ may have any mixture of discrete and continuous components. Using the derived rates, we propose an ensemble estimator of these information measures called GENIE by taking a weighted sum of the plug-in estimators with varied bandwidths. The resulting ensemble estimators achieve the $1/N$ parametric mean squared error convergence rate when the conditional densities of the continuous variables are sufficiently smooth. To the best of our knowledge, this is the first nonparametric mutual information estimator known to achieve the parametric convergence rate for the mixture case, which frequently arises in applications (e.g. variable selection in classification). The estimator is simple to implement and it uses the solution to an offline convex optimization problem and simple plug-in estimators. A central limit theorem is also derived for the ensemble estimators and minimax rates are derived for the continuous case. We demonstrate the ensemble estimator for the mixed case on simulated data and apply the proposed estimator to analyze gene relationships in single cell data.

Uniform convergence rates for wavelet curve estimation in sup-norm loss

This paper presents the rates of uniform strong consistency of wavelet estimation for nonparametric function in sup-norm loss by introducing an empirical process approach. A compact support assumption on the explanatory variable is commonly used in nonparametric regression analysis. In the article, we consider the wavelet estimation analysis without any assumption on the compacity of the support of the explanatory variable. The optimal uniform convergence rates of the wavelet estimators are achieved by suitably choosing resolution level. These results are useful for wavelet theory on nonparametric signal recovery and analysis.

Inference for additive model at response random missing based on principal component imputation

In this paper, we consider statistical inference for the additive model when the response is missing at random. The paper proposes a framework for statistical inference under missing data, which integrates basis function approximation and principal component imputed estimation equation (PCIEE). Inverse probability weighting is used to construct robust estimator. Under mild assumptions, the convergence rate of component function estimator is proved. In addition, when the sample size n tends to infinity, the PCIEE-based variable selection can select the real predictor. We also demonstrated the performance of PCIEE through numerical experiments.

Multivariate wavelet density estimation for strong mixing stratified size-biased sample

This paper considers wavelet estimations of a multivariate density function based on stratified size-biased and strong mixing data. We provide upper bounds of the mean integrated squared error for linear and nonlinear wavelet estimators in Besov space It is shown that the linear estimator achieves the optimal convergence rate in the case of Moreover, the convergence rate of nonlinear estimator coincides with the optimal convergence rate up to a factor for In addition, the nonlinear wavelet estimator is adaptive.

Minimax Rates of ℓp-Losses for High-Dimensional Linear Errors-in-Variables Models over ℓq-Balls.

In this paper, the high-dimensional linear regression model is considered, where the covariates are measured with additive noise. Different from most of the other methods, which are based on the assumption that the true covariates are fully obtained, results in this paper only require that the corrupted covariate matrix is observed. Then, by the application of information theory, the minimax rates of convergence for estimation are investigated in terms of the -losses under the general sparsity assumption on the underlying regression parameter and some regularity conditions on the observed covariate matrix. The established lower and upper bounds on minimax risks agree up to constant factors when , which together provide the information-theoretic limits of estimating a sparse vector in the high-dimensional linear errors-in-variables model. An estimator for the underlying parameter is also proposed and shown to be minimax optimal in the -loss.

Comparison of four bio-inspired algorithms to optimize KNEA for predicting monthly reference evapotranspiration in different climate zones of China

Accurate estimation of reference crop evapotranspiration (ET0) is of great significance to crop water use and agricultural water resources management. This study evaluated the performance of four bio-inspired algorithm optimized kernel-based nonlinear extension of Arps decline (KNEA) models, namely KNEA with grasshopper optimization algorithm (GOA-KNEA), KNEA with grey wolf optimizer algorithm (GWO-KNEA), KNEA with particle swarm optimization algorithm (PSO-KNEA), and KNEA with salp swarm algorithm (SSA-KNEA), on estimating monthly ET0 across China. Monthly meteorological data [including maximum air temperature (Tmax), minimum air temperature (Tmin), extra-terrestrial solar radiation (Ra), relative humidity (RH), global solar radiation (Rs), wind speed (U)] during 1966–2015 from 51 weather stations across the seven different climate zones of China were used for model training and testing. Four different combinations of meteorological data were applied as model input, and results from the FAO-56 Penman-Monteith formula were used as a control. Results showed that the GWO-KNEA model overall performed better than the other three coupling models, of which the GWO-KNEA2 model (i.e., the model with input combination 2) was the best (on average R2 = 0.9814, RMSE = 0.2143 mm d−1). The convergence rate and the population size of the GWO-KNEA model were also superior to the other three models. Among input combinations, models with combination 2 had the best overall performance, while models with combination 3 were the worst on average. In terms of the importance of each meteorological parameter contributing to model accuracy, Rs was greater than Ra, RH, or U. Among different climate zones, station specific models in the semi-arid steppe of Inner Mongolia showed the best estimating performance in general, while models in the Qinghai-Tibetan Plateau overall performed relatively poorly. The GOA-KNEA and SSA-KNEA models with the combination 4 showed large increases (29.7% and 28.7%, respectively), indicating a problem of overfitting. Conversely, the GWO-KNEA model was the most stable model with the smallest increase in RMSE during the testing phase (4.4–15.7%) among all models. In summary, taking the model's estimation accuracy, stability, convergence rate, and climate environment into account, our results suggest that GWO-KNEA would be a suitable model for estimating ET0 in different climate zones of China and the most practical input combination would be combination 2.

Perbandingan Estimator Histogram dan Estimator Kernel

Salah satu hal penting dalam analisis statistik adalah prosedur estimasi suatu fungsi padat peluang yang biasa disebut fungsi densitas. Ada dua metode pendekatan yang biasanya digunakan, yaitu pendekatan parameter yang terkait dengan asumsi distribusi tertentu dan metode estimasi densitas secara non parametrik. Metode non parametrik yang sering kita jumpai adalah metode histogram.
 Beberapa kelemahan metode histogram menjadi acuan untuk dikembangkannya metode yang lain yaitu metode kernel, dimana estimator densitas kernel ini mempunyai parameter yang perlu diestimasi yaitu bandwidth h. Yang menjadi rumusan masalah dalam kajian pustaka ini adalah bagaimana memilih bandwidth dari estimator densitas kernel pada suatu fungsi densitas f di R dan membandingkannya dengan estimator histogram.
 Kesimpulan yang dapat diambil antara lain: (1) Estimasi densitas histogram adalah:
 dengan , (2) Estimator densitas kernel adalah :
 , (3) Estimator kernel mengatasi kelemahan- kelemahan dari estimator histogram, (4) Tingkat konvergensi dari estimator kernel lebih baik dari pada histogram, (5) Pemilihan bandwidth dengan metode unbiased cross validation (least cross validation) secara asymtotik menghasilkan bandwidth yang optimum

One step ahead adaptive control technique featured by fuzzy-based least square estimator for wind systems

The present paper proposes the application to a wind system of the One Step Ahead control scheme featured by a Fuzzy-based Least Square Estimator. The considered wind system power generation supplies an electrical load disconnected from the power supply grid. It is composed of a three bladed horizontal-axis wind turbine which drives a synchronous generator by means of gearbox: the mathematical model for both the horizontal-axis wind-turbine and the synchronous generator will be briefly outlined. The adaptive nature of the One-Step-Ahead control algorithm relies on providing consistent estimation of the controlled system. This is achieved by means of a Least Square Algorithm that is able to provide a good estimation of a linear discrete time model of the controlled system, nevertheless its convergence rate reduces very quickly. For this reason, Least Square Algorithm needs a resetting strategy, which allows the achievement of a compromise between estimation accuracy and convergence rate. This not only represents a very problem-dependent issue but also introduces weaknesses in term of control tracking errors, which in turns needs an extra control contribution (integral correction). The proposed Least Square Algorithm enhancement overcomes these issues managing differently the estimation accuracy and the convergence rate. In the Results section, the achievements of the application of the One-Step-Ahead algorithm to the wind system will prove the reliability of the suggested enhanced control technique.

Homogeneity Test of Many-to-One Risk Differences for Correlated Binary Data under Optimal Algorithms

In clinical studies, it is important to investigate the effectiveness of different therapeutic designs, especially, multiple treatment groups to one control group. The paper mainly studies homogeneity test of many-to-one risk differences from correlated binary data under optimal algorithms. Under Donner’s model, several algorithms are compared in order to obtain global and constrained MLEs in terms of accuracy and efficiency. Further, likelihood ratio, score, and Wald-type statistics are proposed to test whether many-to-one risk differences are equal based on optimal algorithms. Monte Carlo simulations show the performance of these algorithms through the total averaged estimation error, SD, MSE, and convergence rate. Score statistic is more robust and has satisfactory power. Two real examples are given to illustrate our proposed methods.

Convergence rate of estimators of clustered panel models with misclassification

We study kmeans clustering estimation of panel data models with a latent group structure and N units and T time periods under long panel asymptotics. We show that the group-specific coefficients can be estimated at the parametric NT-rate even if error variances diverge as T→∞ and consequently some units are asymptotically misclassified. This limit case approximates empirically relevant settings and is not covered by existing asymptotic results.

Time-varying threshold cointegration with an application to the Fisher hypothesis

Abstract This paper extends the threshold cointegration model developed by Gonzalo, J., and J. Y. Pitarakis. 2006. “Threshold Effects in Cointegrating Relationships.” Oxford Bulletin of Economics & Statistics 68: 813–33 and Chen, H. 2015. “Robust Estimation and Inference for Threshold Models with Integrated Regressors.” Econometric Theory 31 (4): 778–810 to allow for a time-varying threshold, which is a function of candidate variables that affect the separation of regimes. We derive the asymptotic distribution of the proposed least-square estimator of the threshold, and study the convergence rate of the threshold estimator. We also suggest test statistics for threshold effect and threshold constancy. Monte Carlo simulations point out that the convergence rate of the threshold estimator is consistent with the asymptotic theory, and the proposed tests have good size and power properties. The empirical usefulness of the proposed model is illustrated by an application to the US data to investigate the Fisher hypothesis.

ON THE CONVERGENCE RATE OF POTENTIALS OF BRENIER MAPS

The theory of optimal transportation has experienced a sharp increase in interest in many areas of economic research such as optimal matching theory and econometric identification. A particularly valuable tool, due to its convenient representation as the gradient of a convex function, has been the Brenier map: the matching obtained as the optimizer of the Monge–Kantorovich optimal transportation problem with the euclidean distance as the cost function. Despite its popularity, the statistical properties of the Brenier map have yet to be fully established, which impedes its practical use for estimation and inference. This article takes a first step in this direction by deriving a convergence rate for the simple plug-in estimator of the potential of the Brenier map via the semi-dual Monge–Kantorovich problem. Relying on classical results for the convergence of smoothed empirical processes, it is shown that this plug-in estimator converges in standard deviation to its population counterpart under the minimax rate of convergence of kernel density estimators if one of the probability measures satisfies the Poincaré inequality. Under a normalization of the potential, the result extends to convergence in the $L^2$ norm, while the Poincaré inequality is automatically satisfied. The main mathematical contribution of this article is an analysis of the second variation of the semi-dual Monge–Kantorovich problem, which is of independent interest.

Robust estimation of superhedging prices

We consider statistical estimation of superhedging prices using historical stock returns in a frictionless market with $d$ traded assets. We introduce a plug-in estimator based on empirical measures and show it is consistent but lacks suitable robustness. To address this, we propose novel estimators which use a larger set of martingale measures defined through a tradeoff between the radius of Wasserstein balls around the empirical measure and the allowed norm of martingale densities. We then extend our study, in part, to estimation of risk measures, to the case of markets with traded options, to a multi-period setting and to settings with model uncertainty. We also study convergence rates of estimators and convergence of super-hedging strategies.