Estimation For Single-index Models Research Articles

Ultrahigh dimensional single index model estimation via refitted cross-validation

This article develops a new estimation procedure for ultrahigh dimensional sparse single index models. We first use B-spline to approximate the link function, and a naive two-stage estimator can be applied for the estimation of single index models. However, the direct method may not perform well in ultrahigh dimensional data due to the spurious correlations, and the asymptotic results show that it may significantly underestimate the error variance and have greater estimation bias of the regressors coefficients. We further propose an accurate estimate for error variance and parameters in ultrahigh dimensional sparse single index model by effectively integrating sure independence screening and refitted cross-validation (RCV) techniques. The consistency and asymptotic properties of the resulting estimate are established under some regularity conditions. The simulation studies are carried out to study the finite sample performance of the newly proposed methods.

Efficient estimation in heteroscedastic single-index models

In this article, we focus on the efficient estimation in single-index models with heteroscedastic errors. We first develop a nonparametric estimator of the variance function based on a fully nonparametric function or a dimension reduction structure, and the resulting estimator is consistent. Then, we propose a reweighting estimator of the parametric component via taking the estimated variance function into account, and the main results show that it has a smaller asymptotic variance than the naive estimator that neglects the heteroscedasticity. Simulation studies are conducted to evaluate the efficacy of the proposed methodologies, and an analysis of a real data example is provided for illustration.

Robust estimation of single index models with responses missing at random

A single-index regression model is considered, where some responses in the model are assumed to be missing at random. Local linear rank-based estimators of the single-index direction and the unknown link function are proposed. Asymptotic properties of the estimators are established under mild regularity conditions. Monte Carlo simulation experiments show that the proposed estimators are more efficient than their least squares counterparts especially when the data are derived from contaminated or heavy-tailed model error distributions. When the errors follow a normal distribution, the least squares index direction estimator tends to be more efficient than the rank-based index direction estimator; however, the least squares link function estimator remains less efficient than the rank-based link function estimator. A real data example is analyzed and cross-validation studies show that the proposed procedure provides better prediction than the least squares method when the responses contain outliers and are missing at random.

Adaptive semiparametric estimation for single index models with jumps

The single index model is one of the most popular semiparametric models in applied quantitative sciences. This paper studies a single index model with unknown jumps (SIMJ) that occur in the link function. An adaptive semiparametric estimation procedure is proposed for estimating the index coefficient and link function. The asymptotic normality of the resulting estimators for both the parametric and nonparametric parts can be established under some mild conditions and without specifying the error distribution. We show that the resulting estimators are robust and efficient for different error distributions. In particular, a modified EM algorithm is developed to implement the adaptive semiparametric estimation in practice. Numerical simulations and real data analysis are conducted to illustrate the finite sample performance of the proposed approach.

Efficient estimation in single index models through smoothing splines

We consider estimation and inference in a single index regression model with an unknown but smooth link function. In contrast to the standard approach of using kernels or regression splines, we use smoothing splines to estimate the smooth link function. We develop a method to compute the penalized least squares estimators (PLSEs) of the parametric and the nonparametric components given independent and identically distributed (i.i.d.) data. We prove the consistency and find the rates of convergence of the estimators. We establish asymptotic normality under mild assumption and prove asymptotic efficiency of the parametric component under homoscedastic errors. A finite sample simulation corroborates our asymptotic theory. We also analyze a car mileage data set and a Ozone concentration data set. The identifiability and existence of the PLSEs are also investigated.

Estimation of Single-index Models with Fixed Censored Responses

Estimation of Single-index Models with Fixed Censored Responses

Semiparametric jump-detection-based estimation for single-index models with jumps

Semiparametric jump-detection-based estimation for single-index models with jumps

Estimating coefficients of single-index regression models by minimizing variation

ABSTRACTThis paper proposes a novel estimation of coefficients in single-index regression models. Unlike the traditional average derivative estimation [Powell JL, Stock JH, Stoker TM. Semiparametric estimation of index coefficients. Econometrica. 1989;57(6):1403–1430; Hardle W, Thomas M. Investigating smooth multiple regression by the method of average derivatives. J Amer Statist Assoc. 1989;84(408):986–995] and semiparametric least squares estimation [Ichimura H. Semiparametric least squares (sls) and weighted sls estimation of single-index models. J Econometrics. 1993;58(1):71–120; Hardle W, Hall P, Ichimura H. Optimal smoothing in single-index models. Ann Statist. 1993;21(1):157–178], the procedure developed in this paper is to estimate the coefficients directly by minimizing the mean variation function and does not involve estimating the link function nonparametrically. As a result, it avoids the selection of the bandwidth or the number of knots, and its implementation is more robust and easier. The resultant estimator is shown to be consistent. Numerical results and real data analysis also show that the proposed procedure is more applicable against model free assumptions.

Estimation of single-index model with spatial interaction

This article is concerned with the single-index model in spatial dependence data, where the spatial lag effect enters the model linearly and the relationship between variables is a nonparametric function of a linear combination of multivariate regressors. This setup avoids the so-called curse of dimensionality while still capturing important nonlinear features in high dimensional data. It also provides a convenient framework in which to model interactions between the regressors. We propose a two stage estimation strategy where the nonparametric component is established by a local linear approach and the estimation of the parametric part by GMM method, which can be seen as a direct nonlinear least squares method. We derive the asymptotic distributions of the unknowns in our model, and the procedures for constructing simultaneous confidence bands of the nonparametric function are also established. In addition, a simulation study is performed.

Weighted composite quantile regression for single-index models

In this paper we propose a weighted composite quantile regression (WCQR) estimation for single-index models. For parametric part, the WCQR is augmented using a data-driven weighting scheme. With the error distribution unspecified, the proposed estimators share robustness from quantile regression and achieve nearly the same efficiency as the semiparametric maximum likelihood estimator for a variety of error distributions including the Normal, Student’s t, Cauchy distributions, etc. Furthermore, based on the proposed WCQR, we use the adaptive-LASSO to study variable selection for parametric part in the single-index models. For nonparametric part, the WCQR is augmented combining the equal weighted estimators with possibly different weights. Because of the use of weights, the estimation bias is eliminated asymptotically. By comparing asymptotic relative efficiency theoretically and numerically, WCQR estimation all outperforms the CQR estimation and some other estimate methods. Under regularity conditions, the asymptotic properties of the proposed estimations are established. The simulation studies and two real data applications are conducted to illustrate the finite sample performance of the proposed methods.

Identification and estimation of single-index models with measurement error and endogeneity

Economic variables are often measured with an error and may be endogenous. In this paper, we give new identification results for the ratio of partial effects in linear index models with measurement error and endogeneity. The identification restrictions include independence of covariates and error terms, and the derivative of some conditional mean functions being nonzero. We propose a local polynomial regression estimator to estimate the single‐index parameters. We apply these tools to estimate the labour‐supply elasticity and find that the labour‐supply elasticity for married men is positive, while the coefficients for married women are negative for the full sample and positive for the working sample.

Direction estimation in single-index models via distance covariance

We introduce a new method for estimating the direction in single-index models via distance covariance. Our method keeps model-free advantage as a dimension reduction approach. In addition, no smoothing technique is needed, which enables our method to work efficiently when many predictors are discrete or categorical. Under regularity conditions, we show that our estimator is root-n consistent and asymptotically normal. We compare the performance of our method with some dimension reduction methods and the single-index estimation method by simulations and show that our method is very competitive and robust across a number of models. Finally, we analyze the UCI Adult Data Set to demonstrate the efficacy of our method.

Weighted estimation of single index models with right censored responses

In this paper, the unknown link function, the direction parameter, and the heteroscedastic variance in single index models are estimated by the random weight method under the random censorship, respectively. The central limit theory and the convergence rate of the law of the iterated logarithm for the estimator of the direction parameter are derived, respectively. The optimal convergence rates for the estimators of the link function and the heteroscedastic variance are obtained. Simulation results support the theoretical results of the paper.

Estimation of single index model with missing response at random

Under missing at random, we estimate the unknown link function and the direction parameter in a single index model. The central limit theory and the convergence rate of the law of the iterated logarithm for the estimator of the direction parameter are derived, the optimal convergence rate for the estimator of the link function is obtained and simulation results support the theoretical results of the paper.

Curve estimation when the design density is low

In problems where a high-dimensional design is projected into a lower number of dimensions, the density of the new design is typically not bounded away from zero over its support, even if the original one was. Contexts where this problem arises include projection pursuit regression, estimation in single index models and application of the projection-slice method of Radon transform inversion. Theoretical work in these settings typically involves ignoring data toward the ends of the support of the projected design, but in practice that waste of information is not an attractive option. Motivated by these difficulties, we analyze the way in which local linear smoothing is affected by unboundedly sparse design and apply the conclusions of that study to develop empirical, adaptive bandwidth choice methods. Our results even add to knowledge in the familiar case of a design density that is bounded away from zero, where they provide adaptive bandwidth selectors that are optimal right to the ends of the design interval.