Single-index Regression Models Research Articles

Semi-parametric single-index predictive regression models with cointegrated regressors

This paper considers the estimation of a semi-parametric single-index regression model that allows for nonlinear predictive relationships. This model is useful for predicting financial asset returns, whose observed behaviour resembles a stationary process, if the multiple nonstationary predictors are cointegrated. The presence of cointegrated regressors imposes a single-index structure in the model, and this structure not only balances the nonstationarity properties of the multiple predictors with the stationarity properties of asset returns but also avoids the curse of dimensionality associated with nonparametric regression function estimation. An orthogonal series expansion is used to approximate the unknown link function for the single-index component. We consider the constrained nonlinear least squares estimator of the single-index (or the cointegrating) parameters and the plug-in estimator of the link function, and derive their asymptotic properties. In an empirical application, we find some evidence of in-sample nonlinear predictability of U.S. stock returns using cointegrated predictors. We also find that the single-index model in general produces better out-of-sample forecasts than both the historical average benchmark and the linear predictive regression model.

Bayesian composite quantile regression for the single-index model.

By using a Gaussian process prior and a location-scale mixture representation of the asymmetric Laplace distribution, we develop a Bayesian analysis for the composite quantile single-index regression model. The posterior distributions for the unknown parameters are derived, and the Markov chain Monte Carlo sampling algorithms are also given. The proposed method is illustrated by three simulation examples and a real dataset.

Asymptotic normality of the k-NN single index regression estimator for functional weak dependence data*

In this article, we consider the k-Nearest Neighbors (k-NN) method in a single index regression model when the explanatory variable is valued in functional space in the setting of the quasi-association dependence condition. The main result of this work is the establishment of the asymptotic distribution for the k-NN kernel single index estimator. These results are established under fairly general conditions on the underlying models. As an application, the asymptotic confidence bands for the regression model based on the single-index model are presented. Some simulation studies are carried out to show the finite sample performances of the k-NN estimator.

A split-and-conquer variable selection approach for high-dimensional general semiparametric models with massive data

Estimation and variable selection in partially linear models for massive data has been discussed by several authors. However, there does not seem to exist an established procedure for other semiparametric models, such as the semiparametric varying-coefficient linear model, the single index regression model, the partially linear errors-in-variables model, etc. In this paper, we propose a general procedure for variable selection in high-dimensional general semiparametric models by penalized semiparametric estimating equations. Under some regularity conditions, the oracle property is established, which the number of parameters is allowed to diverge. Furthermore, we also propose a split-and-conquer variable selection procedure for high-dimensional general semiparametric models with massive data. Under some weak regularity conditions, we establish the oracle property of the proposed procedure when the number of subsets does not grow too fast. What is more, the split-and-conquer procedure enjoys the oracle property as the penalized estimator by using all the dataset, and can substantially reduce computing time and computer memory requirements. The performance of the proposed method is illustrated via a real data application and numerical simulations.

The k-nearest neighbors method in single index regression model for functional quasi-associated time series data

The k-nearest neighbors method in single index regression model for functional quasi-associated time series data

Varying coefficient single-index regression model with missing responses under rank-based modelling

A varying coefficients single-index regression model with responses missing at random is considered. Rank-based estimators of the index coefficient and the functional coefficients are studied, and their asymptotic properties (consistency and asymptotic normality) are established under mild conditions. To demonstrate the performance of the proposed approach, Monte Carlo simulation experiments are carried out and show that the proposed approach provides robust and more efficient estimators compared to its least-squares counterpart. This is demonstrated under different model error structures, including the standard normal, the t and the contaminated model error distributions. Finally, a real data example is given to illustrate our proposed method.

Semiparametric Efficiency in Convexity Constrained Single-Index Model

We consider estimation and inference in a single-index regression model with an unknown convex link function. We introduce a convex and Lipschitz constrained least-square estimator (CLSE) for both the parametric and the nonparametric components given independent and identically distributed observations. We prove the consistency and find the rates of convergence of the CLSE when the errors are assumed to have only moments and are allowed to depend on the covariates. When , we establish -rate of convergence and asymptotic normality of the estimator of the parametric component. Moreover, the CLSE is proved to be semiparametrically efficient if the errors happen to be homoscedastic. We develop and implement a numerically stable and computationally fast algorithm to compute our proposed estimator in the R package simest. We illustrate our methodology through extensive simulations and data analysis. Finally, our proof of efficiency is geometric and provides a general framework that can be used to prove efficiency of estimators in a wide variety of semiparametric models even when they do not satisfy the efficient score equation directly. Supplementary files for this article are available online.

A constrained single-index regression for estimating interactions between a treatment and covariates.

We consider a single-index regression model, uniquely constrained to estimate interactions between a set of pretreatment covariates and a treatment variable on their effects on a response variable, in the context of analyzing data from randomized clinical trials. We represent interaction effect terms of the model through a set of treatment-specific flexible link functions on a linear combination of the covariates (a single index), subject to the constraint that the expected value given the covariates equals 0, while leaving the main effects of the covariates unspecified. We show that the proposed semiparametric estimator is consistent for the interaction term of the model, and that the efficiency of the estimator can be improved with an augmentation procedure. The proposed single-index regression provides a flexible and interpretable modeling approach to optimizing individualized treatment rules based on patients' data measured at baseline, as illustrated by simulation examples and an application to data from a depression clinicaltrial.

Robust estimation of single index models with responses missing at random

A single-index regression model is considered, where some responses in the model are assumed to be missing at random. Local linear rank-based estimators of the single-index direction and the unknown link function are proposed. Asymptotic properties of the estimators are established under mild regularity conditions. Monte Carlo simulation experiments show that the proposed estimators are more efficient than their least squares counterparts especially when the data are derived from contaminated or heavy-tailed model error distributions. When the errors follow a normal distribution, the least squares index direction estimator tends to be more efficient than the rank-based index direction estimator; however, the least squares link function estimator remains less efficient than the rank-based link function estimator. A real data example is analyzed and cross-validation studies show that the proposed procedure provides better prediction than the least squares method when the responses contain outliers and are missing at random.

4088 Longitudinal Assessment of Metabolic Syndrome as a Modifiable Risk factor of World Trade Center Particulate Matter Exposure Associated Lung Disease

OBJECTIVES/GOALS: Metabolic syndrome (MetSyn) is a risk for World Trade Center-Lung Injury (WTC-LI; defined as developing FEV1<lower limit of normal [LLN]). Metabolic health is a modifiable disease risk factor. We propose to characterize how time-dependent covariates of MetSyn are longitudinally associated with WTC-LI. METHODS/STUDY POPULATION: WTC-particulate exposed firefighters, consented, with pre-9/11 FEV1 LLN (N = 5,746). Data assessed from last pre-9/11 till August 1, 2017. Longitudinal MetSyn characteristics were assessed using 3 models: i. A linear mixed effect model to assess the effect size of longitudinal MetSyn and its components on longitudinal FEV1% predicted as an outcome; ii. a time-dependent Cox regression to assess the associations of MetSyn to time of onset of WTC-LI; iii. a novel, partially linear single index regression model with repeatedly measured MetSyn to assess their joint effects and delineate their relative contribution on the longitudinal lung function in the WTC-FDNY cohort. RESULTS/ANTICIPATED RESULTS: In Model I, BMI 30 kg/m2 had the largest effect size compared to ever-smoking, with −2.524 (95%CI: −2.708,−2.340) compared to −1.681(−2.325,−1.038) respectively. Having MetSyn, defined as 3/5 risk factors, had an effect size of −2.319(−2.526,−2.112). In Model II, hazards of triglycerides 150mg/dL were highest at 1.497(1.336, 1.677), followed by BMI 30 kg/m2 at 1.406(1.256, 1.575), and HDL<40mg/dL 1.355(1.176-1.561), compared to ever-smoking (1.201, p = 0.002). Having high exposure to PM by being present in the morning of 9/11 was a significant covariate only in Model II investigating HDL<40mg/dL or triglycerides 150mg/dL. Model III The proposed methods will be applied to our cohort study. DISCUSSION/SIGNIFICANCE OF IMPACT: MetSyn is both a predictor and concurrent marker of WTC-LI. The single index model can not only reduce dimensionality of the covariates, but also provides efficient estimates of the joint MetSyn effects, allowing linear or nonlinear effects. Future studies will investigate dietary intervention as a potential disease-modifying factor. CONFLICT OF INTEREST DESCRIPTION: NA, nothing to disclose.

Efficient estimation in single index models through smoothing splines

We consider estimation and inference in a single index regression model with an unknown but smooth link function. In contrast to the standard approach of using kernels or regression splines, we use smoothing splines to estimate the smooth link function. We develop a method to compute the penalized least squares estimators (PLSEs) of the parametric and the nonparametric components given independent and identically distributed (i.i.d.) data. We prove the consistency and find the rates of convergence of the estimators. We establish asymptotic normality under mild assumption and prove asymptotic efficiency of the parametric component under homoscedastic errors. A finite sample simulation corroborates our asymptotic theory. We also analyze a car mileage data set and a Ozone concentration data set. The identifiability and existence of the PLSEs are also investigated.

Functional single-index quantile regression models

It is known that functional single-index regression models can achieve better prediction accuracy than functional linear models or fully nonparametric models, when the target is to predict a scalar response using a function-valued covariate. However, the performance of these models may be adversely affected by extremely large values or skewness in the response. In addition, they are not able to offer a full picture of the conditional distribution of the response. Motivated by using trajectories of $$\hbox {PM}_{{10}}$$ concentrations of last day to predict the maximum $$\hbox {PM}_{{10}}$$ concentration of the current day, a functional single-index quantile regression model is proposed to address those issues. A generalized profiling method is employed to estimate the model. Simulation studies are conducted to investigate the finite sample performance of the proposed estimator. We apply the proposed framework to predict the maximal value of $$\hbox {PM}_{{10}}$$ concentrations based on the intraday $$\hbox {PM}_{{10}}$$ concentrations of the previous day.

Estimation and variable selection for partial linear single-index distortion measurement errors models

This paper considers partial linear single-index regression models when all the variables are measured with multiplicative distortion measurement errors. To eliminate the effect caused by the distortion, we propose the conditional absolute mean calibration. This method avoids to use the nonzero expectation conditions imposed on the variables in the literature. Using the calibrated variables, a profile least squares estimator is obtained. For the hypothesis testing of parameter, a restricted estimator under the null hypothesis and a test statistic are proposed. A smoothly clipped absolute deviation penalty is employed to select the relevant variables. The resulting penalized estimators are shown to be asymptotically normal and have the oracle property. Simulation studies demonstrate the performance of the proposed procedure and a real example is analyzed to illustrate its practical usage.

Threshold single index regression model from high-dimensional data

Instability of model parameters is a common phenomenon in areas such as signal processing, quality control, fault detection, finance, security, and clinical medicine. In this paper we focus on the high-dimensional single index model with a possible change point due to a covariate threshold. We propose the $\ell_1$-penalized likelihood estimators for regression coefficients as well as the threshold parameter. A proximal gradient algorithm is proposed for detecting the possible change point. Besides, non-asymptotic Oracle inequalities for both the prediction risk and the $\ell_1$-estimation loss for regression coefficients are obtained under certain sparsity condition.At last, thorough simulation studies are conducted to illustrate the empirical performance of the proposed method.

Robust estimation and selection for single-index regression model

In this paper, we consider a single-index regression model for which we propose a robust estimation procedure for the model parameters and an efficient variable selection of relevant predictors. The proposed method is known as the penalized generalized signed-rank procedure. Asymptotic properties of the proposed estimator are established under mild regularity conditions. Extensive Monte Carlo simulation experiments are carried out to study the finite sample performance of the proposed approach. The simulation results demonstrate that the proposed method dominates many of the existing ones in terms of robustness of estimation and efficiency of variable selection. Finally, a real data example is given to illustrate the method.

An Adaptive-to-Model Test for Parametric Single-Index Errors-in-Variables Models

This paper provides some useful tests for fitting a parametric single-index regression model when covariates are measured with error and validation data is available. We propose two tests whose consistency rates do not depend on the dimension of the covariate vector when an adaptive-to-model strategy is applied. One of these tests has a bias term that becomes arbitrarily large with increasing sample size but its asymptotic variance is smaller, and the other is asymptotically unbiased with larger asymptotic variance. Compared with the existing local smoothing tests, the new tests behave like a classical local smoothing test with only one covariate, and still are omnibus against general alternatives. This avoids the difficulty associated with the curse of dimensionality. Further, a systematic study is conducted to give an insight on the effect of the values of the ratio between the sample size and the size of validation data on the asymptotic behavior of these tests. Simulations are conducted to examine the performance in several finite sample scenarios.

Discussion of “Simple Estimators for Invertible Index Models” by H. Ahn, H. Ichimura, J. Powell, and P. Ruud

ABSTRACTThis is an interesting article that considers the question of inference on unknown linear index coefficients in a general class of models where reduced form parameters are invertible function of one or more linear index. Interpretable sufficient conditions such as monotonicity and or smoothness for the invertibility condition are provided. The results generalize some work in the previous literature by allowing the number of reduced form parameters to exceed the number of indices. The identification and estimation expand on the approach taken in previous work by the authors. Examples include Ahn, Powell, and Ichimura (2004) for monotone single-index regression models to a multi-index setting and extended by Blundell and Powell (2004) and Powell and Ruud (2008) to models with endogenous regressors and multinomial response, respectively. A key property of the inference approach taken is that the estimator of the unknown index coefficients (up to scale) is computationally simple to obtain (relative to other estimators in the literature) in that it is closed form. Specifically, unifying an approach for all models considered in this article, the authors propose an estimator, which is the eigenvector of a matrix (defined in terms of a preliminary estimator of the reduced form parameters) corresponding to its smallest eigenvalue. Under suitable conditions, the proposed estimator is shown to be root-n-consistent and asymptotically normal.

A minimum projected-distance test for parametric single-index Berkson models

In this paper, we propose a minimum projected-distance test for parametric single-index regression models when the predictors are measured with Berkson errors. This test asymptotically behaves like a locally smoothing test as if the null model were with one-dimensional predictor, and is omnibus to detect all global alternative models. The test can also detect local alternative models that converge to the null model at the fastest rate that the existing locally smoothing tests with one-dimensional predictor can achieve. Therefore, the proposed test has potential for alleviating the curse of dimensionality in this field. We also give two bias-correction methods to center the test statistic. Numerical studies are conducted to examine the performance of the proposed test.

Estimating coefficients of single-index regression models by minimizing variation

ABSTRACTThis paper proposes a novel estimation of coefficients in single-index regression models. Unlike the traditional average derivative estimation [Powell JL, Stock JH, Stoker TM. Semiparametric estimation of index coefficients. Econometrica. 1989;57(6):1403–1430; Hardle W, Thomas M. Investigating smooth multiple regression by the method of average derivatives. J Amer Statist Assoc. 1989;84(408):986–995] and semiparametric least squares estimation [Ichimura H. Semiparametric least squares (sls) and weighted sls estimation of single-index models. J Econometrics. 1993;58(1):71–120; Hardle W, Hall P, Ichimura H. Optimal smoothing in single-index models. Ann Statist. 1993;21(1):157–178], the procedure developed in this paper is to estimate the coefficients directly by minimizing the mean variation function and does not involve estimating the link function nonparametrically. As a result, it avoids the selection of the bandwidth or the number of knots, and its implementation is more robust and easier. The resultant estimator is shown to be consistent. Numerical results and real data analysis also show that the proposed procedure is more applicable against model free assumptions.

General rank-based estimation for regression single index models

This study considers rank estimation of the regression coefficients of the single index regression model. Conditions needed for the consistency and asymptotic normality of the proposed estimator are established. Monte Carlo simulation experiments demonstrate the robustness and efficiency of the proposed estimator compared to the semiparametric least squares estimator. A real-life example illustrates that the rank regression procedure effectively corrects model nonlinearity even in the presence of outliers in the response space.