Unknown Link Function Research Articles

Misspecified nonconvex statistical optimization for sparse phase retrieval

Existing nonconvex statistical optimization theory and methods crucially rely on the correct specification of the underlying “true” statistical models. To address this issue, we take a first step towards taming model misspecification by studying the high-dimensional sparse phase retrieval problem with misspecified link functions. In particular, we propose a simple variant of the thresholded Wirtinger flow algorithm that, given a proper initialization, linearly converges to an estimator with optimal statistical accuracy for a broad family of unknown link functions. We further provide extensive numerical experiments to support our theoretical findings.

Identification and estimation in quantile varying-coefficient models with unknown link function

In this paper, we consider the estimation problem of quantile varying-coefficient models when the link function is unspecified, which significantly expands the existing works on varying-coefficient models with unspecified link function focusing only on mean regression. We provide new identification conditions which are weaker than existing ones. Under these identification conditions, we use polynomial splines to estimate both the varying coefficients and the link functions and establish the convergence rate of the estimator. Our simulation studies and a real data application illustrate the finite sample performance of the estimators.

Score estimation in the monotone single‐index model

AbstractWe consider estimation in the single‐index model where the link function is monotone. For this model, a profile least‐squares estimator has been proposed to estimate the unknown link function and index. Although it is natural to propose this procedure, it is still unknown whether it produces index estimates that converge at the parametric rate. We show that this holds if we solve a score equation corresponding to this least‐squares problem. Using a Lagrangian formulation, we show how one can solve this score equation without any reparametrization. This makes it easy to solve the score equations in high dimensions. We also compare our method with the effective dimension reduction and the penalized least‐squares estimator methods, both available on CRAN as R packages, and compare with link‐free methods, where the covariates are elliptically symmetric.

SIMEX estimation for single-index model with covariate measurement error

In this paper, we consider the single-index measurement error model with mismeasured covariates in the nonparametric part. To solve the problem, we develop a simulation-extrapolation (SIMEX) algorithm based on the local linear smoother and the estimating equation. For the proposed SIMEX estimation, it is not needed to assume the distribution of the unobserved covariate. We transform the boundary of a unit ball in $\mathbb{R}^p$ to the interior of a unit ball in $\mathbb{R}^{p-1}$ by using the constraint $\|\beta\|=1$. The proposed SIMEX estimator of the index parameter is shown to be asymptotically normal under some regularity conditions. We also derive the asymptotic bias and variance of the estimator of the unknown link function. Finally, the performance of the proposed method is examined by simulation studies and is illustrated by a real data example.

Estimation in monotone single‐index models

Single‐index models are popular regression models that are more flexible than linear models and still maintain more structure than purely nonparametric models. We consider the problem of estimating the regression parameters under a monotonicity constraint on the unknown link function. In contrast to the standard approach of using smoothing techniques, we review different “non‐smooth” estimators that avoid the difficult smoothing parameter selection. For about 30 years, one has had the conjecture that the profile least squares estimator is an ‐consistent estimator of the regression parameter, but the only non‐smooth argmin/argmax estimators that are actually known to achieve this ‐rate are not based on the nonparametric least squares estimator of the link function. However, solving a score equation corresponding to the least squares approach results in ‐consistent estimators. We illustrate the good behavior of the score approach via simulations. The connection with the binary choice and current status linear regression models is also discussed.

Semiparametric jump-preserving estimation for single-index models

ABSTRACTEstimation of the single-index model with a discontinuous unknown link function is considered in this paper. Existed refined minimum average variance estimation (rMAVE) method can estimate the single-index parameter and unknown link function simultaneously by minimising the average pointwise conditional variance, where the conditional variance can be estimated using the local linear fit method with centred kernel function. When there are jumps in the link function, big biases around jumps can appear. For this reason, we embed the jump-preserving technique in the rMAVE method, then propose an adaptive jump-preserving estimation procedure for the single-index model. Concretely speaking, the conditional variance is obtained by the one among local linear fits with centred, left-sided and right-sided kernel functions who has minimum weighted residual mean squares. The resulting estimators can preserve the jumps well and also give smooth estimates of the continuity parts. Asymptotic properties are established under some mild conditions. Simulations and real data analysis show the proposed method works well.

The single-index support vector regression model to address the problem of high dimensionality

ABSTRACTThe last few years, the applications of Support Vector Machine (SVM) for solving classification and regression problems have been increasing, due to its high performance and ability to transform the non-linear relationships among variables to linear form by employing the kernel idea (kernel function). In this work, we develop a semi-parametric approach to fit single-index models to deal with high-dimensional problems. To achieve this goal, we use support vector regression (SVR) for estimating the unknown nonparametric link function, while the single-index is determined by using the semi-parametric least squares method (Ichimura 1993). This development enhances the ability of SVR to solve high-dimensional problem. We design a three simulation examples with high-dimensional problems (linear and nonlinear). The simulations demonstrate the superior performance of the proposed method versus the standard SVR method. This is further illustrated by applying the real data.

Conditional feature screening for mean and variance functions in models with multiple-index structure

The existing methods for feature screening focus mainly on the mean function of regression models. The variance function, however, plays an important role in statistical theory and application. We thus investigate feature screening for mean and variance functions with multiple-index framework in high dimensional regression models. Notice that some information about predictors can be known in advance from previous investigations and experience, for example, a certain set of predictors is related to the response. Based on the conditional information, together with empirical likelihood, we propose conditional feature screening procedures. Our methods can consistently estimate the sets of active predictors in the mean and variance functions. It is interesting that the proposed screening procedures can avoid estimating the unknown link functions in the mean and variance functions, and moreover, can work well in the case of high correlation among the predictors without iterative algorithm. Therefore, our proposal is of computational simplicity. Furthermore, as a conditional method, our method is robust to the choice of the conditional set. The theoretical results reveal that the proposed procedures have sure screening properties. The attractive finite sample performance of our method is illustrated in simulations and a real data application.

Semiparametric Single-Index Predictive Regression

This paper studies a semi-parametric single-index predictive regression model with multiple nonstationary predictors that exhibit co-movement behaviour. Orthogonal series expansion is employed to approximate the unknown link function in the model and the estimator is derived from an optimization under the constraint of identification condition for index parameter. The main finding includes two types of super-consistency rates of the estimators of the index parameter along two orthogonal directions in a new coordinate system. The central limit theorem is established for a plug-in estimator of the unknown link function. In the empirical studies, we provide evidence in favour of nonlinear predictability of the stock return using long term yield and treasury bill rate.

Variable selection and direction estimation for single-index models via DC-TGDR method

This paper is concerned with selecting important covariates and estimating the index direction simultaneously for high dimensional single-index models. We develop an efficient Threshold Gradient Directed Regularization method via maximizing Distance Covariance (DC-TGDR) between the single index and response variable. Due to the appealing property of distance covariance which can measure nonlinear dependence between random variables, the proposed method avoids estimating the unknown link function of the single index and dramatically reduces computational complexity compared to other methods that use smoothing techniques. It keeps the model-free advantage from the view of sufficient dimension reduction and requires neither predictors nor response variable to be continuous. In addition, the DC-TGDR method encourages a grouping effect. That is, it is capable of choosing highly correlated covariates in or out of the model together. We examine finite-sample performance of the proposed method by Monte Carlo simulations. In a real data analysis, we identify important copy number alterations (CNAs) for gene expression.

Simpler proofs for functional sliced inverse regression

In this short communication, we demonstrate that functional sliced inverse regression obtains the same convergence rate as that presented in Hall, P & Horowitz, JL (2007), ‘Methodology and convergence rates for functional linear regression’, Annals of Statistics, 35(1), 70–91 for functional linear regression, both based on functional principal component analysis. This result is interesting because functional sliced inverse regression imposes far fewer structural constraints than functional linear regression, including an unknown link function. The two proofs provided emphasize the similarity between the two, and the proofs are thus much simpler than typically found in the literature. © 2018 John Wiley & Sons, Ltd.

Sieve maximum likelihood estimation in generalized linear models with an unknown link function

Standard generalized linear models (GLMs) consist of three components: random component referring to a distribution of the response variable that belongs to the exponential family; systematic component referring to the linear predictor; and known link function specifying the relationship between the linear predictor and the mean of the distribution function. A flexible extension of the standard GLMs allows an unknown link function. Classical parametric likelihood approach is not applicable due to a large parameter space. To address this issue, sieve maximum likelihood estimation has been developed in literature in which the estimator of the unknown link function is assumed to lie in a sieve space. Various methods of sieves including the B‐spline and P‐spline based methods are introduced. The numerical implementation and theoretical properties of these methods are also discussed. WIREs Comput Stat 2018, 10:e1425. doi: 10.1002/wics.1425This article is categorized under: Applications of Computational Statistics > Signal and Image Processing and Coding Statistical and Graphical Methods of Data Analysis > Nonparametric Methods Statistical Models > Generalized Linear Models Algorithms and Computational Methods > Maximum Likelihood Methods

Estimation of partially linear single-index additive hazards model with current status data

ABSTRACTIn this paper, we introduce a partially linear single-index additive hazards model with current status data. Both the unknown link function of the single-index term and the cumulative baseline hazard function are approximated by B-splines under a monotonicity constraint on the latter. The sieve method is applied to estimate the nonparametric and parametric components simultaneously. We show that, when the nonparametric link function is an exact B-spline, the resultant estimator of regression parameter vector is asymptotically normal and achieves the semiparametric information bound and the rate of convergence of the estimator for the cumulative baseline hazard function is optimal. Simulation studies are presented to examine the finite sample performance of the proposed estimation method. For illustration, we apply the method to a clinical dataset with current status outcome.

Single-index copulas

We introduce so-called single-index copulas. They are semi-parametric conditional copulas whose parameter is an unknown link function of a univariate index only. We propose estimates of this link function and of the finite-dimensional unknown parameter. The asymptotic properties of the latter estimates are stated. Thanks to some properties of conditional Kendall’s tau, we illustrate our technical conditions with several usual copula families.

Efficient estimation and computation for the generalised additive models with unknown link function

The generalised additive models (GAM) are widely used in data analysis. In the application of the GAM, the link function involved is usually assumed to be a commonly used one without justification. Motivated by a real data example with binary response where the commonly used link function does not work, we propose a generalised additive models with unknown link function (GAMUL) for various types of data, including binary, continuous and ordinal. The proposed estimators are proved to be consistent and asymptotically normal. Semiparametric efficiency of the estimators is demonstrated in terms of their linear functionals. In addition, an iterative algorithm, where all estimators can be expressed explicitly as a linear function of Y, is proposed to overcome the computational hurdle for the GAM type model. Extensive simulation studies conducted in this paper show the proposed estimation procedure works very well. The proposed GAMUL are finally used to analyze a real dataset about loan repayment in China, which leads to some interesting findings.

Composite Estimation for Single‐Index Models with Responses Subject to Detection Limits

AbstractWe propose a semiparametric estimator for single‐index models with censored responses due to detection limits. In the presence of left censoring, the mean function cannot be identified without any parametric distributional assumptions, but the quantile function is still identifiable at upper quantile levels. To avoid parametric distributional assumption, we propose to fit censored quantile regression and combine information across quantile levels to estimate the unknown smooth link function and the index parameter. Under some regularity conditions, we show that the estimated link function achieves the non‐parametric optimal convergence rate, and the estimated index parameter is asymptotically normal. The simulation study shows that the proposed estimator is competitive with the omniscient least squares estimator based on the latent uncensored responses for data with normal errors but much more efficient for heavy‐tailed data under light and moderate censoring. The practical value of the proposed method is demonstrated through the analysis of a human immunodeficiency virus antibody data set.

Estimation and variable selection for quantile partially linear single-index models

Partially linear single-index models are flexible dimension reduction semiparametric tools yet still retain ease of interpretability as linear models. This paper is concerned with the estimation and variable selection for partially linear single-index quantile regression models. Polynomial splines are used to estimate the unknown link function. We first establish the asymptotic properties of the quantile regression estimators. For feature selection, we adopt the smoothly clipped absolute deviation penalty (SCAD) approach to select simultaneously single-index variables and partially linear variables. We show that the regularized variable selection estimators are consistent and possess oracle properties. The consistency and oracle properties are also established under the proposed linear approximation of the nonparametric link function that facilitates fast computation. Furthermore, we show that the proposed SCAD tuning parameter selectors via the Schwarz information criterion can consistently identify the true model. Monte Carlo studies and an application to Boston Housing price data are presented to illustrate the proposed approach.

Partially Linear Additive Models with Unknown Link Functions

AbstractIn this paper, we consider partially linear additive models with an unknown link function, which include single‐index models and additive models as special cases. We use polynomial spline method for estimating the unknown link function as well as the component functions in the additive part. We establish that convergence rates for all nonparametric functions are the same as in one‐dimensional nonparametric regression. For a faster rate of the parametric part, we need to define appropriate ‘projection’ that is more complicated than that defined previously for partially linear additive models. Compared to previous approaches, a distinct advantage of our estimation approach in implementation is that estimation directly reduces estimation in the single‐index model and can thus deal with much larger dimensional problems than previous approaches for additive models with unknown link functions. Simulations and a real dataset are used to illustrate the proposed model.

Latent single-index models for ordinal data

We propose a latent semi-parametric model for ordinal data in which the single-index model is used to evaluate the effects of the latent covariates on the latent response. We develop a Bayesian sampling-based method with free-knot splines to analyze the proposed model. As the index may vary from minus infinity to plus infinity, the traditional spline that is defined on a finite interval cannot be applied directly to approximate the unknown link function. We consider a modified version to address this problem by first transforming the index into the unit interval via a continuously cumulative distribution function and then constructing the spline bases on the unit interval. To obtain a rapidly convergent algorithm, we make use of the partial collapse and parameter expansion and reparameterization techniques, improve the movement step of Bayesian splines with free knots so that all the knots can be relocated each time instead of only one knot, and design a generalized Gibbs step. We check the performance of the proposed model and estimation method by a simulation study and apply them to analyze a real dataset.

GEE analysis for longitudinal single-index quantile regression

We consider a single-index quantile regression model for longitudinal data. Based on generalized estimating equations, an estimation procedure is proposed by taking into account the correlation within subject. Under mild assumptions, we derive the convergence rate of the estimator of the unknown link function and the asymptotic normality of estimator of the index parameter using the “projection” technique. Since the estimating equations are non-continuous, we further adopt the smoothing approach and show that estimators obtained from the smoothed estimating equations are asymptotically equivalent to that from the unsmoothed estimating equations. It is also shown that the estimator is more efficient when the correlation is correctly specified. Finally, we present numerical examples including simulations and analysis of a lung function data.