Oracle Property Of Estimators Research Articles

Variable selection in finite mixture of median regression models using skew-normal distribution

A regression model with skew-normal errors provides a useful extension for traditional normal regression models when the data involve asymmetric outcomes. Moreover, data that arise from a heterogeneous population can be efficiently analysed by a finite mixture of regression models. These observations motivate us to propose a novel finite mixture of median regression model based on a mixture of the skew-normal distributions to explore asymmetrical data from several subpopulations. With the appropriate choice of the tuning parameters, we establish the theoretical properties of the proposed procedure, including consistency for variable selection method and the oracle property in estimation. A productive nonparametric clustering method is applied to select the number of components, and an efficient EM algorithm for numerical computations is developed. Simulation studies and a real data set are used to illustrate the performance of the proposed methodologies.

Variable selection in finite mixture of location and mean regression models using skew-normal distribution

In this article, a variable selection method for the finite mixture of location regression (FMLR) and the finite mixture of mean regression (FMMR) models with a skew-normal error are discussed. The univariate skew-normal distribution was introduced by Sahu et al. will be used in this work, which is attractive because estimation of the skewness parameter does not present the same degree of difficulty as in the case with Azzalini one and, moreover, it allows easy implementation of the EM algorithm. A penalized likelihood approach for variable selection in FMLR and FMMR models was introduced in this article. With a data-adaptive method for selecting tuning parameters, we establish the theoretical properties of our procedure, including consistency in variable selection, the oracle property in estimation. The EM algorithm facilitated by Gauss–Newton method for efficient numerical computations are developed. Simulation studies and a real data set are used to illustrate the proposed methodologies.

Variable selection in finite mixture of regression models using the skew-normal distribution

Variable selection in finite mixture of regression (FMR) models is frequently used in statistical modeling. The majority of applications of variable selection in FMR models use a normal distribution for regression error. Such assumptions are unsuitable for a set of data containing a group or groups of observations with asymmetric behavior. In this paper, we introduce a variable selection procedure for FMR models using the skew-normal distribution. With appropriate choice of the tuning parameters, we establish the theoretical properties of our procedure, including consistency in variable selection and the oracle property in estimation. To estimate the parameters of the model, a modified EM algorithm for numerical computations is developed. The methodology is illustrated through numerical experiments and a real data example.

Efficient parameter estimation and variable selection in partial linear varying coefficient quantile regression model with longitudinal data

Efficient estimation and variable selection in partial linear varying coefficient quantile regression model with longitudinal data is concerned in this paper. To improve estimation efficiency in quantile regression, based on B-spline basis approximation for nonparametric parts, we propose a new estimating function, which can incorporate the correlation structure between repeated measures. In order to reduce computational burdens, the induced smoothing method is used. The new method is empirically shown to be much more efficient and robust than the popular generalized estimating equations based methods. Under mild conditions, the asymptotically normal distribution of the estimators for the parametric components and the optimal convergence rate of the estimators for the nonparametric functions are established. Furthermore, to do variable selection, a smooth-threshold estimating equation is proposed, which can use the correlation structure and select the nonparametric and parametric parts simultaneously. Theoretically, the variable selection procedure works beautifully, including consistency in variable selection and oracle property in estimation. Simulation studies and real data analysis are included to show the finite sample performance.

Robust adaptive model selection and estimation for partial linear varying coefficient models in rank regression

Partial linear varying coefficient models are often used in real data analysis for a good balance between flexibility and parsimony. In this paper, we propose a robust adaptive model selection method based on the rank regression, which can do simultaneous coefficient estimation and three types of selections, i.e., varying and constant effects selection, relevant variable selection. The new method has superiority in robustness and efficiency by inheriting the advantage of the rank regression approach. Furthermore, consistency in the three types of selections and oracle property in estimation are established as well. Simulation studies also confirm our method.

Variable selection in partially linear additive models for modal regression

ABSTRACTBased on B-spline basis functions and smoothly clipped absolute deviation (SCAD) penalty, we present a new estimation and variable selection procedure based on modal regression for partially linear additive models. The outstanding merit of the new method is that it is robust against outliers or heavy-tail error distributions and performs no worse than the least-square-based estimation for normal error case. The main difference is that the standard quadratic loss is replaced by a kernel function depending on a bandwidth that can be automatically selected based on the observed data. With appropriate selection of the regularization parameters, the new method possesses the consistency in variable selection and oracle property in estimation. Finally, both simulation study and real data analysis are performed to examine the performance of our approach.

Variable selection for spatial semivarying coefficient models

Spatial semiparametric varying coefficient models are a useful extension of spatial linear model. Nevertheless, how to conduct variable selection for it has not been well investigated. In this paper, by basis spline approximation together with a general M-type loss function to treat mean, median, quantile and robust mean regressions in one setting, we propose a novel partially adaptive group $$L_{r} (r\ge 1)$$ penalized M-type estimator, which can select variables and estimate coefficients simultaneously. Under mild conditions, the selection consistency and oracle property in estimation are established. The new method has several distinctive features: (1) it achieves robustness against outliers and heavy-tail distributions; (2) it is more flexible to accommodate heterogeneity and allows the set of relevant variables to vary across quantiles; (3) it can keep balance between efficiency and robustness. Simulation studies and real data analysis are included to illustrate our approach.

Robust variable selection in high-dimensional varying coefficient models based on weighted composite quantile regression

In this paper, a new variable selection procedure based on weighted composite quantile regression is proposed for varying coefficient models with a diverging number of parameters. The proposed method is based on basis function approximation and the group SCAD penalty. The new estimation method can achieve both robustness and efficiency. Furthermore, the theoretical properties of our procedure, including consistency in variable selection and the oracle property in estimation are established under some suitable assumptions. Finally, the finite sample behavior of the estimator is evaluated by simulation studies. In addition, some interesting extensions are made to separate constant coefficients from varying coefficients.

Robust estimation for varying index coefficient models

Varying index coefficient models (VICMs) proposed by Ma and Song (J Am Stat Assoc, 2014. doi:10.1080/01621459.2014.903185) are a new class of semiparametric models, which encompass most of the existing semiparametric models. So far, only the profile least squares method and local linear fitting were developed for the VICM, which are very sensitive to the outliers and will lose efficiency for the heavy tailed error distributions. In this paper, we propose an efficient and robust estimation procedure for the VICM based on modal regression which depends on a bandwidth. We establish the consistency and asymptotic normality of proposed estimators for index coefficients by utilizing profile spline modal regression method. The oracle property of estimators for the nonparametric functions is also established by utilizing a two-step spline backfitted local linear modal regression approach. In addition, we discuss the bandwidth selection for achieving better robustness and efficiency and propose a modified expectation---maximization-type algorithm for the proposed estimation procedure. Finally, simulation studies and a real data analysis are carried out to assess the finite sample performance of the proposed method.

Variable selection for generalized varying coefficient models with longitudinal data

In this paper, we apply the penalized quadratic inference function to perform variable selection and estimation simultaneously for generalized varying coefficient models with longitudinal data. The proposed approach is based on basis function approximations and the group SCAD penalty, which can incorporate information on the correlation structure within the same subject to achieve an efficient estimator. Furthermore, we discuss the asymptotic theory of our proposed procedure under suitable conditions, including consistency in variable selection and the oracle property in estimation. Finally, monte carlo simulations and a real data analysis are conducted to examine the finite sample performance of the proposed procedure.

An efficient and robust variable selection method for longitudinal generalized linear models

This paper presents a new efficient and robust smooth-threshold generalized estimating equations for generalized linear models (GLMs) with longitudinal data. The proposed method is based on a bounded exponential score function and leverage-based weights to achieve robustness against outliers both in the response and the covariate domain. Our motivation for the new variable selection procedure is that it enables us to achieve better robustness and efficiency by introducing an additional tuning parameter γ which can be automatically selected using the observed data. Moreover, its performance is near optimal and superior to some recently developed variable selection methods. Under some regularity conditions, the resulting estimator possesses the consistency in variable selection and the oracle property in estimation. Finally, simulation studies and a detailed real data analysis are carried out to assess and illustrate the finite sample performance, which show that the proposed method works better than other existing methods, in particular, when many outliers are included.

Robust variable selection in modal varying-coefficient models with longitudinal

In this article we present a robust and efficient variable selection procedure by using modal regression for varying-coefficient models with longitudinal data. The new method is proposed based on basis function approximations and a group version of the adaptive LASSO penalty, which can select significant variables and estimate the non-zero smooth coefficient functions simultaneously. Under suitable conditions, we establish the consistency in variable selection and the oracle property in estimation. A simulation study and two real data examples are undertaken to assess the finite sample performance of the proposed variable selection procedure.

Walsh-average based variable selection for varying coefficient models

A shrinkage-type variable selection procedure for varying coefficient models is routinely established in the least-squares (LS) framework. Although the LS method has favorable properties for a large class of error distributions, it will break down if the error variance is infinite and is adversely affected by outliers and heavy-tail distributions. To overcome these issues, we propose a robust shrinkage method termed regularized Walsh-average (RWA) that can construct robust nonparametric variable selection and robust coefficient estimation simultaneously. Theoretical analysis reveals RWA works beautifully, including consistency in variable selection and oracle property in estimation, even when error variance is infinite. More important property is that when error variance is finite, compared with the LS based estimators, the asymptotic relative efficiency of the new estimator is at least 0.8896, a relatively high level. Furthermore, a robust BIC-type criterion, which can identify the true model consistently, is suggested for shrinkage parameter selection. Numerical studies also confirm our theories.

Robust variable selection in partially varying coefficient single-index model

By combining basis function approximations and smoothly clipped absolute deviation (SCAD) penalty, this paper proposes a robust variable selection procedure for a partially varying coefficient single-index model based on modal regression. The proposed procedure simultaneously selects significant variables in the parametric components and the nonparametric components. With appropriate selection of the tuning parameters, we establish the theoretical properties of our procedure, including consistency in variable selection and the oracle property in estimation. Furthermore, we also discuss the bandwidth selection and propose a modified expectation–maximization (EM)-type algorithm for the proposed estimation procedure. The finite sample properties of the proposed estimators are illustrated by some simulation examples.

Simultaneous structure estimation and variable selection in partial linear varying coefficient models for longitudinal data

Partial linear varying coefficient models (PLVCM) are often considered for analysing longitudinal data for a good balance between flexibility and parsimony. The existing estimation and variable selection methods for this model are mainly built upon which subset of variables have linear or varying effect on the response is known in advance, or say, model structure is determined. However, in application, this is unreasonable. In this work, we propose a simultaneous structure estimation and variable selection method, which can do simultaneous coefficient estimation and three types of selections: varying and constant effects selection, relevant variable selection. It can be easily implemented in one step by employing a penalized M-type regression, which uses a general loss function to treat mean, median, quantile and robust mean regressions in a unified framework. Consistency in the three types of selections and oracle property in estimation are established as well. Simulation studies and real data analysis also confirm our method.

Variable Selection in Semiparametric Quantile Modeling for Longitudinal Data

We propose a penalized quantile regression for partially linear varying coefficient (VC) model with longitudinal data to select relevant non parametric and parametric components simultaneously. Selection consistency and oracle property are established. Furthermore, if linear part and VC part are unknown, we propose a new unified method, which can do three types of selections: separation of varying and constant effects, selection of relevant variables, and it can be carried out conveniently in one step. Consistency in the three types of selections and oracle property in estimation are established as well. Simulation studies and real data analysis also confirm our method.

Variable selection in robust semiparametric modeling for longitudinal data

This paper considers robust variable selection in semiparametric modeling for longitudinal data with an unspecified dependence structure. First, by basis spline approximation and using a general formulation to treat mean, median, quantile and robust mean regressions in one setting, we propose a weighted M-type regression estimator, which achieves robustness against outliers in both the response and covariates directions, and can accommodate heterogeneity, and the asymptotic properties are also established. Furthermore, a penalized weighted M-type estimator is proposed, which can do estimation and select relevant nonparametric and parametric components simultaneously, and robustly. Without any specification of error distribution and intra-subject dependence structure, the variable selection method works beautifully, including consistency in variable selection and oracle property in estimation. Simulation studies also confirm our method and theories.

Robust variable selection for the varying coefficient model based on composite L 1–L 2 regression

The varying coefficient model (VCM) is an important generalization of the linear regression model and many existing estimation procedures for VCM were built on L 2 loss, which is popular for its mathematical beauty but is not robust to non-normal errors and outliers. In this paper, we address the problem of both robustness and efficiency of estimation and variable selection procedure based on the convex combined loss of L 1 and L 2 instead of only quadratic loss for VCM. By using local linear modeling method, the asymptotic normality of estimation is driven and a useful selection method is proposed for the weight of composite L 1 and L 2. Then the variable selection procedure is given by combining local kernel smoothing with adaptive group LASSO. With appropriate selection of tuning parameters by Bayesian information criterion (BIC) the theoretical properties of the new procedure, including consistency in variable selection and the oracle property in estimation, are established. The finite sample performance of the new method is investigated through simulation studies and the analysis of body fat data. Numerical studies show that the new method is better than or at least as well as the least square-based method in terms of both robustness and efficiency for variable selection.

Variable selection of the quantile varying coefficient regression models

As a useful supplement to mean regression, quantile regression is a completely distribution-free approach and is more robust to heavy-tailed random errors. In this paper, a variable selection procedure for quantile varying coefficient models is proposed by combining local polynomial smoothing with adaptive group LASSO. With an appropriate selection of tuning parameters by the BIC criterion, the theoretical properties of the new procedure, including consistency in variable selection and the oracle property in estimation, are established. The finite sample performance of the newly proposed method is investigated through simulation studies and the analysis of Boston house price data. Numerical studies confirm that the newly proposed procedure (QKLASSO) has both robustness and efficiency for varying coefficient models irrespective of error distribution, which is a good alternative and necessary supplement to the KLASSO method.

Variable Selection in Nonparametric Varying-Coefficient Models for Analysis of Repeated Measurements

Nonparametric varying-coefficient models are commonly used for analyzing data measured repeatedly over time, including longitudinal and functional response data. Although many procedures have been developed for estimating varying coefficients, the problem of variable selection for such models has not been addressed to date. In this article we present a regularized estimation procedure for variable selection that combines basis function approximations and the smoothly clipped absolute deviation penalty. The proposed procedure simultaneously selects significant variables with time-varying effects and estimates the nonzero smooth coefficient functions. Under suitable conditions, we establish the theoretical properties of our procedure, including consistency in variable selection and the oracle property in estimation. Here the oracle property means that the asymptotic distribution of an estimated coefficient function is the same as that when it is known a priori which variables are in the model. The method is illustrated with simulations and two real data examples, one for identifying risk factors in the study of AIDS and one using microarray time-course gene expression data to identify the transcription factors related to the yeast cell-cycle process.