Local Linear Smoothing Research Articles

Multi-dimensional functional principal component analysis

Functional principal component analysis is one of the most commonly employed approaches in functional and longitudinal data analysis and we extend it to analyze functional/longitudinal data observed on a general $d$-dimensional domain. The computational issues emerging in the extension are fully addressed with our proposed solutions. The local linear smoothing technique is employed to perform estimation because of its capabilities of performing large-scale smoothing and of handling data with different sampling schemes (possibly on irregular domain) in addition to its nice theoretical properties. Besides taking the fast Fourier transform strategy in smoothing, the modern GPGPU (general-purpose computing on graphics processing units) architecture is applied to perform parallel computation to save computation time. To resolve the out-of-memory issue due to large-scale data, the random projection procedure is applied in the eigendecomposition step. We show that the proposed estimators can achieve the classical nonparametric rates for longitudinal data and the optimal convergence rates for functional data if the number of observations per sample is of the order $(n/ \log n)^{d/4}$. Finally, the performance of our approach is demonstrated with simulation studies and the fine particulate matter (PM 2.5) data measured in Taiwan.

A new local estimation method for single index models for longitudinal data

Single index models are natural extensions of linear models and overcome the so-called curse of dimensionality. They are very useful for longitudinal data analysis. In this paper, we develop a new efficient estimation procedure for single index models with longitudinal data, based on Cholesky decomposition and local linear smoothing method. Asymptotic normality for the proposed estimators of both the parametric and nonparametric parts will be established. Monte Carlo simulation studies show excellent finite sample performance. Furthermore, we illustrate our methods with a real data example.

Composite quantile regression for varying-coefficient single-index models

ABSTRACTThe varying-coefficient single-index model (VCSIM) is a very general and flexible tool for exploring the relationship between a response variable and a set of predictors. Popular special cases include single-index models and varying-coefficient models. In order to estimate the index-coefficient and the non parametric varying-coefficients in the VCSIM, we propose a two-stage composite quantile regression estimation procedure, which integrates the local linear smoothing method and the information of quantile regressions at a number of conditional quantiles of the response variable. We establish the asymptotic properties of the proposed estimators for the index-coefficient and varying-coefficients when the error is heterogeneous. When compared with the existing mean-regression-based estimation method, our simulation results indicate that our proposed method has comparable performance for normal error and is more robust for error with outliers or heavy tail. We illustrate our methodologies with a real example.

Adaptive jump-preserving estimates in varying-coefficient models

Varying-coefficient models are very important tools to explore the hidden structure between the response variable and its predictors. This article focuses on the estimation of varying-coefficient models with discontinuous coefficient functions. Based on local linear smoothing and jump-preserving regression techniques, an adaptive jump-preserving (AJP) estimation procedure is proposed to estimate the coefficient functions with jumps. This method can automatically accommodate possible jumps of the coefficient functions without knowing the number and locations of jump points or performing any hypothesis tests. Under some mild conditions, the asymptotical properties of the resulting estimators can be established. Furthermore, several numerical studies are conducted to evaluate the finite sample performance of the proposed methodologies. Finally, an application to Australia consumer price index (CPI) data illustrates the validity of the proposed techniques.

Generalized Quasi-Likelihood Ratio Tests for Semiparametric Analysis of Covariance Models in Longitudinal Data

ABSTRACTWe model generalized longitudinal data from multiple treatment groups by a class of semiparametric analysis of covariance models, which take into account the parametric effects of time dependent covariates and the nonparametric time effects. In these models, the treatment effects are represented by nonparametric functions of time and we propose a generalized quasi-likelihood ratio test procedure to test if these functions are identical. Our estimation procedure is based on profile estimating equations combined with local linear smoothers. We find that the much celebrated Wilks phenomenon which is well established for independent data still holds for longitudinal data if a working independence correlation structure is assumed in the test statistic. However, this property does not hold in general, especially when the working variance function is misspecified. Our empirical study also shows that incorporating correlation into the test statistic does not necessarily improve the power of the test. The proposed methods are illustrated with simulation studies and a real application from opioid dependence treatments. Supplementary materials for this article are available online.

Population prevalence of Down's syndrome in the United Kingdom.

Aim was to estimate the age and sex-stratified prevalence of Down's syndrome (DS) in the United Kingdom (UK) general population using a large primary care database. Data source was the Clinical Practice Research Datalink. We divided the number of individuals with a record of DS present on 01/07/2014 by the total number of individuals, and computed Wilson's confidence intervals. Prevalence by age and sex was represented using local linear smoothing plots. On July 1(st) 2014, 1159 females and 1317 males with DS were present in the data, corresponding to a prevalence of 5.9 per 10 000 (95% CI: 5.5; 6.2) in females and 6.8 (6.5; 7.2) per 10 000 in males. Prevalence of DS was increased in individuals aged 40 to 55 years compared to adjacent age groups. A relative peak prevalence of DS at age 40-55 years may be attributed to the combined effects of a rise in life expectancy and the still limited availability of selective abortion.

Partially functional linear varying coefficient model

In this paper, we introduce a new partially functional linear varying coefficient model, where the response is a scalar and some of the covariates are functional. By means of functional principal components analysis and local linear smoothing techniques, we obtain the estimators of coefficient functions of both function-valued variable and real-valued variables. Then the rates of convergence of the proposed estimators and the mean squared prediction error are established under some regularity conditions. Moreover, we develop a hypothesis test for the model and employ the bootstrap procedure to evaluate the null distribution of test statistic and the p-value of the test. At last, we illustrate the finite sample performance of our methods with some simulation studies and a real data application.

Local $M$-estimation for conditional variance function with dependent data

In this paper, a local $M$-estimation for the conditional variance function in heteroscedastic regression models under stationary $\alpha $-mixing dependent samples is developed. The local $M$-estimator is based on the local linear smoothing technique and the $M$-estimation technique, and it is shown to be not only asymptotically equivalent to the local linear estimator but also robust. The weak consistency as well as the asymptotic normality of the local $M$-estimator for the conditional variance function are obtained under mild conditions.

Local linear smoothing for sparse high dimensional varying coefficient models

Varying coefficient models are useful generalizations of parametric linear models. They allow for parameters that depend on a covariate or that develop in time. They have a wide range of applications in time series analysis and regression. In time series analysis they have turned out to be a powerful approach to infer on behavioral and structural changes over time. In this paper, we are concerned with high dimensional varying coefficient models including the time varying coefficient model. Most studies in high dimensional nonparametric models treat penalization of series estimators. On the other side, kernel smoothing is a well established, well understood and successful approach in nonparametric estimation, in particular in the time varying coefficient model. But not much has been done for kernel smoothing in high-dimensional models. In this paper we will close this gap and we develop a penalized kernel smoothing approach for sparse high-dimensional models. The proposed estimators make use of a novel penalization scheme working with kernel smoothing. We establish a general and systematic theoretical analysis in high dimensions. This complements recent alternative approaches that are based on basis approximations and that allow more direct arguments to carry over insights from high-dimensional linear models. Furthermore, we develop theory not only for regression with independent observations but also for local stationary time series in high-dimensional sparse varying coefficient models. The development of theory for local stationary processes in a high-dimensional setting creates technical challenges. We also address issues of numerical implementation and of data adaptive selection of tuning parameters for penalization.The finite sample performance of the proposed methods is studied by simulations and it is illustrated by an empirical analysis of NASDAQ composite index data.

Weighted Local Linear Smoothing Method for Randomly Right Censored Varying Coefficient Models

Weighted Local Linear Smoothing Method for Randomly Right Censored Varying Coefficient Models

Re-weighting estimation of the coefficients in the varying coefficient model with heteroscedastic errors

ABSTRACTThis article explores the estimation problem of the coefficients in the varying coefficient model with heteroscedastic errors. Specifically, we first present a method for estimating the variance function of the error term and the resulting estimator is proved to be consistent. Then, motivated by the generalized least-squares procedure for dealing with heteroscedasticity in the linear regression literature, we re-weight each squared residual term in the local linear smoother with the inverse of the corresponding estimated error variance to construct estimates of the coefficients. Simulation experiments and practical data analysis conducted demonstrate that the re-weighting approach can improve the accuracy of the coefficient estimates under a finite sample size, especially when the error heteroscedasticity is strong.

Fuzzy nonparametric regression based on an adaptive neuro-fuzzy inference system

In this paper, a system, namely the adaptive neuro-fuzzy inference system (ANFIS), is investigated and used for fuzzy nonparametric regression function prediction with crisp input and fuzzy output. The fuzzy least squares problem based on Diamond׳s distance is proposed to optimize the consequent parameters in the hybrid algorithm of the adaptive neuro-fuzzy inference system method. Also, an algorithm is proposed to reduce bias and the boundary effect of the estimates of the underlying regression function. Various examples are used to illustrate and test the performance of this approach. The proposed method is compared with the local linear smoothing method for investigating the accuracy of the approach. The results demonstrate that the proposed method not only gives less biased estimates of the center line, the lower and the upper limit lines of underlying fuzzy regression function but also reduces bias and the boundary effect of the estimates of the underlying regression function by using the proposed algorithm significantly.

Semiparametric GEE analysis in partially linear single-index models for longitudinal data

In this article, we study a partially linear single-index model for longitudinal data under a general framework which includes both the sparse and dense longitudinal data cases. A semiparametric estimation method based on a combination of the local linear smoothing and generalized estimation equations (GEE) is introduced to estimate the two parameter vectors as well as the unknown link function. Under some mild conditions, we derive the asymptotic properties of the proposed parametric and nonparametric estimators in different scenarios, from which we find that the convergence rates and asymptotic variances of the proposed estimators for sparse longitudinal data would be substantially different from those for dense longitudinal data. We also discuss the estimation of the covariance (or weight) matrices involved in the semiparametric GEE method. Furthermore, we provide some numerical studies including Monte Carlo simulation and an empirical application to illustrate our methodology and theory.

Nonparametric Estimation of Conditional Quantile Regression with Mixed Discrete and Continuous Data

In this paper, we investigate the nonlinear quantile regression with mixed discrete and continuous regressors. A local linear smoothing technique with the mixed continuous and discrete kernel function is proposed to estimate the conditional quantile regression function. Under some mild conditions, the asymptotic distribution is established for the proposed nonparametric estimators, which can be seen as a generalisation of some existing theory which only handles the case of purely continuous regressors. We further study the choice of the tuning parameters in the local quantile estimation procedure, and suggest using the cross-validation approach to choose the optimal bandwidths. A simulation study is provided to examine the finite sample behavior of the proposed method, which is also compared with the naive local linear quantile estimation without smoothing the discrete regressors and the nonparametric inverse-CDF method proposed by Li, Lin and Racine (2013).

Numerical discretization-based kernel type estimation methods for ordinary differential equation models

We consider the problem of parameter estimation in both linear and nonlinear ordinary differential equation (ODE) models. Nonlinear ODE models are widely used in applications. But their analytic solutions are usually not available. Thus regular methods usually depend on repetitive use of numerical solutions which bring huge computational cost. We proposed a new two-stage approach which includes a smoothing method (kernel smoothing or local polynomial fitting) in the first stage, and a numerical discretization method (Eulers discretization method, the trapezoidal discretization method, or the Runge-Kutta discretization method) in the second stage. Through numerical simulations, we find the proposed method gains a proper balance between estimation accuracy and computational cost. Asymptotic properties are also presented, which show the consistency and asymptotic normality of estimators under some mild conditions. The proposed method is compared to existing methods in term of accuracy and computational cost. The simulation results show that the estimators with local linear smoothing in the first stage and trapezoidal discretization in the second stage have the lowest average relative errors. We apply the proposed method to HIV dynamics data to illustrate the practicability of the estimator.

Improved Estimation and Uncertainty Quantification Using Monte Carlo-Based Optimization Algorithms

In this article, we present a novel method to obtain both improved estimates and reliable stopping rules for stochastic optimization algorithms such as the Monte Carlo EM (MCEM) algorithm. By characterizing a stationary point, θ*, of the algorithm as the solution to a fixed point equation, we provide a parameter estimation procedure by solving for the fixed point of the update mapping. We investigate various ways to model the update mapping, including the use of a local linear (regression) smoother. This simple approach allows increased stability in estimating the value of θ* as well as providing a natural quantification of the estimation uncertainty. These uncertainty measures can then also be used to construct convergence criteria that reflect the inherent randomness in the algorithm. We establish convergence properties of our modified estimator. In contrast to existing literature, our convergence results do not require the Monte Carlo sample size to go to infinity. Simulation studies are provided to illustrate the improved stability and reliability of our estimator.

Local linear smoothing to estimate accelerated lifetime model with censoring and truncation

A nonparametric accelerated failure time (AFT) model is considered to evaluate the dependability measures of a water supply system laid in a city of the Mediterranean Sea. To do it, a set of breakdown data of sections of pipe of the system is available.Unlike the usual methods that assume a parametric family for the underlying lifetime, we propose an AFT model with an unspecified distribution for the underlying lifetime.To carry out the model fitting we suggest a two-stage procedure. Firstly, we estimate the influence of certain factors over the lifetime of the system. Secondly, we propose modern and flexible statistical tools based on counting processes to construct a smooth estimate the reliability measure of a specific system. We prove the good asymptotic properties of the estimator.

Estimation for Partially Linear Single-index Instrumental Variables Models

In this article, we generalize the partially linear single-index models to the scenario with some endogenous covariates variables. It is well known that the estimators based on the existing methods are often inconsistent because of the endogeneity of covariates. To deal with the endogenous variables, we introduce some auxiliary instrumental variables. A three-stage estimation procedure is proposed for partially linear single-index instrumental variables models. The first stage is to obtain a linear projection of endogenous variables on a set of instrumental variables, the second stage is to estimate the link function by using local linear smoother for given constant parameters, and the last stage is to obtain the estimators of constant parameters based on the estimating equation. Asymptotic normality is established for the proposed estimators. Some simulation studies are undertaken to assess the finite sample performance of the proposed estimation procedure.

Efficient Inference in a Generalized Partially Linear Model with Random Effect for Longitudinal Data

In this article, we study the statistical inference for the generalized partially linear model with random effect. We develop the traditional models that can model generalized longitudinal data and treat categorical data as continuous data by using some transformations. We propose a class of semiparametric estimators for the parametric and variance components. The proposed estimators are data adaptive, which does not require any assumption of working likelihood for the random component or model error. We prove that the resulting estimators for the parametric component are consistent and asymptotic normal, but also remain semiparametrically efficient. The asymptotic normality is established for the proposed estimator of variance component. Moreover, we also propose an estimator for the nonparametric component by using the local linear smoother and present their asymptotic normality. Finite sample performance of the proposed procedures is evaluated by Monte Carlo simulation studies. We further illustrate the proposed procedure by an application.

Local M-estimation for Conditional Variance in Heteroscedastic Regression Models

In this article, we develop a local M-estimation for the conditional variance in heteroscedastic regression models. The estimator is based on the local linear smoothing technique and the M-estimation technique, and it is shown to be not only asymptotically equivalent to the local linear estimator but also robust. The consistency and asymptotic normality of the local M-estimator for the conditional variance in heteroscedastic regression models are obtained under mild conditions. The simulation studies demonstrate that the proposed estimators perform well in robustness.