Smooth Backfitting Research Articles

Online Smooth Backfitting for Generalized Additive Models

We propose an online smoothing backfitting method for generalized additive models coupled with local linear estimation. The idea can be extended to general nonlinear optimization problems. The strategy is to use an appropriate-order expansion to approximate the nonlinear equations and store the coefficients as sufficient statistics which can be updated in an online manner by the dynamic candidate bandwidth method. We investigate the statistical and algorithmic convergences of the proposed method. By defining the relative statistical efficiency and computational cost, we further establish a framework to characterize the tradeoff between estimation performance and computation performance. Simulations and real data examples are provided to illustrate the proposed method and algorithm. Supplementary materials for this article are available online.

Locally polynomial Hilbertian additive regression

In this paper a new additive regression technique is developed for response variables that take values in general Hilbert spaces. The proposed method is based on the idea of smooth backfitting that has been developed mainly for real-valued responses. The local polynomial smoothing device is adopted, which renders various advantages of the technique evidenced in the classical univariate kernel regression with real-valued responses. It is demonstrated that the new technique eliminates many limitations which existing methods are subject to. In contrast to the existing techniques, the proposed approach is equipped with the estimation of the derivatives as well as the regression function itself, and provides options to make the estimated regression function free from boundary effects and possess oracle properties. A comprehensive theory is presented for the proposed method, which includes the rates of convergence in various modes and the asymptotic distributions of the estimators. The efficiency of the proposed method is also demonstrated via simulation study and is illustrated through real data applications.

Nonparametric inference for additive models estimated via simplified smooth backfitting

We investigate hypothesis testing in nonparametric additive models estimated using simplified smooth backfitting (Huang and Yu, Journal of Computational and Graphical Statistics, 28(2), 386–400, 2019). Simplified smooth backfitting achieves oracle properties under regularity conditions and provides closed-form expressions of the estimators that are useful for deriving asymptotic properties. We develop a generalized likelihood ratio (GLR) (Fan, Zhang and Zhang, Annals of statistics, 29(1),153–193, 2001) and a loss function (LF) (Hong and Lee, Annals of Statistics, 41(3), 1166–1203, 2013)-based testing framework for inference. Under the null hypothesis, both the GLR and LF tests have asymptotically rescaled chi-squared distributions, and both exhibit the Wilks phenomenon, which means the scaling constants and degrees of freedom are independent of nuisance parameters. These tests are asymptotically optimal in terms of rates of convergence for nonparametric hypothesis testing. Additionally, the bandwidths that are well suited for model estimation may be useful for testing. We show that in additive models, the LF test is asymptotically more powerful than the GLR test. We use simulations to demonstrate the Wilks phenomenon and the power of these proposed GLR and LF tests, and a real example to illustrate their usefulness.

Backfitting tests in generalized structured models

Summary We introduce bootstrap tests for semiparametric generalized structured models. These can be used for testing different kinds of model specifications like separability, functional forms and homogeneity of effects, or for performing variable selection in a large class of semiparametric models. The test statistics are based on the comparison of non- and semiparametric alternatives in which both the null hypothesis and the alternative are non- or semiparametric. All estimators are obtained by smooth backfitting. Simulation studies show excellent performance of the test procedures.

Package wsbackfit for Smooth Backfitting Estimation of Generalized Structured Models

Package wsbackfit for Smooth Backfitting Estimation of Generalized Structured Models

Smooth Backfitting of Proportional Hazards With Multiplicative Components

Smooth backfitting has proven to have a number of theoretical and practical advantages in structured regression. By projecting the data down onto the structured space of interest smooth backfitting provides a direct link between data and estimator. This article introduces the ideas of smooth backfitting to survival analysis in a proportional hazard model, where we assume an underlying conditional hazard with multiplicative components. We develop asymptotic theory for the estimator. In a comprehensive simulation study, we show that our smooth backfitting estimator successfully circumvents the curse of dimensionality and outperforms existing estimators. This is especially the case in difficult situations like high number of covariates and/or high correlation between the covariates, where other estimators tend to break down. We use the smooth backfitter in a practical application where we extend recent advances of in-sample forecasting methodology by allowing more information to be incorporated, while still obeying the structured requirements of in-sample forecasting. Supplementary materials for this article are available online.

Smooth backfitting for errors-in-variables varying coefficient regression models

Varying coefficient models inherit the simplicity and easy interpretation of classical linear models while enjoying the flexibility of nonparametric models. They are very useful in analyzing the relation between a response and a set of predictors. There has been no study, however, on the estimation of varying coefficients when the predictors, on which the varying coefficients depend, are contaminated by measurement errors. A new kernel smoothing technique that is tailored to the structure of an underlying varying coefficient model as well as corrects for the bias due to the measurement errors is developed here. The estimators of the varying coefficients are given implicitly by solving a system of integral equations, whose implementation requires an iterative backfitting procedure. The existence of a unique solution and the convergence of the associated backfitting algorithm are established theoretically. Some numerical evidences that support the theory and demonstrate the success of the proposed methodology are presented.

Additive Functional Regression for Densities as Responses

We propose and investigate additive density regression, a novel additive functional regression model for situations where the responses are random distributions that can be viewed as random densities and the predictors are vectors. Data in the form of samples of densities or distributions are increasingly encountered in statistical analysis and there is a need for flexible regression models that accommodate random densities as responses. Such models are of special interest for multivariate continuous predictors, where unrestricted nonparametric regression approaches are subject to the curse of dimensionality. Additive models can be expected to maintain one-dimensional rates of convergence while permitting a substantial degree of flexibility. This motivates the development of additive regression models for situations where multivariate continuous predictors are coupled with densities as responses. To overcome the problem that distributions do not form a vector space, we utilize a class of transformations that map densities to unrestricted square integrable functions and then deploy an additive functional regression model to fit the responses in the unrestricted space, finally transforming back to density space. We implement the proposed additive model with an extended version of smooth backfitting and establish the consistency of this approach, including rates of convergence. The proposed method is illustrated with an application to the distributions of baby names in the United States.

A nonparametric approach to identify age, time, and cohort effects

Empirical studies in the social sciences and biometrics often rely on data and models where a number of individuals born at different dates are observed at several points in time, and the relationship of interest centers on the effects of age a, cohort c, and time t. Because of t=a+c, the design is degenerate and one is automatically confronted with the associated (linear) identification problem studied intensively for parametric models (Mason and Fienberg 1985; MaCurdy and Mroz 1995; Kuang, Nielsen and Nielsen 2008a,b). Nonlinear time, age, and cohort effects can be identified in an additive model. The present study seeks to solve the identification problem employing a nonparametric estimation approach: We develop an additive model which is solved using a backfitting algorithm, in the spirit of Mammen et al. (1999). Our approach has the advantage that we do not have to worry about the parametric specification and its impact on the identification problem. The results can easily be interpreted, as the smooth backfitting algorithm is a projection of the data onto the space of additive models. We develop a complete asymptotic distribution theory for nonparametric estimators based on kernel smoothing and apply the method to a study on wage inequality in Germany between 1975 and 2004.

Classical Backfitting for Smooth-Backfitting Additive Models

ABSTRACTSmooth backfitting has been shown to have better theoretical properties than classical backfitting for fitting additive models based on local linear regression. In this article, we show that the smooth backfitting procedure in the local linear case can be alternatively performed as a classical backfitting procedure with a different type of smoother matrices. These smoother matrices are symmetric and shrinking and some established results in the literature are readily applicable. The connections allow the smooth backfitting algorithm to be implemented in a much simplified way, give new insights on the differences between the two approaches in the literature, and provide an extension to local polynomial regression. The connections also give rise to a new estimator at data points. Asymptotic properties of general local polynomial smooth backfitting estimates are investigated, allowing for different orders of local polynomials and different bandwidths. Cases of oracle efficiency are discussed. Computer-generated simulations are conducted to demonstrate finite sample behaviors of the methodology and a real data example is given for illustration. Supplementary materials for this article are available online.

Conditional variance estimation via nonparametric generalized additive models

Generalized additive models provide a way of circumventing curse of dimension in a wide range of nonparametric regression problem. In this paper, we present a multiplicative model for conditional variance functions where one can apply a generalized additive regression method. This approach extends Fan and Yao (1998) to multivariate cases with a multiplicative structure. In this approach, we use squared residuals instead of using log-transformed squared residuals. This idea gives a smaller variance than Yu (2017) when the variance of squared error is smaller than the variance of log-transformed squared error. We provide estimators based on quasi-likelihood and an iterative algorithm based on smooth backfitting for generalized additive models. We also provide some asymptotic properties of estimators and the convergence of proposed algorithm. A numerical study shows the empirical evidence of the theory.

Smooth backfitting for errors-in-variables additive models

In this work, we develop a new smooth backfitting method and theory for estimating additive nonparametric regression models when the covariates are contaminated by measurement errors. For this, we devise a new kernel function that suitably deconvolutes the bias due to measurement errors as well as renders a projection interpretation to the resulting estimator in the space of additive functions. The deconvolution property and the projection interpretation are essential for a successful solution of the problem. We prove that the method based on the new kernel weighting scheme achieves the optimal rate of convergence in one-dimensional deconvolution problems when the smoothness of measurement error distribution is less than a threshold value. We find that the speed of convergence is slower than the univariate rate when the smoothness of measurement error distribution is above the threshold, but it is still much faster than the optimal rate in multivariate deconvolution problems. The theory requires a deliberate analysis of the nonnegligible effects of measurement errors being propagated to other additive components through backfitting operation. We present the finite sample performance of the deconvolution smooth backfitting estimators that confirms our theoretical findings.

Estimating nonlinear additive models with nonstationarities and correlated errors

AbstractIn this paper, we study a nonparametric additive regression model suitable for a wide range of time series applications. Our model includes a periodic component, a deterministic time trend, various component functions of stochastic explanatory variables, and an AR(p) error process that accounts for serial correlation in the regression error. We propose an estimation procedure for the nonparametric component functions and the parameters of the error process based on smooth backfitting and quasimaximum likelihood methods. Our theory establishes convergence rates and the asymptotic normality of our estimators. Moreover, we are able to derive an oracle‐type result for the estimators of the AR parameters: Under fairly mild conditions, the limiting distribution of our parameter estimators is the same as when the nonparametric component functions are known. Finally, we illustrate our estimation procedure by applying it to a sample of climate and ozone data collected on the Antarctic Peninsula.

Smooth backfitting for additive modeling with small errors-in-variables, with an application to additive functional regression for multiple predictor functions

We study smooth backfitting when there are errors-in-variables, which is motivated by functional additive models for a functional regression model with a scalar response and multiple functional predictors that are additive in the functional principal components of the predictor processes. The development of a new smooth backfitting technique for the estimation of the additive component functions in functional additive models with multiple functional predictors requires to address the difficulty that the eigenfunctions and therefore the functional principal components of the predictor processes, which are the arguments of the proposed additive model, are unknown and need to be estimated from the data. The available estimated functional principal components contain an error that is small for large samples but nevertheless affects the estimation of the additive component functions. This error-in-variables situation requires to develop new asymptotic theory for smooth backfitting. Our analysis also pertains to general situations where one encounters errors in the predictors for an additive model, when the errors become smaller asymptotically. We also study the finite sample properties of the proposed method for the application in functional additive regression through a simulation study and a real data example.

Estimation for semiparametric varying coefficient models with different smoothing variables under random right censoring

In this paper we study a semiparametric varying coefficient model when the response is subject to random right censoring. The model gives an easy interpretation due to its direct connectivity to the classical linear model and is very flexible since nonparametric functions which accommodates various nonlinear interaction effects between covariates are admitted in the model. We propose estimators for this model using mean-preserving transformation and establish their asymptotic properties. The estimation procedure is based on the profiling and the smooth backfitting techniques. A simulation study is presented to show the reliability of the proposed estimators and an automatic bandwidth selector is given in a data-driven way.

Nonparametric multiplicative heteroscedasticity in multi-dimensional regression

In many statistical applications, the variability of the data is an important issue. For instance, in the regression analysis, researchers often meet the heteroscedasticity problem. There is a wide body of literature about the nonparametric estimation of the conditional variance function in one-dimensional case. However there are only few papers about the nonparametric estimation of the conditional variance function when there are several regressors in the model. In this paper, we propose a smooth backfitting estimator for the multiplicative conditional variance function and study the asymptotic property and finite sample performance via simulation studies.

Smooth backfitting in additive inverse regression

We consider the problem of estimating an additive regression function in an inverse regression model with a convolution type operator. A smooth backfitting procedure is developed and asymptotic normality of the resulting estimator is established. Compared to other methods for the estimation in additive models the new approach neither requires observations on a regular grid nor the estimation of the joint density of the predictor. It is also demonstrated by means of a simulation study that the backfitting estimator outperforms the marginal integration method at least by a factor of two with respect to the integrated mean squared error criterion. The methodology is illustrated by a problem of live cell imaging in fluorescence microscopy.

Asymptotics for nonparametric and semiparametric fixed effects panel models

In this paper, we investigate the problem of estimating nonparametric and semiparametric panel data models with fixed effects. We focus on establishing the asymptotic results for estimators using smooth backfitting methods. We consider two estimators for the smooth unknown function in nonparametric panel regressions. One is a local linear estimator constructed similar as the one in Mammen et al. (2009) which was proposed for the additive nonparametric panel model. The other is the local profile likelihood based estimator proposed by Henderson et al. (2008) (HCL hereafter). We build the link and compare the difference between these two estimators which are constructed under different sets of conditions. We put both of these estimators in the smooth backfitting algorithm framework discussed in Mammen et al. (1999). Following the recently developed theories on backfitting kernel estimates in Mammen et al. (2009), we establish the asymptotic normality of these estimators, and hence verify the conjectures made by HCL and complement their paper. Further, we consider a partially linear fixed effects panel data model with the nonparametric component estimated using the methods discussed in the first part of the paper. We give the asymptotic result for the estimators of finite dimensional parameters, which shows that the first-step plug-in estimators will not affect the asymptotic variance in the second-step estimation.

Backfitting and smooth backfitting in varying coefficient quantile regression

Summary In this paper, we study ordinary backfitting and smooth backfitting as methods of fitting varying coefficient quantile models. We do this in a unified framework that accommodates various types of varying coefficient models. Our framework also covers the additive quantile model as a special case. Under a set of weak conditions, we derive the asymptotic distributions of the backfitting estimators. We also briefly report on the results of a simulation study.

Efficient estimation for partially linear varying coefficient models when coefficient functions have different smoothing variables

In this paper we consider partially linear varying coefficient models. We provide semiparametric efficient estimators of the parametric part as well as rate-optimal estimators of the nonparametric part. In our model, different nonparametric coefficients have different smoothing variables. This requires employing a projection technique to get proper estimators of the nonparametric coefficients, and thus conventional kernel smoothing cannot give semiparametric efficient estimators of the parametric components. We take the smooth backfitting approach in conjunction with the profiling technique to get semiparametric efficient estimators of the parametric part. We also show that our estimators of the nonparametric part achieve the univariate rate of convergence, regardless of the covariate's dimension. We report the finite sample properties of the semiparametric efficient estimators and compare them with those of other estimators.