Uniform Bahadur representation of the backfitting estimator for additive quantile models and its applications

We consider the efficient estimation of a regression parameter in a partially linear additive nonparametric regression model from repeated measures data when the covariates are multivariate. To date, while there is some literature in the scalar covariate case, the problem has not been addressed in the multivariate additive model case. Ours represents a first contribution in this direction. As part of this work, we first describe the behavior of nonparametric estimators for additive models with repeated measures when the underlying model is not additive. These results are critical when one considers variants of the basic additive model. We apply them to the partially linear additive repeated-measures model, deriving an explicit consistent estimator of the parametric component; if the errors are in addition Gaussian, the estimator is semiparametric efficient. We also apply our basic methods to a unique testing problem that arises in genetic epidemiology; in combination with a projection argument we develop an efficient and easily computed testing scheme. Simulations and an empirical example from nutritional epidemiology illustrate our methods.

Problem statement: Most of Seasonal Autoregressive Integrated Moving Average (SARIMA) models that used for forecasting seasonal time series are multiplicative SARIMA models. These models assume that there is a significant parameter as a result of multiplication between nonseasonal and seasonal parameters without testing by certain statistical test. Moreover, most popular statistical software such as MINITAB and SPSS only has facility to fit a multiplicative model. The aim of this research is to propose a new procedure for indentifying the most appropriate order of SARIMA model whether it involves subset, multiplicative or additive order. In particular, the study examined whether a multiplicative parameter existed in the SARIMA model. Approach: Theoretical derivation about Autocorrelation (ACF) and Partial Autocorrelation (PACF) functions from subset, multiplicative and additive SARIMA model was firstly discussed and then R program was used to create the graphics of these theoretical ACF and PACF. Then, two monthly datasets were used as case studies, i.e. the international airline passenger data and series about the number of tourist arrivals to Bali, Indonesia. The model identification step to determine the order of ARIMA model was done by using MINITAB program and the model estimation step used SAS program to test whether the model consisted of subset, multiplicative or additive order. Results: The theoretical ACF and PACF showed that subset, multiplicative and additive SARIMA models have different patterns, especially at the lag as a result of multiplication between non-seasonal and seasonal lags. Modeling of the airline data yielded a subset SARIMA model as the best model, whereas an additive SARIMA model is the best model for forecasting the number of tourist arrivals to Bali. Conclusion: Both of case studies showed that a multiplicative SARIMA model was not the best model for forecasting these data. The comparison evaluation showed that subset and additive SARIMA models gave more accurate forecasted values at out-sample datasets than multiplicative SARIMA model for airline and tourist arrivals datasets respectively. This study is valuable contribution to the Box-Jenkins procedure particularly at the model identification and estimation steps in SARIMA model. Further work involving multiple seasonal ARIMA models, such as short term load data forecasting in certain countries, may provide further insights regarding the subset, multiplicative or additive orders.

In this article, we propose a model selection and semiparametric estimation method for additive models in the context of quantile regression problems. In particular, we are interested in finding nonzero components as well as linear components in the conditional quantile function. Our approach is based on spline approximation for the components aided by two Smoothly Clipped Absolute Deviation (SCAD) penalty terms. The advantage of our approach is that one can automatically choose between general additive models, partially linear additive models, and linear models in a single estimation step. The most important contribution is that this is achieved without the need for specifying which covariates enter the linear part, solving one serious practical issue for models with partially linear additive structure. Simulation studies as well as a real dataset are used to illustrate our method.

In this paper we study sparse high dimensional additive partial linear models with nonparametric additive components of heterogeneous smoothness. We review several existing algorithms that have been developed for this problem in the recent literature, highlighting the connections between them, and present some computationally efficient algorithms for fitting such models. To achieve optimal rates in large sample situations we use hybrid P-splines and block wavelet penalisation techniques combined with adaptive (group) LASSO-like procedures for selecting the additive components in the nonparametric part of the models. Hence, the component selection and estimation in the nonparametric part may be viewed as a functional version of estimation and grouped variable selection. This allows to take advantage of several oracle results which yield asymptotic optimality of estimators in high-dimensional but sparse additive models. Numerical implementations of our procedures for proximal like algorithms are discussed. Large sample properties of the estimates and of the model selection are presented and the results are illustrated with simulated examples and a real data analysis. Keywords: Additive models, Backfitting, Penalisation, Proximal algorithms, Squared group-LASSO, Splines, Wavelets

This article studies M-type estimators for fitting robust additive models in the presence of anomalous data. The components in the additive model are allowed to have different degrees of smoothness. We introduce a new class of wavelet-based robust M-type estimators for performing simultaneous additive component estimation and variable selection in such inhomogeneous additive models. Each additive component is approximated by a truncated series expansion of wavelet bases, making it feasible to apply the method to nonequispaced data and sample sizes that are not necessarily a power of 2. Sparsity of the additive components together with sparsity of the wavelet coefficients within each component (group), results into a bi-level group variable selection problem. In this framework, we discuss robust estimation and variable selection. A two-stage computational algorithm, consisting of a fast accelerated proximal gradient algorithm of coordinate descend type, and thresholding, is proposed. When using nonconvex redescending loss functions, and appropriate nonconvex penalty functions at the group level, we establish optimal convergence rates of the estimates. We prove variable selection consistency under a weak compatibility condition for sparse additive models. The theoretical results are complemented with some simulations and real data analysis, as well as a comparison to other existing methods.

The additive model is the generalized of multiple linear regression that expresses the mean of a reponse variable as a sum of functional form of predictors. The widely used estimation of additive models described in Hastie and Tibshirani (1990) is backfitting algorithm. However, the algorithm with linear smoothers gave some difficulties when it comes to model selection and its inference. The additive model with P-spline as smooth function admits a mixed model formulation, in which variance components control the non-linearity degree in the smooth function. This research is focused in comparing of estimation additive models using backfitting algorithm and linear mixed model approach. The research results show that estimation of additive models using linear mixed models offer simplicity in the computation, since use low-rank dimension of P-spline, and in the model inference, since based on maximum likelihood method. Estimation additive model using linear mixed model approach can be suggested as an alternative method beside backfitting algorithm

Censored quantile regressions have received a great deal of attention in the literature. In a linear setup, recent research has found that an estimator based on the idea of “redistribution-of-mass” in Efron (1967, Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, vol. 4, pp. 831–853, University of California Press) has better numerical performance than other available methods. In this paper, this idea is combined with the local polynomial kernel smoothing for nonparametric quantile regression of censored data. We derive the uniform Bahadur representation for the estimator and, more importantly, give theoretical justification for its improved efficiency over existing estimation methods. We include an example to illustrate the usefulness of such a uniform representation in the context of sufficient dimension reduction in regression analysis. Finally, simulations are used to investigate the finite sample performance of the new estimator.

Penalized least squares estimation in the additive model with different smoothness for the components

Semiparametric additive regression models relax the common assumption of covariate effects to have a known functional form specified by some polynomial. These models only assume that relationships are smooth and are additively separable and thus are less restrictive than common parametric approaches. Estimation techniques for such models assuming all covariates to be exogenous, i.e. uncorrelated to the error term ruling out the presence of confounding omitted variables, for example, have become widely available. However, additive models with weaker assumptions on the error term and methods for inference such as simultaneous confidence bands and specification tests are still subject to extensive research. Thus, the objectives of the thesis are the development of flexible methods for estimation and inference and their application in various complex data situations. Thereby, particular focus is laid on the computational implementation of all proposed approaches aiming at the provision of user-friendly software packages. First, the determinants of chronic child undernutrition in Kenya are analyzed. Particular research questions include the possibility of catch-up growth, i.e. improvements of the nutritional status over age, and relevance of hypotheses on the functional forms of certain effects. In order to address these questions, simultaneous confidence bands for additive models with locally-adaptive smoothed components and heteroscedastic errors are proposed. These appropriately quantify the estimation uncertainty of function estimates and can be used for assessing the statistical significance of an effect and of certain features in a curve. Further, a powerful nonparametric specification test is introduced that allows to test for polynomial regression versus nonparametric alternatives. Next, the needs-relatedness of relief supply in earthquake-affected communities in Pakistan is studied. Here, non-random sample selection calls for the application of a sample selection model with flexible spatial and time-varying effects accounting for unobserved regional heterogeneity and for varying survivor needs over changing seasonal conditions. A flexible Bayesian approach to correct for the sample selection bias is proposed that allows to simultaneously estimate the determinants of the probability to receive relief supply and of the amount of delivered supply. Finally, the usual assumption of the unobservable error term to be orthogonal to the covariates is relaxed relying on the availability of some instrumental variable. A Bayesian nonparametric instrumental variable approach is proposed where bias correction relies on a simultaneous equations specification with flexible modeling of both the covariate effects and the joint error distribution. The approach is used to analyze the relationship between class size and scholastic achievements of students in Israel.

Econometric modelling in finance and risk management: An overview

In regression models with many potential predictors, choosing an appropriate subset of covariates and their interactions at the same time as determining whether linear or more flexible functional forms are required is a challenging and important task. We propose a spike-and-slab prior structure in order to include or exclude single coefficients as well as blocks of coefficients associated with factor variables, random effects or basis expansions of smooth functions. Structured additive models with this prior structure are estimated with Markov Chain Monte Carlo using a redundant multiplicative parameter expansion. We discuss shrinkage properties of the novel prior induced by the redundant parameterization, investigate its sensitivity to hyperparameter settings and compare performance of the proposed method in terms of model selection, sparsity recovery, and estimation error for Gaussian, binomial and Poisson responses on real and simulated data sets with that of component-wise boosting and other approaches.

Parsimonious additive models

Based on the measured data of hot strip finishing mill, a lateral spread model is proposed using the generalized additive theory. Firstly, according to the framework of generalized additive theory, design procedures including pre-analysis of variables, model setting, model estimation, result analysis and modification are investigated for lateral spread modeling. For each independent variable, suitable Back-fitting algorithm is selected to estimate the smooth function. Then, a large number of strip data covering variety of steel grades is collected for modeling. For each variable, a univariate function is estimated using cubic spline interpolation. On this basis, a generalized additive lateral spread model is developed for hot strip finishing mill. Finally, practical prediction experiments indicate that the accuracy of the proposed model is much better than the online one. In addition, the new model features the characteristics of high calculation precision and wide application scope. As a result, the proposed generalized additive lateral spread model would be a good option for online process control in hot strip finishing mill.

Additive models play an important role in semiparametric statistics. This paper gives learning rates for regularized kernel-based methods for additive models. These learning rates compare favorably in particular in high dimensions to recent results on optimal learning rates for purely nonparametric regularized kernel-based quantile regression using the Gaussian radial basis function kernel, provided the assumption of an additive model is valid. Additionally, a concrete example is presented to show that a Gaussian function depending only on one variable lies in a reproducing kernel Hilbert space generated by an additive Gaussian kernel, but does not belong to the reproducing kernel Hilbert space generated by the multivariate Gaussian kernel of the same variance.

Due to the curse of dimensionality, estimation in a multidimensional nonparametric regression model is in general not feasible. Hence, additional restrictions are introduced, and the additive model takes a prominent place. The restrictions imposed can lead to serious bias. Here, a new estimator is proposed which allows penalizing the nonadditive part of a regression function. This offers a smooth choice between the full and the additive model. As a byproduct, this penalty leads to a regularization in sparse regions. If the additive model does not hold, a small penalty introduces an additional bias compared to the full model which is compensated by the reduced bias due to using smaller bandwidths. For increasing penalties, this estimator converges to the additive smooth backfitting estimator of Mammen, Linton and Nielsen [Ann. Statist. 27 (1999) 1443–1490]. The structure of the estimator is investigated and two algorithms are provided. A proposal for selection of tuning parameters is made and the respective properties are studied. Finally, a finite sample evaluation is performed for simulated and ozone data.