Nonparametric Additive Models Research Articles

Functional Horseshoe Priors for Subspace Shrinkage

We introduce a new shrinkage prior on function spaces, called the functional horseshoe (fHS) prior, that encourages shrinkage toward parametric classes of functions. Unlike other shrinkage priors for parametric models, the fHS shrinkage acts on the shape of the function rather than inducing sparsity on model parameters. We study the efficacy of the proposed approach by showing an adaptive posterior concentration property on the function. We also demonstrate consistency of the model selection procedure that thresholds the shrinkage parameter of the fHS prior. We apply the fHS prior to nonparametric additive models and compare its performance with procedures based on the standard horseshoe prior and several penalized likelihood approaches. We find that the new procedure achieves smaller estimation error and more accurate model selection than other procedures in several simulated and real examples. Supplementary materials for this article, which contain additional simulated and real data examples, MCMC diagnostics, and proofs of the theoretical results, are available online.

Forward Additive Regression for Ultrahigh Dimensional Nonparametric Additive Models

Forward Additive Regression for Ultrahigh Dimensional Nonparametric Additive Models

Can expanding natural gas consumption reduce China's CO2 emissions?

China is now the world's largest emitter of carbon dioxide (CO2), thus leading to China facing enormous pressure on CO2 emission reduction. Natural gas is a high thermal and low-emission energy. Expanding natural gas consumption cannot only meet the growing demand for energy consumption but also optimize energy consumption structure. Therefore, many scholars have investigated the effect of natural gas consumption on CO2 emissions. However, ignoring a large number of nonlinear relationships between economic variables, the vast majority of existing studies use traditional linear models to explore the relationships between natural gas consumption and CO2 emissions. In order to make up for the gap in existing research, this paper uses the nonparametric additive regression model with data-driven features to investigate the relationships between the two. The results show that natural gas consumption has an inverted “U-shaped” nonlinear effect on CO2 emissions in the eastern region, but a positive “U-shaped” nonlinear effect in the central and western regions. The linear impact of natural gas consumption on CO2 emissions in the eastern and central regions is higher than that in the western region, due to the differences in resource availability and energy prices, as well as natural gas consumption. Therefore, during the process of promoting natural gas consumption, the central and local governments should adopt heterogeneous measures at different stages of development.

Influencing factors of the carbon dioxide emissions in China's commercial department: A non-parametric additive regression model

The commercial department plays a significant role in China's energy conservation and carbon dioxide emission mitigation. The study employs a panel data set during 2001–2015 and adopts a non-parametric additive regression model to analyse the drivers of commercial department's carbon dioxide emissions in China. Non-parametric additive regression model based on the provincial panel data is first applied to investigate the carbon dioxide emissions in China's commercial department. The consequences demonstrate that the nonlinear impact of economic growth upon carbon dioxide emissions is in accordance with the Environmental Kuznets Curve (EKC) hypothesis. Energy intensity is currently in the left half of the “overturned U-shaped” relationship. In the long run, the overturned “U-shaped” impact of energy intensity may be due to the discrepant level of technological development in different phases. Nevertheless, the urbanization demonstrates an erected “U-shaped” impact on carbon dioxide emissions owing to the further urban commercial consumption and fixed asset investment in recent times. Aggregate population presents a fast-to-slow growth relationship, which is the result that residents require massive commercial goods with the augment in population and income. Energy structure also reveals an erected “U-shape” relationship on account of slight electricity consumption in former times and the large amount of thermal power consumption in later phases. As a result, we should take into full consideration the disparate fluctuant impacts of these influencing indicators when it comes to carbon dioxide emissions in commercial department.

Non‐parametric statistics‐based predictor enabling online transient stability assessment

Online transient stability assessment (TSA) is of great necessity for fast awareness of transient instability caused by fault contingencies. In this paper, a non-parametric statistics based scheme is proposed for response-based online TSA. A critical clearing time-based stability margin index is defined as the predictive output and 14 kinds of severity indicators are proposed as input features for the TSA predictor. With no prior knowledge of the correlation structure, the non-parametric additive model is used as the basis of the predictor. To screen out the weakly correlated indicators and reduce the dimensionality of the input space, two-stage feature selection is fulfilled by non-parametric independence screening and group Lasso penalised regression successively. The predictor is then learnt by least-squares regression in the reduced multi-feature space. With phasor measurement unit measurements at generator buses, severity indicators can be computed in the real-time and fast evaluation of post-fault stability margin can be made by the offline-trained predictor. The effectiveness of the proposed non-parametric statistics based scheme is demonstrated in a modified New England 39-bus system and a practical 756-bus transmission system in China.

Extended lasso-type MARS (LMARS) model in the description of biological network

ABSTRACTThe multivariate adaptive regression splines (MARS) model is one of the well-known, additive non-parametric models that can deal with highly correlated and nonlinear datasets successfully. From our previous analyses, we have seen that lasso-type MARS (LMARS) can be a strong alternative of the Gaussian graphical model (GGM) which is a well-known probabilistic method to describe the steady-state behaviour of the complex biological systems via the lasso regression. In this study, we extend our original LMARS model by taking into account the second-order interaction effects of genes as the representative of the feed-forward loop in biological networks. By this way, we can describe both linear and nonlinear activations of the genes in the same model. We evaluate the performance of our new model under different dimensional simulated and real systems, and then compare the accuracy of the estimates with GGM and LMARS outputs. The results show the advantage of this new model over its close alternatives.

Additive nonparametric models with time variable and both stationary and nonstationary regressors

This paper considers nonparametric additive models that have a deterministic time trend and both stationary and integrated variables as components. The diverse nature of the regressors caters for applications in a variety of settings. In addition, we extend the analysis to allow the stationary regressor to be instead locally stationary, and we allow the models to include a linear form of the integrated variable. Heteroscedasticity is allowed for in all models. We propose an estimation strategy based on orthogonal series expansion that takes account of the different type of stationarity/nonstationarity possessed by each covariate. We establish pointwise asymptotic distribution theory jointly for all estimators of unknown functions and also show the conventional optimal convergence rates jointly in the L2 sense. In spite of the entanglement of different kinds of regressors, we can separate out the distribution theory for each estimator. We provide Monte Carlo simulations that establish the favorable properties of our procedures in moderate sized samples. Finally, we apply our techniques to the study of a pairs trading strategy.

Estimating nonlinear additive models with nonstationarities and correlated errors

AbstractIn this paper, we study a nonparametric additive regression model suitable for a wide range of time series applications. Our model includes a periodic component, a deterministic time trend, various component functions of stochastic explanatory variables, and an AR(p) error process that accounts for serial correlation in the regression error. We propose an estimation procedure for the nonparametric component functions and the parameters of the error process based on smooth backfitting and quasimaximum likelihood methods. Our theory establishes convergence rates and the asymptotic normality of our estimators. Moreover, we are able to derive an oracle‐type result for the estimators of the AR parameters: Under fairly mild conditions, the limiting distribution of our parameter estimators is the same as when the nonparametric component functions are known. Finally, we illustrate our estimation procedure by applying it to a sample of climate and ozone data collected on the Antarctic Peninsula.

Nonparametric additive beta regression for fractional response with application to body fat data

Fractional data that are restricted in the standard unit interval (0, 1) with a highly skewed distribution are commonly encountered. Such data arise in various areas, such as economics, finance, and medicine, among others. One natural idea to model such data is to use the beta family due to its flexibility to accommodate various density shapes. In this paper, we propose a nonparametric additive beta regression model along with a variable selection procedure, where the mean response is related to covariates through the combination of unknown functions of covariates, which can be approximated on a B-spline basis. By using this approximation method, we transform the problem of variable selection into the problem of selecting the groups of coefficients in the expansion. Based on the penalized likelihood method for group variable selection, we successfully select the significant covariates. Moreover, the estimation and selection consistencies and the properties of the penalized estimators are established. The simulation studies demonstrate that the performance of our proposed method is quite good. Finally, we apply the proposed method to body fat data, and we obtain several important findings with satisfactory selection and prediction performance.

Data-adaptive additive modeling.

In this paper, we consider fitting a flexible and interpretable additive regression model in a data-rich setting. We wish to avoid pre-specifying the functional form of the conditional association between each covariate and the response, while still retaining interpretability of the fitted functions. A number of recent proposals in the literature for nonparametric additive modeling are data adaptive, in the sense that they can adjust the level of flexibility in the functional fits to the data at hand. For instance, the sparse additive model makes it possible to adaptively determine which features should be included in the fitted model, the sparse partially linear additive model allows each feature in the fitted model to take either a linear or a nonlinear functional form, and the recent fused lasso additive model and additive trend filtering proposals allow the knots in each nonlinear function fit to be selected from the data. In this paper, we combine the strengths of each of these recent proposals into a single proposal that uses the data to determine which features to include in the model, whether to model each feature linearly or nonlinearly, and what form to use for the nonlinear functions. We establish connections between our approach and recent proposals from the literature, and we demonstrate its strengths in a simulation study.

Two-step variable selection in partially linear additive models with time series data

ABSTRACTLots of semi-parametric and nonparametric models are used to fit nonlinear time series data. They include partially linear time series models, nonparametric additive models, and semi-parametric single index models. In this article, we focus on fitting time series data by partially linear additive model. Combining the orthogonal series approximation and the adaptive sparse group LASSO regularization, we select the important variables between and within the groups simultaneously. Specially, we propose a two-step algorithm to obtain the grouped sparse estimators. Numerical studies show that the proposed method outperforms LASSO method in both fitting and forecasting. An empirical analysis is used to illustrate the methodology.

Assessing the development of China's new energy industry

As the world's largest carbon dioxide (CO2) emitter, China is facing increasing international pressure to reduce emissions. Actively developing new energy has become a fundamental means to solve the dilemma between environmental pollution and energy consumption growth. Thus, more scholars have conducted a wide range of studies on the new energy industry. However, most of the existing studies use traditional linear models to investigate the relationships between new energy industry and its driving forces, ignoring the objective reality that there are many nonlinear relationships in economic variables. In order to overcome the shortcomings of existing research, this paper uses a data-driven nonparametric additive regression model to study the new energy industry. The results show that the nonlinear effect of agricultural development shows an inverted “U-shaped” pattern due to the changes in crop acreage at different stages of development. The nonlinear impact of foreign energy dependence also indicates an inverted “U–shaped” pattern. However, technological progress produces a positive “U–shaped” nonlinear impact on the new energy industry because of the difference in R&D funding and R&D personnel investments. Similarly, the impact of energy consumption structure also shows a positive “U–shaped” pattern, owing to the gradual decline in coal consumption. Therefore, the government should develop differentiated policies at different stages, in order to effectively promote the development of new energy industries.

Sparse structure selection and estimation

ABSTRACTThis paper studies the sparsity selection and estimation in nonparametric additive models. The sparsity refers to two types — across and within variables. Sparsity across variables corresponds to the irrelevant components in the models; sparsity within variables corresponds to zero function values over the sub-domains of the relevant components. To select and estimate the sparsity, I approximate each component by B-splines and propose a group bridge penalized method, which can simultaneously identify the zero functions and zero structures of the nonzero functions. Simulation studies demonstrate the effectiveness of the proposed method in sparsity selection and estimation across and within variables.

Prediction of the Transient Stability Boundary Based on Nonparametric Additive Modeling

This paper applies modern statistical nonparametric methodology to the problem of prediction of the transient stability boundary of large-scale power engineering systems. The stability issue is characterized by the critical clearing time (CCT) that is employed to determine whether a precontingency steady-state condition is stable for a given fault in the power system. The multidimensional mapping between the precontingency steady-state conditions and the corresponding CCT is modeled as an additive structure of one-dimensional functions. Nonparametric kernel estimation methods are applied to the assumed additive model yielding the boundary prediction algorithm that is easily interpretable and avoids the curse of dimensionality. The precision of our additive nonlinear modeling is demonstrated in the context of fault prediction of the 470-bus power network. For the specified fault type, we demonstrate a stronger prediction accuracy compared to other large-scale machine learning methods that have been used for the transient stability boundary problem so far.

Does the high–tech industry consistently reduce CO2 emissions? Results from nonparametric additive regression model

China is currently the world's largest carbon dioxide (CO2) emitter. Moreover, total energy consumption and CO2 emissions in China will continue to increase due to the rapid growth of industrialization and urbanization. Therefore, vigorously developing the high–tech industry becomes an inevitable choice to reduce CO2 emissions at the moment or in the future. However, ignoring the existing nonlinear links between economic variables, most scholars use traditional linear models to explore the impact of the high–tech industry on CO2 emissions from an aggregate perspective. Few studies have focused on nonlinear relationships and regional differences in China. Based on panel data of 1998–2014, this study uses the nonparametric additive regression model to explore the nonlinear effect of the high–tech industry from a regional perspective. The estimated results show that the residual sum of squares (SSR) of the nonparametric additive regression model in the eastern, central and western regions are 0.693, 0.054 and 0.085 respectively, which are much less those that of the traditional linear regression model (3.158, 4.227 and 7.196). This verifies that the nonparametric additive regression model has a better fitting effect. Specifically, the high–tech industry produces an inverted “U–shaped” nonlinear impact on CO2 emissions in the eastern region, but a positive “U–shaped” nonlinear effect in the central and western regions. Therefore, the nonlinear impact of the high–tech industry on CO2 emissions in the three regions should be given adequate attention in developing effective abatement policies.

Additive Nonparametric Models with Time Variable and Both Stationary and Nonstationary Regressors

This paper considers nonparametric additive models that have deterministic time trend and both stationary and integrated variables as components. The diverse nature of the regressors caters for applications in a variety of settings. In addition, we extend the analysis to allow the stationary regressor to be instead locally stationary, and we allow the models to include a linear form of the integrated variable. Heteroscedasticity is allowed for in all models. We propose an estimation strategy based on orthogonal series expansion that takes account of the different type of stationarity/nonstationarity possessed by each covariate. We establish the large sample distribution theory for all the estimated functions we consider. In spite of the entanglement of different kinds of regressors, we can separate out the distribution theory for each estimator. We provide Monte Carlo simulations that establish the favourable properties of our procedures in moderate sized samples. Finally, we apply our techniques to the study of a pairs trading strategy.

Reducing CO2 emissions in China's manufacturing industry: Evidence from nonparametric additive regression models

Identifying the drivers of carbon dioxide emissions in the manufacturing industry is vital for developing effective environmental policies. This study adopts provincial panel data from 2000 to 2013 and uses nonparametric additive regression models to analyze the drivers of CO2 emissions in the industry. The results show that the nonlinear effect of economic growth on CO2 emissions supports the Environmental Kuznets Curve (EKC) hypothesis. Energy structure has an inverted “U-shape” effect owing to massive coal consumption in the early stages and the optimization of energy structure in the later stage. The inverted “U-shaped” impact of industrialization may be due to the priority development of heavy industry in the early stages and the optimization of industrial structure in the later stages. The impact of urbanization also exhibits an inverted “U-shaped” pattern because of mass consumption of steel and cement products in the early stages and the advancement in clean energy technologies at the later stages. However, specific energy consumption has a positive “U-shaped” impact because of the difference in the speed of technological progress at different times. Thus, the differential effects of these indicators at different times should be taken into consideration when discussing reduction of CO2 emissions in China's manufacturing industry.

Interval-valued data regression using nonparametric additive models

Interval-valued data are observed as ranges instead of single values and frequently appear with advanced technologies in current data collection processes. Regression analysis of interval-valued data has been studied in the literature, but mostly focused on parametric linear regression models. In this paper, we study interval-valued data regression based on nonparametric additive models. By employing one of the current methods based on linear regression, we propose a nonparametric additive approach to properly analyze interval-valued data with a possibly nonlinear pattern. We demonstrate the proposed approach using a simulation study and a real data example, and also compare its performance with those of existing methods.

A dynamic analysis of air pollution emissions in China: Evidence from nonparametric additive regression models

PM2.5 emissions not only have serious adverse health effects, but also impede transportation activities, especially in air and highway transport. As a result, PM2.5 emissions have become a public policy concern in China in recent years. Currently, the vast majority of existing researches on PM2.5 are based on natural science perspective. Very few economic studies on the subject have been conducted with linear models. This paper adopts provincial panel data from 2001 to 2012, and uses the STIRPAT model and nonparametric additive regression models to examine the key driving forces of PM2.5 emissions in China. The results show that the nonlinear effect of economic growth on PM2.5 emissions is consistent with the Environmental Kuznets Curve (EKC) hypothesis. The nonlinear impact of urbanization exhibits an inverted “U-shaped” pattern due to the rapid development of urban real estate in the early stages and the strengthening of environmental protection measures in the latter stage. Coal consumption follows an inverted “U-shaped” relationship with PM2.5 emissions owing to massive coal consumption at the beginning and efforts to optimize the energy structure as well as technological progress in clean energy in the latter stages. The nonlinear inverted “U-shaped” impact of private vehicles may be due to the different roles of scale, structural and technical effects at different stages. However, energy efficiency improvement follows a positive “U-shaped” pattern in relation to PM2.5 emissions because of differences in the scale of the economy and the speed of technological progress at different times. As a result, the differential dynamic effects of the driving forces of PM2.5 emissions at different times should be taken into consideration when initiating policies to reduce PM2.5 emissions in China.

Analysis of short-term forecasting for flight arrival time

We suggest various methodologies to provide short-term forecasting of flight arrival times. Flights arriving at Denver International Airport from various U.S. cities during 2010 are used for the model estimation, and the forecasting is applied to 2011 flights. Forecasting proceeds from the time at which a flight departs from an airport. Prediction models using the spline smoothing-based nonparametric additive techniques are applied and compared with benchmarks. We also provide a method for computing the probability of flight arrival time by fitting the skew t distribution to the models’ residuals. Our empirical results indicate that a nonparametric additive model dominantly outperforms the other models considered. In terms of effect of predictor variables, departure delay time, scheduled airborne time, airlines, and weather conditions significantly improve forecasting accuracy, along with seasonal variables. In particular, departure delay time is the most important factor for substantially improving prediction performance.