Nonparametric Additive Models Research Articles

A high-dimensional additive nonparametric model

Nonparametric additive models are garnering increasing attention in applied research across fields like statistics and economics, attributed to their distinct interpretability, versatility, and their adeptness at addressing the curse of dimensionality. This paper introduces a novel and efficient fully Bayesian method for estimating nonparametric additive models, employing a band matrix smoothness prior. Our methodology leverages unobserved binary indicator parameters, promoting linearity in each additive component while allowing for deviations from it. We validate the efficacy of our approach through experiments on synthetic data derived from ten-component additive models, encompassing diverse configurations of linear, nonlinear, and zero function components. Additionally, the robustness of our algorithm is tested on high-dimensional models featuring up to one hundred components, and models correlated components. The practical utility and computational efficiency of our technique are further underscored by its application to two real-world datasets, showcasing its broad applicability and effectiveness in various scenarios.

Nonparametric screening for additive quantile regression in ultra-high dimension

In practical applications, one often does not know the ‘true’ structure of the underlying conditional quantile function, especially in the ultra-high dimensional setting. To deal with ultra-high dimensionality, quantile-adaptive marginal nonparametric screening methods have been recently developed. However, these approaches may miss important covariates that are marginally independent of the response, or may select unimportant covariates due to their high correlations with important covariates. To mitigate such shortcomings, we develop a conditional nonparametric quantile screening procedure (complemented by subsequent selection) for nonparametric additive quantile regression models. Under some mild conditions, we show that the proposed screening method can identify all relevant covariates in a small number of steps with probability approaching one. The subsequent narrowed best subset (via a modified Bayesian information criterion) also contains all the relevant covariates with overwhelming probability. The advantages of our proposed procedure are demonstrated through simulation studies and a real data example.

A one-covariate-at-a-time multiple testing approach to variable selection in additive models

. This article proposes a One-Covariate-at-a-time Multiple Testing (OCMT) approach to choose significant variables in high-dimensional nonparametric additive regression models. Similarly to Chudik, Kapetanios, and Pesaran, we consider the statistical significance of individual nonparametric additive components one at a time and take into account the multiple testing nature of the problem. Both one-stage and multiple-stage procedures are considered. The former works well in terms of the true positive rate only if the net effects of all signals are strong enough; the latter helps to pick up hidden signals that have weak net effects. Simulations demonstrate the good finite-sample performance of the proposed procedures. As an empirical illustration, we apply the OCMT procedure to a dataset extracted from the Longitudinal Survey on Rural Urban Migration in China. We find that our procedure works well in terms of out-of-sample root mean square forecast errors, compared with competing methods such as adaptive group Lasso (AGLASSO).

An assessment on the new impetus of green energy development and its impact on climate change: a non-linear perspective.

The purpose of this article is to investigate the new driving forces behind China's green energy and further assess the impact of green energy on climate change. The existing literature has used linear methods to investigate green energy, ignoring the non-linear relationships between economic variables. The nonparametric models can accurately simulate nonlinear relationships between economic variables. This paper constructs a nonparametric additive model and uses it to explore green energy. The empirical results show that the impact of green finance on green energy is more prominent in the later stage (a U-shaped impact). Fiscal decentralization also exerts a positive U-shaped impact, meaning that expanding local fiscal autonomy has contributed to green energy growth in the later stage. Similarly, the impact of oil prices and foreign direct investment demonstrates a positive U-shaped pattern. However, the nonlinear impact of environmental pressure displays an inverted U-shaped pattern. Furthermore, this article explores the impact of green energy on climate change and its impact mechanisms. The results exhibit green energy generates a positive U-shaped impact on climate change, meaning that the role of green energy in mitigating climate change gradually becomes prominent over time. Mechanism analysis exhibits that industrial structure and energy structure both produce a nonlinear influence on climate change.

Exploring the role of green finance in wind power development: using the nonparametric model

In the context of the “dual carbon” strategy, how to leverage green finance to promote China's wind power industry is a hot topic. Unlike existing literature, this article uses a nonparametric additive model to investigate the impact and mechanism of green finance on wind power development. Research has found that green finance has an inverted U-shaped nonlinear impact on wind power development, indicating that green finance has a more prominent contribution to the wind power industry in the early stages. Further mechanism research indicates that green finance affects the wind power industry through foreign direct investment and green technology innovation. Specifically, with the relaxation of foreign direct investment conditions in the energy sector, the role of foreign direct investment in promoting the wind power industry more prominent in the later stages. In the early stages, government support was greater, and green technology patents grew rapidly, driving green technology innovation to have a more significant impact on the wind power industry. In addition, the impact of fiscal decentralization, wind power prices, and environmental regulations on the wind power industry also exhibits significant nonlinear characteristics. This article helps to comprehensively understand the mechanism and impact of green finance on wind power development, and provides a reliable basis for optimizing green finance policy and effectively promoting wind power.

Nonparametric Additive Regression for High-Dimensional Group Testing Data

Group testing has been verified as a cost-effective and time-efficient approach, where the individual samples are pooled with a predefined group size for subsequent testing. Recent research has explored the integration of covariate information to improve the modeling of the group testing data. While existing works for high-dimensional data primarily focus on parametric models, this study considers a more flexible generalized nonparametric additive model. Nonlinear components are approximated using B-splines and model estimation under the sparsity assumption is derived employing group lasso. Theoretical results demonstrate that our method selects the true model with a high probability and provides consistent estimates. Numerical studies are conducted to illustrate the good performance of our proposed method, using both simulated and real data.

Nonparametric Additive Models for Billion Observations

The nonparametric additive model (NAM) is a widely used nonparametric regression method. Nevertheless, due to the high computational burden, classic statistical techniques for fitting NAMs are not well-equipped to handle massive data with billions of observations. To address this challenge, we develop a scalable element-wise subset selection method, referred to as Core-NAM, for fitting penalized regression spline based NAMs. Specifically, we first propose an approximation of the penalized least squares estimation, based on which we develop an efficient variant of generalized cross-validation (GCV) to select the smoothing parameter and approximate the Bayesian confidence intervals for statistical inference. Theoretically, we show that the proposed estimator approximately minimizes an upper bound of the estimation mean squared error. Moreover, we provide a non-asymptotic approximation guarantee for the proposed estimator and establish the asymptotic optimality of the proposed variant of GCV. Extensive simulations demonstrate the superior accuracy and efficiency of the Core-NAM method. We also apply the proposed method to a total column ozone dataset containing nearly one billion observations, and the results indicate a speed-up by almost a thousand times with comparable performance compared to the full data approach. Supplementary materials for this article are available online.

Environmental regulations, technological innovation, and low carbon transformation: A case of the logistics industry in China

China is the world's largest emitter of carbon dioxide. The logistics industry is the second largest source of CO2 emissions, second only to the manufacturing industry. Therefore, developing effective environmental regulations to achieve low-carbon transformation in the logistics industry is an urgent task for government managers. Based on the 2005–2021 panel data of China's 30 provinces, this article adopts a data-driven nonparametric additive model to investigate the role of environmental regulations in CO2 emission reduction in the logistics industry. The results display that overall, environmental regulations contribute to carbon reduction in the logistics industry (an inverted N-shaped effect). After endogeneity test and various robustness tests such as replacing the variable value of core variable, this conclusion still has good reliability. Further heterogeneity analysis indicates that the carbon reduction effect of mandatory environmental regulations fluctuates frequently (a W-shaped impact). However, the carbon reduction impact of incentive environmental regulations has gradually shifted from insignificant in the early stages to prominent in the later stages (an inverted U-shaped pattern). Regional heterogeneity analysis shows that the impact of environmental regulations on carbon reduction in the eastern, central, and western regions also has significant nonlinear characteristics. Finally, this article verifies that technological innovation and energy consumption structure are effective intermediate mechanisms for environmental regulations to affect emission reduction. The research findings provide practical inspiration for the government to formulate relevant environmental policies and promote the green development of the logistics industry.

Wavelet estimation for the nonparametric additive model in random design and long-memory dependent errors

We investigate the nonparametric additive regression estimation in random design and long-memory errors and construct adaptive thresholding estimators based on wavelet series. The proposed approach achieves asymptotically near-optimal convergence rates when the unknown function and its univariate additive components belong to Besov space. We consider the problem under two noise structures; (1) homoskedastic Gaussian long memory errors and (2) heteroskedastic Gaussian long memory errors. In the homoskedastic long-memory error case, the estimator is completely adaptive with respect to the long-memory parameter. In the heteroskedastic long-memory case, the estimator may not be adaptive with respect to the long-memory parameter unless the heteroskedasticity is of polynomial form. In either case, the convergence rates depend on the long-memory parameter only when long-memory is strong enough, otherwise, the rates are identical to those under i.i.d. errors. In addition, convergence rates are free from the curse of dimensionality.

Financial decentralization, renewable energy technologies, energy subsidies and wind power development in China: An analysis of nonparametric model

Currently, financial development and renewable energy technologies have become new tools to stimulate wind power development, attracting attention from the government and academia. However, most existing literature uses linear methods to investigate wind power, masking the nonlinear relationship between these factors and wind power development. The objective of this study is to reveal the nonlinear relationships between these economic factors and wind power development, providing a basis for the government to design differentiated phased policies. Compared to traditional linear methods, the nonparametric models can truly simulate the nonlinear relationship between these economic factors and wind power development. Given this, this article uses the nonparametric additive regression model to investigate these links. The result of this study shows: (1) financial decentralization produces a U-shaped impact on wind power, indicating that expanding the autonomy of local financial institutions has driven wind power development in the later stage. (2) The impact of renewable energy technologies demonstrates an inverted U-shaped pattern, signifying that the driving effect of green technology has shifted from being prominent in the early stage to being reduced in the later stage. (3) Energy subsidies also exert an inverted U-shaped effect, meaning that its driving role is prominent in the early stage. (4) In addition, the impacts of CO2 emissions, wind power prices, environmental governance investments, and urbanization on wind power development also exhibit non-linear characteristics. Finally, this article proposes policies in response to the stage where economic tools don't play a role in driving wind power development.

Solar energy investment, technological innovation and carbon emission reduction: Evidence from China

The aim of this paper is to investigate the impact of solar energy investment on carbon emissions. The STIRPAT model, a non-parametric additive regression model, and the vector autoregression model are built to investigate the comprehensive effect of solar energy investment on China’s carbon emissions. Solar energy investment and other factors related to carbon emissions are examined. The empirical study shows that it will take about 8 years for the solar energy investment to promote carbon emission reductions. The moderation analysis indicates that technological innovation has a moderating effect in the facilitation of carbon emission reduction by solar energy investment. The finding of this study has some meaningful policy implications. In order to achieve the goal of carbon emission reduction, China should keep solar energy investment continuous and steady and improve technological innovation.

Frequentist Model Averaging for the Nonparametric Additive Model

Frequentist Model Averaging for the Nonparametric Additive Model

A Flexibly Conditional Screening Approach via a Nonparametric Quantile Partial Correlation

Considering the influence of conditional variables is crucial to statistical modeling, ignoring this may lead to misleading results. Recently, Ma, Li and Tsai proposed the quantile partial correlation (QPC)-based screening approach that takes into account conditional variables for ultrahigh dimensional data. In this paper, we propose a nonparametric version of quantile partial correlation (NQPC), which is able to describe the influence of conditional variables on other relevant variables more flexibly and precisely. Specifically, the NQPC firstly removes the effect of conditional variables via fitting two nonparametric additive models, which differs from the conventional partial correlation that fits two parametric models, and secondly computes the QPC of the resulting residuals as NQPC. This measure is very useful in the situation where the conditional variables are highly nonlinearly correlated with both the predictors and response. Then, we employ this NQPC as the screening utility to do variable screening. A variable screening procedure based on NPQC (NQPC-SIS) is proposed. Theoretically, we prove that the NQPC-SIS enjoys the sure screening property that, with probability going to one, the selected subset can recruit all the truly important predictors under mild conditions. Finally, extensive simulations and an empirical application are carried out to demonstrate the usefulness of our proposal.

Model Selection for High Dimensional Nonparametric Additive Models via Ridge Estimation

In ultrahigh dimensional data analysis, to keep computational performance well and good statistical properties still working, nonparametric additive models face increasing challenges. To overcome them, we introduce a methodology of model selection for high dimensional nonparametric additive models. Our approach is to propose a novel group screening procedure via nonparametric smoothing ridge estimation (GRIE) to find the importance of each covariate. It is then combined with the sure screening property of GRIE and the model selection property of extended Bayesian information criteria (EBIC) to select the suitable sub-models in nonparametric additive models. Theoretically, we establish the strong consistency of model selection for the proposed method. Extensive simulations and two real datasets illustrate the outstanding performance of the GRIE-EBIC method.

Nonparametric estimation of additive models with errors-in-variables

In the estimation of nonparametric additive models, conventional methods, such as backfitting and series approximation, cannot be applied when measurement error is present in a covariate. This paper proposes a two-stage estimator for such models. In the first stage, to adapt to the additive structure, we use a series approximation together with a ridge approach to deal with the ill-posedness brought by mismeasurement. We derive the uniform convergence rate of this first-stage estimator and characterize how the measurement error slows down the convergence rate for ordinary/super smooth cases. To establish the limiting distribution, we construct a second-stage estimator via one-step backfitting with a deconvolution kernel using the first-stage estimator. The asymptotic normality of the second-stage estimator is established for ordinary/super smooth measurement error cases. Finally, a Monte Carlo study and an empirical application highlight the applicability of the estimator.

Variable selection for nonparametric additive Cox model with interval-censored data.

The standard Cox model is perhaps the most commonly used model for regression analysis of failure time data but it has some limitations such as the assumption on linear covariate effects. To relax this, the nonparametric additive Cox model, which allows for nonlinear covariate effects, is often employed, and this paper will discuss variable selection and structure estimation for this general model. For the problem, we propose a penalized sieve maximum likelihood approach with the use of Bernstein polynomials approximation and group penalization. To implement the proposed method, an efficient group coordinate descent algorithm is developed and can be easily carried out for both low- and high-dimensional scenarios. Furthermore, a simulation study is performed to assess the performance of the presented approach and suggests that it works well in practice. The proposed method is applied to an Alzheimer's disease study for identifying important and relevant genetic factors.

Nonparametric inference for additive models estimated via simplified smooth backfitting

We investigate hypothesis testing in nonparametric additive models estimated using simplified smooth backfitting (Huang and Yu, Journal of Computational and Graphical Statistics, 28(2), 386–400, 2019). Simplified smooth backfitting achieves oracle properties under regularity conditions and provides closed-form expressions of the estimators that are useful for deriving asymptotic properties. We develop a generalized likelihood ratio (GLR) (Fan, Zhang and Zhang, Annals of statistics, 29(1),153–193, 2001) and a loss function (LF) (Hong and Lee, Annals of Statistics, 41(3), 1166–1203, 2013)-based testing framework for inference. Under the null hypothesis, both the GLR and LF tests have asymptotically rescaled chi-squared distributions, and both exhibit the Wilks phenomenon, which means the scaling constants and degrees of freedom are independent of nuisance parameters. These tests are asymptotically optimal in terms of rates of convergence for nonparametric hypothesis testing. Additionally, the bandwidths that are well suited for model estimation may be useful for testing. We show that in additive models, the LF test is asymptotically more powerful than the GLR test. We use simulations to demonstrate the Wilks phenomenon and the power of these proposed GLR and LF tests, and a real example to illustrate their usefulness.

Government Health Expenditure, Economic Growth, and Regional Development Differences-Analysis Based on a Non-parametric Additive Model.

Government health expenditure plays an important role in guaranteeing people's livelihood and in improving the quality of human capital, and it also plays an important role in affecting economic development. In order to characterize the specific trend of government health spending affecting economic growth, and analyze the impact difference in different regions, this paper uses a non-parametric additive model to analyze the impact of government health expenditure on economic development and regional development differences based on three aspects: linear, non-linear and comprehensive effects. From the perspective of linear effects, the results show that the proportion of government health expenditure to GDP nationwide and in the western regions has a positive impact on economic development, while the impact in the eastern and central regions is negative. From the perspective of non-linear effects, in the national and in the eastern, central and western regions, the proportion of government health expenditure to GDP has a significant non-linear impact on economic development. From the perspective of comprehensive effects, the proportion of government health expenditure to GDP has a significant non-linear positive effect on economic development nationwide and in all regions. In addition, the proportion of fixed assets investment to GDP, the proportion of exports to GDP, and the proportion of residents' income to GDP promote non-linear effects to different extents of non-linear promotion, but there are slight differences in different regions. Based on estimation results, the paper recommends that the government further expands the proportion of government health expenditure in GDP, promotes investment in fixed assets, stimulates exports in the eastern region, and continues to implement the western support policy.

Variable Selection for Nonparametric Additive Cox Model with Interval-censored Data

Variable Selection for Nonparametric Additive Cox Model with Interval-censored Data

Kernel Knockoffs Selection for Nonparametric Additive Models

Thanks to its fine balance between model flexibility and interpretability, the nonparametric additive model has been widely used, and variable selection for this type of model has been frequently studied. However, none of the existing solutions can control the false discovery rate (FDR) unless the sample size tends to infinity. The knockoff framework is a recent proposal that can address this issue, but few knockoff solutions are directly applicable to nonparametric models. In this article, we propose a novel kernel knockoffs selection procedure for the nonparametric additive model. We integrate three key components: the knockoffs, the subsampling for stability, and the random feature mapping for nonparametric function approximation. We show that the proposed method is guaranteed to control the FDR for any sample size, and achieves a power that approaches one as the sample size tends to infinity. We demonstrate the efficacy of our method through intensive simulations and comparisons with the alternative solutions. Our proposal thus, makes useful contributions to the methodology of nonparametric variable selection, FDR-based inference, as well as knockoffs. Supplementary materials for this article are available online.