Nonparametric Regression Method Research Articles

Association between vitamin B2 intake and prostate-specific antigen in American men: 2003–2010 National Health and Nutrition Examination Survey

BackgroundAccumulating evidence suggests a pivotal role of vitamin B2 in the pathogenesis and progression of prostate cancer (PCa). Vitamin B2 intake has been postulated to modulate the screening rate for PCa by altering the concentration of prostate-specific antigen(PSA). However, the relationship between vitamin B2 and PSA remains indeterminate. Hence, we conducted a comprehensive evaluation of the association between vitamin B2 intake and PSA levels, utilizing data from the National Health and Nutrition Examination Survey (NHANES) database.MethodsFrom a pool of 20,371 participants in the NHANES survey conducted between 2003 and 2010, a cohort of 2,323 participants was selected for the present study. The male participants were classified into four distinct groups based on their levels of vitamin B2 intake. We employed a multiple linear regression model and a non-parametric regression method to investigate the relationship between vitamin B2 and PSA levels.ResultsThe study cohort comprised of 2,323 participants with a mean age of 54.95 years (± 11.73). Our findings revealed a statistically significant inverse correlation between vitamin B2 intake (mg) and PSA levels, with a reduction of 0.13 ng/ml PSA concentration for every unit increase in vitamin B2 intake. Furthermore, we employed a fully adjusted model to construct a smooth curve to explore the possible linear relationship between vitamin B2 intake and PSA concentration.ConclusionsOur study in American men has unveiled a notable inverse association between vitamin B2 intake and PSA levels, potentially posing a challenge for the identification of asymptomatic prostate cancer. Specifically, our findings suggest that individuals with higher vitamin B2 intake may be at a greater risk of being diagnosed with advanced prostate cancer in the future, possibly indicating a detection bias. These results may offer a novel explanation for the observed positive correlation between vitamin B2 intake and prostate cancer.

Non-parametric Regression Among Factor Scores: Motivation and Diagnostics for Nonlinear Structural Equation Models.

We provide a framework for motivating and diagnosing the functional form in the structural part of nonlinear or linear structural equation models when the measurement model is a correctly specified linear confirmatory factor model. A mathematical population-based analysis provides asymptotic identification results for conditional expectations of a coordinate of an endogenous latent variable given exogenous and possibly other endogenous latent variables, and theoretically well-founded estimates of this conditional expectation are suggested. Simulation studies show that these estimators behave well compared to presently available alternatives. Practically, we recommend the estimator using Bartlett factor scores as input to classical non-parametric regression methods.

Modeling Spatiotemporal Heterogeneity of Customer Preferences With Small-Scale Aggregated Data: A Spatial Panel Modeling Approach

Abstract Customer preferences are found to evolve over time and correlate with geographical locations. Studying the spatiotemporal heterogeneity of customer preferences is crucial to engineering design as it provides a dynamic perspective for understanding the trend of customer preferences. However, existing choice models for demand modeling do not take the spatiotemporal heterogeneity of customer preferences into consideration. Learning-based spatiotemporal data modeling methods usually require large-scale datasets for model training, which are not applicable to small aggregated data, such as the sale records of a product in several regions and years. To fill this research gap, we propose a spatial panel modeling approach to investigate the spatiotemporal heterogeneity of customer preferences. Product and regional attributes varying in time are included as model inputs to support demand forecasting in engineering design. With case studies using the dataset of small SUVs and compact sedans in China's automotive market, we demonstrate that the spatial panel modeling approach outperforms other statistical spatiotemporal data models and non-parametric regression methods in goodness of fit and prediction accuracy. We also illustrate a potential design application of the proposed approach in a portfolio optimization of two vehicles from the same producer. While the spatial panel modeling approach exists in econometrics, applying this approach to support engineering decisions by considering spatiotemporal heterogeneity and introducing engineering attributes in demand forecasting is the contribution of this work. Our paper is focused on presenting the approach rather than the results per se.

Unveiling the effect of composition on nuclear waste immobilization glasses’ durability by nonparametric machine learning

Ensuring the long-term chemical durability of glasses is critical for nuclear waste immobilization operations. Durable glasses usually undergo qualification for disposal based on their response to standardized tests such as the product consistency test or the vapor hydration test (VHT). The VHT uses elevated temperature and water vapor to accelerate glass alteration and the formation of secondary phases. Understanding the relationship between glass composition and VHT response is of fundamental and practical interest. However, this relationship is complex, non-linear, and sometimes fairly variable, posing challenges in identifying the distinct effect of individual oxides on VHT response. Here, we leverage a dataset comprising 654 Hanford low-activity waste (LAW) glasses across a wide compositional envelope and employ various machine learning techniques to explore this relationship. We find that Gaussian process regression (GPR), a nonparametric regression method, yields the highest predictive accuracy. By utilizing the trained model, we discern the influence of each oxide on the glasses’ VHT response. Moreover, we discuss the trade-off between underfitting and overfitting for extrapolating the material performance in the context of sparse and heterogeneous datasets.

FINKER: Frequency Identification through Nonparametric KErnel Regression in astronomical time series

Optimal frequency identification in astronomical datasets is crucial for variable star studies, exoplanet detection, and asteroseismology. Traditional period-finding methods often rely on specific parametric assumptions, employ binning procedures, or overlook the regression nature of the problem, limiting their applicability and precision. We introduce a universal, nonparametric kernel regression method for optimal frequency determination that is generalizable, efficient, and robust across various astronomical data types. FINKER uses nonparametric kernel regression on folded datasets at different frequencies, selecting the optimal frequency by minimising squared residuals. This technique inherently incorporates a weighting system that accounts for measurement uncertainties and facilitates multi-band data analysis. We evaluated our method's performance across a range of frequencies pertinent to diverse data types and compared it with an established period-finding algorithm, conditional entropy. The method demonstrates superior performance in accuracy and robustness compared to existing algorithms, requiring fewer observations to reliably identify significant frequencies. It exhibits resilience against noise and adapts well to datasets with varying complexity.

Independence-Encouraging Subsampling for Nonparametric Additive Models

The additive model is a popular nonparametric regression method due to its ability to retain modeling flexibility while avoiding the curse of dimensionality. The backfitting algorithm is an intuitive and widely used numerical approach for fitting additive models. However, its application to large datasets may incur a high computational cost and is thus infeasible in practice. To address this problem, we propose a novel approach called independence-encouraging subsampling (IES) to select a subsample from big data for training additive models. Inspired by the minimax optimality of an orthogonal array (OA) due to its pairwise independent predictors and uniform coverage for the range of each predictor, the IES approach selects a subsample that approximates an OA to achieve the minimax optimality. Our asymptotic analyses demonstrate that an IES subsample converges to an OA and that the backfitting algorithm over the subsample converges to a unique solution even if the predictors are highly dependent in the full data. The proposed IES method is shown to be numerically appealing via simulations and a real data application. Theoretical proofs, R codes, and supplementary numerical results are accessible online as supplementary materials.

Nonparametric Additive Models for Billion Observations

The nonparametric additive model (NAM) is a widely used nonparametric regression method. Nevertheless, due to the high computational burden, classic statistical techniques for fitting NAMs are not well-equipped to handle massive data with billions of observations. To address this challenge, we develop a scalable element-wise subset selection method, referred to as Core-NAM, for fitting penalized regression spline based NAMs. Specifically, we first propose an approximation of the penalized least squares estimation, based on which we develop an efficient variant of generalized cross-validation (GCV) to select the smoothing parameter and approximate the Bayesian confidence intervals for statistical inference. Theoretically, we show that the proposed estimator approximately minimizes an upper bound of the estimation mean squared error. Moreover, we provide a non-asymptotic approximation guarantee for the proposed estimator and establish the asymptotic optimality of the proposed variant of GCV. Extensive simulations demonstrate the superior accuracy and efficiency of the Core-NAM method. We also apply the proposed method to a total column ozone dataset containing nearly one billion observations, and the results indicate a speed-up by almost a thousand times with comparable performance compared to the full data approach. Supplementary materials for this article are available online.

Can spatial patterns mitigate the urban heat island effect? Evidence from German metropolitan regions

This study examines the efficacy of urban spatial patterns at alleviating the urban heat island (UHI) effect in Germany’s city regions ( Großstadtregionen) using multivariate and non-parametric regression methods. Urban spatial patterns are quantified using five landscape metrics that capture the spatial arrangement of urban footprints and greenspaces, along with a polycentricity index that measures the distribution of human activities. The results indicate that certain features of urban fabric, including fragmentation, mixed land use, and regular-shaped urban patches, have the potential to mitigate the UHI effect. Moreover, dispersing multiple smaller greenspaces throughout the urban area demonstrates a greater cooling effect compared to having a single large and more aggregated park. In addition, our analysis reveals that a doubling (100%) of the polycentricity degree corresponds to a significant decrease in both day- and night-time UHI effects, with reductions of 10.4% and 24.6%, respectively. This study confirms that polycentric development yields greater benefits in reducing urban heat for large-sized city regions compared to medium- and small-sized ones; and its effectiveness is mostly pronounced near urban center(s). These findings suggest that polycentric development represents an efficient and feasible strategy for urban thermal planning of large-sized city regions, surpassing other commonly discussed urban configurations, such as compact or dispersed urban development.

ESTIMATION OF A BI-RESPONSE TRUNCATED SPLINE NONPARAMETRIC REGRESSION MODEL ON LIFE EXPECTANCY AND PREVALENCE OF UNDERWEIGHT CHILDREN IN INDONESIA

Researchers use the nonparametric regression method because it provides excellent flexibility in the modeling process. Nonparametric regression procedures can be used if the relationship pattern between the predictor and response variables is unknown. The truncated spline method is one of the most frequently used nonparametric regression methods. A truncated spline is a polynomial slice with continuous segmented properties, and the resulting curve is relatively smooth. The advantage of truncated splines is that they can be used on data that experience behavior changes at specific intervals. The nonparametric spline truncated bi-response regression approach is used when one or more predictor variables affect the two response variables with the assumption that there is a correlation between the response variables. This study aimed to obtain the best spline truncated bi-response nonparametric regression model on life expectancy data and the prevalence of underweight children in Indonesia in 2021. The data used comes from the Central Bureau of Statistics and the Indonesian Ministry of Health. The optimal knot point selection method uses the Generalized Cross Validation (GCV) method. The results showed that the best model formed was obtained using three-knot points based on a minimum GCV value of 22.77 and a coefficient of determination of 99.58%.

Sampling Importance Resampling Algorithm with Nonignorable Missing Response Variable Based on Smoothed Quantile Regression

The presence of nonignorable missing response variables often leads to complex conditional distribution patterns that cannot be effectively captured through mean regression. In contrast, quantile regression offers valuable insights into the conditional distribution. Consequently, this article places emphasis on the quantile regression approach to address nonrandom missing data. Taking inspiration from fractional imputation, this paper proposes a novel smoothed quantile regression estimation equation based on a sampling importance resampling (SIR) algorithm instead of nonparametric kernel regression methods. Additionally, we present an augmented inverse probability weighting (AIPW) smoothed quantile regression estimation equation to reduce the influence of potential misspecification in a working model. The consistency and asymptotic normality of the empirical likelihood estimators corresponding to the above estimating equations are proven under the assumption of a correctly specified parameter working model. Furthermore, we demonstrate that the AIPW estimation equation converges to an IPW estimation equation when a parameter working model is misspecified, thus illustrating the robustness of the AIPW estimation approach. Through numerical simulations, we examine the finite sample properties of the proposed method when the working models are both correctly specified and misspecified. Furthermore, we apply the proposed method to analyze HIV—CD4 data, thereby exploring variations in treatment effects and the influence of other covariates across different quantiles.

A new bandwidth selection method for nonparametric modal regression based on generalized hyperbolic distributions

A new bandwidth selection method for nonparametric modal regression based on generalized hyperbolic distributions

Comparative Analysis of Predictive Performance in Nonparametric Functional Regression: A Case Study of Spectrometric Fat Content Prediction

Objective: This research aims to compare two nonparametric functional regression models, the Kernel Model and the K-Nearest Neighbor (KNN) Model, with a focus on predicting scalar responses from functional covariates. Two semi-metrics, one based on second derivatives and the other on Functional Principle Component Analysis, are employed for prediction. The study assesses the accuracy of these models by computing Mean Square Errors (MSE) and provides practical applications for illustration.
 Method: The study delves into the realm of nonparametric functional regression, where the response variable (Y) is scalar, and the covariate variable (x) is a function. The Kernel Model, known as funopare.kernel.cv, and the KNN Model, termed funopare.knn.gcv, are used for prediction. The Kernel Model employs automatic bandwidth selection via Cross-Validation, while the KNN Model employs a global smoothing parameter. The performance of both models is evaluated using MSE, considering two different semi-metrics.
 Results: The results indicate that the KNN Model outperforms the Kernel Model in terms of prediction accuracy, as supported by the computed MSE. The choice of semi-metric, whether based on second derivatives or Functional Principle Component Analysis, impacts the model's performance. Two real-world applications, Spectrometric Data for predicting fat content and Canadian Weather Station data for predicting precipitation, demonstrate the practicality and utility of the models.
 Conclusion: This research provides valuable insights into nonparametric functional regression methods for predicting scalar responses from functional covariates. The KNN Model, when compared to the Kernel Model, offers superior predictive performance. The selection of an appropriate semi-metric is essential for model accuracy. Future research may explore the extension of these models to cases involving multivariate responses and consider interactions between response components.

Calibrating Sales Forecasts in a Pandemic Using Competitive Online Nonparametric Regression

Motivated by our collaboration with Anheuser-Busch InBev (AB InBev), a consumer packaged goods (CPG) company, we consider the problem of forecasting sales under the coronavirus disease 2019 (COVID-19) pandemic. Our approach combines nonparametric regression, game theory, and pandemic modeling to develop a data-driven competitive online nonparametric regression method. Specifically, the method takes the future COVID-19 case estimates, which can be simulated via the susceptible-infectious-removed (SIR) epidemic model as an input, and outputs the level of calibration for the baseline sales forecast generated by AB InBev. In generating the calibration level, we focus on an online learning setting where our algorithm sequentially predicts the label (i.e., the level of calibration) of a random covariate (i.e., the current number of active cases) given past observations and the generative process (i.e., the SIR epidemic model) of future covariates. To provide robust performance guarantee, we derive our algorithm by minimizing regret, which is the difference between the squared [Formula: see text]-norm associated with labels generated by the algorithm and labels generated by an adversary and the squared [Formula: see text]-norm associated with labels generated by the best isotonic (nondecreasing) function in hindsight and the adversarial labels. We develop a computationally efficient algorithm that attains the minimax-optimal regret over all possible choices of the labels (possibly non-i.i.d. and even adversarial). We demonstrate the performances of our algorithm on both synthetic and AB InBev’s data sets of three different markets (each corresponds to a country) from March 2020 to March 2021. The AB InBev’s numerical experiments show that our method is capable of reducing the forecast error in terms of weighted mean absolute percentage error (WMAPE) and mean squared error (MSE) by more than 37% for the company. This paper was accepted by J. George Shanthikumar, data science. Funding: This work was partially supported by the Massachusetts Institute of Technology Data Science Lab and AB-InBev Corporation. Supplemental Material: The e-companion and data are available at https://doi.org/10.1287/mnsc.2023.4969 .

Bayesian and Classical Semi-parametric Estimation of the Balanced Longitudinal Data Model

The primary objective of this study is to employ semi-parametric regression techniques in the balanced longitudinal data model. Where the parametric regression models are plagued by the problem of strict constraints, while non-parametric regression models, despite their flexibility, suffer from the problem of the curse of dimensionality. Consequently, semi-parametric regression presents a suitable solution to address the problems in parametric and non-parametric regression methods. The advantage of this model is that it contains all the positive properties included in the previous two models such as containing strict restrictions in its parametric component, complete flexibility in its non-parametric component, and clarity of the interaction between its parametric and non-parametric components. According to the above, two methods were used to estimate a semi-parametric balanced longitudinal data model. The first is the Bayesian estimating method; the second is the Speckman method, which estimated the unknown nonparametric smoothing function by employing the kernel smoothing Nadaraya & Watson method. The Aim was to make a comparison between the Bayesian estimation method and the classical estimation method. Based on simulation experiments conducted on three different sample sizes (50, 100, and 200), it was concluded that the Bayes method is best at the variance levels (1,5). In contrast, the Profile least square method is best at the variance level (10).

Forecasting Day-Ahead Electricity Prices for the Italian Electricity Market Using a New Decomposition—Combination Technique

Over the last 30 years, day-ahead electricity price forecasts have been critical to public and private decision-making. This importance has increased since the global wave of deregulation and liberalization in the energy sector at the end of the 1990s. Given these facts, this work presents a new decomposition–combination technique that employs several nonparametric regression methods and various time-series models to enhance the accuracy and efficiency of day-ahead electricity price forecasting. For this purpose, first, the time-series of the original electricity prices deals with the treatment of extreme values. Second, the filtered series of the electricity prices is decomposed into three new subseries, namely the long-term trend, a seasonal series, and a residual series, using two new proposed decomposition methods. Third, we forecast each subseries using different univariate and multivariate time-series models and all possible combinations. Finally, the individual forecasting models are combined directly to obtain the final one-day-ahead price forecast. The proposed decomposition–combination forecasting technique is applied to hourly spot electricity prices from the Italian electricity-market data from 1 January 2014 to 31 December 2019. Hence, four different accuracy mean errors—mean absolute error, mean squared absolute percent error, root mean squared error, and mean absolute percent error; a statistical test, the Diebold–Marino test; and graphical analysis—are determined to check the performance of the proposed decomposition–combination forecasting method. The experimental findings (mean errors, statistical test, and graphical analysis) show that the proposed forecasting method is effective and accurate in day-ahead electricity price forecasting. Additionally, our forecasting outcomes are comparable to those described in the literature and are regarded as standard benchmark models. Finally, the authors recommended that the proposed decomposition–combination forecasting technique in this research work be applied to other complicated energy market forecasting challenges.

SELECTION OF THE BEST B-SPLINE REGRESSION MODEL FOR ESTIMATING BITCOIN PRICE INCREASES BASED ON ORDER AND OPTIMAL KNOT POINT

The cryptographic virtual currency, bitcoin, is considered the main originator of cryptocurrencies that emerged due United States financial crisis in 2008. The idea was sparked by Nakamoto by introducing an alternative currency system that really refers to the strength of supply and demand. Based on INDODAX data, the bitcoin exchange rate during October 2020 to February 2021 is a condition of a large increase in a short time with a percentage increase of 450%. The increase in bitcoin prices can be modelled using the b-spline nonparametric regression method based on order and optimal knot points based on the smallest Generalized Cross Validation value. The resulting b-spline 4 degree and the number of knots points 5 as the best model with each bases described recursively.

Model selection in multivariate adaptive regressions splines (MARS) using alternative information criteria

Multivariate Adaptive Regression Splines (MARS) is a useful non-parametric regression analysis method that can be used for model selection in high-dimensional data. Since MARS can identify and model complex, non-linear relationships between the dependent variable and independent variables without requiring any assumptions, it has advantage over simple linear regression techniques. Also, for simplifying the model building process and preventing overfitting, MARS can select automatically the variables to be included in the model, which is useful for datasets with many variables. While MARS is a flexible non-parametric regression method, generalized cross validation (GCV) technique is used within the MARS framework to avoid overfitting and to select the best model. GCV criterion is widely used and can be effective in many situations, however it has some criticism. These criticism are the arbitrary value of the smoothing parameter used in the algorithm of the GCV criterion and the models obtained using this criterion are high-dimensional. In this paper, it is aimed to obtain the barest model that best explains the relationship between the dependent variable and independent variables by using alternative information criteria (Akaike information criterion (AIC), Schwarz Bayesian criterion (SBC) and information complexity criterion (ICOMP(IFIM)PEU)) instead of the use of smoothing parameters in order to put an end to the criticism. To achieve this goal, a simulation study was first conducted with a data set composed of variables that do and do not contribute to the dependent variable to test the success of the information criteria. As a consequence of this simulation work, when variables (which do not contribute to the dependent variable) are not included in the regression model, it demonstrates the success of the criteria in model selection. As a real data set, the reasons for loan defaults were investigated between the years 2005–2019 by utilizing data from 18 banks operating in Türkiye. The results obtained reveal the success of ICOMP(IFIM)PEU criterion in model selection.

A unified model specification for sparse and dense functional/longitudinal data

The semi varying-coefficient additive model is a flexible nonparametric regression method, including varying-coefficient model and additive model as its special cases. However, a complex model may lead over-fitting phenomenon, which motivates us to develop a set of testing procedure to judge whether a parsimonious submodel is sufficient or not. Specifically, we propose hypothesis testings to check time-varying property of varying-coefficient component functions and linearity of additive component functions, respectively. For repeated measurements data, it is a subjective choice between sparse and dense case, which may lead to wrong statistical inference. One major contribution of this paper is to introduce consistent testing methodologies in a unified framework of sparse, dense and ultra dense cases of the data, which avoids a subjective choice of data types in practical applications. Extensive Monte Carlo simulation studies investigating the finite sample performance of the proposed methodologies confirm our asymptotic results. We further illustrate our methodologies via two real-life data applications.

Estimation of Winter Wheat Plant Nitrogen Concentration from UAV Hyperspectral Remote Sensing Combined with Machine Learning Methods

Nitrogen is one of the most important macronutrients and plays an essential role in the growth and development of winter wheat. It is very crucial to diagnose the nitrogen status timely and accurately for applying a precision nitrogen management (PNM) strategy to the guidance of nitrogen fertilizer in the field. The main purpose of this study was to use three different prediction methods to evaluate winter wheat plant nitrogen concentration (PNC) at booting, heading, flowering, filling, and the whole growth stage in the Guanzhong area from unmanned aerial vehicle (UAV) hyperspectral imagery. These methods include (1) the parametric regression method; (2) linear nonparametric regression methods (stepwise multiple linear regression (SMLR) and partial least squares regression (PLSR)); and (3) machine learning methods (random forest regression (RFR), support vector machine regression (SVMR), and extreme learning machine regression (ELMR)). The purpose of this study was also to pay attention to the impact of different growth stages on the accuracy of the model. The results showed that compared with parametric regression and linear nonparametric regression, the machine learning regression method could evidently improve the estimation accuracy of winter wheat PNC, especially using SVMR and RFR, the training set of the model at flowering and filling stage explained 93% and 92% of the PNC variability respectively. The testing set of the model at flowering and filling stages explained 88% and 91% of the PNC variability, the root mean square error of the validation set (RMSEtesting) was 0.82 and 1.23, and the relative prediction deviation (RPD) was 2.58 and 2.40, respectively. Therefore, a conclusion was drawn that it was the best choice to estimate winter wheat PNC at the flowering and filling stage from UAV hyperspectral imagery. Using machine learning methods, SVMR and RFR, respectively, could achieve the most outstanding estimation performance, which could provide a theoretical basis for putting forward the PNM strategy.

Kernel Nonparametric Regression for Forecasting Local Original Income

Regional Original Revenue (ROR) is an income collected based on regional regulations under statutory regulations. ROR aims to give authority to Regional Governments to sponsor the implementation of regional autonomy following regional potential. Every year, the Central Lombok Regency government sets ROR targets to assist the government in formulating regional policies. The targets set by the government are sometimes not following their realization. This study aims to determine a model that can be used in forecasting ROR targets. One way to predict the value of ROR is by using a nonparametric regression approach. This approach is flexible since it is not dependent on a particular model. The use of the nonparametric kernel regression method with the Gaussian kernel function obtained a minimum GCV value of 1,769688931 with an optimum bandwidth value of of 0,212740452 and of 0,529682589. Modeling with optimum bandwidth produces a coefficient of determination of 87,55%. The best model is used for forecasting and produces a MAPE value of 5,4%. The analysis results show that what influences the value of ROR is ROR receipts in the previous month and the previous 12 months.