Nonparametric Regression Function Research Articles

Properties of Test Statistics for Nonparametric Cointegrating Regression Functions Based on Subsamples

Nonparametric cointegrating regression models have been extensively used in financial markets, stock prices, heavy traffic, climate data sets, and energy markets. Models with parametric regression functions can be more appealing in practice compared to non-parametric forms, but do result in potential functional misspecification. Thus, there exists a vast literature on developing a model specification test for parametric forms of regression functions. In this paper, we develop two test statistics which are applicable for the endogenous regressors driven by long memory and semi-long memory input shocks in the regression model. The limit distributions of the test statistics under these two scenarios are complicated and cannot be effectively used in practice. To overcome this difficulty, we use the subsampling method and compute the test statistics on smaller blocks of the data to construct their empirical distributions. Throughout, Monte Carlo simulation studies are used to illustrate the properties of test statistics. We also provide an empirical example of relating gross domestic product to total output of carbon dioxide in two European countries.

Comparative Analysis of Predictive Performance in Nonparametric Functional Regression: A Case Study of Spectrometric Fat Content Prediction

Objective: This research aims to compare two nonparametric functional regression models, the Kernel Model and the K-Nearest Neighbor (KNN) Model, with a focus on predicting scalar responses from functional covariates. Two semi-metrics, one based on second derivatives and the other on Functional Principle Component Analysis, are employed for prediction. The study assesses the accuracy of these models by computing Mean Square Errors (MSE) and provides practical applications for illustration.
 Method: The study delves into the realm of nonparametric functional regression, where the response variable (Y) is scalar, and the covariate variable (x) is a function. The Kernel Model, known as funopare.kernel.cv, and the KNN Model, termed funopare.knn.gcv, are used for prediction. The Kernel Model employs automatic bandwidth selection via Cross-Validation, while the KNN Model employs a global smoothing parameter. The performance of both models is evaluated using MSE, considering two different semi-metrics.
 Results: The results indicate that the KNN Model outperforms the Kernel Model in terms of prediction accuracy, as supported by the computed MSE. The choice of semi-metric, whether based on second derivatives or Functional Principle Component Analysis, impacts the model's performance. Two real-world applications, Spectrometric Data for predicting fat content and Canadian Weather Station data for predicting precipitation, demonstrate the practicality and utility of the models.
 Conclusion: This research provides valuable insights into nonparametric functional regression methods for predicting scalar responses from functional covariates. The KNN Model, when compared to the Kernel Model, offers superior predictive performance. The selection of an appropriate semi-metric is essential for model accuracy. Future research may explore the extension of these models to cases involving multivariate responses and consider interactions between response components.

Semi-parametric single-index predictive regression models with cointegrated regressors

This paper considers the estimation of a semi-parametric single-index regression model that allows for nonlinear predictive relationships. This model is useful for predicting financial asset returns, whose observed behaviour resembles a stationary process, if the multiple nonstationary predictors are cointegrated. The presence of cointegrated regressors imposes a single-index structure in the model, and this structure not only balances the nonstationarity properties of the multiple predictors with the stationarity properties of asset returns but also avoids the curse of dimensionality associated with nonparametric regression function estimation. An orthogonal series expansion is used to approximate the unknown link function for the single-index component. We consider the constrained nonlinear least squares estimator of the single-index (or the cointegrating) parameters and the plug-in estimator of the link function, and derive their asymptotic properties. In an empirical application, we find some evidence of in-sample nonlinear predictability of U.S. stock returns using cointegrated predictors. We also find that the single-index model in general produces better out-of-sample forecasts than both the historical average benchmark and the linear predictive regression model.

Detecting change structures of nonparametric regressions

This research investigates detecting change points of general nonparametric regression functions by introducing a novel criterion. It is based on the moving sums of conditional expectation to avoid both computationally expensive algorithms, exhaustive search methods need, and false positives hypothesis testing-based approaches encounter. This new criterion can simultaneously and consistently, in a certain sense, detect multiple change points and their locations even when, as the sample size goes to infinity, the number of changes grows up to infinity, and some changes tend to zero. Further, because of its visualization nature, in practice, the locations can be relatively more easily identified, by plotting its signal statistic, than existing methods in the literature. Numerical studies are conducted to examine its performance in finite sample scenarios, and a real data example is analyzed for illustration.

Disparities in sleep-wake patterns by labor force status: Population-based findings

ABSTRACT Sleep disturbances have been associated with unemployment, but variation in sleep-wake patterns by labor force status has rarely been examined. With a population-based sample, we investigated differences in sleep-wake patterns by labor force status (employed, unemployed, and not-in-the-labor-force) and potential disparities by sociodemographic variables. The analysis included 130,602 adults aged 25–60 y, who participated in the American Time Use Survey between 2003 and 2019. Individual sleep-wake pattern was extracted from time use logs in a strict 24-h period (04:00 h–03:59 h). Functional nonparametric regression models based on dimensionality reduction and neighborhood matching were applied to model the relationship between sleep-wake patterns and labor force status. Specifically, we predicted changes in intra-person sleep-wake patterns under hypothetical changes of labor force status from employed to unemployed or not-in-the-labor-force. We then studied moderations of this association by gender, race/ethnicity and educational attainment. In comparison to the employed state, unemployed and not-in-the-labor-force states were predicted to have later wake-times, later bedtimes, and higher tendency for taking midday naps. Changes in labor force status led to more apparent shifts in wake-times than in bedtimes. Additionally, sleep schedules of Hispanics and those with higher education level were more vulnerable to the change of labor force status from employed to unemployed.

A spline-assisted semiparametric approach to nonparametric measurement error models

A spline-assisted approach is proposed to handle the measurement error problem in treating the pollution and asthma data. It is well known that the minimax rate of convergence in nonparametric regression function estimation of a random variable measured with error is much slower than the rate in the error free case. A different problem is considered. It is shown that if one is willing to impose a relatively mild assumption in requiring that the error-prone variable has a compact support, then standard nonparametric results are obtainable for measurement error models. New and constructive methods to take full advantage of the compact support assumption via spline-assisted semiparametric methods are proposed. It is proven that the new estimator achieves the usual nonparametric rate as if there were no measurement error. Furthermore it is shown that similar spline approach can be used to assess the probability density function on a compact set, using observations with error, and the resulting estimator differs from the true density function only by a constant scale. In addition, it retains the nonparametric density convergence properties of the error free case. The performance of the new methods is demonstrated through simulations and the methods are implemented to analyze the relation between asthma and pollution.

Generalized kernel regularized least squares estimator with parametric error covariance

A two-step estimator of a nonparametric regression function via Kernel regularized least squares (KRLS) with parametric error covariance is proposed. The KRLS, not considering any information in the error covariance, is improved by incorporating a parametric error covariance, allowing for both heteroskedasticity and autocorrelation, in estimating the regression function. A two step procedure is used, where in the first step, a parametric error covariance is estimated by using KRLS residuals and in the second step, a transformed model using the error covariance is estimated by KRLS. Theoretical results including bias, variance, and asymptotics are derived. Simulation results show that the proposed estimator outperforms the KRLS in both heteroskedastic errors and autocorrelated errors cases. An empirical example is illustrated with estimating an airline cost function under a random effects model with heteroskedastic and correlated errors. The derivatives are evaluated, and the average partial effects of the inputs are determined in the application.

A Functional Approach for Analyzing Time-Dependent Driver Response Behavior to Real-World Connected Vehicle Warnings

Understanding driver response behavior to Connected Vehicle (CV) warnings is one of the fundamental tasks in improving road safety. However, previous studies often used aggregated safety measures and simulation environments to accomplish this task. Consequently, time-dependent driver response behavior under real-world conditions has rarely been investigated. This study proposes a new functional data analysis (FDA) approach to analyze time-dependent driver response behavior to CV warnings obtained from the New York City CV Pilot Deployment (NYC CVPD) project. Sparse functional design that can account for irregularly spaced functional measurements is adopted to accommodate the potential noise contaminated real-world CV data. Functional principal component analysis and nonparametric functional linear regression are used to smooth raw driver behavior profile and estimate time-dependent effect of driver response behavior, respectively. The speed compliance application implemented in the NYC CVPD project is used as the case study. By modeling time series of speed data after receiving warnings as continuous functions, the proposed FDA approach reveals new patterns of driver response behavior over time, such as the diminishing effect of drivers’ speed reduction and the increase in variability of driver response behavior over time, which have rarely been explored using traditional approaches. Moreover, the proposed FDA approach supports the extraction of certain traditional safety measures and has the potential to be generalized to analyze various CV applications. The findings of this study can support the calibration of detailed driver behavior in CV environments and facilitate better CV application design and further investigation of the benefits of CVs.

Distributional data analysis of accelerometer data from the NHANES database using nonparametric survey regression models

Abstract The aim of this paper is twofold. First, a new functional representation of accelerometer data of a distributional nature is introduced to build a complete individualized profile of each subject’s physical activity levels. Second, we extend two nonparametric functional regression models, kernel smoothing and kernel ridge regression, to handle survey data and obtain reliable conclusions about the influence of physical activity. The advantages of the proposed distributional representation are demonstrated through various analyses performed on the NHANES cohort, which possesses a complex sampling design.

THE USE OF PENALIZED WEIGHTED LEAST SQUARE TO OVERCOME CORRELATIONS BETWEEN TWO RESPONSES

The non-parametric regression model can consider two correlated responses. However, for these conditions, we cannot use the usual estimation process because there are violations of assumptions. To solve this problem, we use a penalized weighted least square involving knots, smoothing parameters, and weighting in the estimation criteria simultaneously. The estimation process involves a weighted criteria matrix in the estimation criteria. Estimation results show that the estimated two-response non-parametric regression function with penalized spline is a linear estimation class in y response observation and depends on the knot point and smoothing parameter. Furthermore, the use of the model on toddler growth data shows some changes in the pattern of weight and height gain. The pattern segmentation that experienced a gradual increase was age 7-43 months for weight and age 6-54 months for height

Nonparametric Functional Data Analysis for Forecasting Container Throughput: The Case of Shanghai Port

Transportation is one of the major carbon sources in China. Container throughput is one of the main influencing factors of ports’ carbon emission budget, and accurate prediction of container throughput is of great significance to the study of carbon emissions. Time series methods are key techniques and frequently used for container throughput. However, the existing time series methods treat container throughput data as discrete points and ignore the functional characteristics of the data. There has recently been interest in developing new statistical methods to predict time series by taking into account a continuous set of past values as predictors. In addition, to eliminate the linear constraint in the functional time series prediction approach, we propose a functional version of a nonparametric model that allows using a continuous path in the past to predict future values of the process, including functional nonparametric regression and functional conditional quantile and functional conditional mode models, to forecast the container throughput of Shanghai Port. For the purpose better forecasting, an experiment was conducted to compare our functional data analysis approaches with other forecasting methods. The results indicated that nonparametric functional forecasting methods exhibit more significant performance than other classical models, including the functional linear regression model, nonparametric regression model, and autoregressive integrated moving-average model. At the same time, we also compared the prediction accuracy of the three nonparametric functional methods.

The trimmed mean in non-parametric regression function estimation

This article studies a trimmed version of the Nadaraya–Watson estimator for the unknown non-parametric regression function. The characterization of the estimator through the minimization problem is established, and its pointwise asymptotic distribution is derived. The robustness property of the proposed estimator is also studied through the breakdown point. Moreover, similar to the trimmed mean in the location model, and for a wide range of trimming proportion, the proposed estimator possesses good efficiency and high breakdown point, which is out of the ordinary properties for any estimator. Furthermore, the usefulness of the proposed estimator is shown for two benchmark real data and various simulated data.

Nonparametric Testing for Hammerstein Systems

We examine the problem of testing a parametric form of the nonlinearity of Hammerstein systems against a nonparametric alternative. This article proposes test statistics relying on nonparametric orthogonal series regression function estimates. An asymptotic theory of the proposed tests is established including the asymptotic normality under the null hypothesis that the parametric form of the system nonlinearity is correct. Moreover, it is shown that under nonparametric alternatives, the probability of detecting that the parametric model is incorrect tends to one. The rate of detecting local alternatives is also established.

Nonparametric inverse-probability-weighted estimators based on the highly adaptive lasso.

Inverse-probability-weighted estimators are the oldest and potentially most commonly used class of procedures for the estimation of causal effects. By adjusting for selection biases via a weighting mechanism, these procedures estimate an effect of interest by constructing a pseudopopulation in which selection biases are eliminated. Despite their ease of use, these estimators require the correct specification of a model for the weighting mechanism, are known to be inefficient, and suffer from the curse of dimensionality. We propose a class of nonparametric inverse-probability-weighted estimators in which the weighting mechanism is estimated via undersmoothing of the highly adaptive lasso, a nonparametric regression function proven to converge at nearly -rate to the true weighting mechanism. We demonstrate that our estimators are asymptotically linear with variance converging to the nonparametric efficiency bound. Unlike doubly robust estimators, our procedures require neither derivation of the efficient influence function nor specification of the conditional outcome model. Our theoretical developments have broad implications for the construction of efficient inverse-probability-weighted estimators in large statistical models and a variety of problem settings. We assess the practical performance of our estimators in simulation studies and demonstrate use of our proposed methodology with data from a large-scale epidemiologicstudy.

Testing stochastic dominance with many conditioning variables

We propose tests of the conditional first- and second-order stochastic dominance in the presence of growing numbers of covariates. Our approach builds on a semiparametric location-scale model, where the conditional distribution of the outcome given the covariates is characterized by nonparametric mean and skedastic functions with independent innovations from an unknown distribution. The nonparametric regression functions are estimated by utilizing the ℓ1-penalized nonparametric series estimation with thresholding. Deviation bounds for the regression functions and series coefficients estimates are obtained allowing for the time series dependence. We propose test statistics, which are the maximum (integrated) deviation of a composite of the estimated regression functions and the residual empirical distribution, and introduce a smooth stationary bootstrap to compute p-values. We investigate the finite sample performance of the bootstrap critical values by a set of Monte Carlo simulations. Finally, our method is illustrated by an application to stochastic dominance among portfolio returns given all the past information.

Estimation and Model Selection for Nonparametric Function-on-Function Regression

Regression models with a functional response and functional covariate have received significant attention recently. While various nonparametric and semiparametric models have been developed, there is an urgent need for model selection and diagnostic methods. In this article, we develop a unified framework for estimation and model selection in nonparametric function-on-function regression. We propose a general nonparametric functional regression model with the model space constructed through smoothing spline analysis of variance (SS ANOVA). The proposed model reduces to some of the existing models when selected components in the SS ANOVA decomposition are eliminated. We propose new estimation procedures under either L 1 or L 2 penalty and show that the combination of the SS ANOVA decomposition and L 1 penalty provides powerful tools for model selection and diagnostics. We establish consistency and convergence rates for estimates of the regression function and each component in its decomposition under both the L 1 and L 2 penalties. Simulation studies and real examples show that the proposed methods perform well. Technical details and additional simulation results are available in online supplementary materials.

K-Nearest Neighbor Estimation of Functional Nonparametric Regression Model under NA Samples

Functional data, which provides information about curves, surfaces or anything else varying over a continuum, has become a commonly encountered type of data. The k-nearest neighbor (kNN) method, as a nonparametric method, has become one of the most popular supervised machine learning algorithms used to solve both classification and regression problems. This paper is devoted to the k-nearest neighbor (kNN) estimators of the nonparametric functional regression model when the observed variables take values from negatively associated (NA) sequences. The consistent and complete convergence rate for the proposed kNN estimator is first provided. Then, numerical assessments, including simulation study and real data analysis, are conducted to evaluate the performance of the proposed method and compare it with the standard nonparametric kernel approach.

Distribution free prediction intervals for multiple functional regression

My research aims to establish a method of constructing prediction intervals for a scalar response of interest when predictors are functional data, with minimal distributional and modeling assumptions. To accommodate flexible regression relationships, we integrated nonparametric functional regression based on functional principal component analysis into our conformal prediction method, and we developed nonparametric functional regression approaches based on the signature method, which is a mathematical tool to represent the information contained in the functions by a collection of iterated integrals. The prediction intervals constructed by the conformal method have guaranteed coverage (confidence) without the heavy restrictions on the error distribution and on the regression function, while the efficiency (implied by the length of the intervals) will depend on the representation and information compression of the functional predictors. Conditions necessary for efficiency and contiguity of the prediction set are discussed. Finally, our methods are illustrated using simulated and real data examples.

K-Nearest Neighbor Method with Principal Component Analysis for Functional Nonparametric Regression

This paper proposed a new method to study functional non-parametric regression data analysis with conditional expectation in the case that the covariates are functional and the Principal Component Analysis was utilized to de-correlate the multivariate response variables. It utilized the formula of the Nadaraya Watson estimator (K-Nearest Neighbour (KNN)) for prediction with different types of the semi-metrics, (which are based on Second Derivative and Functional Principal Component Analysis (FPCA)) for measureing the closeness between curves. Root Mean Square Errors is used for the implementation of this model which is then compared to the independent response method. R program is used for analysing data. Then, when the covariates are functional and the Principal Component Analysis was utilized to de-correlate the multivariate response variables model, results are more preferable than the independent response method. The models are demonstrated by both a simulation data and real data.

Using the wavelet analysis to estimate the nonparametric regression model in the presence of associated errors

The wavelet shrink estimator is an attractive technique when estimating the nonparametric regression functions, but it is very sensitive in the case of a correlation in errors. In this research, a polynomial model of low degree was used for the purpose of addressing the boundary problem in the wavelet reduction in addition to using flexible threshold values in the case of Correlation in errors as it deals with those transactions at each level separately, unlike the comprehensive threshold values that deal with all levels simultaneously, as (Visushrink) methods, (False Discovery Rate) method, (Improvement Thresholding) and (Sureshrink method), as the study was conducted on real monthly data represented in the rates of theft crimes for juveniles in Iraq, specifically the Baghdad governorate, and the risk ratios about those crimes for the years 2008-2018, with a sample size of (128) (Sureshrink) The study also showed an increase in the rate of theft crimes for juveniles in recent years.