Kernel Regression Estimator Research Articles

A conditional distribution function-based measure for independence and [formula omitted]-sample tests in multivariate data

We introduce a new index to measure the degree of dependence and test for independence between two random vectors. The index is obtained by generalizing the Cramér-von Mises distances between the conditional and marginal distribution functions via the projection-averaging technique. If one of the random vectors is categorical with K categories, we propose slicing estimators to estimate our index. We conduct an asymptotic analysis for the slicing estimators, considering both situations where K is fixed and where K is allowed to increase with the sample size. When both random vectors are continuous, we introduce a kernel regression estimator for the proposed index, demonstrating that its asymptotic null distribution follows a normal distribution and conducting a local power analysis for the kernel estimator-based independence test. The proposed tests are studied via simulation, with a real data application presented to illustrate our methods.

Robust tests for equality of regression curves based on characteristic functions

This paper focuses on the problem of testing the null hypothesis that the regression functions of several populations are equal under a general nonparametric homoscedastic regression model. It is well known that linear kernel regression estimators are sensitive to atypical responses. These distorted estimates will influence the test statistic constructed from them so the conclusions obtained when testing equality of several regression functions may also be affected. In recent years, the use of testing procedures based on empirical characteristic functions has shown good practical properties. For that reason, to provide more reliable inferences, we construct a test statistic that combines characteristic functions and residuals obtained from a robust smoother under the null hypothesis. The asymptotic distribution of the test statistic is studied under the null hypothesis and under root − n contiguous alternatives. A Monte Carlo study is performed to compare the finite sample behaviour of the proposed test with the classical one obtained using local averages. The reported numerical experiments show the advantage of the proposed methodology over the one based on Nadaraya–Watson estimators for finite samples. An illustration to a real data set is also provided and enables to investigate the sensitivity of the p − value to the bandwidth selection.

Asymptotic normality of Nadaraya–Waton kernel regression estimation for mixing high-frequency data

High-frequency data is widely used and studied in many fields, especially in the econometrics and statistics. In this paper, the asymptotic normality of Nadaraya–Waton (NW) kernel regression estimator under ρ-mixing high-frequency data is studied. We first derive some moment inequalities for ρ-mixing high-frequency data, and then use them to study the asymptotic normality of NW kernel regression estimator, and give Berry–Esseen upper bounds. The numerical simulations report that the kernel regression estimator of high-frequency data has good asymptotic normality. Our empirical analysis is to fit the correlation between the five minute interval price increment and the corresponding trading volume for the Shanghai Stock Exchange Index, Entrepreneurship Index and Real Estate Index. These kernel regression curves better reflect people's investment behaviour.

Application of the evolutionary approach to structural and parametric identification of dynamic objects

The paper examines the evolutionary approach to structural and parametric identification of dynamic systems in the form of differential equations. The approach is based on a genetic programming algorithm to determine a structure of the equation and differential evolution method for parameters selection. The author proposed approach based on such evolutionary algorithms as genetic programming and differential evolution. The search for the structure is carried out by genetic programming. The selection of numerical parameters and initial conditions is implemented by a method of differential evolution. The problem of finding a model that describes changes in the efficiency of a hydraulic system is solved with the help of this approach. The proposed approach is compared with a recurrent neural network and a nonparametric kernel regression estimation.

Adaptive Nadaraya-Watson kernel regression estimators utilizing some non-traditional and robust measures: a numerical application of British food data

In nonparametric regression research, estimation of regression function is a prime concern. Recently, researchers developed some modified Nadaraya-Watson (N-W) regression estimators utilizing robust mean, median and harmonic mean. In this paper, we propose to utilize the additive combination of non-traditional measures i.e. (Hodges-Lehmann, Mid-Range, Tri-Mean, Quartile-Deviation) with the robust minimum covariance determinant (MCD) scale estimator in (N-W) regression estimator. Utilizing these measures, we get some new versions of (N-W) regression estimator. We also attempted to derive the properties of the proposed versions, such as bias, variance, mean square error (MSE), and mean integrated square error (MISE). The proposed estimators are compared with some of the existing estimators available in literature through a simulation study, utilizing two artificial populations. We also incorporated real-life application by taking British food data set denoted as Engel95, and assess the predictive ability of nonparametric regression, based on proposed and existing N-W estimators.

Regional forecasting of PM2.5 concentrations: A novel model based on the empirical orthogonal function analysis and Nadaraya–Watson kernel regression estimator

Predicting regional air pollutant concentrations quickly and accurately is still a challenge when meteorological data that affect pollutant concentrations are not available. To avoid modeling single points, which is tedious, it is necessary to explore more efficient and reliable regional methods with fewer input data. In this study, the principle and procedure of a new hybrid model, EOF-NWKRE, based on empirical orthogonal function (EOF) decomposition and the N–W kernel regression estimator (NWKRE) are proposed, then, the PM2.5 concentrations of 13 cities in Jiangsu Province is simulated by the EOF-NWKRE. The results show that the average prediction accuracy of EOF-NWKRE model is 74.38%, with more than 92% of the cumulative variance. The model is completely self-driven by PM2.5 data without covariate data input, which reduces the computational burden, improves the computational efficiency, and results in better prediction accuracy. It is suitable for the prediction of pollutant concentrations in a large area.

Methodology for nonparametric bias reduction in kernel regression estimation

Abstract In this paper, we propose and investigate two new kernel regression estimators based on a bias reduction transformation technique. We study the properties of these estimators and compare them with Nadaraya–Watson’s regression estimator and Slaoui’s (2016) regression estimator. It turns out that, with an adequate choice of the parameters of the two proposed estimators, the rate of convergence of two estimators will be faster than the two classical estimators, and the asymptotic MISE (mean integrated squared error) will be smaller than the two classical estimators. We corroborate these theoretical results through simulations and a real Malaria dataset.

Weak consistency for the nonparametric kernel regression estimator based on negatively associated random errors

In this article, we mainly study the weak consistency for the Gasser and Müller (G-M) kernel estimator (a weighted kernel estimator with integral form) in a nonparametric regression model based on negatively associated random errors under some suitable conditions. In addition, a simulation to study the numerical performance of the consistency for the G-M estimator is provided.

Fractal Perturbation of the Nadaraya–Watson Estimator

One of the main tasks in the problems of machine learning and curve fitting is to develop suitable models for given data sets. It requires to generate a function to approximate the data arising from some unknown function. The class of kernel regression estimators is one of main types of nonparametric curve estimations. On the other hand, fractal theory provides new technologies for making complicated irregular curves in many practical problems. In this paper, we are going to investigate fractal curve-fitting problems with the help of kernel regression estimators. For a given data set that arises from an unknown function m, one of the well-known kernel regression estimators, the Nadaraya–Watson estimator m^, is applied. We consider the case that m is Hölder-continuous of exponent β with 0<β≤1, and the graph of m is irregular. An estimation for the expectation of |m^−m|2 is established. Then a fractal perturbation f[m^] corresponding to m^ is constructed to fit the given data. The expectations of |f[m^]−m^|2 and |f[m^]−m|2 are also estimated.

A stock options metaphor for content delivery networks

We focus on the timely issue of Content Delivery Network (CDN) resource management. We introduce a multi-stage scheme that leverages solutions from the capital market. The mechanism of Stock Options (SOs) is used to address a potential scarcity of resources, not adequately addressed by other predictive mechanisms previously introduced by the authors. Using a Predictive Reservation Scheme (PRS), network resources offered by a CDN are monitored through established techniques (Kernel Regression Estimators) in a given time frame. Next, a Secondary Market (SM) significantly reduces resource waste by allowing the fast exchange of unused (remaining) resources granted to Origin Servers (OSs). This exchange occurs either by implementing socially optimal practices or by allowing automatic electronic auctions at the end of the day (EoD) or shorter time intervals. Finally, we further enhance our Load Prediction Mechanism (LPM); SOs are purchased and exercised, depending on the lack of resources at the EoD. As a result, OSs may acquire resources (if required) at a standard price. The effectiveness of the proposed stock market-based CDN resource management framework further improves.

Non-parametric Multivariate Kernel Regression Estimation to Describe Cognitive Processes and Mental Representations

In this research paper, we set forward a non-parametric multivariate recursive kernel regression estimator under missing data using the propensity score approach in order to describe writing word production. Our main objective is to explore cognitive processes and mental representations mobilized when a human being prepares to write a word according to the idea developed in Perret and Olive (2019). We investigate the asymptotic properties of the proposed recursive estimator and compare them to the well known Nadaraya-Watson’s regression estimator. We calculate the bias and the variance of the proposed estimator which depend on the choice of some parameters such as the stepsize and the bandwidth. We examine some data-driven procedures to select these parameters. Thus, we demonstrate that, under some optimal choices of these parameters, the MSE (Mean Squared Error) of the proposed estimator can be smaller than the one obtained by using Nadaraya Watson’s regression estimator. The elaborated estimator is then applied to the behavioral data to classify some participants in groups. This classification may stand for a departure point to tackle written behavior variations.

Some non‐parametric regression models for interval‐valued functional data

The interval‐valued functional data belong to the category of symbolic data in which the values of each observation consist of two functions, a lower limit function and an upper limit function. This study introduces some functional non‐parametric approaches to fit a functional regression model on interval‐valued functional data. First, using kernel functions, two non‐parametric regression models are presented to predict a scalar interval‐valued response variable based on interval‐valued functional predictors. For this purpose, a local linear regression and a kernel regression estimator of the Nadaraya–Watson type are proposed. These approaches are based on two independent non‐parametric models, which may not hold in practice. Therefore, some non‐parametric methods are introduced, including bivariate functional Nadaraya–Watson, bivariate functional locally linear smoothing and another method based on bootstrap, which maintain selecting points randomly in the intervals of the proposed data. The applicability and advantages of the proposed models are investigated through simulation experiments and a real data example.

Recursive regression estimation based on the two-time-scale stochastic approximation method and Bernstein polynomials

Abstract In this paper, we propose a recursive estimators of the regression function based on the two-time-scale stochastic approximation algorithms and the Bernstein polynomials. We study the asymptotic properties of this estimators. We compare the proposed estimators with the classic regression estimator using the Bernstein polynomial defined by Tenbusch. Results showed that, our proposed recursive estimators can overcome the problem of the edges associated with kernel regression estimation with a compact support. The proposed recursive two-time-scale estimators are compared to the non-recursive estimator introduced by Tenbusch and the performance of the two estimators are illustrated via simulations as well as two real datasets.

Kernel regression estimation for LTRC and associated data

This paper focuses on nonparametric regression modeling of time-series and incomplete observations. In this sense, the observations are subject to both left truncation and right censoring (LTRC) and satisfy an association dependency. Using the well-known kernel estimation method, we establish the strong uniform consistency with a rate of the kernel estimator proposed in this paper. Simulation studies are conducted to assess the impact of both incompleteness and association dependency on the estimation.

Potential Trade Vulnerability of Bangladesh: An Exploratory Analysis

Bangladesh's integration in the global economy has hitherto proven to be an important force behind its economic growth. However, exclusively trade dependent growth is also trade vulnerable growth. This paper explores the factors that may potentially affect the trade vulnerability of Bangladesh. Results of the bivariate non-parametric local linear kernel regression estimation reveal that the minimum wage of readymade garment workers, manufacturing cost, utilities cost, crude oil prices, and climate change are positively related to trade volatility, while improved political stability is negatively related to trade volatility. Multivariate kernel regressions with composite indicators show that macroeconomic factors, such as exchange rate, foreign exchange reserves and budget balance, have a statistically significant impact on the trade volatility of Bangladesh. In view of these preliminary results, a number of policy recommendations are suggested to deal with the short run and long run trade vulnerability of Bangladesh.

Two-time-scale nonparametric recursive regression estimator for independent functional data

In this paper, we propose and investigate a new kernel regression estimators based on the two-time-scale stochastic approximation algorithm in the case of independent functional data. We study the properties of the proposed recursive estimators and compare them with the recursive estimators based on single-time-scale stochastic algorithm proposed by Slaoui and to the non-recursive estimator proposed by Slaoui. It turns out that, with an adequate choice of the parameters, the proposed two-time-scale estimators perform better than the recursive estimators constructed using single-time-scale stochastic algorithm. We corroborate these theoretical results through some simulations and two real datasets.

A Comparison Between New Modification of ANWK and Classical ANWK Methods in Nonparametric Regression

Nonparametric kernel estimators are mostly used in a variety of statistical research fields. Nadaraya-Watson kernel estimator (NWK) is one of the most important nonparametric kernel estimator that is often used in regression models with a fixed bandwidth. In this article, we consider the four new Proposed Adaptive Nadaraya-Watson Kernel Regression Estimators (Interquartile Range, Standard Deviation, Mean Absolute Devotion, and Median Absolute Deviation) rather than (Fixed Bandwidth, Adaptive Geometric, Adaptive Mean, Adaptive Range, and Adaptive Median). The outcomes in both simulation and actual data in Leukemia Cancer show that the four new ANW Kernel Estimators (Interquartile Range, Standard Deviation, Mean Absolute devotion, and Median Absolute Deviation) is more effective than the kernel estimations with fixed bandwidth in previous studies using Mean Square Error (MSE) Criterion.

Novel kernel density estimator based on ensemble unbiased cross-validation

Unbiased cross-validation (UCV) is a commonly-used method to calculate the optimal bandwidth for the kernel density estimator (KDE), which estimates the underlying probability density function (PDF) for a given data set. Since the UCV method was proposed, there have been few studies that have pointed out its instability when determining the KDE bandwidth. Following the principle of stability improvement, this paper presents a novel ensemble UCV based KDE (EUCV-KDE), which determines the expectation of an estimated PDF using an ensemble of data-block based UCVs rather than a single data-point based UCV. To derive the optimal bandwidth, a novel objective function is designed for EUCV-KDE by considering the empirical and structural risk of KDE together. We validate the rationality and effectiveness of EUCV-KDE on 10 probability distributions. The experimental results show that EUCV-KDE is convergent as the number of data-block based UCVs increases and can obtain a more stable and better prediction performance than the classical UCV-KDE and the revisited cross-validation (RCV) based KDE (RCV-KDE). In addition, a real-world application based on UK climate data is provided to further validate the effectiveness of EUCV-KDE by determining the optimal bandwidth for Nadaraya-Watson kernel regression estimator.

A robust strategy for sensor fault detection in nuclear power plants based on principal component analysis

The principal component analysis (PCA) method has been widely used in sensor fault detection. However, outliers of training data may affect the projection directions of both principal component (PC) and residual space, thereby reducing the fault detection rate (FDR). The high sensitivity of PCA to random noise in the test data can also lead to an increase in the false alarm rate (FAR). To improve the performance of the PCA, this paper proposes a robust PCA approach for sensor fault detection in nuclear power plants (NPPs). A statistical method based on Euclidean distance is used to clean outliers in the training data pre-processing phase. Subsequently in the fault detection phase, the moving average (MA) filtering method is adopted to process Q-statistic to reduce false alarms caused by random noise in the test data. Simulation and plant signals are used to verify the effectiveness of the proposed method. Finally, comparisons with the conventional PCA, auto-associative kernel regression (AAKR) and multivariate state estimation technique (MSET) highlight the advantages of the proposed method.

Monotonicity preservation properties of kernel regression estimators

Three common classes of kernel regression estimators are considered: the Nadaraya–Watson (NW) estimator, the Priestley–Chao (PC) estimator, and the Gasser–Müller (GM) estimator. It is shown that (i) the GM estimator has a certain monotonicity preservation property for any kernel K, (ii) the NW estimator has this property if and only the kernel K is log concave, and (iii) the PC estimator does not have this property for any kernel K. Other related properties of these regression estimators are discussed.