Abstract
Many current regression algorithms have unsatisfactory prediction accuracy with small samples. To solve this problem, a regression algorithm based on Nadaraya-Watson kernel regression (NWKR) is proposed. The proposed method advocates parameter selection directly from the standard deviation of training data, optimized with leave-one-out cross- validation (LOO-CV). Good generalization performance of the proposed parameter selection is demonstrated empirically using small sample regression problems with Gaussian noise. The results show that proposed parameter optimization method is more robust and accurate than other methods for different noise levels and different sample sizes, and indicate the importance of Vapnik’s e-insensitive loss for regression problems with small samples.
Highlights
This template, modified in MS Word 2007 and saved as a “Word 97-2003 Document” for the PC, provides authors with most of the formatting specifications needed for preparing electronic versions of their papers
This paper describes practical recommendations for setting meta-parameters for Nadaraya-Watson kernel regression (NWKR) regression with small samples
Empirical comparisons suggest that the proposed parameter selection method (Eq (12)) yields good generalization performance for NWKR estimates under different noise levels and sample sizes
Summary
This template, modified in MS Word 2007 and saved as a “Word 97-2003 Document” for the PC, provides authors with most of the formatting specifications needed for preparing electronic versions of their papers. ANNs (artificial neural networks) and k-nearest neighbor are widely used, and have good performance in many applications (Maxwell & Stinchcombe, 1995; Su, Jing, et al, 2008; Cho, Ishida, et al, 2011; La, Guo, et al, 2012). With small samples, the noise variance cannot be precisely estimated by any well-known approach (such as polynomial or k-nearest-neighbor regression).
Sign-in/Register to view highlights & summary 
Published Version (
Free)
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call