Parameter Optimization for Nadaraya-Watson Kernel Regression Method with Small Samples

Li Fengping,Xue Wei,Zhou Yuqing

doi:10.14569/ijarai.2016.050501

Abstract

Many current regression algorithms have unsatisfactory prediction accuracy with small samples. To solve this problem, a regression algorithm based on Nadaraya-Watson kernel regression (NWKR) is proposed. The proposed method advocates parameter selection directly from the standard deviation of training data, optimized with leave-one-out cross- validation (LOO-CV). Good generalization performance of the proposed parameter selection is demonstrated empirically using small sample regression problems with Gaussian noise. The results show that proposed parameter optimization method is more robust and accurate than other methods for different noise levels and different sample sizes, and indicate the importance of Vapnik’s e-insensitive loss for regression problems with small samples.

Highlights

This template, modified in MS Word 2007 and saved as a “Word 97-2003 Document” for the PC, provides authors with most of the formatting specifications needed for preparing electronic versions of their papers
This paper describes practical recommendations for setting meta-parameters for Nadaraya-Watson kernel regression (NWKR) regression with small samples
Empirical comparisons suggest that the proposed parameter selection method (Eq (12)) yields good generalization performance for NWKR estimates under different noise levels and sample sizes

Summary

INTRODUCTION

This template, modified in MS Word 2007 and saved as a “Word 97-2003 Document” for the PC, provides authors with most of the formatting specifications needed for preparing electronic versions of their papers. ANNs (artificial neural networks) and k-nearest neighbor are widely used, and have good performance in many applications (Maxwell & Stinchcombe, 1995; Su, Jing, et al, 2008; Cho, Ishida, et al, 2011; La, Guo, et al, 2012). With small samples, the noise variance cannot be precisely estimated by any well-known approach (such as polynomial or k-nearest-neighbor regression).

NADARAYA-WATSON KERNEL REGRESSION

PARAMETER OPTIMIZATION WITH CROSS-VALIDATION

PREPARE YOUR PAPER BEFORE STYLING EXPERIMEENTAL RESULTS WITH GAUSSIAN NOISE

SELECTION METHODS AND SEVERAL NOISE LEVELS

Parameter estimation with regression model

Comparisons with other parameter selection methods

CONCLUSIONS