Regularized Kernel Forms of Minimum Squared Error Method

Abstract

Minimum squared error (MSE) algorithm is one of the classical pattern recognition and regression analysis methods, whose objective is to minimize the squared error summation between the output of linear function and the desired output. In this paper, the MSE algorithm is modified by using kernel functions satisfying the Mercer condition and regularization technique; and the nonlinear MSE algorithms based on kernels and regularization term, that is, the regularized kernel forms of MSE algorithm, are proposed. Their objective functions include the squared error summation between the output of nonlinear function based on kernels and the desired output and a proper regularization term. The regularization technique can handle ill-posed problems, reduce the solution space, and control the generalization. Three squared regularization terms are utilized in this paper. In accordance with the probabilistic interpretation of regularization terms, the difference among three regularization terms is given in detail. The synthetic and real data are used to analyze the algorithm performance.