Kernel PLS Regression II: Kernel Partial Least Squares Regression by Projecting Both Independent and Dependent Variables into Reproducing Kernel Hilbert Space

Abstract

we propose a regression method using partial least square (PLS) technique and kernel method, which we refer to as kernel partial least square regression II to distinguish the conventional kernel PLS regression. The motivation of the conventional kernel PLS regression is to establish a coordinator system where the independent and dependent variables have a stronger correlation, and which models a regression model using the coordinator system. However, it projects independent variables into a reproducing kernel Hilbert space (RKHS) alone. The proposal extends the basic framework of conventional kernel PLS regression. The proposed method does not only project independent variables into the RKHS, but also dependent variables, and establishes a coordinator system where the independent and dependent variables have a stronger correlation. We use two function regression cases to evaluate the proposed method compared with the conventional kernel PLS regression. The regression performance of the proposed method has almost the same regression accuracy arising from the evaluation result, and this depends on the regression tasks. We explain the correlation calculations of our proposed method, conventional kernel PLS regression, and PLS regression. The meaning of correlation depends on the application in question. We also analyse and discuss the algorithm implementation, correlation meaning, and other issues for further development of the proposal.