Abstract
Kernel regression models have been used as non-parametric methods for fitting experimental data. However, due to their non-parametric nature, they belong to the so-called “black box” models, indicating that the relation between the input variables and the output, depending on the kernel selection, is unknown. In this paper we propose a new methodology to retrieve the relation between each input regressor variable and the output in a least squares support vector machine (LS-SVM) regression model. The method is based on oblique subspace projectors (ObSP), which allows to decouple the influence of input regressors on the output by including the undesired variables in the null space of the projection matrix. Such functional relations are represented by the nonlinear transformation of the input regressors, and their subspaces are estimated using appropriate kernel evaluations. We exploit the properties of ObSP in order to decompose the output of the obtained regression model as a sum of the partial nonlinear contributions and interaction effects of the input variables, we called this methodology Nonlinear ObSP (NObSP). We compare the performance of the proposed algorithm with the component selection and smooth operator (COSSO) for smoothing spline ANOVA models. We use as benchmark 2 toy examples and a real life regression model using the concrete strength dataset from the UCI machine learning repository. We showed that NObSP is able to outperform COSSO, producing stable estimations of the functional relations between the input regressors and the output, without the use of prior-knowledge. This methodology can be used in order to understand the functional relations between the inputs and the output in a regression model, retrieving the physical interpretation of the regression models.
Highlights
Non-parametric regression is an important field of data analysis
If a basis for the subspaces of each individual input regressor and their interaction effects can be found, appropriate projection matrices can be constructed in order to retrieve the interpretability of the models. This idea has been previously exploited in the case of linear models, where we have proposed to use a Hankel expansion of the input variables to construct a basis for their subspaces, and construct oblique subspace projectors in order to effectively decouple the dynamics of undesired input variables, representing the output as a sum of the partial linear contributions of each input [13]
We show how to create a basis for the subspaces spanned by the nonlinear transformation of the input regressors, main effects as well as interaction effects, using kernel evaluations
Summary
These models retrieve interpretability to the case of generalized linear models [8], where a direct link between the contribution of each input in the output is estimated explicitly These models require the use of prior knowledge in terms of which variables should be included in the regression models, as well as which interaction effects are of importance. From a geometry point of view, interpretability of non-parametric regression models can be addressed as the decomposition of a target observation vector into additive components [12] Each of these components should lie in the subspaces that are spanned by the respective input regressors. With some abuse of notation, we will refer to the projector matrix onto the subspace of the lth input regressors, along the subspace spanned by the other regressors as Pl/(l), where the subindex l represents the subspace of the regressor where the output will be projected, the subindex (l) represents all the input regressors excluding the lth, and the symbol / represents the oblique projection
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
