Abstract
This paper discusses the role of primal and (Lagrange) dual model representations in problems of supervised and unsupervised learning. The specification of the estimation problem is conceived at the primal level as a constrained optimization problem. The constraints relate to the model which is expressed in terms of the feature map. From the conditions for optimality one jointly finds the optimal model representation and the model estimate. At the dual level the model is expressed in terms of a positive definite kernel function, which is characteristic for a support vector machine methodology. It is discussed how least squares support vector machines are playing a central role as core models across problems of regression, classification, principal component analysis, spectral clustering, canonical correlation analysis, dimensionality reduction and data visualization.
Highlights
The use of kernel methods has a long history and tradition in mathematics and statistics with fundamental contributions made by Moore, Aronszajn, Krige, Parzen, Kimeldorf and Wahba, and others [7, 20, 45, 56, 69, 86]
The specification of the estimation problem at the primal level is done by formulating a constrained optimization problem, where the model is expressed in terms of a feature map
In this paper we have discussed problems in supervised and unsupervised learning which can be conceived in terms of primal and dual model representations, respectively involving a high dimensional feature map and positive definite kernel functions
Summary
The use of kernel methods has a long history and tradition in mathematics and statistics with fundamental contributions made by Moore, Aronszajn, Krige, Parzen, Kimeldorf and Wahba, and others [7, 20, 45, 56, 69, 86]. Through the choice of an appropriate loss function one obtains a sparse representation in SVMs. A subset of the given training data constitutes the set of support vectors, which follows from solving a convex quadratic programming problem. Extend support vector machine methodologies to a wide range of problems in supervised and unsupervised learning (regression, classification, principal component analysis, canonical correlation analysis, spectral clustering) and in dynamical systems (identification of different model structures, recurrent networks, optimal control) and others;. The emphasis of this paper is on illustrating the main concepts and potential of models with primal and dual representations, in particular for LS-SVMs and in connection to other methods In general this may contribute to achieving an integrative understanding of the subject given its multi-disciplinary nature, being at the intersection of machine learning and neural networks, mathematics and statistics, pattern recognition and signal processing, systems and control, optimization and others.
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
