Abstract
Relevance vector machine (RVM) is a Bayesian sparse kernel method for regression in statistical learning theory, which can avoid principal limitations of the support vector machine (SVM) and result in faster performance on test data while maintaining comparable generalization error. In this paper, we develop a logic framework of the evidence function approximation associated with RVM based on Taylor expansion instead of traditional technology called “completing the square.” While constructing the term of completing the square, we have to find the term of completing the square by making use of some skill, which in practice increases the difficulty in dealing with the evidence function approximation associated with RVM. The logical framework in this paper based on Taylor expansion shows some advantages compared with the conventional method of completing the square, which is easier to be enforced due to the clear logical framework and avoids the difficulty in looking for the term of completing the square intentionally. From symmetry of covariance in a multivariate Gaussian distribution and algebraic knowledge, we derive approximation and maximization of the evidence function associated with RVM, which is consistent with the previous result using the novel logical framework. Finally, we derive the EM algorithm for RVM, which is also consistent with the previous result except that we use the precision matrix as the covariance.
Highlights
When fitting polynomial curve to a given training data, we encounter the problem of choosing the order of the polynomial, which make us discuss an important concept called model selection
The method of completing the square has difficulty in looking for the term of completing the square, especially in complex problems in the machine learning field. e paper will develop a novel logical framework based on Taylor expansion, which is different from the classic technique of completing the square over parameter of the model for evaluating the evidence function associated with relevance vector machine shortly. e method presented in this paper is convenient to evaluate the evidence function with relevance vector machine (RVM)
We will build up a novel logical framework and discuss its merit compared with the traditional method when dealing with evaluating the evidence function with relevance vector machine (RVM) shortly
Summary
When fitting polynomial curve to a given training data, we encounter the problem of choosing the order of the polynomial, which make us discuss an important concept called model selection. To avoid overfitting problem, we add regularization term [1, 2] to the sum of the squares of the errors between the prediction for each data point and the corresponding target values or partition the available data into a training set which determines the coefficient and a separate validation set, used to optimize the model complexity, called a hold-out set. This is too wasteful of valuable training data and we have to look for more sophisticated methods. The method of completing the square has difficulty in looking for the term of completing the square, especially in complex problems in the machine learning field. e paper will develop a novel logical framework based on Taylor expansion, which is different from the classic technique of completing the square over parameter of the model for evaluating the evidence function associated with relevance vector machine shortly. e method presented in this paper is convenient to evaluate the evidence function with relevance vector machine (RVM)
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
