Lagrangian Support Vector Machine Research Articles

커널 함수를 도입한 새로운 추천 시스템

In this paper, a new kernelized approach for use in a recommender system (RS) is proposed. Using a machine learning technique, the proposed method predicts the user`s preferences for unknown items and recommends items which are likely to be preferred by the user. Since the ratings of the users are generally inconsistent and noisy, a robust binary classifier called a dual margin Lagrangian support vector machine (DMLSVM) is employed to suppress the noise. The proposed method is applied to MovieLens databases, and its effectiveness is demonstrated via simulations.

Dual margin approach on a Lagrangian support vector machine

In this paper, we propose a new support vector machine (SVM) called dual margin Lagrangian support vectors machine (DMLSVM). Unlike other SVMs which use only support vectors to determine the separating hyperplanes, DMLSVM utilizes all the available training data for training the classifier, thus producing robust performance. The training data are weighted differently depending on whether they are in a marginal region or surplus region. For fast training, DMLSVM borrows its training algorithm from Lagrangian SVM (LSVM) and tailors the algorithm to its formulation. The convergence of our training method is rigorously proven and its validity is tested on a synthetic test set and UCI dataset. The proposed method can be used in a variety of applications such as a recommender systems for web contents of IPTV services.

A new weighted approach to imbalanced data classification problem via support vector machine with quadratic cost function

In this paper, a new weighted approach on Lagrangian support vector machine for imbalanced data classification problem is proposed. The weight parameters are embedded in the Lagrangian SVM formulation. The training method for weighted Lagrangian SVM is presented and its convergence is proven. The weighted Lagrangian SVM classifier is tested and compared with some other SVMs using synthetic and real data to show its effectiveness and feasibility.

On Lagrangian support vector regression

Prediction by regression is an important method of solution for forecasting. In this paper an iterative Lagrangian support vector machine algorithm for regression problems has been proposed. The method has the advantage that its solution is obtained by taking the inverse of a matrix of order equals to the number of input samples at the beginning of the iteration rather than solving a quadratic optimization problem. The algorithm converges from any starting point and does not need any optimization packages. Numerical experiments have been performed on Bodyfat and a number of important time series datasets of interest. The results obtained are in close agreement with the exact solution of the problems considered clearly demonstrates the effectiveness of the proposed method.

An incremental learning algorithm for Lagrangian support vector machines

Incremental learning has attracted more and more attention recently, both in theory and application. In this paper, the incremental learning algorithms for Lagrangian support vector machine (LSVM) are proposed. LSVM is an improvement to the standard linear SVM for classifications, which leads to the minimization of an unconstrained differentiable convex programming. The solution to this programming is obtained by an iteration scheme with a simple linear convergence. The inversion of the matrix in the solving algorithm is converted to the order of the original input space’s dimensionality plus one at the beginning of the algorithm. The algorithm uses the Sherman–Morrison–Woodbury identity to reduce the computation time. The incremental learning algorithms for LSVM presented in this paper include two cases that are namely online and batch incremental learning. Because the inversion of the matrix after increment is solved based on the previous computed information, it is unnecessary to repeat the computing process. Experimental results show that the algorithms are superior to others.

An extended Lagrangian support vector machine for classifications*

Abstract Lagrangian support vector machine (LSVM) cannot solve large problems for nonlinear kernel classifiers. In order to extend the LSVM to solvevery large problems, an extended Lagrangian support vector machine (ELSVM) for classifications based on LSVM and SVMlight is presented in this paper. Our idea for the ELSVM is to divide a large quadratic programming problem into a series of subproblems with small size and to solve them via LSVM. Since the LSVM can solve small and medium problems for nonlinear kernel classifiers, the proposed ELSVM can be used to handle large problems very efficiently. Numerical experiments on different types of problems are performed to demonstrate the high efficiency of the ELSVM.

Finite Newton method for Lagrangian support vector machine classification

An implicit Lagrangian [Math. Programming Ser. B 62 (1993) 277] formulation of a support vector machine classifier that led to a highly effective iterative scheme [J. Machine Learn. Res. 1 (2001) 161] is solved here by a finite Newton method. The proposed method, which is extremely fast and terminates in 6 or 7 iterations, can handle classification problems in very high dimensional spaces, e.g. over 28,000, in a few seconds on a 400 MHz Pentium II machine. The method can also handle problems with large datasets and requires no specialized software other than a commonly available solver for a system of linear equations. Finite termination of the proposed method is established in this work.

10.1162/15324430152748218

An implicit Lagrangian for the dual of a simple reformulation of the standard quadratic program of a linear support vector machine proposed. This leads to the minimization of an unconstrained differentiable convex function in a space of dimensionality equal to the number of classified points. This problem solvable by an extremely simple linearly convergent Lagrangian support vector machine (LSVM) algorithm. LSVM requires the inversion at the outset of a single matrix of the order of the much smaller dimensionality of the original input space plus one. The full algorithm given in this paper in 11 lines of MATLAB code without any special optimization tools such linear or quadratic programming solvers. This LSVM code can be used as is to solve classification problems with millions of points. For example, 2 million points in 10 dimensional input space were classified by a linear surface in 82 minutes on a Pentium III 500 MHz notebook with 384 megabytes of memory (and additional swap space), and in 7 minutes on a 250 MHz UltraSPARC II processor with 2 gigabytes of memory. Other standard classification test problems were also solved. Nonlinear kernel classification can also be solved by LSVM. Although it does not scale up to very large problems, it can handle any positive semidefinite kernel and guaranteed to converge. A short MATLAB code also given for nonlinear kernels and tested on a number of problems.