Scale Support Vector Machine Research Articles

A New Large Scale SVM for Classification of Imbalanced Evolving Streams

Classification from imbalanced evolving streams possesses a combined challenge of class imbalance and concept drift (CI-CD). However, the state of imbalance is dynamic, a kind of virtual concept drift. The imbalanced distributions and concept drift hinder the online learner’s performance as a combined or individual problem. A weighted hybrid online oversampling approach,”weighted online oversampling large scale support vector machine (WOOLASVM),” is proposed in this work to address this combined problem. The WOOLASVM is an SVM active learning approach with new boundary weighing strategies such as (i) dynamically oversampling the current boundary and (ii) dynamic weighing of the cost parameter of the SVM objective function. Thus at any time step, WOOLASVM maintains balanced class distributions so that the CI-CD problem does not hinder the online learner performance. Over extensive experiments on synthetic and real-world streams with the static and dynamic state of imbalance, the WOOLASVM exhibits better online classification performances than other state-of-the-art methods.

A Fast Algorithm for Training Large Scale Support Vector Machines

The manuscript presents an augmented Lagrangian—fast projected gradient method (ALFPGM) with an improved scheme of working set selection, pWSS, a decomposition based algorithm for training support vector classification machines (SVM). The manuscript describes the ALFPGM algorithm, provides numerical results for training SVM on large data sets, and compares the training times of ALFPGM and Sequential Minimal Minimization algorithms (SMO) from Scikit-learn library. The numerical results demonstrate that ALFPGM with the improved working selection scheme is capable of training SVM with tens of thousands of training examples in a fraction of the training time of some widely adopted SVM tools.

Adaptive kernel scaling support vector machine with application to a prostate cancer image study

The support vector machine (SVM) is a popularly used classifier in applications such as pattern recognition, texture mining and image retrieval owing to its flexibility and interpretability. However, its performance deteriorates when the response classes are imbalanced. To enhance the performance of the support vector machine classifier in the imbalanced cases we investigate a new two stage method by adaptively scaling the kernel function. Based on the information obtained from the standard SVM in the first stage, we conformally rescale the kernel function in a data adaptive fashion in the second stage so that the separation between two classes can be effectively enlarged with incorporation of observation imbalance. The proposed method takes into account the location of the support vectors in the feature space, therefore is especially appealing when the response classes are imbalanced. The resulting algorithm can efficiently improve the classification accuracy, which is confirmed by intensive numerical studies as well as a real prostate cancer imaging data application.

A Coordinate-Descent Primal-Dual Algorithm with Large Step Size and Possibly Nonseparable Functions

This paper introduces a coordinate descent version of the V\~u-Condat algorithm. By coordinate descent, we mean that only a subset of the coordinates of the primal and dual iterates is updated at each iteration, the other coordinates being maintained to their past value. Our method allows us to solve optimization problems with a combination of differentiable functions, constraints as well as non-separable and non-differentiable regularizers. We show that the sequences generated by our algorithm converge to a saddle point of the problem at stake, for a wider range of parameter values than previous methods. In particular, the condition on the step-sizes depends on the coordinate-wise Lipschitz constant of the differentiable function's gradient, which is a major feature allowing classical coordinate descent to perform so well when it is applicable. We then prove a sublinear rate of convergence in general and a linear rate of convergence if the objective enjoys strong convexity properties. We illustrate the performances of the algorithm on a total-variation regularized least squares regression problem and on large scale support vector machine problems.

Blind image forensics using reciprocal singular value curve based local statistical features

In this article, passive contrast enhancement detection technique is presented using block based reciprocal singular value curve features. Contrast enhancement operation changes the natural statistics of the image and variation in singular value curve is exploited for constructing the feature vector for forgery detection. Various statistical features using reciprocal singular value curve are extracted after multilevel 2-Dimensional wavelet decomposition. Fisher criterion is employed to choose the most discriminating and to discard the redundant features. Experimental results are presented using gray scale, G component and C b image database and support vector machine classifier. Robustness against anti-forensic algorithm and JPEG compression is also presented. The algorithm outperforms all the existing feature based blind contrast enhancement detection methods in terms of detection accuracy.

Uncertainty-safe large scale support vector machines

The issue of large scale binary classification when data is subject to random perturbations is addressed. The proposed model integrates a learning framework that adjusts its robustness to noise during learning. The method avoids over-conservative situations that can be encountered with worst-case robust support vector machine formulations. The algorithm could be seen as a technique to learn the support-set of the noise distribution during the training process. This is achieved by introducing optimization variables that control the magnitude of the noise perturbations that should be taken into account. The magnitude is tuned by optimizing a generalization error. Only rough estimates of perturbations bounds are required. Additionally, a stochastic bi-level optimization technique is proposed to solve the resulting formulation. The algorithm performs very cheap stochastic subgradient moves and is therefore well suited to large datasets. Encouraging experimental results show that the technique outperforms robust second order cone programming formulations.

Bi-level stochastic gradient for large scale support vector machine

We propose a new bi-level stochastic optimization algorithm for training large scale support vector machine (SVM) with automatic selection of the C hyperparameter. We show that in the proposed bi-level formulation, the variation of the inner objective with respect to the outer variable can be nicely expressed. Gradient estimates are computed for both inner and outer objectives in order to perform stochastic moves with low complexity. Extension to nonlinear SVM is also proposed. We further discuss the possibility to integrate the technique within an automatic k-fold cross validation framework. Preliminary results on several datasets show that the method is finding the optimum hyperplane while adjusting the penalty parameter with significant computational time savings when compared to the classic cross validation procedure.

An algorithm for training a large scale support vector machine for regression based on linear programming and decomposition methods

This paper presents a method to train a Support Vector Regression (SVR) model for the large-scale case where the number of training samples supersedes the computational resources. The proposed scheme consists of posing the SVR problem entirely as a Linear Programming (LP) problem and on the development of a sequential optimization method based on variables decomposition, constraints decomposition, and the use of primal–dual interior point methods. Experimental results demonstrate that the proposed approach has comparable performance with other SV-based classifiers. Particularly, experiments demonstrate that as the problem size increases, the sparser the solution becomes, and more computational efficiency can be gained in comparison with other methods. This demonstrates that the proposed learning scheme and the LP-SVR model are robust and efficient when compared with other methodologies for large-scale problems.

Analysis of Large-Scale SVM Training Algorithms for Language and Speaker Recognition

This paper compares a set of large scale support vector machine (SVM) training algorithms for language and speaker recognition tasks. We analyze five approaches for training phonetic and acoustic SVM models for language recognition. We compare the performance of these approaches as a function of the training time required by each of them to reach convergence, and we discuss their scalability towards large corpora. Two of these algorithms can be used in speaker recognition to train a SVM that classifies pairs of utterances as either belonging to the same speaker or to two different speakers. Our results show that the accuracy of these algorithms is asymptotically equivalent, but they have different behavior with respect to the time required to converge. Some of these algorithms not only scale linearly with the training set size, but are also able to give their best results after just a few iterations. State-of-the-art performance has been obtained in the female subset of the NIST 2010 Speaker Recognition Evaluation extended core test using a single SVM system.

A New Working Set Selection Method for Solving Large Scale Support Vector Machine

In this work we consider nonlinear minimization problems with a single linear equality constraint and box constraints. We are especially interested in solving problems where the number of variables is so large that traditional optimization methods cannot be directly applied. The Support Vector Machines (SVM) is a technique for machine learning problems. In this paper, we define a descent search direction selected among a suitable set of sparse feasible directions which have q (q>2, even) components to reduce the iteration numbers. Thus we put forward a new working set selection method for solving large scale support vector machines.

Identification of NARX Hammerstein Models Based on Large Scale Support Vector Machine Regression

Methods for the identification of Hammerstein models consisting of a Support Vector Machine nonlinearity followed by an ARX linear system are developed. The models are identified by minimizing epsilon insensitive cost functions based on either the sum of absolute residuals or the sum of squared residuals. Large scale implementations of these techniques are then derived using subset selection methods, and used to identify a model of the stretch reflex electromyogram from experimental data. The effects of the various cost functions and tuning parameters are demonstrated with the experimental results.

Parallel Software for Training Large Scale Support Vector Machines on Multiprocessor Systems

Parallel software for solving the quadratic program arising in training support vector machines for classification problems is introduced. The software implements an iterative decomposition techniq...