Properties Of Support Vector Machines Research Articles

Support vector machine and optimal parameter selection for high-dimensional imbalanced data

In this article, we consider asymptotic properties of support vector machine (SVM) in high-dimension, low-sample-size (HDLSS) settings. In particular, we treat high-dimensional imbalanced data. We investigate behaviors of SVM for a regularization parameter C in a framework of kernel functions. We show that SVM cannot handle imbalanced classification, and SVM is very biased in HDLSS settings. In order to overcome such difficulties, we propose robust SVM (RSVM), which gives excellent performances in HDLSS settings. We also give a pre-selection method for parameters included in a kernel function without cross-validation. Finally, we check the performance of RSVM and the optimality of the choice in numerical simulation and actual data analyses.

Robust support vector machine for high-dimensional imbalanced data

In this paper, we consider asymptotic properties of support vector machine (SVM) in high-dimension, low-sample-size (HDLSS) settings. In particular, we investigate the behavior of soft-margin SVM for the regularization parameter C. We show that SVM cannot handle imbalanced classification and SVM is very biased in HDLSS settings. In order to overcome such difficulties, we propose a robust SVM (RSVM). We show that RSVM gives preferable performances in HDLSS settings. Finally, we check the performance of RSVM in actual data analyses.

Predictive-Maintenance Approach Uses Machine Learning To Increase Efficiency

This article, written by JPT Technology Editor Chris Carpenter, contains highlights of paper OTC 28990, “Increasing Production Efficiency Through Compressor-Failure Predictive Analytics Using Machine Learning,” by D. Pandya, A. Srivastava, A. Doherty, S. Sundareshwar, C. Needham, A. Chaudry, and S. Krishnalyer, Shell, prepared for the 2018 Offshore Technology Conference, Houston, 30 April–3 May. The paper has not been peer reviewed. Copyright 2018 Offshore Technology Conference. Reproduced by permission. This paper focuses on compressor systems associated with major production deferments. An advanced machine-learning approach is presented for determining anomalous behavior to predict a potential trip and probable root cause with sufficient warning to allow for intervention. This predictive-maintenance approach has the potential to reduce downtime associated with rotating-equipment failures. Introduction The first step in using a machine-learning system is to train the model to identify normal and abnormal operating conditions. The model can then classify real-time data from the equipment and indicate when the equipment’s performance strays outside the identified steady state. The ability to identify anomalies is a major difference between the proposed approach and traditional monitoring tools. With advances in digital technologies, correlations and warnings can be achieved in a matter of minutes, allowing engineers to take appropriate preventative action when they receive a failure warning. The authors used historical data for 2016 in their analysis of system efficiency in predicting failures. The proof-of-concept system correctly predicted 11 trip events over the course of the year, almost 50% of the 23 failures that occurred during that period. One of the more important findings was that the machine-learning model predicted many failures hours in advance. In one case, it gave 36 hours’ notice. The median period of notice for eight events that were subsequently analyzed was approximately 7 hours. Support Vector Machines (SVMs) SVMs are used in this study as a classifier for detecting abnormal machine states. SVMs were developed for binary classification. Some authors have argued that the SVM classifier has better results compared with techniques such as linear discriminant analysis and back-propagation neural networks. The compressor usually operates under normal working conditions. This poses a highly unbalanced problem for a two-class classification. Because of this, one-class classification using an SVM is implemented. The algorithm is trained on only normal data and creates a representation of this data. When the new points inferred are substantially different from the modeled class, they are labeled as outliers. Linear as well as radial kernel functions are explored. One of the properties of SVMs is that they may create a nonlinear decision boundary by projecting the data through nonlinear function to a space in higher dimension (Fig. 1). The one-class SVM creates a binary function that captures the region in the input space where most of the data exist. The function, thus formed, returns +1 for the region defined by training data points and –1 everywhere else.

Robust variable selection based on bagging classification tree for support vector machine in metabonomic data analysis

In metabonomics, metabolic profiles of high complexity bring out tremendous challenges to existing chemometric methods. Variable selection (ie, biomarker discovery) and pattern recognition (ie, classification) are two important tasks of chemometrics in metabonomics, especially biomarker discovery that can be potentially used for disease diagnosis and pathology discovery. Typically, the informative variables are elicited from a single classifier; however, it is often unreliable in practice. To rectify this, in the current study, bagging and classification tree (CT) were combined to form a general framework (ie, BAGCT) for robustly selecting the informative variables, based on the advantages of CT in automatically carrying out variable selection as well as measuring variable importance and the properties of bagging in improving the reliability and robustness of a single model. In BAGCT, a set of parallel CT models were established based on the idea of bagging, each CT providing some endowed information such as the splitting variables and their corresponding importance values. The informative variables can be successfully spied via inspecting the variable importance values over all CTs in BAGCT. Taking the promising properties of support vector machine (SVM) into account, we used the informative variables identified by BAGCT as the inputs of SVM, forming a new classification tool abbreviated as BAGCT‐SVM. A metabonomic dataset by hydrogen‐1 nuclear magnetic resonance from the patients with lung cancer and the healthy controls was used to validate BAGCT‐SVM with CT and SVM as comparisons. Results showed that BAGCT‐SVM with less number of variables can give better predictive ability than CT and SVM.

A support vector machine method for bid/no bid decision making

The bid/no bid decision is an important and complex process, and is impacted by numerous variables that are related to the contractor, project, client, competitors, tender and market conditions. Despite the complexity of bid decision making process, in the construction industry the majority of bid/no bid decisions is made informally based on experience, judgment, and perception. In this paper, a procedure based on support vector machines and backward elimination regression is presented for improving the existing bid decision making methods. The method takes advan­tage of the strong generalization properties of support vector machines and attempts to further enhance generalization performance by eliminating insignificant input variables. The method is implemented for bid/no bid decision making of offshore oil and gas platform fabrication projects to achieve a parsimonious support vector machine classifier. The performance of the support vector machine classifier is compared with the performances of the worth evaluation model, linear regression, and neural network classifiers. The results show that the support vector machine classifier outperforms existing methods significantly, and the proposed procedure provides a powerful tool for bid/no bid decision making. The results also reveal that elimination of the insignificant input variables improves generalization performance of the sup­port vector machines.

Maximum-margin sparse coding

This work devises a maximum-margin sparse coding algorithm, jointly considering reconstruction loss and hinge loss in the model. The sparse representation along with maximum-margin constraint is analogous to kernel trick and maximum-margin properties of support vector machine (SVM), giving a base for the proposed algorithm to perform well in classification tasks. The key idea behind the proposed method is to use labeled and unlabeled data to learn discriminative representations and model parameters simultaneously, making it easier to classify data in the new space. We propose to use block coordinate descent to learn all the components of the proposed model and give detailed derivation for the update rules of the model variables. Theoretical analysis on the convergence of the proposed MMSC algorithm is provided based on Zangwill’s global convergence theorem. Additionally, most previous research studies on dictionary learning suggest to use an overcomplete dictionary to improve classification performance, but it is computationally intensive when the dimension of the input data is huge. We conduct experiments on several real data sets, including Extended YaleB, AR face, and Caltech101 data sets. The experimental results indicate that the proposed algorithm outperforms other comparison algorithms without an overcomplete dictionary, providing flexibility to deal with high-dimensional data sets.

Oracle properties of SCAD-penalized support vector machine

In many scientific investigations, a large number of input variables are given at the early stage of modeling and identifying the variables predictive of the response is often a main purpose of such investigations. Recently, the support vector machine has become an important tool in classification problems of many fields. Several variants of the support vector machine adopting different penalties in its objective function have been proposed. This paper deals with the Fisher consistency and the oracle property of support vector machines in the setting where the dimension of inputs is fixed. First, we study the Fisher consistency of the support vector machine over the class of affine functions. It is shown that the function class for decision functions is crucial for the Fisher consistency. Second, we study the oracle property of the penalized support vector machines with the smoothly clipped absolute deviation penalty. Once we have addressed the Fisher consistency of the support vector machine over the class of affine functions, the oracle property appears to be meaningful in the context of classification. A simulation study is provided in order to show small sample properties of the penalized support vector machines with the smoothly clipped absolute deviation penalty.

Embedded Speech Recognition Based on Multiclass Support Vector Machine

This paper introduces the characteristics and requirements of speech recognition technology based on embedded platform. It also describes the basic theory and related properties of Support Vector Machine. The advantages and disadvantages of the Multiclass SVM algorithms are analyzed, providing the algorithms principles for training and recognition of SVM application in the embedded speech recognition system. Finally, we proposed a design strategy based on multiclass SVM decision tree classifier, combined with the features of the embedded speech recognition.

A review on consistency and robustness properties of support vector machines for heavy-tailed distributions

Support vector machines (SVMs) belong to the class of modern statistical machine learning techniques and can be described as M-estimators with a Hilbert norm regularization term for functions. SVMs are consistent and robust for classification and regression purposes if based on a Lipschitz continuous loss and a bounded continuous kernel with a dense reproducing kernel Hilbert space. For regression, one of the conditions used is that the output variable Y has a finite first absolute moment. This assumption, however, excludes heavy-tailed distributions. Recently, the applicability of SVMs was enlarged to these distributions by considering shifted loss functions. In this review paper, we briefly describe the approach of SVMs based on shifted loss functions and list some properties of such SVMs. Then, we prove that SVMs based on a bounded continuous kernel and on a convex and Lipschitz continuous, but not necessarily differentiable, shifted loss function have a bounded Bouligand influence function for all distributions, even for heavy-tailed distributions including extreme value distributions and Cauchy distributions. SVMs are thus robust in this sense. Our result covers the important loss functions \({\epsilon}\) -insensitive for regression and pinball for quantile regression, which were not covered by earlier results on the influence function. We demonstrate the usefulness of SVMs even for heavy-tailed distributions by applying SVMs to a simulated data set with Cauchy errors and to a data set of large fire insurance claims of Copenhagen Re.

A comparison of reduced support vector machines

In this paper, we first briefly review some knowledge of Support Vector Machines (SVMs). This includes not only SVM's dual forms and solution forms but also the general process and properties of SVM. Then aiming at speeding up SVMs, some kinds of Reduced SVMs (RSVMs) are discussed in detailed. A comparison among them is presented in several aspects. Finally, we show research issues that need to be resolved or investigated further. A number of future trends are also briefly sketched.

Robust ASR using Support Vector Machines

The improved theoretical properties of Support Vector Machines with respect to other machine learning alternatives due to their max-margin training paradigm have led us to suggest them as a good technique for robust speech recognition. However, important shortcomings have had to be circumvented, the most important being the normalisation of the time duration of different realisations of the acoustic speech units. In this paper, we have compared two approaches in noisy environments: first, a hybrid HMM–SVM solution where a fixed number of frames is selected by means of an HMM segmentation and second, a normalisation kernel called Dynamic Time Alignment Kernel (DTAK) first introduced in Shimodaira et al. [Shimodaira, H., Noma, K., Nakai, M., Sagayama, S., 2001. Support vector machine with dynamic time-alignment kernel for speech recognition. In: Proc. Eurospeech, Aalborg, Denmark, pp. 1841–1844] and based on DTW (Dynamic Time Warping). Special attention has been paid to the adaptation of both alternatives to noisy environments, comparing two types of parameterisations and performing suitable feature normalisation operations. The results show that the DTA Kernel provides important advantages over the baseline HMM system in medium to bad noise conditions, also outperforming the results of the hybrid system.

An asymptotic statistical analysis of support vector machines with soft margins

The generalization properties of support vector machines (SVMs) are examined. From a geometrical point of view, the estimated parameter of an SVM is the one nearest the origin in the convex hull formed with given examples. Since introducing soft margins is equivalent to reducing the convex hull of the examples, an SVM with soft margins has a different learning curve from the original. In this paper we derive the asymptotic average generalization error of SVMs with soft margins in simple cases, that is, only when the dimension of inputs is one, and quantitatively show that soft margins increase the generalization error.

Time series prediction using support vector machines, the orthogonal and the regularized orthogonal least-squares algorithms

Generalization properties of support vector machines, orthogonal least squares and zero-order regularized orthogonal least squares algorithms are studied using simulation. For high signal-to-noise ratios (40 dB), mixed results are obtained, but for a low signal-to-noise ratio, the prediction performance of support vector machines is better than the orthogonal least squares algorithm in the examples considered. However, the latter can usually give a parsimonious model with very fast training and testing time. Two new algorithms are therefore proposed that combine the orthogonal least squares algorithm with support vector machines to give a parsimonious model with good prediction accuracy in the low signal-to-noise ratio case.

Properties of support vector machines.

Support vector machines (SVMs) perform pattern recognition between two point classes by finding a decision surface determined by certain points of the training set, termed support vectors (SV). This surface, which in some feature space of possibly infinite dimension can be regarded as a hyperplane, is obtained from the solution of a problem of quadratic programming that depends on a regularization parameter. In this article, we study some mathematical properties of support vectors and show that the decision surface can be written as the sum of two orthogonal terms, the first depending on only the margin vectors (which are SVs lying on the margin), the second proportional to the regularization parameter. For almost all values of the parameter, this enables us to predict how the decision surface varies for small parameter changes. In the special but important case of feature space of finite dimension m, we also show that m + 1 SVs are usually sufficient to determine the decision surface fully. For relatively small m, this latter result leads to a consistent reduction of the SV number.

On consistency and robustness properties of support vector machines for heavy-tailed distributions

Support Vector Machines (SVMs) are known to be consistent and robust for classification and regression if they are based on a Lipschitz continuous loss function and on a bounded kernel with a dense and separable reproducing kernel Hilbert space. These facts are even true in the regression context for unbounded output spaces, if the target function f is integrable with respect to the marginal distribution of the input variable X and if the output variable Y has a finite first absolute moment. The latter assumption clearly excludes distributions with heavy tails, e. g., several stable distributions or some extreme value distributions which occur in financial or insurance projects. The main point of this paper is that we can enlarge the applicability of SVMs even to heavy-tailed distributions, which violate this moment condition. Results on existence, uniqueness, representation, consistency, and statistical robustness are given.