Maximal Margin Hyperplane Research Articles

Prediction of recurrent spontaneous abortion using evolutionary machine learning with joint self-adaptive sime mould algorithm

Recurrent spontaneous abortion (RSA) is a frequent abnormal pregnancy with long-term psychological repercussions that disrupt the peace of the whole family. In the diagnosis and treatment of RSA worsened by thyroid disorders, recurrent spontaneous abortion is also a significant obstacle. The pathogenesis and possible treatment methods for RSA are yet unclear. Using clinical information, vitamin D and thyroid function measurements from normal pregnant women with RSA, we attempt to build a framework for conducting an effective analysis for RSA in this research. The framework is presented by combining the joint self-adaptive sime mould algorithm (JASMA) with the common kernel learning support vector machine with maximum-margin hyperplane theory, abbreviated as JASMA-SVM. The JASMA has a complete set of adaptive parameter change methods, which improves the algorithm’s global search and optimization capabilities and guarantees that it speeds convergence and departs from the local optimum. On CEC 2014 benchmarks, the property of JASMA is validated, and then it is utilized to concurrently optimize parameters and select optimal features for SVM on RSA data from VitD, thyroid hormone levels, and thyroid autoantibodies. The statistical results demonstrate that the proposed JASMA-SVM can be treated as a potential tool for RSA with accuracy of 92.998%, MCC of 0.92425, sensitivity of 93.286%, specificity of 93.064%.

Structural Fuzzy Multi-class Support Vector Machine

Support vector machine is a kind of generalized linear classifier that classifies data in a supervised learning manner, and its decision boundary is the maximum margin hyperplane for solving learning samples. It is used to solve the problem of binary classification. For multi-classification problems, a combination method of multiple binary classifiers is usually used to solve, but such methods are prone to generate inseparable regions of data. Therefore, on the basis of constructing a multi-class problem directly, using the pinball loss function, and introducing the structural information of different classes in the data and the role of different samples, a new support vector machine algorithm Pin-SFSimMSVM for solving multi-classification problems is proposed. It not only retains the advantages of avoiding the existence of inseparable regions and fast calculation speed of multiple types of data, but also is insensitive to noise and resampling data, and has greatly improved the accuracy. The effectiveness of the proposed algorithm is verified by experimenting on UCI standard data sets and comparing with some multi-classification algorithms.

On support vector machines under a multiple-cost scenario

Support vector machine (SVM) is a powerful tool in binary classification, known to attain excellent misclassification rates. On the other hand, many realworld classification problems, such as those found in medical diagnosis, churn or fraud prediction, involve misclassification costs which may be different in the different classes. However, it may be hard for the user to provide precise values for such misclassification costs, whereas it may be much easier to identify acceptable misclassification rates values. In this paper we propose a novel SVM model in which misclassification costs are considered by incorporating performance constraints in the problem formulation. Specifically, our aim is to seek the hyperplane with maximal margin yielding misclassification rates below given threshold values. Such maximal margin hyperplane is obtained by solving a quadratic convex problem with linear constraints and integer variables. The reported numerical experience shows that our model gives the user control on the misclassification rates in one class (possibly at the expense of an increase in misclassification rates for the other class) and is feasible in terms of running times.

Minimum spectral connectivity projection pursuit

We study the problem of determining the optimal low-dimensional projection for maximising the separability of a binary partition of an unlabelled dataset, as measured by spectral graph theory. This is achieved by finding projections which minimise the second eigenvalue of the graph Laplacian of the projected data, which corresponds to a non-convex, non-smooth optimisation problem. We show that the optimal univariate projection based on spectral connectivity converges to the vector normal to the maximum margin hyperplane through the data, as the scaling parameter is reduced to zero. This establishes a connection between connectivity as measured by spectral graph theory and maximal Euclidean separation. The computational cost associated with each eigen problem is quadratic in the number of data. To mitigate this issue, we propose an approximation method using microclusters with provable approximation error bounds. Combining multiple binary partitions within a divisive hierarchical model allows us to construct clustering solutions admitting clusters with varying scales and lying within different subspaces. We evaluate the performance of the proposed method on a large collection of benchmark datasets and find that it compares favourably with existing methods for projection pursuit and dimension reduction for data clustering. Applying the proposed approach for a decreasing sequence of scaling parameters allows us to obtain large margin clustering solutions, which are found to be competitive with those from dedicated maximum margin clustering algorithms.

Clustering by Minimum Cut Hyperplanes

Minimum normalised graph cuts are highly effective ways of partitioning unlabeled data, having been made popular by the success of spectral clustering. This work presents a novel method for learning hyperplane separators which minimise this graph cut objective, when data are embedded in Euclidean space. The optimisation problem associated with the proposed method can be formulated as a sequence of univariate subproblems, in which the optimal hyperplane orthogonal to a given vector is determined. These subproblems can be solved in log-linear time, by exploiting the trivial factorisation of the exponential function. Experimentation suggests that the empirical runtime of the overall algorithm is also log-linear in the number of data. Asymptotic properties of the minimum cut hyperplane, both for a finite sample, and for an increasing sample assumed to arise from an underlying probability distribution are discussed. In the finite sample case the minimum cut hyperplane converges to the maximum margin hyperplane as the scaling parameter is reduced to zero. Applying the proposed methodology, both for fixed scaling, and the large margin asymptotes, is shown to produce high quality clustering models in comparison with state-of-the-art clustering algorithms in experiments using a large collection of benchmark datasets.

Study on Prediction of Mackey-Glass Chaotic Time Series Using Support Vector Machines

Support Vector Machines (SVM), which is a new generation learning method based on advances in statistical learning theory, is characterized by the use of many standard technologies of machine learning such as maximal margin hyperplane, Mercel kernels and the quadratic programming. Because the best performance is obtained in many currently challenging applications, SVM has sustained wide attention, and has been become the standard tools of machine learning and data mining. But as a developing technology, SVM still have some problems and its applications are limited. In this paper, SVM and its applications in chaotic time series including predicting chaotic time series, focus on comparison in regression type selection, and kernel type selection in the same regression machine type.

Analysis of Hantaviruses Glycoprotein Sequence Using SVM Algorithm

Hantaviruses are single-stranded, enveloped, negative sense RNA viruses of the Bunyaviridae family. They induce deadly hemorrhagic fever with fatality up to 40%. Currently, there is no specific cure for Hantaviruses, so more researches about this mortal virus should be done. In order to effectively analyze a variety of different Hantaviruses, we utilize a model called the support vector machine (also known as SVM) which is generally used for analyzing and classifying binary data.The basic mechanism of the SVM is to find the most optimal hyperplane, or the maximum-margin hyperplane, which can separate different types of data with the least error bound. Out of all of the hyperplanes that may be used to classify the data points, the most optimal hyperplane is the one that has the largest margin, or separation among different types of data. In other words, the optimal hyperplane is chosen in the case where the distance between the nearest points of each group of data is maximized. Ultimately, using the selected hyperplane, SVM classifies the data points and computes values such as accuracy and sensitivity. At the end of its operation, the SVM algorithm prints out the computed values. Such SVM algorithms can be used to learn the characteristics of each Hantavirus such as sequence patterns and abundance of amino acids. Since we are the first time to scientifically investigate the Hantavirus with SVM, it is expected that the results of this research will be greatly helpful for further in-depth researching and the development of the cure for the virus.

The efficacy of support vector machines (SVM) in robust determination of earthquake early warning magnitudes in central Japan

This work deals with a methodology applied to seismic early warning systems which are designed to provide real-time estimation of the magnitude of an event. We will reappraise the work of Simons et al. (2006), who on the basis of wavelet approach predicted a magnitude error of ±1. We will verify and improve upon the methodology of Simons et al. (2006) by applying an SVM statistical learning machine on the time-scale wavelet decomposition methods. We used the data of 108 events in central Japan with magnitude ranging from 3 to 7.4 recorded at KiK-net network stations, for a source–receiver distance of up to 150 km during the period 1998–2011. We applied a wavelet transform on the seismogram data and calculating scale-dependent threshold wavelet coefficients. These coefficients were then classified into low magnitude and high magnitude events by constructing a maximum margin hyperplane between the two classes, which forms the essence of SVMs. Further, the classified events from both the classes were picked up and linear regressions were plotted to determine the relationship between wavelet coefficient magnitude and earthquake magnitude, which in turn helped us to estimate the earthquake magnitude of an event given its threshold wavelet coefficient. At wavelet scale number 7, we predicted the earthquake magnitude of an event within 2.7 seconds. This means that a magnitude determination is available within 2.7 s after the initial onset of the P-wave. These results shed light on the application of SVM as a way to choose the optimal regression function to estimate the magnitude from a few seconds of an incoming seismogram. This would improve the approaches from Simons et al. (2006) which use an average of the two regression functions to estimate the magnitude.

Classification of companies using maximal margin ellipsoidal surfaces

We recently proposed a data mining approach for classifying companies into several groups using ellipsoidal surfaces. This problem can be formulated as a semi-definite programming problem, which can be solved within a practical amount of computation time by using a state-of-the-art semi-definite programming software. It turned out that this method performs better for this application than earlier methods based on linear and general quadratic surfaces. In this paper we will improve the performance of ellipsoidal separation by incorporating the idea of maximal margin hyperplane developed in the field of support vector machine. It will be demonstrated that the new method can very well simulate the rating of a leading rating company of Japan by using up to 18 financial attributes of 363 companies. This paper is expected to provide another evidence of the importance of ellipsoidal separation approach in credit risk analysis.

Combination of kernel PCA and linear support vector machine for modeling a nonlinear relationship between bioactivity and molecular descriptors

AbstractIn this paper, a two‐step nonlinear classification algorithm is proposed to model the structure–activity relationship (SAR) between bioactivities and molecular descriptors of compounds, which consists of kernel principal component analysis (KPCA) and linear support vector machines (KPCA + LSVM). KPCA is used to remove some uninformative gradients such as noises and then exactly capture the latent structure of the training dataset using some new variables called the principal components in the kernel‐defined feature space. LSVM makes full use of the maximal margin hyperplane to give the best generalization performance in the KPCA‐transformed space. The combination of KPCA and LSVM can effectively improve the prediction performance compared with the linear SVM as well as two nonlinear methods. Three datasets related to different categorical bioactivities of compounds are used to evaluate the performance of KPCA + LSVM. The results show that our algorithm is competitive. Copyright © 2011 John Wiley & Sons, Ltd.

Linear potential proximal support vector machines for pattern classification

Support vector machine (SVM) classifiers attempt to find a maximum margin hyperplane by solving a convex optimization problem. The conventional SVM approach involves the minimization of a quadratic function subject to linear inequality constraints. However, the margin is not scale invariant, and therefore a linear transformation of the data tends to affect the classification accuracy. Recently, potential SVMs attempted to address the issue of scale variance by using an appropriate scaling to improve the classification accuracy. In this paper, we propose a novel SVM formulation that is in the spirit of potential SVM, but requires a single matrix inversion to find the classifier. Experimental results bear out the efficacy of the classifier.

Hypersphere support vector machines based on generalized multiplicative updates

This paper proposes a novel hypersphere support vector machines (HSVMs) based on generalized multiplicative updates. This algorithm can obtain the boundary of hypersphere containing one class of samples by the description of the training samples from one class and use this boundary to classify the test samples. The generalized multiplicative updates are applied to solving boundary optimization programming. Multiplicative updates available are suited for nonnegative quadratic convex programming. The generalized multiplicative updates are derived to box and sum constrained quadratic programming in this paper. They provide an extremely straightforward way to implement support vector machines (SVMs) where all variables are updated in parallel. The generalized multiplicative updates converge monotonically to the solution of the maximum margin hyperplane. The experiments show the superiority of our new algorithm.

An iterative algorithm learning the maximal margin classifier

A simple learning algorithm for maximal margin classifiers (also support vector machines with quadratic cost function) is proposed. We build our iterative algorithm on top of the Schlesinger–Kozinec algorithm (S–K-algorithm) from 1981 which finds a maximal margin hyperplane with a given precision for separable data. We suggest a generalization of the S–K-algorithm (i) to the non-linear case using kernel functions and (ii) for non-separable data. The requirement in memory storage is linear to the data. This property allows the proposed algorithm to be used for large training problems. The resulting algorithm is simple to implement and as the experiments showed competitive to the state-of-the-art algorithms. The implementation of the algorithm in Matlab is available. We tested the algorithm on the problem aiming at recognition poor quality numerals.

Active learning with support vector machines in the drug discovery process.

We investigate the following data mining problem from computer-aided drug design: From a large collection of compounds, find those that bind to a target molecule in as few iterations of biochemical testing as possible. In each iteration a comparatively small batch of compounds is screened for binding activity toward this target. We employed the so-called "active learning paradigm" from Machine Learning for selecting the successive batches. Our main selection strategy is based on the maximum margin hyperplane-generated by "Support Vector Machines". This hyperplane separates the current set of active from the inactive compounds and has the largest possible distance from any labeled compound. We perform a thorough comparative study of various other selection strategies on data sets provided by DuPont Pharmaceuticals and show that the strategies based on the maximum margin hyperplane clearly outperform the simpler ones.

A new approximate maximal margin classification algorithm

A new incremental learning algorithm is described which approximates the maximal margin hyperplane w.r.t. norm p 2 for a set of linearly separable data. Our algorithm, called ALMA_p (Approximate La...

Covering numbers for support vector machines

Support vector (SV) machines are linear classifiers that use the maximum margin hyperplane in a feature space defined by a kernel function. Previously, the only bounds on the generalization performance of SV machines (within Valiant's probably approximately correct framework) took no account of the kernel used except in its effect on the margin and radius. It has been shown that one can bound the relevant covering numbers using tools from functional analysis. In this paper, we show that the resulting bound can be greatly simplified. The new bound involves the eigenvalues of the integral operator induced by the kernel. It shows that the effective dimension depends on the rate of decay of these eigenvalues. We present an explicit calculation of covering numbers for an SV machine using a Gaussian kernel, which is significantly better than that implied by previous results.

DirectSVM: A Simple Support Vector Machine Perceptron

We propose a very simple learning algorithm, DirectSVM, for constructing support vector machine classifiers. This new algorithm is based on the proposition that the two closest training points of opposite class in a training set are support vectors, on condition that the training points in the set are linearly independent. The latter condition is always satisfied for soft-margin support vector machines with quadratic penalties. Other support vectors are found using the following conjecture: the training point that maximally violate the current hyperplane is also a support vector. We show that DirectSVM converges to a maximal margin hyperplane in M − 2 iterations, if the number of support vectors is M. DirectSVM is evaluated empirically on a number of standard databases. Performance-wise, the algorithm generalizes similarly as other implementations. Speed-wise, the proposed method is faster than a standard quadratic programming approach, while it has the potential to be competitive with current state-of-the-art SVM implementations.

Structural risk minimization over data-dependent hierarchies

The paper introduces some generalizations of Vapnik's (1982) method of structural risk minimization (SRM). As well as making explicit some of the details on SRM, it provides a result that allows one to trade off errors on the training sample against improved generalization performance. It then considers the more general case when the hierarchy of classes is chosen in response to the data. A result is presented on the generalization performance of classifiers with a large margin. This theoretically explains the impressive generalization performance of the maximal margin hyperplane algorithm of Vapnik and co-workers (which is the basis for their support vector machines). The paper concludes with a more general result in terms of luckiness functions, which provides a quite general way for exploiting serendipitous simplicity in observed data to obtain better prediction accuracy from small training sets. Four examples are given of such functions, including the Vapnik-Chervonenkis (1971) dimension measured on the sample.