Multicategory Support Vector Machine Research Articles

A hybrid multicategory framework for generating automated trading systems

AbstractThis work proposes a framework for producing automatic trading systems that suggest making trading decisions in financial markets. The trading systems thus generated utilize multicategory classifiers. A collection of important technical indicators are considered as input features for the underlying multicategory classifiers. The proposed trading systems also have an option to use random forest (RF) algorithms for feature selection. The trading range breakout strategy is used to train the proposed trading systems thus generating “BUY/SELL/WAIT” trading signals on daily open prices. The performances of the proposed trading systems are evaluated over five future indices. Empirical findings suggest that the day trading systems based on the proposed multicategory SVM classifiers along with the RF technique outperform the day trading systems built using the multicategory classifiers taken from the literature. The trading range breakout strategy‐based systems are found to be superior to the traditional BUY‐HOLD strategy and that of RF‐PSVM (BUY/SELL) based strategy.

Simplex-Based Proximal Multicategory Support Vector Machine

The multicategory support vector machine (MSVM) has been widely used for multicategory classification. Despite its widespread popularity, regular MSVM cannot provide direct probabilistic results and suffers from excessive computational cost, as it is formulated on the hinge loss function and it solves a sum-to-zero constrained quadratic programming problem. In this study, we propose a general refinement of regular MSVM, termed as the simplex-based proximal MSVM (SPMSVM). Our SPMSVM uses a novel family of squared error loss functions in place of the hinge loss and it removes the explicit sum-to-zero constraint by the simplex structure. Consequently, the SPMSVM only requires solving an unconstrained linear system, leading to closed-form solutions. In addition, the SPMSVM can be cast into a weighted regression problem so that it is scalable for large-scale applications. Moreover, the SPMSVM naturally yields an estimate of the conditional category probability, which is more informative than regular MSVM. Theoretically, the SPMSVM is shown to include many existing MSVMs as its special cases, and its asymptotic and finite-sample statistical properties are well established. Simulations and real examples show that the proposed SPMSVM is a stable, scalable and competitive classifier.

Statistical inference and distributed implementation for linear multicategory SVM

Support vector machine (SVM) is one of the most prevalent classification techniques due to its excellent performance. The standard binary SVM has been well‐studied. However, a large number of multicategory classification problems in the real world are equally worth attention. In this paper, focusing on the computationally efficient multicategory angle‐based SVM model, we first study the statistical properties of model coefficient estimation. Notice that the new challenges posed by the widespread presence of distributed data, this paper further develops a distributed smoothed estimation for the multicategory SVM and establishes its theoretical guarantees. Through the derived asymptotic properties, it can be seen that our distributed smoothed estimation can achieve the same statistical efficiency as the global estimation. Numerical studies are performed to demonstrate the highly competitive performance of our proposed distributed smoothed method.

Frank-Wolfe Algorithm for Learning SVM-Type Multi-Category Classifiers

Multi-category support vector machine (MC-SVM) is one of the most popular machine learning algorithms. There are lots of variants of MC-SVM, although different optimization algorithms were developed for different learning machines. In this study, we developed a new optimization algorithm that can be applied to many of MC-SVM variants. The algorithm is based on the Frank-Wolfe framework that requires two subproblems, direction finding and line search, in each iteration. The contribution of this study is the discovery that both subproblems have a closed form solution if the Frank-Wolfe framework is applied to the dual problem. Additionally, the closed form solutions on both for the direction finding and for the line search exist even for the Moreau envelopes of the loss functions. We use several large datasets to demonstrate that the proposed optimization algorithm converges rapidly and thereby improves the pattern recognition performance.

Sparse Learning with Non-convex Penalty in Multi-classification

Multi-classification is commonly encountered in data science practice, and it has broad applications in many areas such as biology, medicine, and engineering. Variable selection in multiclass problems is much more challenging than in binary classification or regression problems. In addition to estimating multiple discriminant functions for separating different classes, we need to decide which variables are important for each individual discriminant function as well as for the whole set of functions. In this paper, we address the multi-classification variable selection problem by proposing a new form of penalty, supSCAD, which first groups all the coefficients of the same variable associated with all the discriminant functions altogether and then imposes the SCAD penalty on the supnorm of each group. We apply the new penalty to both soft and hard classification and develop two new procedures: the supSCAD multinomial logistic regression and the supSCAD multi-category support vector machine. Our theoretical results show that, with a proper choice of the tuning parameter, the supSCAD multinomial logistic regression can identify the underlying sparse model consistently and enjoys oracle properties even when the dimension of predictors goes to infinity. Based on the local linear and quadratic approximation to the non-concave SCAD and nonlinear multinomial log-likelihood function, we show that the new procedures can be implemented efficiently by solving a series of linear or quadratic programming problems. Performance of the new methods is illustrated by simulation studies and real data analysis of the Small Round Blue Cell Tumors and the Semeion Handwritten Digit data sets.

High-quality topological structures selection for smart city land spatial understanding and governance

Due to the acceleration of urbanization process in modern society, there are many metropolises throughout the world, such as New York, Tokyo, and Shanghai. Such large cities typically have a carefully designed space planning. For example, the residential area should be distant from the industrial area due to the potential environmental pollution issue. In order to cope with public health emergencies, the infrastructure needed for modern urban governance needs to be improved. Notably, establishing a quality visual model to exploit such sophisticated spatial configurations is an important but challenging task. Such a model can facilitate many applications such as urban planning, environmental evaluation, smart transportation, and urban governance. However, the flexible spatial interactions among multiple regions make it difficult to apply a traditional visual model to encode them. In this work, a quality-guided feature selection framework is proposed to obtain a set of high-quality topologies to model the discriminative structures from different land spatial city regions. Given a city region from a metropolitan area, the well-known super pixel algorithm SLIC is used to decompose each land spatial city region image into multiple atomic regions. Based on this, a binary graph is used to model the spatial interactions among these regions. Each binary graph is then decomposed into multiple subgraphs and a topology selection algorithm is proposed to discover subgraphs with highly discriminative topologies. By leveraging these high-quality subgraphs, the image kernel machine is used to convert the high quality subgraphs from each city region’s image into a feature vector. Afterward, a multi-category support vector machine (SVM) is learned to classify each city region’s image into one particular category. Comprehensive experimental results by comparing with many state-of-the-art have shown the competitiveness of this method. Furthermore, the selected high-quality topologies have demonstrated that highly representative spatial interactions are nicely encoded. The results are of great significance to the establishment of the land spatial database management system.

Support Vector Machine Based Route Classification and Description

Traveling is very much important in one's life. Location-based services have developed a lot due to the development of communication technologies. It confines services that execute programs that use geographical data. Map services authorize travelers to look for the information surrounding them and organize an outing to his/her best-loved spot. Google Maps API is convenient for locating the shortest route information. However, the map system doesn't supply any illustration about air quality or congestion in a path. At times, a substitute route with less congestion can take you quickly to your spot than a shorter route. Numerous crucial health concerns for human beings are caused due to pollution. The motive is to develop a system that offers the textual explanation of routes utilizing the sub-routes information from Google map and BreezoMeter. The end-user can choose the starting and ending points of his/her travel and the route map showing various routes from source to destination is exhibited along with a small description of each route. The illustration of the routes is obtained depending on three factors such as air quality, congestion and distance gathered from BreezoMeter, Google map traffic API and distance matrix API respectively. Multicategory Support Vector Machine (SVM) is an organized and guided categorization technique and is used here to classify factors into various levels. Since the textual illustration of the route is accessible, the end-user can effortlessly understand the details about the route and they can choose a particular route.

Support Vector Machine Based Route Classification and Description

Traveling is very much important in one's life. Location-based services have developed a lot due to the development of communication technologies. It confines services that execute programs that use geographical data. Map services authorize travelers to look for the information surrounding them and organize an outing to his/her best-loved spot. Google Maps API is convenient for locating the shortest route information. However, the map system doesn't supply any illustration about air quality or congestion in a path. At times, a substitute route with less congestion can take you quickly to your spot than a shorter route. Numerous crucial health concerns for human beings are caused due to pollution. The motive is to develop a system that offers the textual explanation of routes utilizing the sub-routes information from Google map and BreezoMeter. The end-user can choose the starting and ending points of his/her travel and the route map showing various routes from source to destination is exhibited along with a small description of each route. The illustration of the routes is obtained depending on three factors such as air quality, congestion and distance gathered from BreezoMeter, Google map traffic API and distance matrix API respectively. Multicategory Support Vector Machine (SVM) is an organized and guided categorization technique and is used here to classify factors into various levels. Since the textual illustration of the route is accessible, the end-user can effortlessly understand the details about the route and they can choose a particular route.

Extended Bayesian Framework for Multicategory Support Vector Machine

Extended Bayesian Framework for Multicategory Support Vector Machine

Multicategory large margin classification with unequal costs

In this paper, we propose a multicategory large margin classification with unequal costs. In addition to extending the standard multicategory SVM to the case of unequal costs, we also develop an unequal costs classification for ψ-loss and propose an efficient algorithm for computation. Besides commonly used L 2 penalty, the adaptive LASSO is also examined to remove irrelevant variables. Theoretically, we derive the Bayes rules under the generalized cost and reduce the infinite sum-to-zero constraint to a finite constraint. Numerically, we demonstrate the good performance of our methodology on two simulated examples and one real-life dataset.

Inconsistent Cluster Analysis With Disease Feature Enhancement (ICADFE) For American Cotton Leaf Disease Recognition

The broad leaves of cotton plant carry various visible disease symptoms. The ability of visual analysis by experts motivated the development of the plant disease recognition model. There are several visual feature descriptors, which can be primarily distinguished on the basis of pattern, texture or color. This system has been developed for the convenience of the farmers, who can avail the benefit by submitting the pictures of infected cotton leaves on the interface and the plant disease recognition system will return type of disease. In this paper, a dynamic feature descriptor is designed with inconsistent cluster analysis (ICA) and disease feature enhancement (DFE), which are combined as hybrid descriptor known as ICADFE for the recognition of the cotton plant disease. The ICADFE is found to improve the detection accuracy (approx. 80%), precision (approx 95%) and f1-measure (approx. 88%) on average in comparison with traditional shape and texture based feature descriptors such as scale invariant feature transform (SIFT), speeded up robust features (SURF) and fast retina keypoints (FREAK) with multi-category SVM (mSVM) for disease recognition.

Data Augmentation and Bayesian Methods for Multicategory Support Vector Machines

The support vector machine (SVM) has become very popular within the machine learning literature. Recently, SVM has received much attention from statisticians. It is well known that for multicategory classification problem, the commonly used multicategory SVM is based on the frequentist framework. In this paper, we develop a multi-class support vector machine under the Bayesian framework. Numerical studies were performed by EM and the Bayesian algorithm Gibbs sampler. Our results have shown that the classification accuracy of the Bayesian approach is comparable to that of frequentist approaches, while Bayesian approach also has the advantage of providing estimates of uncertainty in predictions.

Robust multicategory support vector machines using difference convex algorithm

The Support Vector Machine (SVM) is one of the most popular classification methods in the machine learning literature. Binary SVM methods have been extensively studied, and have achieved many successes in various disciplines. However, generalization to Multicategory SVM (MSVM) methods can be very challenging. Many existing methods estimate k functions for k classes with an explicit sum-to-zero constraint. It was shown recently that such a formulation can be suboptimal. Moreover, many existing MSVMs are not Fisher consistent, or do not take into account the effect of outliers. In this paper, we focus on classification in the angle-based framework, which is free of the explicit sum-to-zero constraint, hence more efficient, and propose two robust MSVM methods using truncated hinge loss functions. We show that our new classifiers can enjoy Fisher consistency, and simultaneously alleviate the impact of outliers to achieve more stable classification performance. To implement our proposed classifiers, we employ the difference convex algorithm (DCA) for efficient computation. Theoretical and numerical results obtained indicate that for problems with potential outliers, our robust angle-based MSVMs can be very competitive among existing methods.

A new algorithm for computation of a regularization solution path for reinforced multicategory support vector machines

AbstractThe recently proposed Reinforced Multicategory Support Vector Machine (RMSVM) has been proven to have desirable theoretical properties as well as competitive numerical accuracy for multi‐class classification problems. Currently solving the RMSVM is based on a grid search approach for selecting the tuning parameter , which dramatically increases its computational complexity. To overcome this hurdle we develop a new algorithm RMSVMPATH to compute a regularization solution path for RMSVM. We relax the commonly used continuity assumption and propose a new linear programming approach. Numerical simulations and real data analyses demonstrate that the proposed algorithm can yield a valid solution path at a low computational cost. The Canadian Journal of Statistics 45: 149–163; 2017 © 2017 Statistical Society of Canada

Reinforced Angle-Based Multicategory Support Vector Machines

The support vector machine (SVM) is a very popular classification tool with many successful applications. It was originally designed for binary problems with desirable theoretical properties. Although there exist various multicategory SVM (MSVM) extensions in the literature, some challenges remain. In particular, most existing MSVMs make use of k classification functions for a k-class problem, and the corresponding optimization problems are typically handled by existing quadratic programming solvers. In this article, we propose a new group of MSVMs, namely, the reinforced angle-based MSVMs (RAMSVMs), using an angle-based prediction rule with k − 1 functions directly. We prove that RAMSVMs can enjoy Fisher consistency. Moreover, we show that the RAMSVM can be implemented using the very efficient coordinate descent algorithm on its dual problem. Numerical experiments demonstrate that our method is highly competitive in terms of computational speed, as well as classification prediction performance. Supplemental materials for the article are available online.

Diabetic peripheral neuropathy class prediction by multicategory support vector machine model: a cross- sectional study

Diabetic peripheral neuropathy class prediction by multicategory support vector machine model: a cross- sectional study

Multi-category laplacian least squares twin support vector machine

In this paper, we have formulated a Laplacian Least Squares Twin Support Vector Machine called Lap-LST-KSVC for semi-supervised multi-category k-class classification problem. Similar to Least Squares Twin Support Vector Machine for multi-classification(LST-KSVC), Lap-LST-KSVC, evaluates all the training samples into “1-versus-1-versus-rest” classification paradigm, so as to generate ternary output {−1, 0, +1}. Experimental results prove the efficacy of the proposed method over other inline Laplacian Twin Support Vector Machine(Lap-TWSVM) in terms of classification accuracy and computational time.

Tree based multi-category Laplacian TWSVM for content based image retrieval

This paper attempts to combine Laplacian Twin Support Vector Machine (Lap-TWSVM) and Decision Tree Twin Support Vector Machine classifier to obtain an effective tree based classifier for semi-supervised multi-category classification. This classifier is termed as Tree Based Multi-Category Laplacian Twin Support Vector Machine (TB-Lap-TWSVM) classifier. TB-Lap-TWSVM is an improved One Against All (OAA) partition based tree classifier which takes advantage of both Lap-TWSVM and Decision Tree methodology. This paper concentrates on the application of TB-Lap-TWSVM to Content Based Image Retrieval problem. Further, extensive experiments on color images are carried out on different databases to establish the efficacy of the proposed model vis a vis TB-Lap-SVM and OAA-Lap-TWSVM.

Methods for multi-category cancer diagnosis from gene expression data: a comprehensive evaluation to inform decision support system development.

Cancer diagnosis is a major clinical applications area of gene expression microarray technology. We are seeking to develop a system for cancer diagnostic model creation based on microarray data. In order to equip the system with the optimal combination of data modeling methods, we performed a comprehensive evaluation of several major classification algorithms, gene selection methods, and cross-validation designs using 11 datasets spanning 74 diagnostic categories (41 cancer types and 12 normal tissue types). The Multi-Category Support Vector Machine techniques by Crammer and Singer, Weston and Watkins, and one-versus-rest were found to be the best methods and they outperform other learning algorithms such as K-Nearest Neighbors and Neural Networks often to a remarkable degree. Gene selection techniques are shown to significantly improve classification performance. These results guided the development of a software system that fully automates cancer diagnostic model construction with quality on par with or better than previously published results derived by expert human analysts.

A novel multi-class SVM model using second-order cone constraints

In this work we present a novel maximum-margin approach for multi-class Support Vector Machines based on second-order cone programming. The proposed method consists of a single optimization model to construct all classification functions, in which the number of second-order cone constraints corresponds to the number of classes. This is a key difference from traditional SVM, where the number of constraints is usually related to the number of training instances. This formulation is extended further to kernel-based classification, while the duality theory provides an interesting geometric interpretation: the method finds an equidistant point between a set of ellipsoids. Experiments on benchmark datasets demonstrate the virtues of our method in terms of predictive performance compared with various other multicategory SVM approaches.