Support Vector Machine Learning Algorithm Research Articles

A novel method to identify boundaries of basins of attraction in a dynamical system using Lyapunov exponents and Monte Carlo techniques

Stability analysis is a critical part of the study of a dynamical system that includes: finding the attractors of the system, determining their stability or instability, and identifying basins of attraction (BoAs) of the attractors. There are some classical methods for finding parts of BoAs, like Lyapunov-based and Non-Lyapunov-based methods; however, due to the limitations of the current methods, the identification of a complex boundary of a BoA is not feasible in addition to the high computational load of these methods. The concept of Lyapunov exponents is a powerful tool for performing stability analysis of highly nonlinear systems. In this paper, a new and intelligent approach for identifying the boundary of the entire BoA using Lyapunov exponents, Monte Carlo techniques, and support vector machine learning algorithm is developed. The proposed approach can identify the boundary of BoAs due to the intelligent sampling brought by Monte Carlo technique, which also dramatically reduces the computational load. Three illustrative examples are presented to demonstrate the effectiveness of the proposed approach.

Classification of EEG signals using a multiple kernel learning support vector machine.

In this study, a multiple kernel learning support vector machine algorithm is proposed for the identification of EEG signals including mental and cognitive tasks, which is a key component in EEG-based brain computer interface (BCI) systems. The presented BCI approach included three stages: (1) a pre-processing step was performed to improve the general signal quality of the EEG; (2) the features were chosen, including wavelet packet entropy and Granger causality, respectively; (3) a multiple kernel learning support vector machine (MKL-SVM) based on a gradient descent optimization algorithm was investigated to classify EEG signals, in which the kernel was defined as a linear combination of polynomial kernels and radial basis function kernels. Experimental results showed that the proposed method provided better classification performance compared with the SVM based on a single kernel. For mental tasks, the average accuracies for 2-class, 3-class, 4-class, and 5-class classifications were 99.20%, 81.25%, 76.76%, and 75.25% respectively. Comparing stroke patients with healthy controls using the proposed algorithm, we achieved the average classification accuracies of 89.24% and 80.33% for 0-back and 1-back tasks respectively. Our results indicate that the proposed approach is promising for implementing human-computer interaction (HCI), especially for mental task classification and identifying suitable brain impairment candidates.

An Approach for Sentiment Tendency Analysis on Comment Text

With the rapid development of network, texts which contain position, views and opinions of events are exploding. Texts of review contain author’s feelings, views and tendencies the author wants to express. People need to analyze these texts automatically to acquire sentiment tendency of the author. This paper presents a model for automatic text analysis about sentiment tendency on comment text. The model combines algorithms based on emotional dictionary and Support Vector Machine learning algorithm together, which takes advantage of both algorithms.

Video object matching across multiple non-overlapping camera views based on multi-feature fusion and incremental learning

Matching objects across multiple cameras with non-overlapping views is a necessary but difficult task in the wide area video surveillance. Owing to the lack of spatio-temporal information, only the visual information can be used in some scenarios, especially when the cameras are widely separated. This paper proposes a novel framework based on multi-feature fusion and incremental learning to match the objects across disjoint views in the absence of space–time cues. We first develop a competitive major feature histogram fusion representation (CMFH11CMFH is the abbreviation of Competitive Major Feature Histogram fusion representation.) to formulate the appearance model for characterizing the potentially matching objects. The appearances of the objects can change over time and hence the models should be continuously updated. We then adopt an improved incremental general multicategory support vector machine algorithm (IGMSVM22IGMSVM is the abbreviation of Incremental General Multicategory Support Vector Machine learning algorithm.) to update the appearance models online and match the objects based on a classification method. Only a small amount of samples are needed for building an accurate classification model in our method. Several tests are performed on CAVIAR, ISCAPS and VIPeR databases where the objects change significantly due to variations in the viewpoint, illumination and poses. Experimental results demonstrate the advantages of the proposed methodology in terms of computational efficiency, computation storage, and matching accuracy over that of other state-of-the-art classification-based matching approaches. The system developed in this research can be used in real-time video surveillance applications.

A Hybrid Optimization Technique for Fault Classification in Power Transformer Using Dissolved Gas Analysis

To allow utilities to fulfill self-imposed and regulative performance targets the demand for new optimized tools and techniques to Estimate the performance of modern Transformers has increased. The modern power transformers has subjected to different types of faults, which affect the continuity of power supply which in turn causes serious economic losses. To avoid the interruption of power supply, various fault diagnosis approaches are adopted to detect faults in the power transformer and has to eliminate the impacts of the faults at the initial stage. Among the fault diagnosis methods, the hybrid technique of Particle Swarm Optimization (PSO) with Support Vector Machine (SVM) learning algorithm is simple conceptually derived and its implementation process is faster with better scaling properties for complex problems with non linearity and load variations but performance factor related to accuracy has a declined value in case of correlations implicit . In order to obtain better fault diagnosis to improve the service of the power transformer, SVM is optimized with Improved PSO technique to achieve high interpretation accuracy for Dissolved Gas Analysis (DGA) of power transformer through the extracting positive features from both the techniques. Primary SVM is applied to establish classification features for faults in the transformer through DGA. The features are applied as input data to Autonomous optimized Technique for faults analysis. The proposed methodology obtains the DGA data set from diagnostic gas in oil of 500 KV main transformers of Pingguo Substation in South China Electric Power Company. The simulations are carried out in MATLAB software with an Intel core 3 processor with speed of 3 GHZ and 4 GB RAM PC. The result obtained by Autonomous optimized Technique (IPSO-SVM) is compared against PSO-SVM to estimate the performance of the classifiers in terms of execution time and quality of classification for precision. The test results indicate that the Autonomous optimization of IPSO-SVM approach has significantly improved the classification accuracy and computational time for power transformer fault classification. Keywords: Transformer Fault Analysis, Improved Particle Swarm Optimization, Hybrid Optimization, Dissolved Gas Analysis, Support Vector Machine

Feature subset selection by gravitational search algorithm optimization

A new method for feature subset selection in machine learning, FSS-MGSA (Feature Subset Selection by Modified Gravitational Search Algorithm), is presented. FSS-MGSA is an evolutionary, stochastic search algorithm based on the law of gravity and mass interactions, and it can be executed when domain knowledge is not available. A wrapper approach, over Naive-Bayes, ID3, K-Nearest Neighbor and Support Vector Machine learning algorithms, is used to evaluate the goodness of each visited solution. The key to the success of the MGSA is to utilize the piecewise linear chaotic map for increasing its diversity of species, and to use sequential quadratic programming for accelerating local exploitation. Promising results are achieved in a variety of tasks where domain knowledge is not available. The experimental results show that the proposed method has the ability of selecting the discriminating input features correctly and can achieve high accuracy of classification, which is comparable to or better than well-known similar classifier systems. Furthermore, the MGSA is tested on ten functions provided by CEC 2005 special session and compared with various modified Gravitational Search Algorithm, Particle Swarm Optimization, and Genetic Algorithm. The obtained results confirm the high performance of the MGSA in solving various problems in optimization.

Symmetrical SURF and Its Applications to Vehicle Detection and Vehicle Make and Model Recognition

Speeded-Up Robust Features (SURF) is a robust and useful feature detector for various vision-based applications but it is unable to detect symmetrical objects. This paper proposes a new symmetrical SURF descriptor to enrich the power of SURF to detect all possible symmetrical matching pairs through a mirroring transformation. A vehicle make and model recognition (MMR) application is then adopted to prove the practicability and feasibility of the method. To detect vehicles from the road, the proposed symmetrical descriptor is first applied to determine the region of interest of each vehicle from the road without using any motion features. This scheme provides two advantages: there is no need for background subtraction and it is extremely efficient for real-time applications. Two MMR challenges, namely multiplicity and ambiguity problems, are then addressed. The multiplicity problem stems from one vehicle model often having different model shapes on the road. The ambiguity problem results from vehicles from different companies often sharing similar shapes. To address these two problems, a grid division scheme is proposed to separate a vehicle into several grids; different weak classifiers that are trained on these grids are then integrated to build a strong ensemble classifier. The histogram of gradient and SURF descriptors are adopted to train the weak classifiers through a support vector machine learning algorithm. Because of the rich representation power of the grid-based method and the high accuracy of vehicle detection, the ensemble classifier can accurately recognize each vehicle.

Performance Comparison of SVM and ANN in Predicting Compressive Strength of Concrete

Concrete compressive strength prediction is very important in structure and building design, particularly in specifying the quality and measuring performance of concrete as well as determination of its mix proportion. The conventional method of determining the strength of concrete is complicated and time consuming hence artificial neural network (ANN) is widely proposed in lieu of this method. However, ANN is an unstable predictor due to the presence of local minima in its optimization objective. Hence, in this paper we have studied the performance of support vector machine (SVM), a stable and robust learning algorithm, in concrete strength prediction and compare the result to that of ANN. It is found that SVM displayed a slightly better performance compared to ANN and is highly stable.

SVM 학습 알고리즘을 이용한 자동차 썬루프의 부품 유무 비전검사 시스템

This paper presents a learning-based visual inspection method that addresses the need for an improved adaptability of a visual inspection system for parts verification in panorama sunroof assembly lines. It is essential to ensure that the many parts required (bolts and nuts, etc.) are properly installed in the PLC sunroof manufacturing process. Instead of human inspectors, a visual inspection system can automatically perform parts verification tasks to assure that parts are properly installed while rejecting any that are improperly assembled. The proposed visual inspection method is able to adapt to changing inspection tasks and environmental conditions through an efficient learning process. The proposed system consists of two major modules: learning mode and test mode. The SVM (Support Vector Machine) learning algorithm is employed to implement part learning and verification. The proposed method is very robust for changing environmental conditions, and various experimental results show the effectiveness of the proposed method.

Energy Efficient Smartphone-Based Activity Recognition Using Fixed-Point Arithmetic

In this paper we propose a novel energy efficient approach for the recog- nition of human activities using smartphones as wearable sensing devices, targeting assisted living applications such as remote patient activity monitoring for the disabled and the elderly. The method exploits fixed-point arithmetic to propose a modified multiclass Support Vector Machine (SVM) learning algorithm, allowing to better pre- serve the smartphone battery lifetime with respect to the conventional floating-point based formulation while maintaining comparable system accuracy levels. Experiments show comparative results between this approach and the traditional SVM in terms of recognition performance and battery consumption, highlighting the advantages of the proposed method.

Parallel geometric classification of stem cells by their three-dimensional morphology

There is a need for tools to classify cells based on their three-dimensional (3D) shape. Cells exist in vivo in 3D, cells are frequently cultured within 3D scaffolds in vitro and 3D scaffolds are used for cell delivery in tissue engineering therapies. Recent work indicates that the physical structure of a tissue engineering scaffold can direct stem cell function by driving stem cells into morphologies that induce their differentiation. Thus, we have developed a rapid method for classifying cells based on their 3D shape. First, random lines are intersected with 3D Z-stacks of confocal images of stem cells. The intersection lengths are stored in histograms, which are then used to train a support vector machine (SVM) learning algorithm to distinguish between stem cells cultured on differentiation-inducing 3D scaffolds and those cultured on non-differentiating flat substrates. The trained SVM is able to properly classify the ?new? query cells over 80% of the time. The algorithm is easily parallelizable and we demonstrate its implementation on a commodity graphics processing unit (GPU). Use of a GPU to run the algorithm increases throughput by over 100-fold as compared to use of a CPU. The algorithm is also progressive, providing an approximate answer quickly and refining the answer over time. This allows further increase in the throughput of the algorithm by allowing the SVM classification scheme to terminate early if it becomes confident enough of the class of the cell being analyzed. These results demonstrate a rapid method for classifying stem cells based on their 3D shape that can be used by tissue engineers for identifying 3D tissue scaffold structures that drive stem cells into shapes that correlate with differentiation.

Support Vector Machines for Multi-Attribute ABC Analysis

This paper examined the classification performance of Support Vector Machines (SVMs) on multi-criteria inventory analysis. The ABC analysis using the Simple Additive Weighting (SAW) method was employed to determine inventory classes of items held in inventory of a large scale automobile company operating in Turkey. The provided data set was analyzed with SVMs to obtain classification performance of the SVM learning algorithm. The results showed that SVM is highly applicable to the inventory classification problem.

Learning control of nonhonolomic robot based on support vector machine

A learning controller of nonhonolomic robot in real-time based on support vector machine (SVM) is presented. The controller includes two parts: one is kinematic controller based on nonlinear law, and the other is dynamic controller based on SVM. The kinematic controller is aimed to provide desired velocity which can make the steering system stable. The dynamic controller is aimed to transform the desired velocity to control torque. The parameters of the dynamic system of the robot are estimated through SVM learning algorithm according to the training data of sliding windows in real time. The proposed controller can adapt to the changes in the robot model and uncertainties in the environment. Compared with artificial neural network (ANN) controller, SVM controller can converge to the reference trajectory more quickly and the tracking error is smaller. The simulation results verify the effectiveness of the method proposed.

E-Mail Classification Based Learning Algorithm Using Support Vector Machine

Due to the distribution of personal computers and the internet, E-mail has become one of the most widely used communicative means. However, a massive amount of spam mail is polluting mailboxes everyday, taking advantage of the ability to send mail to any number of random people through the internet. In this paper we will introduce an efficient method of classifying E-mails using the SVM(Support Vector Machine) learning algorithm, which is recently showing high performance in the field of classifying documents. The disposition of the words inside the E-mail documents are extracted, and the performance of classification is compared and examined through the learning based on the change of DF value which occurs to reduce the disposition space in the learning level. To assess the performance of the SVM, the SVM is compared to the Naïve Bayes classifier (which uses probability methods) and a vector model classifier in order to verify that the method of using the learning algorithm of SVM shows better performance.

Identification of apnea during respiratory monitoring using support vector machine classifier: a pilot study

To determine the use of photoplethysmography (PPG) as a reliable marker for identifying respiratory apnea based on time-frequency features with support vector machine (SVM) classifier. The PPG signals were acquired from 40 healthy subjects with the help of a simple, non-invasive experimental setup under normal and induced apnea conditions. Artifact free segments were selected and baseline and amplitude variabilities were derived from each recording. Frequency spectrum analysis was then applied to study the power distribution in the low frequency (0.04-0.15 Hz) and high frequency (0.15-0.40 Hz) bands as a result of respiratory pattern changes. Support vector machine (SVM) learning algorithm was used to distinguish between the normal and apnea waveforms using different time-frequency features. The algorithm was trained and tested (780 and 500 samples respectively) and all the simulations were carried out using linear kernel function. Classification accuracy of 97.22 % was obtained for the combination of power ratio and reflection index features using SVM classifier. The pilot study indicates that PPG can be used as a cost effective diagnostic tool for detecting respiratory apnea using a simple, robust and non-invasive experimental setup. The ease of application and conclusive results has proved that such a system can be further developed for use in real-time monitoring under critical care conditions.

A New Incremental Support Vector Machine Algorithm

Support vector machine is a popular method in machine learning. Incremental support vector machine algorithm is ideal selection in the face of large learning data set. In this paper a new incremental support vector machine learning algorithm is proposed to improve efficiency of large scale data processing. The model of this incremental learning algorithm is similar to the standard support vector machine. The goal concept is updated by incremental learning. Each training procedure only includes new training data. The time complexity is independent of whole training set. Compared with the other incremental version, the training speed of this approach is improved and the change of hyperplane is reduced.

In-sample and out-of-sample model selection and error estimation for support vector machines.

In-sample approaches to model selection and error estimation of support vector machines (SVMs) are not as widespread as out-of-sample methods, where part of the data is removed from the training set for validation and testing purposes, mainly because their practical application is not straightforward and the latter provide, in many cases, satisfactory results. In this paper, we survey some recent and not-so-recent results of the data-dependent structural risk minimization framework and propose a proper reformulation of the SVM learning algorithm, so that the in-sample approach can be effectively applied. The experiments, performed both on simulated and real-world datasets, show that our in-sample approach can be favorably compared to out-of-sample methods, especially in cases where the latter ones provide questionable results. In particular, when the number of samples is small compared to their dimensionality, like in classification of microarray data, our proposal can outperform conventional out-of-sample approaches such as the cross validation, the leave-one-out, or the Bootstrap methods.

A Study on the Mathematic Model of Nox Emission in Coal-Fired Boiler with Opposed Combustion Pattern

The perfect mathematic model is critical to reseaech the NOx emission charateristic. In this paper, a new statistical learning algorithm SVM(support vector machine) was used to establish the model, based on the mechanism analysis of the NOx emission characteristics, and grid optimization method was applied to determine the model parameters. The model was tested on a 660MW power plant ,and the result indicated that SVM was a good tool for building NOx emission model and had better generalization ability and higher calculation speed comparing with BP modeling approaches.

A novel SVM modeling approach for highly imbalanced and overlapping classification

Traditional classification algorithms can be limited in their performance on highly imbalanced and overlapping data sets, In this paper, we focus on modifying support vector machines (SVMs) to make it suitable for highly imbalanced and overlapping (HIO) classification. Based on the analysis of most SVM learning algorithms for imbalanced classification, we argue that in SVM-based algorithms, due to the linearity property of SVM, the key problem is that the increase of the number of correctly predicted minority samples will lead to even more majority samples be misclassified. Then a novel algorithm HIO-SVM is developed, it can recognize all minority samples while minimizing the error rate of majority ones. The proposed approach can identify the non-overlapping samples in one feature space, furthermore, by iteratively shifting kernel spaces, all non-overlapping samples in different kernel spaces are recognized. Because of the highly imbalanced distribution, the remaining overlapping samples can be regarded as minority. Then all minority samples can be predicted correctly and the error rate of majority samples can be guaranteed minimized simultaneously. Finally, numerous case studies show the properties and effectiveness of the proposed HIO-SVM algorithm.

Active learning with adaptive regularization

In classification problems, active learning is often adopted to alleviate the laborious human labeling efforts, by finding the most informative samples to query the labels. One of the most popular query strategy is selecting the most uncertain samples for the current classifier. The performance of such an active learning process heavily relies on the learned classifier before each query. Thus, stepwise classifier model/parameter selection is quite critical, which is, however, rarely studied in the literature. In this paper, we propose a novel active learning support vector machine algorithm with adaptive model selection. In this algorithm, before each new query, we trace the full solution path of the base classifier, and then perform efficient model selection using the unlabeled samples. This strategy significantly improves the active learning efficiency with comparatively inexpensive computational cost. Empirical results on both artificial and real world benchmark data sets show the encouraging gains brought by the proposed algorithm in terms of both classification accuracy and computational cost. ► Model selection is very important for active learning. ► An adaptive model is more suitable for active learning than a fixed model. ► We propose a novel active learning SVM algorithm with adaptive model selection. ► We use unlabeled data to validate the model parameter in our algorithm. ► We investigate how to properly use the data in active learning scenario.