Set Of Support Vectors Research Articles

Active Incremental Learning for Health State Assessment of Dynamic Systems With Unknown Scenarios

This article is concerned with the data-driven health state assessment task with unknown scenarios. Unknown scenarios are inaccessible in the stage of model training but they can appear unexpectedly during the running stage. A new problem called the within-class distribution mismatch is raised by assuming that unknown scenarios still belong to known classes. To tackle this challenging problem, a novel active incremental learning scheme with a classifier and an out-of-distribution (OOD) detector is proposed. A <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">k</i> -fast incremental support vector data description ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">k</i> -FISVDD) model is put forward as the OOD detector to recognize distribution mismatch samples online for label annotation. Specifically, it integrates the clustering algorithm to build local support vector sets, based on which an active query strategy is developed. An incremental learning mechanism is also designed to reduce the labeling cost. Then, the new labeled data can simultaneously refine the classifier and OOD detector. Two cases, including a bearing benchmark dataset and the operation data of a practical deep-sea manned submersible, are studied to demonstrate the effectiveness of the proposed scheme.

Autonomous Navigation in Unknown Environments With Sparse Bayesian Kernel-Based Occupancy Mapping

This article focuses on online occupancy mapping and real-time collision checking onboard an autonomous robot navigating in a large unknown environment. Commonly used voxel and octree map representations can be easily maintained in a small environment but have increasing memory requirements as the environment grows. We propose a fundamentally different approach for occupancy mapping, in which the boundary between occupied and free space is viewed as the decision boundary of a machine learning classifier. This work generalizes a kernel perceptron model which maintains a very sparse set of support vectors to represent the environment boundaries efficiently. We develop a probabilistic formulation based on relevance vector machines, handling measurement noise, and probabilistic occupancy classification, supporting autonomous navigation. We provide an online training algorithm, updating the sparse Bayesian map incrementally from streaming range data, and an efficient collision-checking method for general curves, representing potential robot trajectories. The effectiveness of our mapping and collision checking algorithms is evaluated in tasks requiring autonomous robot navigation and active mapping in unknown environments.

A working likelihood approach to support vector regression with a data-driven insensitivity parameter

The insensitivity parameter in support vector regression determines the set of support vectors that greatly impacts the prediction. A data-driven approach is proposed to determine an approximate value for this insensitivity parameter by minimizing a generalized loss function originating from the likelihood principle. This data-driven support vector regression also statistically standardizes samples using the scale of noises different from conventional response scaling method. Statistical standardization together with probabilistic regularization based on a working likelihood function produces data-dependent values for the hyperparameters including the insensitivity parameter. The exact asymptotical solutions are provided when the noises are normally distributed. Nonlinear and linear numerical simulations with three types of noises (epsilon-Laplacian distribution, normal distribution, and uniform distribution), and in addition, five real benchmark data sets, are used to test the capacity of the proposed method. Based on all the simulations and the five case studies, the proposed support vector regression using a working likelihood, data-driven insensitivity parameter is superior and has lower computational costs.

Intelligent Diagnosis Method for New Diseases Based on Fuzzy SVM Incremental Learning.

The diagnosis of new diseases is a challenging problem. In the early stage of the emergence of new diseases, there are few case samples; this may lead to the low accuracy of intelligent diagnosis. Because of the advantages of support vector machine (SVM) in dealing with small sample problems, it is selected for the intelligent diagnosis method. The standard SVM diagnosis model updating needs to retrain all samples. It costs huge storage and calculation costs and is difficult to adapt to the changing reality. In order to solve this problem, this paper proposes a new disease diagnosis method based on Fuzzy SVM incremental learning. According to SVM theory, the support vector set and boundary sample set related to the SVM diagnosis model are extracted. Only these sample sets are considered in incremental learning to ensure the accuracy and reduce the cost of calculation and storage. To reduce the impact of noise points caused by the reduction of training samples, FSVM is used to update the diagnosis model, and the generalization is improved. The simulation results on the banana dataset show that the proposed method can improve the classification accuracy from 86.4% to 90.4%. Finally, the method is applied in COVID-19's diagnostic. The diagnostic accuracy reaches 98.2% as the traditional SVM only gets 84%. With the increase of the number of case samples, the model is updated. When the training samples increase to 400, the number of samples participating in training is only 77; the amount of calculation of the updated model is small.

A support vector approach based on penalty function method

Support vector machine (SVM) models are usually trained by solving the dual of a quadratic programming, which is time consuming. Using the idea of penalty function method from optimization theory, this paper combines the objective function and the constraints in the dual, obtaining an unconstrained optimization problem, which could be solved by a generalized Newton method, yielding an approximate solution to the original model. Extensive experiments on pattern classification were conducted, and compared to the quadratic programming-based models, the proposed approach is much more computationally efficient (tens to hundreds of times faster) and yields similar performance in terms of receiver operating characteristic curve. Furthermore, the proposed method and quadratic programming-based models extract almost the same set of support vectors.

Noniterative Sparse LS-SVM Based on Globally Representative Point Selection.

A least squares support vector machine (LS-SVM) offers performance comparable to that of SVMs for classification and regression. The main limitation of LS-SVM is that it lacks sparsity compared with SVMs, making LS-SVM unsuitable for handling large-scale data due to computation and memory costs. To obtain sparse LS-SVM, several pruning methods based on an iterative strategy were recently proposed but did not consider the quantity constraint on the number of reserved support vectors, as widely used in real-life applications. In this article, a noniterative algorithm is proposed based on the selection of globally representative points (global-representation-based sparse least squares support vector machine, GRS-LSSVM) to improve the performance of sparse LS-SVM. For the first time, we present a model of sparse LS-SVM with a quantity constraint. In solving the optimal solution of the model, we find that using globally representative points to construct the reserved support vector set produces a better solution than other methods. We design an indicator based on point density and point dispersion to evaluate the global representation of points in feature space. Using the indicator, the top globally representative points are selected in one step from all points to construct the reserved support vector set of sparse LS-SVM. After obtaining the set, the decision hyperplane of sparse LS-SVM is directly computed using an algebraic formula. This algorithm only consumes O(N2) in computational complexity and O(N) in memory cost which makes it suitable for large-scale data sets. The experimental results show that the proposed algorithm has higher sparsity, greater stability, and lower computational complexity than the traditional iterative algorithms.

Machine learning method based on improved drosophila optimisation algorithm

Aiming at the problems of poor classification effect and high CPU ratio of traditional machine learning methods, a machine learning method based on improved drosophila optimisation algorithm was proposed. The rank one data mapping and the low order data are established. In low rank support vector set, CP rank organisation of traditional support vector machine is used to improve data security. The traditional drosophila algorithm was improved and optimised to increase the number of data iterations, ensure the compatibility of rank one data, improve the optimal calculation of drosophila, and increase the density clustering. The decomposition process is designed to evaluate the objective function value of the optimal solution. In the evaluation process, support vector machine is used to complete the label classification of learning data. Experimental data show that this method performs well in data classification effect, low-rank data storage dimension characteristic performance and CPU operation proportion performance.

Efficient nearest neighbors methods for support vector machines in high dimensional feature spaces

In the context of support vector machines, identifying the support vectors is a key issue when dealing with large data sets. In Camelo et al. (Ann Oper Res 235:85–101, 2015), the authors present a promising approach to finding or approximating most of the support vectors through a procedure based on sub-sampling and enriching the support vector sets by nearest neighbors. This method has been shown to improve the computational efficiency of support vector machines on large data sets with low or intermediate feature space dimension. In the present article we discuss ways of adapting the nearest neighbor enriching methodology to the context of very high dimensional data, such as text data or other high dimensional data types, for which nearest neighbor queries involve, in principle, a high computational cost. Our approach incorporates the proximity preserving order search algorithm of Chavez et al. (MICAI 2005: advances in artificial intelligence, Springer, Berlin, pp 405–414, 2005), into the nearest neighbor enriching method of Camelo et al. (2015), in order to adapt this procedure to the high dimension setting. For the required set of pivots, both random pivots and the base prototype pivot set of Mico et al. (Pattern Recogn Lett 15:9–17, 2015), are considered. The methodology proposed is evaluated on real data sets.

Point and Interval Solar Power Forecasting Using Hybrid Empirical Wavelet Transform and Robust Wavelet Kernel Ridge Regression

In this paper, a new and efficient hybrid empirical wavelet transform (EWT)-based reduced robust Mexican hat wavelet kernel ridge regression (RMHWK) model is proposed to achieve both point and interval forecasting of solar power in a smart grid scenario. Initially, the actual nonlinear solar power data series was decomposed by the EWT method. A reduced robust kernel ridge regression (RKRR) approach was incorporated that shows a notable decrease in training time without appreciable loss in forecasting accuracy. The reduction in the size of the kernel matrix was achieved by selecting a set of random support vectors from the training data set. For validating the superior performance of the proposed EWT-RMHWK forecasting model, a numerical experimentation implementing a real-time data set of 1 MW solar power plant (Odisha, India) as well as an online historical data set (Florida, USA) was considered and compared with other hybrid models using either empirical mode decomposition- or wavelet decomposition-based RKRR and EWT-ELM, etc. The kernel parameters were optimized with the chaotic water cycle algorithm to boost the performance of the proposed prediction model. Further, the proposed EWT-RKRR method was used to construct prediction interval forecasting with three different confidence levels with 90%, 95%, and 99% for Florida solar power plant using different time horizons of 15 min, 1 h, and 1 day, respectively.

A low rank robust kernel ridge regression classifier for power quality disturbance pattern recognition

This paper presents the development of a new low rank robust kernel ridge regression (KRR) classifier for power quality (PQ) disturbance pattern recognition. It is well known that the kernel methods are used extensively for regression and classification problems using support vector machines (SVM) by mapping the original nonlinear data into a high-dimensional space thereby increasing the generalization performance, and accuracy of classification. On the other hand, the nonlinear kernel ridge regression approach is known for its simple implementation, fast processing speed and accuracy in comparison to the widely used least square support vector machine (LS-SVM). However, for large scale training patterns, the size of kernel matrix becomes large thereby the execution time becomes prohibitive. Besides, the presence of noise and outliers in the data affects the accuracy of the conventional KRR classifier. This paper, therefore, attempts to develop a dimensionally reduced but robust KRR classifier by choosing a random set of support vectors from the training subset to reduce substantially the training time at the cost of a slight loss in accuracy. Further, the KRR algorithm is modified by using a new objective function to minimize the mean and variance of the error to provide robustness and accuracy. For applying this new classifier to power quality disturbance events three relatively new signal processing techniques like the Short-time modified Hilbert Transform (STMHT), Morphological filters, and Fourier kernel S-transform are used to extract the relevant features from the data samples. The simulation results imply that the proposed methods have a higher recognition rate compared with other established techniques such as ELM and Poly SVM while classifying the PQ disturbances. A PC integrated hardware assembly has been used to verify the PQ events classification in real-time.

Evaluation of Machine Learning Models for Predicting Antimicrobial Resistance of Actinobacillus pleuropneumoniae From Whole Genome Sequences.

Antimicrobial resistance (AMR) is becoming a huge problem in countries all over the world, and new approaches to identifying strains resistant or susceptible to certain antibiotics are essential in fighting against antibiotic-resistant pathogens. Genotype-based machine learning methods showed great promise as a diagnostic tool, due to the increasing availability of genomic datasets and AST phenotypes. In this article, Support Vector Machine (SVM) and Set Covering Machine (SCM) models were used to learn and predict the resistance of the five drugs (Tetracycline, Ampicillin, Sulfisoxazole, Trimethoprim, and Enrofloxacin). The SVM model used the number of co-occurring k-mers between the genome of the isolates and the reference genes to learn and predict the phenotypes of the bacteria to a specific antimicrobial, while the SCM model uses a greedy approach to construct conjunction or disjunction of Boolean functions to find the most concise set of k-mers that allows for accurate prediction of the phenotype. Five-fold cross-validation was performed on the training set of the SVM and SCM model to select the best hyperparameter values to avoid model overfitting. The training accuracy (mean cross-validation score) and the testing accuracy of SVM and SCM models of five drugs were above 90% regardless of the resistant mechanism of which were acquired resistant or point mutation in the chromosome. The results of correlation between the phenotype and the model predictions of the five drugs indicated that both SVM and SCM models could significantly classify the resistant isolates from the sensitive isolates of the bacteria (p < 0.01), and would be used as potential tools in antimicrobial resistance surveillance and clinical diagnosis in veterinary medicine.

Fast Incremental SVDD Learning Algorithm with the Gaussian Kernel

Support vector data description (SVDD) is a machine learning technique that is used for single-class classification and outlier detection. The idea of SVDD is to find a set of support vectors that defines a boundary around data. When dealing with online or large data, existing batch SVDD methods have to be rerun in each iteration. We propose an incremental learning algorithm for SVDD that uses the Gaussian kernel. This algorithm builds on the observation that all support vectors on the boundary have the same distance to the center of sphere in a higher-dimensional feature space as mapped by the Gaussian kernel function. Each iteration involves only the existing support vectors and the new data point. Moreover, the algorithm is based solely on matrix manipulations; the support vectors and their corresponding Lagrange multiplier αi’s are automatically selected and determined in each iteration. It can be seen that the complexity of our algorithm in each iteration is only O(k2), where k is the number of support vectors. Experimental results on some real data sets indicate that FISVDD demonstrates significant gains in efficiency with almost no loss in either outlier detection accuracy or objective function value.

Distributed Traversability Analysis of Flow Field Under Communication Constraints

Traversability analysis, previously developed for ground robots, is extended to be performed by a group of marine vehicles over ocean flow fields. To enable distributed path planning against flow, a systematic method is developed to determine a flow map that is significant for sharing among vehicles. Using traversability analysis, each vehicle finds a set of support vectors to approximate the boundary of flow regions with strong average flow speed. These support vectors, together with the parameters determined by the traversability analysis, are then shared over the communication links. The shared information is used by each vehicle to construct a flow map that supports path planning algorithms running on each vehicle. Underwater communication limits the amount of information contained in a map that can be shared among vehicles. A novel method for information reduction is developed so that only reduced flow maps are shared. The length of the data packets can be adjusted through an optimization process that balance between the fidelity of the approximated boundary and the number of bits that need to be transmitted. Simulation results demonstrate that the flow maps constructed from the shared information can have sufficient quality to allow near-optimal paths to be generated.

Between-subclass piece-wise linear solutions in large scale kernel SVM learning

The paper proposes a novel approach for learning kernel Support Vector Machines (SVM) from large scale data with reduced computation time. The proposed approach, termed as Subclass Reduced Set SVM (SRS-SVM), utilizes the subclass structure of data to effectively estimate the candidate support vector set. Since the candidate support vector set cardinality is only a fraction of the training set cardinality, learning SVM from the former requires less time without significantly changing the decision boundary. SRS-SVM depends on a domain knowledge related input parameter, i.e., number of subclasses. To reduce the domain knowledge dependency and to make the approach less sensitive to the subclass parameter, we extend the proposed SRS-SVM to create a robust and improved hierarchical model termed as the Hierarchical Subclass Reduced Set SVM (HSRS-SVM). Since SRS-SVM and HSRS-SVM splits non-linear optimization problem into multiple (smaller) linear optimization problems, both of them are amenable to parallelization. The effectiveness of the proposed approaches is evaluated on four synthetic and six real-world datasets. The performance is also compared with traditional solver (LibSVM) and state-of-the-art approaches such as divide-and-conquer SVM, FastFood, and LLSVM. The experimental results demonstrate that the proposed approach achieves similar classification accuracies while requiring fewer folds of reduced computation time as compared to existing solvers. We further demonstrate the suitability and improved performance of the proposed HSRS-SVM with deep learning features for face recognition using Labeled Faces in the Wild (LFW) dataset.

Fixed least squares support vector machines for flashover modelling of outdoor insulators

Many theoretical and experimental studies have been performed to optimize the maintenance times of outdoor insulators. But most of the proposed methods are both time consuming and difficult to apply in practice and also they rely mainly on pollution charts and they are not reliable to weather variations. Artificial intelligence has been extensively applied to solve a large number of electrical and high voltage engineering modeling and optimization problems. In this paper, a methodology based on modified LS-SVM strategy using a fixed set of support vectors is proposed to evaluate the flashover performance of outdoor insulators under contaminated conditions, where the candidate support vectors are selected from the training set according to a quadratic Renyi criterion. The obtained results are promising and ensure that the presented technique can help high voltage engineers to assess insulator performance including such important factors as the flashover behavior, the insulator geometrical parameters, aging, and contamination accumulation. Further comparative analysis of the estimated results with the measured data collected from the site measurement demonstrate the effectiveness of the use of fixed-size LS-SVMs models for flashover prediction.

A hybrid transfer learning algorithm incorporating TrSVM with GASEN

Traditional machine learning is generally committed to obtaining classifiers which are well-performed over unlabeled test data. This usually relies on two critical assumptions: firstly, sufficient labeled training data are available; secondly, training and testing data are drawn from the same distribution and the same feature space. Unfortunately, in most cases, the actual situation is difficult to meet the above conditions. Transfer learning scheme is naturally proposed to alleviate this problem. In order to get robust classifiers with relatively lower computational costs, we incorporate the rationale of Support Vector Machine (SVM) into transfer learning scheme and propose a novel SVM-based transfer learning model, abbreviated as TrSVM. In this method, support vector sets are extracted to represent the source domain. New training datasets are respectively constructed by combining each support vector set and target labeled dataset. On the basis of these training datasets, a number of new base classifiers can be acquired. Since performance of a classifiers ensemble is generally superior to that of individual classifiers, ensemble selection is utilized in our work. A hybrid transfer learning algorithm, integrating the Genetic Algorithm based Selective Ensemble (GASEN) with TrSVM, is proposed, and abbreviated as TrGASVM, naturally. GASEN is a genetic algorithm-based heuristic algorithm for solving combinatorial optimization problems. It can not only enhance the generalization ability of an ensemble, but also alleviate the local minimum problem of greedy ensemble pruning methods. Since TrGASVM is under frame of TrSVM and GASEN, it inevitably inherits the advantages of both algorithms. The reasonable incorporation of TrSVM with GASEN endows TrGASVM with favorable transfer learning capability, with its effectiveness being demonstrated by the experimental results on three real-world text classification datasets.

Variational approach to cancerous tissue identification from in vivo Raman spectra

In biomedical applications, the human factor represents the major obstacle in the processing and classification of in vivo Raman spectra. Although several methods of statistical classification have been employed to address this issue, the outcomes thereof vary depending on the selection of spectral bands used for the evaluation. To avoid the use of such discrete values, we developed a variational classifier based on spectrum weighting, weight smoothing and regularization and the selection of a support vector set. Embedded regularization of the variational classifier smooths weight function and therefore original Raman spectra preprocessing don’t need to include spectra smoothing. Due to a small data set, we proposed a hierarchical cross-validation technique performing better data generalization and overfitting avoiding. Accuracy of variational classifier obtained by hierarchical cross-validation for the data set, involving 30 Raman spectra of which 13 represent cancerous and 17 healthy tissues, was 93% with sensitivity of 92% and specificity of 94%.

Parameter identification of unmanned marine vehicle manoeuvring model based on extended Kalman filter and support vector machine

To predict the manoeuvrability of unmanned marine vehicle and improve its manoeuvrability, the parameters of the manoeuvring model of unmanned marine vehicle need to be obtained. Aiming at the inconvenience of obtaining model parameters under the traditional experimental method, this article studies the parameter identification of unmanned marine vehicle’s manoeuvring model based on extended Kalman filter and support vector machine. Firstly, the second-order nonlinear manoeuvring response model of unmanned marine vehicle is discretized by the difference method, and the corresponding data are collected by the manoeuvring motion simulation of the response model. Secondly, the discrete response model is transformed into an augmented state vector based on extended Kalman filter, and the optimal estimation of the state vector is calculated to identify the parameters. And then, the discrete response model is transformed into a support vector machine-based regression model, the collected data are processed and a set of support vectors are obtained to further identify the parameters of the response model. Finally, by comparing the simulation experiments’ results from the original model and the identification model, the recognition results-based extended Kalman filter and support vector machine are analysed and some research results are obtained. The results of this article will provide a powerful reference for the design of unmanned marine vehicle’s motion control algorithm.

Research on Incremental Learning of SVM Based on Robustness

The following problems exist in the process of incremental learning of support vector machines. If the number of support vectors will increase with the increase of the increment sample, the time of training will become longer and longer; if the vector that has no effect on the hyperplane of the classification is abandoned, this part of the vector may become a support vector in the subsequent training. This will have an impact on the effect of the classification. In this paper, a support vector machine incremental learning algorithm based on fuzzy C mean clustering and central density is used to determine support vector set by fuzzy C mean clustering, and the selected non support vector sets are obtained by using the ratio of center density to determine non support vectors. The effect of non support vector on the robustness of support vector incremental learning is studied by comparing the classification efficiency of the four kinds of classification, and the conclusion of this paper is finally obtained.

Separation of Handwritten and Machine-Printed Texts from Noisy Documents Using Contourlet Transform

To make paperless environment in office, document image analysis, where optical character recognition is mostly used, plays a major role. The documents such as bank cheques, admission forms, application forms, memorandums and letters generally consist of text material in mixed form, i.e., handwritten and machine-printed texts along with some noises. Because of this mixture, significant issues raise in the recognition process. By separating out handwritten and machine-printed texts, a solution is offered to overcome this problem. In this paper, a new algorithm is proposed for separation of machine-printed and handwritten texts using correlation coefficients and probabilities-based moments features. Contourlet transform is used to extract these significant features because it has excellent directional and isotropic properties. Finally, set of support vector machines classifiers is used to identify machine-printed text, handwritten text and noise. Comprehensive experiments on different databases show that the proposed algorithm is robust and it provides a better text separation performance compared to that of other state-of-the-art approaches. In benchmarking analysis, maximum identification recall rate of 98.9% is obtained from the proposed technique, which demonstrates its effectiveness.