Core Vector Machine Research Articles

Fast AUC Maximization Learning Machine With Simultaneous Outlier Detection.

While AUC maximizing support vector machine (AUCSVM) has been developed to solve imbalanced classification tasks, its huge computational burden will make AUCSVM become impracticable and even computationally forbidden for medium or large-scale imbalanced data. In addition, minority class sometimes means extremely important information for users or is corrupted by noises and/or outliers in practical application scenarios such as medical diagnosis, which actually inspires us to generalize the AUC concept to reflect such importance or upper bound of noises or outliers. In order to address these issues, by means of both the generalized AUC metric and the core vector machine (CVM) technique, a fast AUC maximizing learning machine, called ρ-AUCCVM, with simultaneous outlier detection is proposed in this study. ρ-AUCCVM has its notorious merits: 1) it indeed shares the CVM's advantage, that is, asymptotically linear time complexity with respect to the total number of sample pairs, together with space complexity independent on the total number of sample pairs and 2) it can automatically determine the importance of the minority class (assuming no noise) or the upper bound of noises or outliers. Extensive experimental results about benchmarking imbalanced datasets verify the above advantages of ρ-AUCCVM.

The Application of Machine Learning in Predicting Outcome of Cryotherapy and Immunotherapy for Wart Removal.

BackgroundWarts can be extremely painful conditions that may be associated with localised bleeding and discharge. They are commonly treated by cryotherapy or immunotherapy. However, each of these therapies have discomforting side effects and are no official dermatological guideline that exist that may be used to determine which of these methods would work on an individual patient.ObjectiveThis study aimed at developing a machine learning algorithm that improved the prediction of the outcome of wart removing using cryotherapy and immunotherapy.MethodsSupport vector machines, core vector machines, random forest, k-nearest neighbours, multilayer perceptron and binary logistic regression was applied on datasets in to create a model that predicted the outcome of an immunotherapy and cryotherapy treatments based on sex, age, time that has passed since last treatment, number of warts, type, area, diameter and result of treatment.ResultsThe average accuracy of the immunotherapy prediction was 88.6%±8.0% while the same measure for cryotherapy prediction was 94.6%±4.0%. The most efficient immunotherapy and cryotherapy model had an accuracy of 100%, predicating the correct treatment outcome when applied to all test cases.ConclusionThis study successfully created a machine learning model that improved the prediction ability of the outcome of immunotherapy and cryotherapy for wart removal. This model created a more in-depth guideline for understanding is immunotherapy would work and took a new approach to cryotherapy.

Deep Kernel machines: a survey

The emergence of deep learning frameworks paves the way for achieving higher-level data abstractions and possess the potential in consolidating both supervised and unsupervised learning paradigms. Researchers have made many successful explorations in the field of deep learning, with applications in the fields of face recognition, text mining, language translation, image prediction, and action recognition. Kernel machines act as a bridge between the linearity and nonlinearity for many machine learning algorithms such as support vector machines, extreme learning machines, and core vector machines. These Kernel machines play a vital role in mapping the data in the input space to a Kernel-induced high-dimensional feature space to obtain a better distribution of the data. In this Kernel-induced high-dimensional feature space, the distribution of data points will be more amenable to the classification problem under consideration. The Kernel trick facilitates in transforming the machine learning algorithms that require only inner product computations between the data vectors into a Kernel-based approach by selecting an appropriate Kernel function. In Kernel-based approaches, the Kernel functions can thus be utilized for accomplishing the inner product computations between the transformed data vectors in an implicitly defined Kernel-induced feature space. Unlike neural networks, the Kernel machines guarantee structural risk minimization and global optimal solutions. Also, the Kernel machines exhibit capabilities such as theoretical tractability and excellent performance in practical applications. These attempts motivated the researchers towards utilizing the emerging trends of deep learning with Kernel methods for building deep Kernel machines. Researchers integrate Kernel methods and deep learning networks for maintaining their advantages and make up their limitations, then apply the deep Kernel learning approaches for improving the performance of the learning algorithm in different applications. Different ways of building deep Kernel machines by integrating the Kernel methods and deep learning architectures include utilizing Kernel machines as the final classifier of deep learning networks, Kernelization in deep neural networks for better feature enrichment, and building deep Kernel machines by utilizing deep or multiple Kernels in different tasks. This survey attempts to provide an overview of different approaches in building several deep Kernel learning architectures for enhancing the learning algorithm properties and their performance in practical applications.

A Practical Model Based on Anomaly Detection for Protecting Medical IoT Control Services Against External Attacks

The application of the Internet of Things (IoT) in medical field has brought unprecedented convenience to human beings. However, attackers can use device configuration vulnerabilities to hijack devices, control services, steal medical data, or make devices operate illegally. These restrictions have led to huge security risks for IoT, and have challenged the management of critical infrastructure services. Based on these problems, this article proposes an anomaly detection system for detecting illegal behavior (DIB) in medical IoT environment.The DIB system can analyze data packets transmitted by medical IoT devices, learn operation rules by itself, and remind management personnel that the device is in an abnormal operation state to ensure the safety of control service. We further propose a model that is based on rough set theory and fuzzy core vector machine (FCVM) to improve the accuracy of DIB classification anomalies. Experimental results show that the R-FCVM is effective.

Classification of Benign and Malignant Breast Masses on Mammograms for Large Datasets using Core Vector Machines.

Breast cancer is one of the most leading causes of cancer deaths among women. Early detection of cancer increases the survival rate of the affected women. Machine learning approaches that are used for classification of breast cancer usually takes a lot of processing time during the training process. This paper attempts to propose a Machine Learning approach for breast cancer detection in mammograms, which does not depend on the number of training samples. The paper aims to develop a core vector machine-based diagnosis system for breast cancer detection using the date from MIAS. The main motivation behind using this system is to reduce the computational and memory requirement for large training data and to improve the classification accuracy. The proposed method has four stages: 1) Pre-processing is done to extract the breast region using global thresholding and enhancement using histogram equalization; 2) identification of potential mass using Otsu thresholding; 3) feature extraction using Laws Texture energy measures; and 4) mass detection is done using Core vector machine (CVM) classifier. Comparative analysis was done with different existing algorithms: Artificial Neural Network (ANN), Support Vector Machine (SVM), and Fuzzy Support Vector Machines (FSVM). The results illustrate that the proposed Core Vector Machine (CVM) classifier produced a promising result in terms of sensitivity (96.9%), misclassification rate (0.0443) and accuracy (95.89%). The time taken for training process is 0.0443, which is less when compared with other machine learning algorithms. Performance analysis shows that CVM classifier is superior to other classifiers like ANN, SVM and FSVM. The computational time of the CVM classifier during the training process was also analysed and found to be better than other discussed algorithms. The results achieved show that CVM classifier is the best algorithm for breast mass detection in mammograms.

Detection of Pulsar Candidates using Bagging Method

Abstract The pulsar classification represents a major issue in the astrophysical area. The Bagging Algorithm is an ensemble method widely used to improve the performance of classification algorithms, especially in the case of pulsar search. In this way, our paper tries to prove how the Bagging Method can improve the performance of pulsar candidate detection in connection with four basic classifiers: Core Vector Machines (CVM), the K-Nearest-Neighbors (KNN), the Artificial Neural Network (ANN), and Cart Decision Tree (CDT). The Error Rate, Area Under the Curve (AUC), and Computation Time (CT) are measured to compare the performance of different classifiers. The High Time Resolution Universe (HTRU2) dataset, collected from the UCI Machine Learning Repository, is used in the experimentation phase.

Sparsification of core set models in non-metric supervised learning

Supervised learning employing positive semi definite kernels has gained wide attraction and lead to a variety of successful machine learning approaches. The restriction to positive semi definite kernels and a hilbert space is common to simplify the mathematical derivations of the respective learning methods, but is also limiting because more recent research indicates that non-metric, and therefore non positive semi definite, data representations are often more effective. This challenge is addressed by multiple approaches and recently dedicated algorithms for so called indefinite learning have been proposed. Along this line, the Krĕin space Support Vector Machine (KSVM) and variants are very efficient classifiers for indefinite learning problems, but with a non-sparse decision function. This very dense decision function prevents practical applications due to a costly out of sample extension. We focus on this problem and provide two post processing techniques to sparsify models as obtained by a Krĕin space SVM approach. In particular we consider the indefinite Core Vector Machine and indefinite Core Vector Regression Machine which are both efficient for psd kernels, but suffer from the same dense decision function, if the Krĕin space approach is used. We evaluate the influence of different levels of sparsity and employ a Nyström approach to address large scale problems. Experiments show that our algorithm is similar efficient as the non-sparse Krĕin space Support Vector Machine but with substantially lower costs, such that also problems of larger scale can be processed.

Self-Adjusting Fuzzy Support Vector Machine Based on Analysis of Potential Support Vector Sample Point

Fuzzy support vector machine (FSVM) is a part of machine learning with its good classification effect. So far, there are two most commonly used FSVM models: FSVM on account of class core and fuzzy support vector machine on account of hyperplane that is over class core. Each has its own problems: FSVM on account of class core are dependent on the geometric shape of sample sets. Although FSVM on account of hyperplane that is over class core can solve the above problems to some extent. However, this algorithm has low generalization ability and high time complexity. Therefore, Inspired by these two common models, the paper proposes an improved membership function method. By analyzing and calculating the potential support vector sample points, adjustment factor is obtained, which drives the class core to adjust along the direction away from the outliers. In this way, membership of noise and outliers are reduced and the membership of support vector will also increase to some extent. In this paper, a new experimental comparison method is used, which can make the comparison of classification effect more obvious and convincing. The experimental part compares the proposed FSVM model with the above two FSVM models. It shows that the proposed algorithm improves the stability and classification accuracy to some extent.

Identification of Bio-Markers for Breast Cancer Detection through Data Mining Methods

Worldwide, breast cancer is the leading type of cancer in women accounting for 25% of all cases. Survival rates in the developed countries are comparatively higher with that of developing countries. This had led to the importance of computer aided diagnostic methods for early detection of breast cancer disease. This eventually reduces the death rate. This paper intents the scope of the biomarker that can be used to predict the breast cancer from the anthropometric data. This experimental study aims at computing and comparing various classification models (Binary Logistic Regression, Ball Vector Machine (BVM), C4.5, Partial Least Square (PLS) for Classification, Classification Tree, Cost sensitive Classification Tree, Cost sensitive Decision Tree, Support Vector Machine for Classification, Core Vector Machine, ID3, K-Nearest Neighbor, Linear Discriminant Analysis (LDA), Log-Reg TRIRLS, Multi Layer Perceptron (MLP), Multinomial Logistic Regression (MLR), Naïve Bayes (NB), PLS for Discriminant Analysis, PLS for LDA, Random Tree (RT), Support Vector Machine SVM) for the UCI Coimbra breast cancer dataset. The feature selection algorithms (Backward Logit, Fisher Filtering, Forward Logit, ReleifF, Step disc) are worked out to find out the minimum attributes that can achieve a better accuracy. To ascertain the accuracy results, the Jack-knife cross validation method for the algorithms is conducted and validated. The Core vector machine classification algorithm outperforms the other nineteen algorithms with an accuracy of 82.76%, sensitivity of 76.92% and specificity of 87.50% for the selected three attributes, Age, Glucose and Resistin using ReleifF feature selection algorithm.

A physiological and behavioral feature authentication scheme for medical cloud based on fuzzy-rough core vector machine

The medical cloud brings in connivance to share medical data within the same hospital or between different hospitals. However, sharing medical data on the cloud faces challenges such as (1) the medical cloud platform is vulnerable to cyber attacks, (2) medical data can be illegally accessed by the doctor without patient’s authorization, and (3) unauthorized access from an attacker. Currently, common security authentication schemes generally use fingerprint or face for identification, but these biometric data are easily copied and reused. At present, widely used biometric authentications such as fingerprint and faceID are easy to be copied and reused. The latest GAN algorithm has broken into almost all face recognition systems, the success proportion of the attack reached 95%. Therefore, a secure login authentication scheme is needed. In this paper, we propose a physiological and behavioral feature authentication scheme based on fuzzy-rough theory to limit the access right of medical devices. Such a scheme requires the doctor’s own gesture for the authorization to access the medical device. Fuzzy-rough core vector machine (FRCVM) approach is adopted in our scheme to achieve high classification accuracy and efficiency. The results have shown that our solutions are highly secure and practical. To secure the cloud platform and ensure only the authorized doctors can access the patient data, we have designed an efficient data sharing solution that enables medical data stored in the cloud to be hierarchically authorized for patient access. The solution exploits proxy re-encryption to protect patient-centric medical data sharing.

A Network Intrusion Detection System Based On Ensemble CVM Using Efficient Feature Selection Approach

The growth and development of web services is drastic these days. Many things which we did manually once can be done through the Internet today. But with these types of services, the concern for security also increases. Many efficient Intrusion Detection systems proposed by researchers are in practice; but still hackers manage to attack the systems successfully. This paper proposes an efficient intrusion detection system using Ensemble Core Vector Machine (CVM) approach. CVMs are algorithms which work on the basis of Minimum Enclosing Ball concept. It detects the attacks like: U2R attack, R2L attack, Probe attack and DoS attack. For each type of attack, a CVM classifier is modeled. KDD Cup’99 dataset is used for training and testing the classifiers. This approach uses Chi-square test for selecting the relevant features for each attack and a weighted function is applied to these features for the dimensionality reduction. The test results verify that this model achieves high efficiency in all the four attacks with less computation time compared to the existing approaches.

Deep multiple multilayer kernel learning in core vector machines

Over the last few years, we have been witnessing a dramatic progress of deep learning in many real world applications. Deep learning concepts have been originated in the area of neural network and show a quantum leap in effective feature learning techniques such as auto-encoders, convolutional neural networks, recurrent neural networks etc. In the case of kernel machines, there are several attempts to model learning machines that mimic deep neural networks. In this direction, Multilayer Kernel Machines (MKMs) was an attempt to build a kernel machine architecture with multiple layers of feature extraction. It composed of many layers of kernel PCA based feature extraction units with support vector machine having arc-cosine kernel as the final classifier. The other approaches like Multiple Kernel Learning (MKL) and deep core vector machines solve the fixed kernel computation problem and scalability aspects of MKMs respectively. In addition to this, there are lot of avenues where the use of unsupervised MKL with both single and multilayer kernels in the multilayer feature extraction framework have to be evaluated. In this context, this paper attempts to build a scalable deep kernel machines with multiple layers of feature extraction. Each kernel PCA based feature extraction layer in this framework is modeled by the combination of both single and multilayer kernels in an unsupervised manner. Core vector machine with arc-cosine kernel is used as the final layer classifier which ensure the scalability in this model. The major contribution of this paper is a novel effort to build a deep structured kernel machine architecture similar to deep neural network architecture for classification. It opens up an extendable research avenue for researchers in deep learning based intelligent system leveraging the principles of kernel theory. Experiments show that the proposed method consistently improves the generalization performances of existing deep core vector machine.

Indefinite Core Vector Machine

The recently proposed Krĕin space Support Vector Machine (KSVM) is an efficient classifier for indefinite learning problems, but with quadratic to cubic complexity and a non-sparse decision function. In this paper a Krĕin space Core Vector Machine (iCVM) solver is derived. A sparse model with linear runtime complexity can be obtained under a low rank assumption. The obtained iCVM models can be applied to indefinite kernels without additional preprocessing. Using iCVM one can solve CVM with usually troublesome kernels having large negative eigenvalues or large numbers of negative eigenvalues. Experiments show that our algorithm is similar efficient as the Krĕin space Support Vector Machine but with substantially lower costs, such that also large scale problems can be processed.

An unbalanced spectra classification method based on entropy

How to solve the problem of distinguishing the minority spectra from the majority of the spectra is quite important in astronomy. In view of this, an unbalanced spectra classification method based on entropy (USCM) is proposed in this paper to deal with the unbalanced spectra classification problem. USCM greatly improves the performances of the traditional classifiers on distinguishing the minority spectra as it takes the data distribution into consideration in the process of classification. However, its time complexity is exponential with the training size, and therefore, it can only deal with the problem of small- and medium-scale classification. How to solve the large-scale classification problem is quite important to USCM. It can be easily obtained by mathematical computation that the dual form of USCM is equivalent to the minimum enclosing ball (MEB), and core vector machine (CVM) is introduced, USCM based on CVM is proposed to deal with the large-scale classification problem. Several comparative experiments on the 4 subclasses of K-type spectra, 3 subclasses of F-type spectra and 3 subclasses of G-type spectra from Sloan Digital Sky Survey (SDSS) verify USCM and USCM based on CVM perform better than $k$ NN ( $k$ nearest neighbor) and SVM (support vector machine) in dealing with the problem of rare spectra mining respectively on the small- and medium-scale datasets and the large-scale datasets.

Image coding algorithm based on Hadamard transform and simple vector quantization

Transform coding is commonly used in image processing algorithms to provide high compression ratios, often at the expense of processing time and simplicity of the system. We have recently proposed a pixel value prediction scheme in order to exploit adjacent pixel correlation, providing a low-complexity model for image coding. However, the proposed model was unable to reach high compression ratios retaining high quality of reconstructed image at the same time. In this paper we propose a new segmentation algorithm which further utilizes adjacent pixel correlation, provides higher compression ratios and it is based on application of Hadamard transform coding. Additional compression is provided by using vector quantization for a low number of quantization levels and by simplifying generalized Lloyd’s algorithm where the special attention is paid to determination of optimal partitions for vector quantization, making a fixed quantizer. The proposed method is quite simple and experimental results show that it ensures better or similar rate-distortion ratio for very low bit-rates, comparing to the other similar methods that are based on wavelet or curvelet transform coding and support or core vector machine application. Furthermore, the proposed method requires very low processing time since the proposed quantizers are fixed, much less than the required time for the aforementioned methods that we compare with as well as much less than the time required for fractal image coding. In the end, the appropriate discussion is provided comparing the results with a scheme based on linear prediction and dual-mode quantization.

Deep kernel learning in core vector machines

In machine learning literature, deep learning methods have been moving toward greater heights by giving due importance in both data representation and classification methods. The recently developed multilayered arc-cosine kernel leverages the possibilities of extending deep learning features into the kernel machines. Even though this kernel has been widely used in conjunction with support vector machines (SVM) on small-size datasets, it does not seem to be a feasible solution for the modern real-world applications that involve very large size datasets. There are lot of avenues where the scalability aspects of deep kernel machines in handling large dataset need to be evaluated. In machine learning literature, core vector machine (CVM) is being used as a scaling up mechanism for traditional SVMs. In CVM, the quadratic programming problem involved in SVM is reformulated as an equivalent minimum enclosing ball problem and then solved by using a subset of training sample (Core Set) obtained by a faster \((1+\epsilon )\) approximation algorithm. This paper explores the possibilities of using principles of core vector machines as a scaling up mechanism for deep support vector machine with arc-cosine kernel. Experiments on different datasets show that the proposed system gives high classification accuracy with reasonable training time compared to traditional core vector machines, deep support vector machines with arc-cosine kernel and deep convolutional neural network.

Power System Transient Stability Assessment Based on Big Data and the Core Vector Machine

In this paper, an online power system transient stability assessment (TSA) problem is mapped as a two-class classification problem and a novel data mining algorithm the core vector machine (CVM) is proposed to solve the problem based on phasor measurement units (PMUs) big data. First of all, an offline training, online application framework is proposed, which contained four sub-steps, namely features selection, offline training, online application, and assessment evaluation. First, 24 features are selected to present the system status. Then in the offline training procedure, the PMU big data is generated by time domain simulation, and a CVM model is trained and tested. In the online application procedure, an interface between PMU data center and feature calculation program is set up to collect real time specific PMU big data and the CVM trained is applied to the TSA problem. Last but not least, the evaluation indices are calculated. Compared with other support vector machines, the proposed CVM based assessment algorithm has the higher precision, meanwhile, it has the least time consumption and space complexity. As long as online PMU big data are received, TSA can be done simultaneously. Case studies on the IEEE New England 39-bus system, and real systems in China and the U.S., exhibit the speed and effectiveness of the proposed algorithm.

Scalable learning method for feedforward neural networks using minimal-enclosing-ball approximation

Training feedforward neural networks (FNNs) is one of the most critical issues in FNNs studies. However, most FNNs training methods cannot be directly applied for very large datasets because they have high computational and space complexity. In order to tackle this problem, the CCMEB (Center-Constrained Minimum Enclosing Ball) problem in hidden feature space of FNN is discussed and a novel learning algorithm called HFSR-GCVM (hidden-feature-space regression using generalized core vector machine) is developed accordingly. In HFSR-GCVM, a novel learning criterion using L2-norm penalty-based ε-insensitive function is formulated and the parameters in the hidden nodes are generated randomly independent of the training sets. Moreover, the learning of parameters in its output layer is proved equivalent to a special CCMEB problem in FNN hidden feature space. As most CCMEB approximation based machine learning algorithms, the proposed HFSR-GCVM training algorithm has the following merits: The maximal training time of the HFSR-GCVM training is linear with the size of training datasets and the maximal space consumption is independent of the size of training datasets. The experiments on regression tasks confirm the above conclusions.

Multi-view L2-SVM and its multi-view core vector machine

In this paper, a novel L2-SVM based classifier Multi-view L2-SVM is proposed to address multi-view classification tasks. The proposed Multi-view L2-SVM classifier does not have any bias in its objective function and hence has the flexibility like μ-SVC in the sense that the number of the yielded support vectors can be controlled by a pre-specified parameter. The proposed Multi-view L2-SVM classifier can make full use of the coherence and the difference of different views through imposing the consensus among multiple views to improve the overall classification performance. Besides, based on the generalized core vector machine GCVM, the proposed Multi-view L2-SVM classifier is extended into its GCVM version MvCVM which can realize its fast training on large scale multi-view datasets, with its asymptotic linear time complexity with the sample size and its space complexity independent of the sample size. Our experimental results demonstrated the effectiveness of the proposed Multi-view L2-SVM classifier for small scale multi-view datasets and the proposed MvCVM classifier for large scale multi-view datasets.

Multi-label Lagrangian support vector machine with random block coordinate descent method

When all training instances and labels are considered all together in a single optimization problem, multi-label support and core vector machines (i.e., Rank-SVM and Rank-CVM) are formulated as quadratic programming (QP) problems with equality and bounded constraints, whose training procedures have a sub-linear convergence rate. Therefore it is highly desirable to design and implement a novel efficient SVM-type multi-label algorithm. In this paper, through applying pairwise constraints between relevant and irrelevant labels, and defining an approximate ranking loss, we generalize binary Lagrangian support vector machine (LSVM) to construct its multi-label form (Rank-LSVM), resulting into a strictly convex QP problem with non-negative constraints only. Particularly, each training instance is associated with a block of variables and all variables are divided naturally into manageable blocks. Consequently we build an efficient training procedure for Rank-LSVM using random block coordinate descent method with a linear convergence rate. Moreover a heuristic strategy is applied to reduce the number of support vectors. Experimental results on twelve data sets demonstrate that our method works better according to five performance measures, and averagely runs 15 and 107 times faster and has 9 and 15% fewer support vectors, compared with Rank-CVM and Rank-SVM.