Cost-sensitive Support Vector Machine Research Articles

Cost-Sensitive Support Vector Machine Using Randomized Dual Coordinate Descent Method for Big Class-Imbalanced Data Classification

Cost-sensitive support vector machine is one of the most popular tools to deal with class-imbalanced problem such as fault diagnosis. However, such data appear with a huge number of examples as well as features. Aiming at class-imbalanced problem on big data, a cost-sensitive support vector machine using randomized dual coordinate descent method (CSVM-RDCD) is proposed in this paper. The solution of concerned subproblem at each iteration is derived in closed form and the computational cost is decreased through the accelerating strategy and cheap computation. The four constrained conditions of CSVM-RDCD are derived. Experimental results illustrate that the proposed method increases recognition rates of positive class and reduces average misclassification costs on real big class-imbalanced data.

Early Detection of Human Epileptic Seizures Based on Intracortical Local Field Potentials.

The unpredictability of re-occurring seizures dramatically impacts the quality of life and autonomy of people with epilepsy. Reliable early seizure detection could open new therapeutic possibilities and thus substantially improve quality of life and autonomy. Though many seizure detection studies have shown the potential of scalp electroencephalogram (EEG) and intracranial EEG (iEEG) signals, reliable early detection of human seizures remains elusive in practice. Here, we examined the use of intracortical local field potentials (LFPs) recorded from 4×4-mm2 96-microelectrode arrays (MEA) for early detection of human epileptic seizures. We adopted a framework consisting of (1) sampling of intracortical LFPs; (2) denoising of LFPs with the Kalman filter; (3) spectral power estimation in specific frequency bands using 1-sec moving time windows; (4) extraction of statistical features, such as the mean, variance, and Fano factor (calculated across channels) of the power in each frequency band; and (5) cost-sensitive support vector machine (SVM) classification of ictal and interictal samples. We tested the framework in one-participant dataset, including 4 seizures and corresponding interictal recordings preceding each seizure. The participant was a 52-year-old woman suffering from complex partial seizures. LFPs were recorded from an MEA implanted in the participant's left middle temporal gyrus. In this participant, spectral power in 0.3-10 Hz, 20-55 Hz, and 125-250 Hz changed significantly between ictal and interictal epochs. The examined seizure detection framework provided an event-wise sensitivity of 100% (4/4) and only one 20-sec-long false positive event in interictal recordings (likely an undetected subclinical event under further visual inspection), and a detection latency of 4.35 ± 2.21 sec (mean ± std) with respect to iEEG-identified seizure onsets. These preliminary results indicate that intracortical MEA recordings may provide key signals to quickly and reliably detect human seizures.

Finding falls in ambulatory care clinical documents using statistical text mining

To determine how well statistical text mining (STM) models can identify falls within clinical text associated with an ambulatory encounter. 2241 patients were selected with a fall-related ICD-9-CM E-code or matched injury diagnosis code while being treated as an outpatient at one of four sites within the Veterans Health Administration. All clinical documents within a 48-h window of the recorded E-code or injury diagnosis code for each patient were obtained (n=26 010; 611 distinct document titles) and annotated for falls. Logistic regression, support vector machine, and cost-sensitive support vector machine (SVM-cost) models were trained on a stratified sample of 70% of documents from one location (dataset Atrain) and then applied to the remaining unseen documents (datasets Atest-D). All three STM models obtained area under the receiver operating characteristic curve (AUC) scores above 0.950 on the four test datasets (Atest-D). The SVM-cost model obtained the highest AUC scores, ranging from 0.953 to 0.978. The SVM-cost model also achieved F-measure values ranging from 0.745 to 0.853, sensitivity from 0.890 to 0.931, and specificity from 0.877 to 0.944. The STM models performed well across a large heterogeneous collection of document titles. In addition, the models also generalized across other sites, including a traditionally bilingual site that had distinctly different grammatical patterns. The results of this study suggest STM-based models have the potential to improve surveillance of falls. Furthermore, the encouraging evidence shown here that STM is a robust technique for mining clinical documents bodes well for other surveillance-related topics.

The Research on Volleyball Trajectory Simulation Based on Cost-Sensitive Support Vector Machine

Aim of study of the study is to effectively solve the imbalanced data sets classification problem, the field of machine learning proposed many effective algorithms. On Seabrook’s view, now the classification methods for class imbalance problem can be broadly divided into two categories, one class is to create new methods or improve the existing methods based on the characteristics of class imbalance. Another is to reduce class imbalance effect re- use of existing methods by resembling technique. But there are also some drawbacks in resembling method. Sample set sampling may lead to excessive learning; the next sample may result in the training set. Therefore, by dividing the training set, it does not increase the number of training samples; also it is without loss of useful information in the sample to obtain a certain degree of balance in a subset of the problem.

Introduction to the special issue on information technologies in logistics and service science

Information technology has been playing an increasingly important role in every walk of life since the third industrial revolution. In order to realize more efficient and smarter manufacturing, working and living spaces, enabling information technologies such as cloud computing and big data are emerging themes among global research scholars. Logistics is a typical sector that is regularly enhanced by every revolution of information technology. On the one hand, information technology makes more kinds of information available in supply chains, many of which used not to be available. Also, technology accelerates the information sharing across supply chains, and among supply chain partners. For instance, radio frequency identification devices (RFID) and wireless sensor networks (WSN) as well as other enabling technologies bring the supply chain visibility, realizing ‘‘real-time logistics’’ by sharing more immediate information about products and goods. To achieve satisfied service users in cyber space or daily digital life, proper use of information technology is of tantamount importance as the availability of data grows exponentially. Many research projects on data mining, big data analysis and cloud computing, or related technologies, could effectively facilitate qualified service by catching and providing valuable information to individuals from an ocean of data. This special issue includes 4 articles on different aspects of information technologies adaptable in logistics or service science. The first paper, ‘‘A resource integration approach for HTML5 mobile applications’’ by Zhu et al., focuses on an Internet information integration method. The authors propose an integration approach for mobile web applications, which can assist application service providers to provide HTML5-based mobile hybrid applications for various types of mobile ends. And correspondent prototype platform that constructs more flexible mechanism of integration is developed in the article. The paper entitled ‘‘An Inter-PAN Mobility Support Scheme for IP-based Wireless Sensor Networks and its Applications’’ by Shang et al. presents a novel mobile management scheme used in internet protocol (IP) based WSN. This scheme is able to support many exciting applications such as healthcare system, logistics and storage, as demonstrated by the authors. In order to deal with the problem of unbalanced data classification and asymmetry misclassification cost in loan default discrimination problem, Cao et al. in their paper ‘‘A loan default discrimination model using cost-sensitive support vector machine improved by particle swarm optimization (PSO)’’ propose a novel PSO–CS–SVM model that hybridizes the PSO and cost sensitive support vector machine (CS– SVM). This is a research on data classification technology, available to underpin high quality finical service. In the last paper ‘‘A new evolution model for B2C e-commerce market’’, Tian et al. study B2C e-commerce market from the view of network. A new proposed model reveals a ‘‘bubble-stable’’ evolutionary process, which is correspondent with real e-commerce market from an initial H. Liu (&) School of Traffic and Transportation, Beijing Jiaotong University, Beijing 100044, China e-mail: hlliu@bjtu.edu.cn

A loan default discrimination model using cost-sensitive support vector machine improved by PSO

This study proposes a novel PSO---CS-SVM model that hybridizes the particle swarm optimization (PSO) and cost sensitive support vector machine (CS-SVM) to deal with the problem of unbalanced data classification and asymmetry misclassification cost in loan default discrimination problem. Cost sensitive learning is applied to the standard SVM by integrating misclassification cost of each sample into standard SVM and PSO is employed for parameter determination of the CS-SVM. Meantime, the financial data are discretized by using the self-organizing mapping neural network. And the evaluation indices are reduced without information loss by genetic algorithm for decreasing the complexity of the model. The effectiveness of integrated model of CS-SVM and PSO is verified by three experiments comparing with traditional CS-SVM, PSO---SVM, SVM and BP neural network through real loan default data of companies in China. The corresponding results indicate that the accuracy rate, hit rate, covering rate and lift coefficient are improved dramatically by the developed approach. The proposed method can control the different types of errors distribution with various cost of misclassification accurately, reduce the total misclassification cost largely, and distinguish the loan default problems effectively.

Semi-Supervised Novelty Detection Using SVM Entire Solution Path

Very often, the only reliable information available to perform change detection is the description of some “unchanged” regions. Since, sometimes, these regions do not contain all the relevant information to identify their counterpart (the changes), we consider the use of unlabeled data to perform semi-supervised novelty detection (SSND). SSND can be seen as an unbalanced classification problem solved using the cost-sensitive support vector machine (CS-SVM), but this requires a heavy parameter search. Here, we propose the use of entire solution path algorithms for the CS-SVM in order to facilitate and accelerate parameter selection for SSND. Two algorithms are considered and evaluated. The first algorithm is an extension of the CS-SVM algorithm that returns the entire solution path in a single optimization. This way, optimization of a separate model for each hyperparameter set is avoided. The second algorithm forces the solution to be coherent through the solution path, thus producing classification boundaries that are nested (included in each other). We also present a low-density (LD) criterion for selecting optimal classification boundaries, thus avoiding recourse to cross validation (CV) that usually requires information about the “change” class. Experiments are performed on two multitemporal change detection data sets (flood and fire detection). Both algorithms tracing the solution path provide similar performances than the standard CS-SVM while being significantly faster. The proposed LD criterion achieves results that are close to the ones obtained by CV but without using information about the changes.

Detection of Pulmonary Nodules in CT Images Based on Fuzzy Integrated Active Contour Model and Hybrid Parametric Mixture Model

The segmentation and detection of various types of nodules in a Computer-aided detection (CAD) system present various challenges, especially when (1) the nodule is connected to a vessel and they have very similar intensities; (2) the nodule with ground-glass opacity (GGO) characteristic possesses typical weak edges and intensity inhomogeneity, and hence it is difficult to define the boundaries. Traditional segmentation methods may cause problems of boundary leakage and “weak” local minima. This paper deals with the above mentioned problems. An improved detection method which combines a fuzzy integrated active contour model (FIACM)-based segmentation method, a segmentation refinement method based on Parametric Mixture Model (PMM) of juxta-vascular nodules, and a knowledge-based C-SVM (Cost-sensitive Support Vector Machines) classifier, is proposed for detecting various types of pulmonary nodules in computerized tomography (CT) images. Our approach has several novel aspects: (1) In the proposed FIACM model, edge and local region information is incorporated. The fuzzy energy is used as the motivation power for the evolution of the active contour. (2) A hybrid PMM Model of juxta-vascular nodules combining appearance and geometric information is constructed for segmentation refinement of juxta-vascular nodules. Experimental results of detection for pulmonary nodules show desirable performances of the proposed method.

A Cost-Sensitive Ensemble Method for Class-Imbalanced Datasets

In imbalanced learning methods, resampling methods modify an imbalanced dataset to form a balanced dataset. Balanced data sets perform better than imbalanced datasets for many base classifiers. This paper proposes a cost-sensitive ensemble method based on cost-sensitive support vector machine (SVM), and query-by-committee (QBC) to solve imbalanced data classification. The proposed method first divides the majority-class dataset into several subdatasets according to the proportion of imbalanced samples and trains subclassifiers using AdaBoost method. Then, the proposed method generates candidate training samples by QBC active learning method and uses cost-sensitive SVM to learn the training samples. By using 5 class-imbalanced datasets, experimental results show that the proposed method has higher area under ROC curve (AUC), F-measure, and G-mean than many existing class-imbalanced learning methods.

Cost-Sensitive Support Vector Machine for Semi-Supervised Learning

Cost-sensitive learning has been a hot research topic in machine learning. Many cost-sensitive methods have been successfully applied in many real-world applications such as disease diagnosis, fraud detection and business decision making. In this paper, we proposed a new Cost-Sensitive Laplacian Support Vector Machine(called Cos-LapSVM), which can deal with the cost- sensitive problem in Semi-Supervised Learning. The effectiveness of the proposed method is demonstrated via experiments on UCI datasets

A maximum-margin genetic algorithm for misclassification cost minimizing feature selection problem

We consider a feature selection problem where the decision-making objective is to minimize overall misclassification cost by selecting relevant features from a training dataset. We propose a two-stage solution approach for solving misclassification cost minimizing feature selection (MCMFS) problem. Additionally, we propose a maximum-margin genetic algorithm (MMGA) that maximizes margin of separation between classes by taking into account all examples as opposed to maximizing margin of separation using a few support vectors. Feature selection is carried out by either an exhaustive or a heuristic simulated annealing approach in the first stage and a cost sensitive classification using either MMGA or cost sensitive support vector machines (SVM) in the second stage. Using simulated and real-world data sets and different misclassification cost matrices, we test our two-stage approach for solving the MCMFS problem. Our results indicate that feature selection plays an important role when misclassification cost asymmetries increase and the MMGA shows equal or better performance than the SVM.

Cost Sensitive Semi-Supervised Laplacian Support Vector Machine

Cost sensitive learning is the hot research area in machine learning.In practical real applications,the datasets are usually class-imbalanced,most of the samples are unlabeled,only a few of the samples are labeled,and noise samples are existed.Although some progress has been made in cost sensitive learning for such situation,it needs further solved.For that we propose a semi-supervised Laplacian support vector machine based on cost sensitive learning.On the basis of label propagation,the proposed model integrates the misclassification costs considering class-imbalance into the hinge loss and Laplacian regularization of the Laplacian support vector machine.At the same time,considering the effect on the decision hypersphere of the noise samples,we define an example-dependent cost which makes the weights of noise samples lower.The experimental results on 7 UCI,8 NASA datasets demonstrate the superiority of our proposed algorithm.

Intelligent Recognition of Lung Nodule Combining Rule-based and C-SVM Classifiers

Abstract Computer-aided detection(CAD) system for lung nodules plays the important role in the diagnosis of lung cancer. In this paper, an improved intelligent recognition method of lung nodule in HRCT combing rule-based and cost-sensitive support vector machine(C-SVM) classifiers is proposed for detecting both solid nodules and ground-glass opacity(GGO) nodules(part solid and nonsolid). This method consists of several steps. Firstly, segmentation of regions of interest(ROIs), including pulmonary parenchyma and lung nodule candidates, is a difficult task. On one side, the presence of noise lowers the visibility of low-contrast objects. On the other side, different types of nodules, including small nodules, nodules connecting to vasculature or other structures, part-solid or nonsolid nodules, are complex, noisy, weak edge or difficult to define the boundary. In order to overcome the difficulties of obvious boundary-leak and slow evolvement speed problem in segmentatioin of weak edge, an overall segmentatio...

Seizure prediction with spectral power of EEG using cost-sensitive support vector machines

We propose a patient-specific algorithm for seizure prediction using multiple features of spectral power from electroencephalogram (EEG) and support vector machine (SVM) classification. The proposed patient-specific algorithm consists of preprocessing, feature extraction, SVM classification, and postprocessing. Preprocessing removes artifacts of intracranial EEG recordings and they are further preprocessed in bipolar and/or time-differential methods. Features of spectral power of raw, or bipolar and/or time-differential intracranial EEG (iEEG) recordings in nine bands are extracted from a sliding 20-s-long and half-overlapped window. Nine bands are selected based on standard EEG frequency bands, but the wide gamma bands are split into four. Cost-sensitive SVMs are used for classification of preictal and interictal samples, and double cross-validation is used to achieve in-sample optimization and out-of-sample testing. We postprocess SVM classification outputs using the Kalman Filter and it removes sporadic and isolated false alarms. The algorithm has been tested on iEEG of 18 patients of 20 available in the Freiburg EEG database who had three or more seizure events. To investigate the discriminability of the features between preictal and interictal, we use the Kernel Fisher Discriminant analysis. The proposed patient-specific algorithm for seizure prediction has achieved high sensitivity of 97.5% with total 80 seizure events and a low false alarm rate of 0.27 per hour and total false prediction times of 13.0% over a total of 433.2 interictal hours by bipolar preprocessing (92.5% sensitivity, a false positive rate of 0.20 per hour, and false prediction times of 9.5% by time-differential preprocessing). This high prediction rate demonstrates that seizures can be predicted by the patient-specific approach using linear features of spectral power and nonlinear classifiers. Bipolar and/or time-differential preprocessing significantly improves sensitivity and specificity. Spectral powers in high gamma bands are the most discriminating features between preictal and interictal. High sensitivity and specificity are achieved by nonlinear classification of linear features of spectral power. Power changes in certain frequency bands already demonstrated their possibilities for seizure prediction indicators, but we have demonstrated that combining those spectral power features and classifying them in a multivariate approach led to much higher prediction rates. Employing only linear features is advantageous, especially when it comes to an implantable device, because they can be computed rapidly with low power consumption.

Cost-sensitive SVM with Error Cost and Class-dependent Reject Cost

In such real data mining applications as medical diagnosis, fraud detection and fault classification, and so on, the two problems that the error cost is expensive and the reject cost is class-dependent are often encountered. In order to overcome those problems, firstly, the general mathematical description of the Binary Classification Problem with Error Cost and Class-dependent Reject Cost (BCP-EC2RC) is proposed. Secondly, as one of implementation methods of BCP-EC2RC, the new algorithm, named as Cost-sensitive Support Vector Machines with the Error Cost and the Class-dependent Reject Cost (CSVM-EC2RC), is presented. The CSVM-EC2RC algorithm involves two stages: estimating the classification reliability based on trained SVM classifier, and determining the optimal reject rate of positive class and negative class by minimizing the average cost based on the given error cost and class-dependent reject cost. The experiment studies based on a benchmark data set illustrate that the proposed algorithm is effective.

Tuning Support Vector Machines for Minimax and Neyman-Pearson Classification

This paper studies the training of support vector machine (SVM) classifiers with respect to the minimax and Neyman-Pearson criteria. In principle, these criteria can be optimized in a straightforward way using a cost-sensitive SVM. In practice, however, because these criteria require especially accurate error estimation, standard techniques for tuning SVM parameters, such as cross-validation, can lead to poor classifier performance. To address this issue, we first prove that the usual cost-sensitive SVM, here called the 2C-SVM, is equivalent to another formulation called the 2nu-SVM. We then exploit a characterization of the 2nu-SVM parameter space to develop a simple yet powerful approach to error estimation based on smoothing. In an extensive experimental study, we demonstrate that smoothing significantly improves the accuracy of cross-validation error estimates, leading to dramatic performance gains. Furthermore, we propose coordinate descent strategies that offer significant gains in computational efficiency, with little to no loss in performance.

Prostate Cancer Localization With Multispectral MRI Using Cost-Sensitive Support Vector Machines and Conditional Random Fields

Prostate cancer is a leading cause of cancer death for men in the United States. Fortunately, the survival rate for early diagnosed patients is relatively high. Therefore, in vivo imaging plays an important role for the detection and treatment of the disease. Accurate prostate cancer localization with noninvasive imaging can be used to guide biopsy, radiotherapy, and surgery as well as to monitor disease progression. Magnetic resonance imaging (MRI) performed with an endorectal coil provides higher prostate cancer localization accuracy, when compared to transrectal ultrasound (TRUS). However, in general, a single type of MRI is not sufficient for reliable tumor localization. As an alternative, multispectral MRI, i.e., the use of multiple MRI-derived datasets, has emerged as a promising noninvasive imaging technique for the localization of prostate cancer; however almost all studies are with human readers. There is a significant inter and intraobserver variability for human readers, and it is substantially difficult for humans to analyze the large dataset of multispectral MRI. To solve these problems, this study presents an automated localization method using cost-sensitive support vector machines (SVMs) and shows that this method results in improved localization accuracy than classical SVM. Additionally, we develop a new segmentation method by combining conditional random fields (CRF) with a cost-sensitive framework and show that our method further improves cost-sensitive SVM results by incorporating spatial information. We test SVM, cost-sensitive SVM, and the proposed cost-sensitive CRF on multispectral MRI datasets acquired from 21 biopsy-confirmed cancer patients. Our results show that multispectral MRI helps to increase the accuracy of prostate cancer localization when compared to single MR images; and that using advanced methods such as cost-sensitive SVM as well as the proposed cost-sensitive CRF can boost the performance significantly when compared to SVM.

Seizure Prediction With Spectral Power of EEG Using Cost-Sensitive Support Vector Machines

A patient-specific seizure prediction algorithm is proposed using a classifier to differentiate pre-ictal from inter-ictal EEG signals. The spectral power of EEG processed in four different fashions is used as features: raw, time-differential, space-differential, and time/space-differential EEG. The features are classified using cost-sensitive support vector machines by the double cross-validation methodology. The proposed algorithm has been applied to EEG recordings of 18 patients in the Freiburg EEG database, totaling 80 seizures and 437 h long inter-ictal recordings. Classification with the feature obtained from time/space-differential ECoG demonstrates the performance of 86.25% sensitivity and 0.1281 false positives per hour in out-of-sample testing.

Nested Support Vector Machines

One-class and cost-sensitive support vector machines (SVMs) are state-of-the-art machine learning methods for estimating density level sets and solving weighted classification problems, respectively. However, the solutions of these SVMs do not necessarily produce set estimates that are nested as the parameters controlling the density level or cost-asymmetry are continuously varied. Such nesting not only reflects the true sets being estimated, but is also desirable for applications requiring the simultaneous estimation of multiple sets, including clustering, anomaly detection, and ranking. We propose new quadratic programs whose solutions give rise to nested versions of one-class and cost-sensitive SVMs. Furthermore, like conventional SVMs, the solution paths in our construction are piecewise linear in the control parameters, although here the number of breakpoints is directly controlled by the user. We also describe decomposition algorithms to solve the quadratic programs. These methods are compared to conventional (non-nested) SVMs on synthetic and benchmark data sets, and are shown to exhibit more stable rankings and decreased sensitivity to parameter settings.

On the role of cost-sensitive learning in multi-class brain-computer interfaces

Brain-computer interfaces (BCIs) present an alternative way of communication for people with severe disabilities. One of the shortcomings in current BCI systems, recently put forward in the fourth BCI competition, is the asynchronous detection of motor imagery versus resting state. We investigated this extension to the three-class case, in which the resting state is considered virtually lying between two motor classes, resulting in a large penalty when one motor task is misclassified into the other motor class. We particularly focus on the behavior of different machine-learning techniques and on the role of multi-class cost-sensitive learning in such a context. To this end, four different kernel methods are empirically compared, namely pairwise multi-class support vector machines (SVMs), two cost-sensitive multi-class SVMs and kernel-based ordinal regression. The experimental results illustrate that ordinal regression performs better than the other three approaches when a cost-sensitive performance measure such as the mean-squared error is considered. By contrast, multi-class cost-sensitive learning enables us to control the number of large errors made between two motor tasks.