Cost-sensitive Support Vector Machine Research Articles

Credit risk assessment method driven by asymmetric loss function

Credit risk assessment is significantly hindered by the problem of class imbalance, and cost-sensitive methods represent an effective strategy to address this issue. However, most algorithms tend to approach the imbalance from a class perspective, overlooking the finer details at the sample level. Moreover, such methods are susceptible to interference from noise and outliers. In response to these challenges, this paper proposes an asymmetric cost-sensitive support vector machine (QTSVM) that combines the quadratic type squared error loss function (QTSELF) with support vector machine (SVM). It not only leverages the asymmetry of the loss function by applying varying penalties based on classification errors but also employs different processing measures for samples from different classes. Additionally, it enhances model robustness by imposing a tiny penalty on noise or outliers. The adaptive moment estimation (Adam) algorithm is employed to optimize QTSVM. Extensive experiments and statistical tests profoundly demonstrate the superior performance of QTSVM.

A multi-strategy ontology mapping method based on cost-sensitive SVM

As the core of ontology integration, the task of ontology mapping is to find the semantic relationship between ontologies. Nevertheless, most existing ontology mapping methods only rely ontext information to calculate entity similarity, thereby disregarding semantic nuances and necessitating substantial manual intervention. Therefore, this paper introduces an ontology mapping method. Based on the traditional ontology mapping method, the process employs a deep learning model to mine the semantic information of entity concepts, entity properties and ontology structure to obtain the embedding vector. We use the similarity mechanism to calculate the similarity between different embedding vectors, and combine the similarity values obtained from multiple strategy entities into a similarity matrix. The similarity matrix serves as input to the support vector machine (SVM), and the ontology mapping problem is finally transformed into a binary classification problem. However, since the number of mapped pairs is much larger than the number of non-mapped pairs, the number of positive samples in the data set is much smaller than the number of negative samples. Therefore, based on the traditional SVM, the paper adopts cost-sensitive strategy to deal with the class imbalance problem. In comparative evaluations against contemporary ontology mapping techniques, our method demonstrates a noteworthy 5.0% enhancement in recall and a 3.0% improvement in F1 score when tested on both public benchmark datasets and domain-specific datasets.

MVQS: Robust multi-view instance-level cost-sensitive learning method for imbalanced data classification

Multi-view imbalanced learning is to handle the datasets with multi-view representations and imbalanced classes. Existing multi-view imbalanced learning methods can be divided into two main categories: multi-view ensemble learning and multi-view cost-sensitive learning. However, these methods suffer from the following problems: 1) neglecting either consensus or complementary information, 2) complex preprocessing and information fusion in multi-view ensemble learning and manual assignment of misclassification costs in multi-view cost-sensitive learning, and 3) limited ability to handle noisy samples. Therefore, we aim to design a concise and unified framework to grapple with the multi-view representations, imbalanced classes and noisy samples simultaneously. Inspired by the merits of support vector machine (SVM) and quadratic type squared error (QTSE) loss function, we propose a robust multi-view instance-level cost-sensitive SVM with QTSE loss (MVQS) for imbalanced data classification. The consensus regularization term and combination weight strategy are employed to fully exploit multi-view information. The QTSE loss can adaptively impose instance-level penalties to the misclassification of samples, and make MVQS be robust to noisy samples. We solve MVQS with the alternating direction method of multipliers (ADMM) and the gradient descent (GD) algorithm. Comprehensive experiments validate that MVQS is more competitive and robust than other benchmark approaches.

Identification of market power abuse in Chinese electricity market based on an improved cost-sensitive support vector machine

Accurate and real-time identification of market power abuse is a key task in the management of electricity market violations. However, there are few effective monitoring methods for extremely imbalanced datasets and progressively increasing amounts of data in actual market transactions. To address the aforementioned problems, this paper proposes an improved support vector machine by considering the index system, which can not only realize identification automatically but also minimize the credit risk of power market transactions. Firstly, the dataset is composed of an indicator system for measuring market power abuses. Secondly, a comprehensive algorithm for identifying offending data is proposed, which combines the K-Nearest Bound Neighbor and the distance between the means of two classes methods to overcome the shortcomings of traditional support vector machines with long training time due to the high dimensionality and progressively increasing amounts of data in actual market transactions, and the Cost-sensitive Support Vector Machine to tackle the problem of inefficient identification due to few tags in transaction data. Finally, five different features of constructed datasets and a power market synthetic dataset are tested, and results indicate that the proposed method can ensure high classification accuracy while significantly improving recognition speed and recall for violation data, which is more suitable for Chinese electricity market data and provide a dynamic detection method to identify market power abuse precisely and quickly.

Three novel cost-sensitive machine learning models for urban growth modelling

This article addresses the class imbalance problem in urban gain modelling (UGM) of Tabriz and Isfahan megacities in Iran by proposing novel cost-sensitive machine learning models, namely cost-sensitive support vector machine (CSVM), random forest (CRF) and artificial neural network (CANN). Random sampling, a frequently utilized method, fails to effectively tackle this issue by biasing models towards no change samples, which outnumber change samples. The results showed that CRF exhibited the highest accuracy (AUC = 0.560), followed by CANN (AUC = 0.557) and CSVM (AUC = 0.448) in Isfahan. In Tabriz, CRF (AUC = 0.809) and CANN (AUC = 0.818) excelled, outperforming balanced sampling models constructed with ANN, RF and SVM with the AUROC of ANN and RF boosted by 15% and 2% in validation. By emphasizing the significance of addressing class imbalance appropriately, this research highlights the improvement in modelling outcomes achievable through the cost-sensitive models especially in Tabriz case.

An Imbalanced Data Classification Method Based on Hybrid Resampling and Fine Cost Sensitive Support Vector Machine

An Imbalanced Data Classification Method Based on Hybrid Resampling and Fine Cost Sensitive Support Vector Machine

IoT-Enabled Fault Prediction and Maintenance for Smart Charging Piles

With the application of the Internet of Things (IoT), smart charging piles, which are important facilities for new energy electric vehicles (NEVs), have become an important part of the smart grid. Since the smart charging piles are generally deployed in complex environments and prone to failure, it’s significant to perform efficient fault diagnosis and timely maintenance for them. One of the key problems to be solved is how to conduct fault prediction based on limited data collected through IoT in the early stage and develop reasonable preventive maintenance strategies. In this paper, a real-time fault prediction method combining costsensitive logistic regression (CS-LR) and cost-sensitive support vector machine classification (CS-SVM) is proposed. CS-LR is firstly used to classify the fault data of smart charging piles, then the CS-SVM is adopted to predict the faults based on the classified data. The feasibility of the proposed model is illustrated through the case study on fault prediction of real-world smart charging piles. To demonstrate the advantage in prediction accuracy, the proposed fault prediction model is compared with the classic baseline models such as LR, SVM, decision tree (DT), K-nearest neighbor (KNN), and back propagation neural network (BPNN). Finally, based on the proposed fault prediction method, preventive maintenance based on a probability threshold with the minimum total expected cost is proposed. Simulation results show that the proposed maintenance strategy has a better performance in reducing the total maintenance cost compared with traditional periodic maintenance. This is valuable for the development of preventive maintenance strategies for repairable systems under early real-time monitoring data.

Consumer Purchasing Power Prediction of Interest E-Commerce Based on Cost-Sensitive Support Vector Machine

The traditional e-commerce business chain is being reconstructed around the content of short videos and live streams, and the interest e-commerce is thriving as a new trend in the e-commerce industry. Diversified content promotes the rapid development of interest e-commerce. For consumers, their preferences for different content reflect their consumption level to a certain extent. The purpose of this study is to accurately predict the purchasing power level with the consumer content preference, and provide new ideas for interest e-commerce business. In this paper, the new swarm intelligence algorithm is used to find the optimal misclassification cost, and three cost-sensitive models are established. On this basis, the content preference of interest e-commerce consumers is used to predict the level of purchasing power. The results show that the content preference of interest e-commerce consumers, such as “fashion”, “photography” and “interpretation”, have a significant effect on the prediction of purchasing power at the 95% confidence level. The accuracies of the optimized cost-sensitive support vector machine in predicting consumer purchasing power are all above 0.9, and the highest is 0.9792. This study effectively alleviates the problem that the classification results tend to be biased towards negative samples, especially when the imbalanced rate of the sample is high. It not only provides researchers with an efficient parameter optimization method, but also reflects the relationship between consumer content preference and purchasing power, providing data support for interest e-commerce operations.

Handling Imbalanced Classification Problems With Support Vector Machines via Evolutionary Bilevel Optimization.

Support vector machines (SVMs) are popular learning algorithms to deal with binary classification problems. They traditionally assume equal misclassification costs for each class; however, real-world problems may have an uneven class distribution. This article introduces EBCS-SVM: evolutionary bilevel cost-sensitive SVMs. EBCS-SVM handles imbalanced classification problems by simultaneously learning the support vectors and optimizing the SVM hyperparameters, which comprise the kernel parameter and misclassification costs. The resulting optimization problem is a bilevel problem, where the lower level determines the support vectors and the upper level the hyperparameters. This optimization problem is solved using an evolutionary algorithm (EA) at the upper level and sequential minimal optimization (SMO) at the lower level. These two methods work in a nested fashion, that is, the optimal support vectors help guide the search of the hyperparameters, and the lower level is initialized based on previous successful solutions. The proposed method is assessed using 70 datasets of imbalanced classification and compared with several state-of-the-art methods. The experimental results, supported by a Bayesian test, provided evidence of the effectiveness of EBCS-SVM when working with highly imbalanced datasets.

CANCER PREDICTION IN INFLAMMATORY BOWEL DISEASE PATIENTS BY USING MACHINE LEARNING ALGORITHMS

Colon cancer is one of the most common spread cancers in the world, which leads to total death of 10%. Prediction of onset of cancer, and the cause of its development in these patients can be of an enormous help and relief to those affected, as they can get back their “normal” life. Data mining and machine learning are important intelligent tools for classification, prediction and hidden relation extraction between patient information. We collected data from Shahid Faghihi Hospital in Shiraz. Features collected are as follows: Gender, age, duration of cancer before surgery, number of times the patients used bathroom, taking anti-inflammatory drug prednisolone, duration of drug use and dosage, kind of surgery and number of times consulted and retreatment of surgery, incontinence, etc. After pre-processing and data cleaning stages, effective features were extracted, and also occurrence of cancer predicts by using different classification algorithms. Then association rule mining algorithms like Apriori were used for obtaining any internal hidden relation between entries. Approaching them with different algorithms and assessing them with support vector machine was with highest prediction accuracy (84%). Due to unbalanced dataset, we chose cost sensitive support vector machine. In another aspect, after applying Apriori algorithm, the conditions of non-inflammation were extracted based on dataset features. Some significant outcomes are in what follows. If surgery treatment or diagnosed was less than 5 years, the possibility of developing colon cancer is lower. Also, as the duration of disease increases, the possibility of reoperation increases, as confirmed by the interiors. Since this issue with these features was raised for the first time in this paper at the suggestion of internists, early detection of cancer and also the extraction of effective laws can be of help to the medical community. In future, to get higher accuracy, the improvement of the dataset in terms of number of samples and colonoscopy image features is considered.

An Integrated System of Multifaceted Machine Learning Models to Predict If and When Hospital-Acquired Pressure Injuries (Bedsores) Occur

Hospital-Acquired Pressure Injury (HAPI), known as bedsore or decubitus ulcer, is one of the most common health conditions in the United States. Machine learning has been used to predict HAPI. This is insufficient information for the clinical team because knowing who would develop HAPI in the future does not help differentiate the severity of those predicted cases. This research develops an integrated system of multifaceted machine learning models to predict if and when HAPI occurs. Phase 1 integrates Genetic Algorithm with Cost-Sensitive Support Vector Machine (GA-CS-SVM) to handle the high imbalance HAPI dataset to predict if patients will develop HAPI. Phase 2 adopts Grid Search with SVM (GS-SVM) to predict when HAPI will occur for at-risk patients. This helps to prioritize who is at the highest risk and when that risk will be highest. The performance of the developed models is compared with state-of-the-art models in the literature. GA-CS-SVM achieved the best Area Under the Curve (AUC) (75.79 ± 0.58) and G-mean (75.73 ± 0.59), while GS-SVM achieved the best AUC (75.06) and G-mean (75.06). The research outcomes will help prioritize at-risk patients, allocate targeted resources and aid with better medical staff planning to provide intervention to those patients.

An evolutionary cost‐sensitive support vector machine for carbon price trend forecasting

AbstractThis paper aims at the imbalanced characteristics and proposes a novel evolutionary cost‐sensitive support vector machine (CSSVM) by integrating cost‐sensitive learning, support vector machine, and genetic algorithm for carbon price trend prediction. First, carbon price trend prediction is converted into a binary‐class prediction problem for CSSVM, in which a higher misclassification cost is imposed on the minority samples. In comparison, a more negligible misclassification cost is imposed on most samples. Second, a genetic algorithm (GA) is used to optimize all parameters of CSSVM synchronously. Taking Beijing, Hubei, and Guangdong carbon markets as samples, the empirical results show that the proposed model has a higher classification accuracy and lower misclassification costs compared with other popular prediction models. Furthermore, the sensitivity analysis verifies that the proposed approach is robust.

Financial Early Warning Model for Listed Companies Based on the Smart Sensor Data Network

This paper uses intelligent sensors to build a data network, collects information about listed enterprises in all aspects, performs frequency statistics and semantic analysis based on the financial domain lexicon on information related to listed enterprises, and introduces big data variables into a nonlinear support vector machine early warning model of enterprises combined with financial indicators. This paper introduces online reviews as big data indicators based on financial early warning theories and methods. Suitable financial indicators and big data indicators are selected for the financial early warning model to filter the available indicators. The prediction results of only financial indicators are compared with the prediction results of incorporating big data indicators. Quantify the comment information related to enterprises on the platform through sentiment classification and statistics on the number of comment information posted. The cost-sensitive support vector machine is used as the base classifier of the improved AdaBoost algorithm to build a dynamic imbalance warning model. To address the problems of large complexity and computation of unbalanced big data classification, high reliance on a priori knowledge, and classification performance to be improved, the classification and detection method of the intelligent sensor data network is proposed. By introducing migration learning, the problem of knowledge acquisition and training efficiency of high-dimensional complex data feature extraction in a big data environment is effectively solved, and the network performance is optimized by a conjugate gradient descent algorithm. Through simulation experiments, the prediction accuracy of the model for positive class samples is significantly improved after the nonequilibrium processing, the recall rate of the network model reaches 84.15%, and the prediction accuracy of the network model under different time steps reaches more than 90%. The experiment proves that the model can send out financial alert information more quickly and efficiently and accurately when a financial crisis may occur, relative to the traditional financial forecasting methods.

A Parkinson’s disease diagnosis approach for nonequilibrium gait data

An important feature of Parkinson’s disease (PD) patients is dyskinesia, and gait signal analysis can provide a strong basis for the diagnosis and rehabilitation of Parkinson’s disease. Traditional machine learning methods are not suitable for directly classifying imbalanced data. In order to accurately distinguish healthy people from Parkinson’s disease patients, a cost-sensitive support vector machine (CS-SVM) method is designed in this paper, which is used to construct the model for classification of gait signals between Kinson’s disease patients and healthy individuals. Gait data for the entire subject was extracted from a real U-shaped electronic walkway. The extracted features are converted to a dimensionless form, the classification performance can be improved. The experimental results show that the prediction accuracy and F-measure obtained by the CS-SVM method are 94.16% and 87.08%, respectively.

A Novel Classification Method Based on a Two-Phase Technique for Learning Imbalanced Text Data

The problem of imbalanced data has a heavy impact on the performance of learning models. In the case of an imbalanced text dataset, minority class data are often classified to the majority class, resulting in a loss of minority information and low accuracy. Thus, it is a serious challenge to determine how to tackle the high imbalance ratio distribution of datasets. Here, we propose a novel classification method for learning tasks with imbalanced test data. It aims to construct a method for data preprocessing that researchers can apply to their learning tasks with imbalanced text data and save the efforts to search for more dedicated learning tools. In our proposed method, there are two core stages. In stage one, balanced datasets are generated using an asymmetric cost-sensitive support vector machine; in stage two, the balanced dataset is classified using the symmetric cost-sensitive support vector machine. In addition, the learning parameters in both stages are adjusted with a genetic algorithm to create an optimal model. A Yelp review dataset was used to validate the effectiveness of the proposed method. The experimental results showed that the proposed method led to a better performance subject to the targeted dataset, with at least 75% accuracy, and revealed that this new method significantly improved the learning approach.

Linear Cost-sensitive Max-margin Embedded Feature Selection for SVM

The information needed for a certain machine application can be often obtained from a subset of the available features. Strongly relevant features should be retained to achieve desirable model performance. This research focuses on selecting relevant independent features for Support Vector Machine (SVM) classifiers in a cost-sensitive manner. A review of recent literature about feature selection for SVM revealed a lack of linear programming embedded SVM feature selection models. Most reviewed models were mixed-integer linear or nonlinear. Further, the review highlighted a lack of cost-sensitive SVM feature selection models. Cost sensitivity improves the generalization of SVM feature selection models, making them applicable to various cost-of-error situations. It also helps with handling imbalanced data. This research introduces an SVM-based filter method named Knapsack Max-Margin Feature Selection (KS-MMFS), which is a proposed linearization of the quadratic Max-Margin Feature Selection (MMFS) model. MMFS provides explicit estimates of feature importance in terms of relevance and redundancy. KS-MMFS was then used to develop a linear cost-sensitive SVM embedded feature selection model. The proposed model was tested on a group of 11 benchmark datasets and compared to relevant models from the literature. The results and analysis showed that different cost sensitivity (i.e., sensitivity–specificity tradeoff) requirements influence the features selected. The analysis demonstrated the competitive performance of the proposed model compared with relevant models. The model achieved an average improvement of 31.8% on classification performance with a 22.4% average reduction in solution time. The results and analysis in this research demonstrated the competitive performance of the proposed model as an efficient cost-sensitive embedded feature selection method.

A hyper-parameter tuning approach for cost-sensitive support vector machine classifiers

AbstractIn machine learning, hyperparameter tuning is strongly useful to improve model performance. In our research, we concentrate our attention on classifying imbalanced data by cost-sensitive support vector machines. We propose a multi-objective approach that optimizes model’s hyper-parameters. The approach is devised for imbalanced data. Three SVM model’s performance measures are optimized. We present the algorithm in a basic version based on genetic algorithms, and as an improved version based on genetic algorithms combined with decision trees. We tested the basic and the improved approach on benchmark datasets either as serial and parallel version. The improved version strongly reduces the computational time needed for finding optimized hyper-parameters. The results empirically show that suitable evaluation measures should be used in assessing the classification performance of classification models with imbalanced data.

Efficient electro encephelogram classification system using support vector machine classifier and adaptive learning technique

Complex <span>modern signal processing is used to automate the analysis of electro encephelogram (EEG) signals. For the diagnosis of seizures, approaches that are simple and precise may be preferable rather than difficult and time-consuming. In this paper, efficient EEG classification system using support vector machine (SVM) and Adaptive learning technique is proposed. The database EEG signals are subjected to temporal and spatial filtering to remove unwanted noise and to increase the detection accuracy of the classifier by selecting the specific bands in which most of the EEG data are present. The neural network based SVM is used to classify the test EEG data with respect to training data. The cost-sensitive SVM with proposed Adaptive learning classifies the EEG signals where the adaptive learning with probability based function helps in prediction of the future samples and this leads in improving the accuracy with detection time. The detection accuracy of the proposed algorithm is compared with existing which shows that the proposed algorithm can classify the EEG signal more </span>effectively.

Identification of Critical Components in the Complex Technical Infrastructure of the Large Hadron Collider Using Relief Feature Ranking and Support Vector Machines

This work proposes a data-driven methodology for identifying critical components in Complex Technical Infrastructures (CTIs), for which the functional logic and/or the system structure functions are not known due the CTI’s complexity and evolving nature. The methodology uses large amounts of CTI monitoring data acquired over long periods of time and under different operating conditions. The critical components are identified as those for which the condition monitoring signals permit the optimal classification of the CTI functioning or failed state. The methodology includes two stages: in the first stage, a feature selection filter method based on the Relief technique is used to rank the monitoring signals according to their importance with respect to the CTI functioning or failed state; the second stage identifies the subset of signals among those highlighted by the Relief technique that are most informative with respect to the CTI state. This identification is performed on the basis of evaluating the performance of a Cost-Sensitive Support Vector Machine (CS-SVM) classifier trained with several subsets of the candidate signals. The capabilities of the methodology proposed are assessed through its application to different benchmarks of highly imbalanced datasets, showing performances that are competitive to those obtained by other methods presented in the literature. The methodology is finally applied to the monitoring signals of the Large Hadron Collider (LHC) of the European Organization for Nuclear Research (CERN), a CTI for experiments of physics; the criticality of the identified components has been confirmed by CERN experts.

Integration of aggressive bound tightening and Mixed Integer Programming for Cost-sensitive feature selection in medical diagnosis

Silent diseases is an umbrella term that captures a spectrum of chronic illnesses that produce no clinically obvious signs and are diagnosed at advanced stages when the damage is irreversible. Current diagnostic strategies of silent diseases depend on self-reported symptoms and observed behavior through extended periods of time, and until now there are no specific clinical tests to diagnose silent diseases. Scientific research suggests the importance of early diagnosis to restore the functionality and reduce diseases-related complications. Previous studies primarily focused on feature selection methods to aid in medical diagnosis. Traditional feature selection methods are primarily focused on correct classification and often ignore features’ costs; the cost of clinical tests required to acquire the feature value. However, in medical diagnosis, features have different associated costs. Because ignoring features’ costs may result in a high cost diagnostic strategy that cannot be used in practice, developing a low-cost diagnostic strategy remains a subject of much interest. In this paper, new Mixed Integer Programming (MIP) models, namely, Cost-sensitive Support Vector Machine (CS-SVM) and Cost-sensitive Multi-surface Method Tree (CS-MSMT) that allow for simultaneous selection of low-cost and informative features are proposed. The CS-SVM and CS-MSMT are superior because they have the ability to account for shared costs. The CS-SVM and CS-MSMT were modified to embed shared costs across feature groups, and are termed Discounted CS-SVM (dCS-SVM) and Discounted CS-MSMT (dCS-MSMT), respectively. Computationally effective algorithm that integrates aggressive bound tightening with the MIP formulation is proposed. To demonstrate the effectiveness of the proposed models, different analysis paradigms are conducted on six UCI medical datasets; Chronic Kidney Disease, Hepatitis, Heart Disease, Thyroid, Diabetes and Leukemia. The results demonstrate the efficiency and robustness of the CS-SVM and CS-MSMT (and consequently the dCS-SVM and dCS-MSMT) under various conditions. The CS-SVM and CS-MSMT improved accuracy by 10.3% and 3.4% and reduced costs by 94.3% and 72.4% in the leukemia dataset, respectively.