Cost-sensitive Learning Algorithm Research Articles

Dynamic Treatment Strategy of Chinese Medicine for Metastatic Colorectal Cancer Based on Machine Learning Algorithm.

To establish the dynamic treatment strategy of Chinese medicine (CM) for metastatic colorectal cancer (mCRC) by machine learning algorithm, in order to provide a reference for the selection of CM treatment strategies for mCRC. From the outpatient cases of mCRC in the Department of Oncology at Xiyuan Hospital, China Academy of Chinese Medical Sciences, 197 cases that met the inclusion criteria were screened. According to different CM intervention strategies, the patients were divided into 3 groups: CM treatment alone, equal emphasis on Chinese and Western medicine treatment (CM combined with local treatment of tumors, oral chemotherapy, or targeted drugs), and CM assisted Western medicine treatment (CM combined with intravenous regimen of Western medicine). The survival time of patients undergoing CM intervention was taken as the final evaluation index. Factors affecting the choice of CM intervention scheme were screened as decision variables. The dynamic CM intervention and treatment strategy for mCRC was explored based on the cost-sensitive classification learning algorithm for survival (CSCLSurv). Patients' survival was estimated using the Kaplan-Meier method, and the survival time of patients who received the model-recommended treatment plan were compared with those who received actual treatment plan. Using the survival time of patients undergoing CM intervention as the evaluation index, a dynamic CM intervention therapy strategy for mCRC was established based on CSCLSurv. Different CM intervention strategies for mCRC can be selected according to dynamic decision variables, such as gender, age, Eastern Cooperative Oncology Group score, tumor site, metastatic site, genotyping, and the stage of Western medicine treatment at the patient's first visit. The median survival time of patients who received the model-recommended treatment plan was 35 months, while those who receive the actual treatment plan was 26.0 months (P=0.06). The dynamic treatment strategy of CM, based on CSCLSurv for mCRC, plays a certain role in providing clinical hints in CM. It can be further improved in future prospective studies with larger sample sizes.

Classification of tracheal stenosis with asymmetric misclassification errors from EMG signals using an adaptive cost-sensitive learning method

Upper airway obstruction is characterized by loss of normal airway architecture resulting from various disorders such as infections and asthma. Early detection of airway obstruction is essential to prevent medical deterioration. The objective of this study was to non-invasively identify early-stage tracheal stenosis, using electromyography (EMG) signals of inspiratory muscles. The identification of tracheal stenosis has been defined as an asymmetric misclassification cost problem. Specifically, the EMG signals are used as input to a ResNet-like architecture for tabular data with an Adaptive Cost-Sensitive Learning (AdaCSL) algorithm. The electrical activity of the external intercostal muscle of four healthy individuals was recorded while they breathed through two different tubes, one simulating a narrowed airway and the other simulating a normal airway. Two experiment settings were designed. The first setting aimed to classify tracheal stenosis in a specific subject by training the model on data from other subjects, reflecting the case of diagnosing a new subject. To overcome multi-subject variations, the second setting aimed to classify tracheal stenosis by mixing all subjects’ training and test data. The ResNet-like architecture with an AdaCSL algorithm was significantly better in the first experiment setting with costs that were 43%, 48%, and 59% lower than the cost of the second-best alternative for three different misclassification cost values. It also achieved a lower cost in the second experiment setting over other classifiers. The experiments emphasize the capability of using inspiratory muscle EMG signals to diagnose respiratory disease and demonstrate the usefulness of the AdaCSL algorithm for personalized monitoring.

Spatial rain probabilistic prediction performance using costsensitive learning algorithm

The use of machine learning in weather prediction is growing rapidly as an alternative to conventional numerical weather prediction. However, predictions using machine learning such as Long Short Term Memory (LSTM) based on neural networks have weaknesses in predicting extreme events with a high ratio of unbalanced data. This research examines the performance of using focal loss in LSTM to obtain a machine-learning model that is cost-sensitive. The model used the Global Forecasting System Data and the Global Satellite Measurement of Precipitation for the years 2017-2020. Testing the hyperparameter configuration was carried out using the hyperband method on the number of nodes and the number of iterations with 3 scenarios (2, 3, and 4 classes). The results showed an increased performance against noncost sensitive LSTM with an average increase of 25% accuracy and 11% F1-score on 2 classes scenario, 15% accuracy increase and 21% F1-score for scenario 3 classes, as well as an increase in accuracy of 15% and F1-score 26% for scenario 4 class. It also provides the idea of how cost-sensitive properties can help machine learning models detect classes with extreme ratios, based on an increase in average performance as the number of classification scenarios increases.

Bayesian Optimization Cost-Sensitive XGBoost Learning Algorithm for Imbalanced Data in Semiconductor Industry

Bayesian Optimization Cost-Sensitive XGBoost Learning Algorithm for Imbalanced Data in Semiconductor Industry

Cross-city crash severity analysis with cost-sensitive transfer learning algorithm

Due to the difficulties of data collection in some cities and the relatively small portion of more severe crashes, this paper proposes a cost-sensitive transfer learning framework for more robust crash severity analysis. Specifically, to address the class imbalance issue, cost-sensitive learning method is adopted by assigning unequal cost values to different crash severity levels to obtain unbiased classification results. Moreover, due to the existence of city-irrelevant common crash contributing factors, cross-city crash severity analysis is developed, i.e., transferring common crash severity knowledge from other cities with transfer learning algorithm to assist crash severity modelling in target city that suffers from data scarcity problem. In this study, the crash datasets of Victorian Australia and Seattle are selected as the target and source domains respectively. To address the issue of heterogeneous explanatory features among these two datasets and extract the interpretable common crash severity knowledge, a feature alignment approach is proposed which can represent the cross-city data with unified feature representations on their original explanatory feature spaces. The two proposed models, i.e., cost-sensitive transfer logistic regression (CST-LR) and cost-sensitive transfer support vector machine (CST-SVM), have demonstrated better performance in comparison with twelve commonly used crash severity models, especially when target city crash data is scarce. The most significant crash contributing factors extracted by proposed model also show higher degree of consistency with the true contributing factors obtained from the target dataset, in comparison with the model built without transfer learning. The results could provide policy implications and counter measures for crash severity mitigation.

Predicting Tree Mortality Using Spectral Indices Derived from Multispectral UAV Imagery

Past research has shown that remotely sensed spectral information can be used to predict tree health and vitality. Recent developments in unmanned aerial vehicles (UAVs) have now made it possible to derive such information at the tree and stand scale from high-resolution imagery. We used visible and multispectral bands from UAV imagery to calculate a set of spectral indices for 52,845 individual tree crowns within 38 forest stands in western Canada. We then used those indices to predict the mortality of these canopy trees over the following year. We evaluated whether including multispectral indices leads to more accurate predictions than indices derived from visible wavelengths alone and how the performance varies among three different tree species (Picea glauca, Pinus contorta, Populus tremuloides). Our results show that spectral information can be effectively used to predict tree mortality, with a random forest model producing a mean area under the receiver operating characteristic curve (AUC) of 89.8% and a balanced accuracy of 83.3%. The exclusion of multispectral indices worsened the model performance, but only slightly (AUC = 87.9%, balanced accuracy = 81.8%). We found variation in model performance among species, with higher accuracy for the broadleaf species (balanced accuracy = 85.2%) than the two conifer species (balanced accuracy = 73.3% and 77.8%). However, all models overpredicted tree mortality by a major degree, which limits the use for tree mortality predictions on an individual level. Further improvements such as long-term monitoring, the use of hyperspectral data and cost-sensitive learning algorithms, and training the model with a larger and more balanced data set are necessary. Nevertheless, our results demonstrate that imagery from UAVs has strong potential for predicting annual mortality for individual canopy trees.

A cost-sensitive active learning algorithm: toward imbalanced time series forecasting

A cost-sensitive active learning algorithm: toward imbalanced time series forecasting

CSE-IDS: Using cost-sensitive deep learning and ensemble algorithms to handle class imbalance in network-based intrusion detection systems

In recent times, Network-based Intrusion Detection Systems (NIDSs) have become very popular for detecting intrusions in computer networks. Existing NIDSs can easily identify those intrusions that have been frequently witnessed in the network (majority attacks), but they cannot identify new and infrequent intrusions (minority attacks) accurately. Moreover, such systems solely focus on maximizing the overall Attack Detection Rate while overlooking the number of false alarms. To address these issues, this paper proposes CSE-IDS, a three-layer NIDS, based on Cost-Sensitive Deep Learning and Ensemble algorithms. Layer 1 of the proposed CSE-IDS uses Cost-Sensitive Deep Neural Network to separate normal traffic from suspicious network traffic. These suspicious samples are then sent to Layer 2, which uses the eXtreme Gradient Boosting algorithm to classify them into normal class, different majority attack classes, and a single class representing all minority attack classes. At last, Random Forest is used at Layer 3 to classify the minority attacks identified at Layer 2 into their respective classes. The performance of the proposed CSE-IDS was evaluated on the NSL-KDD, CIDDS-001, and CICIDS2017 datasets with respect to Accuracy, Recall, Precision, F1-score, ROC curve, AUC values, and computational times. The proposed system outperforms its counterparts by achieving a high Attack Detection Rate for both majority attacks and minority attacks present in the network. Further, it minimizes the number of false alarms by correctly segregating normal traffic from attack traffic. The obtained results confirm that the proposed CSE-IDS can be deployed in the real world for performing network-based intrusion detection.

A novel adaptive weighted kernel extreme learning machine algorithm and its application in wind turbine blade icing fault detection

The conventional weighted kernel extreme learning machine (WKELM) is a typical cost-sensitive learning algorithm, which can provide a solution for the data imbalance problem in wind turbine blade icing fault detection. However, WKELM algorithm uses a fixed weighting strategy that only considers the number of samples to achieve imbalanced learning, which leads to its insufficient performance in data imbalance classification and fault detection. In this study, the support vector data description (SVDD) method is used to obtain sample distribution information to construct a novel adaptive weighting strategy. The adaptive weighting strategy is combined with the conventional fixed weighting strategy to construct an adaptive weighted kernel extreme learning machine (AWKELM) algorithm. The effectiveness and superiority of the AWKELM algorithm is verified through three benchmark cases, it is demonstrated that AWKELM algorithm can effectively solve the problem of data imbalance and further improve the effect of wind turbine blade icing detection.

Using Machine Learning to Predict Invasive Bacterial Infections in Young Febrile Infants Visiting the Emergency Department.

Background: The aim of this study was to develop and evaluate a machine learning (ML) model to predict invasive bacterial infections (IBIs) in young febrile infants visiting the emergency department (ED). Methods: This retrospective study was conducted in the EDs of three medical centers across Taiwan from 2011 to 2018. We included patients age in 0–60 days who were visiting the ED with clinical symptoms of fever. We developed three different ML algorithms, including logistic regression (LR), supportive vector machine (SVM), and extreme gradient boosting (XGboost), comparing their performance at predicting IBIs to a previous validated score system (IBI score). Results: During the study period, 4211 patients were included, where 126 (3.1%) had IBI. A total of eight, five, and seven features were used in the LR, SVM, and XGboost through the feature selection process, respectively. The ML models can achieve a better AUROC value when predicting IBIs in young infants compared with the IBI score (LR: 0.85 vs. SVM: 0.84 vs. XGBoost: 0.85 vs. IBI score: 0.70, p-value < 0.001). Using a cost sensitive learning algorithm, all ML models showed better specificity in predicting IBIs at a 90% sensitivity level compared to an IBI score > 2 (LR: 0.59 vs. SVM: 0.60 vs. XGBoost: 0.57 vs. IBI score >2: 0.43, p-value < 0.001). Conclusions: All ML models developed in this study outperformed the traditional scoring system in stratifying low-risk febrile infants after the standardized sensitivity level.

Using Dual Neural Network Architecture to Detect the Risk of Dementia With Community Health Data: Algorithm Development and Validation Study.

BackgroundRecent studies have revealed lifestyle behavioral risk factors that can be modified to reduce the risk of dementia. As modification of lifestyle takes time, early identification of people with high dementia risk is important for timely intervention and support. As cognitive impairment is a diagnostic criterion of dementia, cognitive assessment tools are used in primary care to screen for clinically unevaluated cases. Among them, Mini-Mental State Examination (MMSE) is a very common instrument. However, MMSE is a questionnaire that is administered when symptoms of memory decline have occurred. Early administration at the asymptomatic stage and repeated measurements would lead to a practice effect that degrades the effectiveness of MMSE when it is used at later stages.ObjectiveThe aim of this study was to exploit machine learning techniques to assist health care professionals in detecting high-risk individuals by predicting the results of MMSE using elderly health data collected from community-based primary care services.MethodsA health data set of 2299 samples was adopted in the study. The input data were divided into two groups of different characteristics (ie, client profile data and health assessment data). The predictive output was the result of two-class classification of the normal and high-risk cases that were defined based on MMSE. A dual neural network (DNN) model was proposed to obtain the latent representations of the two groups of input data separately, which were then concatenated for the two-class classification. Mean and k-nearest neighbor were used separately to tackle missing data, whereas a cost-sensitive learning (CSL) algorithm was proposed to deal with class imbalance. The performance of the DNN was evaluated by comparing it with that of conventional machine learning methods.ResultsA total of 16 predictive models were built using the elderly health data set. Among them, the proposed DNN with CSL outperformed in the detection of high-risk cases. The area under the receiver operating characteristic curve, average precision, sensitivity, and specificity reached 0.84, 0.88, 0.73, and 0.80, respectively.ConclusionsThe proposed method has the potential to serve as a tool to screen for elderly people with cognitive impairment and predict high-risk cases of dementia at the asymptomatic stage, providing health care professionals with early signals that can prompt suggestions for a follow-up or a detailed diagnosis.

Data-Driven Prediction Method for Power Grid State Subjected to Heavy-Rain Hazards

This study presents a machine learning-based method for predicting the power grid state subjected to heavy-rain hazards. Machine learning models can recognize key knowledge from a dataset without any preliminary knowledge about the dataset. Hence, machine learning methods have been utilized for solving power grid-related problems. Two sets of historical data were used herein: Local weather data and power grid outage data. First, we investigated the heavy-rain-related outage distribution and analyzed the correlated characteristics between weather and outages to characterize the heavy rain events. The analysis results show that multiple weather effects are significant in causing power outages, even under heavy-rain conditions. Furthermore, this study proposes a cost-sensitive prediction method using a support vector machine (SVM) model. The accuracy of the model was improved by applying a cost-sensitive learning algorithm to the SVM model, which was subsequently used to predict the state of the grid. The developed model was evaluated using G-mean values. The proposed method was verified via actual data of a heavy rain event that occurred in South Korea.

Boosting label weighted extreme learning machine for classifying multi-label imbalanced data

As a flexible and efficient cost sensitive learning algorithm, the label weighted extreme learning machine (LW-ELM) has been proposed to address the class imbalance learning problem on the multi-label data. However, due to the adoption of empirical costs, the classification performance of LW-ELM can't be guaranteed enough. To solve this problem, an improved algorithm called BLW-ELM, which integrates LW-ELM into the Boosting ensemble learning framework, is presented in this paper. Specifically, BLW-ELM designates the appropriate cost for each training label belonging to each training instance according to the iterative feedbacks of the training results, further gets rid of exploring the intricate distribution of multi-label data directly. That is to say, BLW-ELM is an universal and self-adapting algorithm that can promotes the robustness of classification regardless of the data distribution types. Twelve multi-label data sets are used to verify the effectiveness and superiority of the proposed algorithm. Experimental results indicate that the proposed BLW-ELM algorithm is significantly superior to LW-ELM algorithm and many other state-of-the-art multi-label imbalance learning algorithms, as well it generally needs far less training time than those sophisticated algorithms.

The effectiveness of cost sensitive machine learning algorithms in classifying Zeus flows

The effectiveness of cost sensitive machine learning algorithms in classifying Zeus flows

Using Cost-Sensitive Learning and Feature Selection Algorithms to Improve the Performance of Imbalanced Classification

Imbalanced data problem is widely present in network intrusion detection, spam filtering, biomedical engineering, finance, science, being a challenge in many real-life data-intensive applications. Classifier bias occurs when traditional classification algorithms are used to deal with imbalanced data. As already known, the General Vector Machine (GVM) algorithm has good generalization ability, though it does not work well for the imbalanced classification. Additionally, the state-of-the-art Binary Ant Lion Optimizer (BALO) algorithm has high exploitability and fast convergence rate. Based on these facts, we have proposed in this paper a Cost-sensitive Feature selection General Vector Machine (CFGVM) algorithm based on GVM and BALO algorithms to tackle the imbalanced classification problem, delivering different cost weights to different classes of samples. In our method, the BALO algorithm determines the cost weights and extract more significant features to improve the classification performance. Experiments conducted on eleven imbalanced data sets have shown that the CFGVM algorithm significantly improves the classification performance of minority class samples. By comparing with similar algorithms and state-of-the-art algorithms, the proposed algorithm significantly outperforms in performance and produces better classification results.

Detecting cyberattacks in industrial control systems using online learning algorithms

Industrial control systems are critical to the operation of industrial facilities, especially for critical infrastructures, such as refineries, power grids, and transportation systems. Similar to other information systems, a significant threat to industrial control systems is the attack from cyberspace—the offensive maneuvers launched by “anonymous” in the digital world that target computer-based assets with the goal of compromising a system’s functions or probing for information. Owing to the importance of industrial control systems, and the possibly devastating consequences of being attacked, significant endeavors have been attempted to secure industrial control systems from cyberattacks. Among them are intrusion detection systems that serve as the first line of defense by monitoring and reporting potentially malicious activities. Classical machine-learning-based intrusion detection methods usually generate prediction models by learning modest-sized training samples all at once. Such approach is not always applicable to industrial control systems, as industrial control systems must process continuous control commands with limited computational resources in a nonstop way. To satisfy such requirements, we propose using online learning to learn prediction models from the controlling data stream. We introduce several state-of-the-art online learning algorithms categorically, and illustrate their efficacies on two typically used testbeds—power system and gas pipeline. Further, we explore a new cost-sensitive online learning algorithm to solve the class-imbalance problem that is pervasive in industrial intrusion detection systems. Our experimental results indicate that the proposed algorithm can achieve an overall improvement in the detection rate of cyberattacks in industrial control systems.

Cost-sensitive active learning through statistical methods

Clustering-based active learning splits the data into a number of blocks and queries the labels of most the representative instances. When the cost of labeling and misclassification are considered, we also face a key issue: How many labels should be queried for a given block. In this paper, we present theoretical and practical statistical methods to handle this issue. The theoretical statistical method calculates the optimal number of query labels for a predefined label distribution. Considering label distributions for different clustering qualities, we obtain three hypothetical models, namely Gaussian, Uniform, and V models. The practical statistical method calculates empirical label distribution of the cluster blocks. Considering four popular clustering algorithms, we use symmetry and curve fitting techniques on 30 datasets to obtain empirical distributions. Inspired by three-way decision, we design an algorithm called the cost-sensitive active learning through statistical methods (CATS). Experiments were performed on 12 binary-class datasets for both the distribution evaluation and learning task. The results of significance tests verify the effectiveness of CATS and its superior performance with respect to state-of-the-art cost-sensitive active learning algorithms.

Annotation cost-sensitive active learning by tree sampling

Active learning is an important machine learning setup for reducing the labelling effort of humans. Although most existing works are based on a simple assumption that each labelling query has the same annotation cost, the assumption may not be realistic. That is, the annotation costs may actually vary between data instances. In addition, the costs may be unknown before making the query. Traditional active learning algorithms cannot deal with such a realistic scenario. In this work, we study annotation cost-sensitive active learning algorithms, which need to estimate the utility and cost of each query simultaneously. We propose a novel algorithm, the cost-sensitive tree sampling algorithm, that conducts the two estimation tasks together and solve it with a tree-structured model motivated from hierarchical sampling, a famous algorithm for traditional active learning. Extensive experimental results using datasets with simulated and true annotation costs validate that the proposed method is generally superior to other annotation cost-sensitive algorithms.

Second-Order Online Active Learning and Its Applications

The goal of online active learning is to learn predictive models from a sequence of unlabeled data given limited label query budget. Unlike conventional online learning tasks, online active learning is considerably more challenging because of two reasons. First, it is difficult to design an effective query strategy to decide when is appropriate to query the label of an incoming instance given limited query budget. Second, it is also challenging to decide how to update the predictive models effectively whenever the true label of an instance is queried. Most existing approaches for online active learning are often based on a family of first-order online learning algorithms, which are simple and efficient but fall short in the slow convergence and sub-optimal solution in exploiting the labeled training data. To solve these issues, this paper presents a novel framework of Second-order Online Active Learning (SOAL) by fully exploiting both the first-order and second-order information. The proposed algorithms are able to achieve effective online learning efficacy, maximize the predictive accuracy, and minimize the labeling cost. To make SOAL more practical for real-world applications, especially for class-imbalanced online classification tasks (e.g., malicious web detection), we extend the SOAL framework by proposing the Cost-sensitive Second-order Online Active Learning algorithm named “SOAL $_{CS}$ ”, which is devised by maximizing the sum of weighted sensitivity and specificity or minimizing the cost of weighted mistakes of different classes. We conducted both theoretical analysis and empirical studies, including an extensive set of experiments on a variety of large-scale real-world datasets, in which the promising empirical results validate the efficacy and scalability of the proposed algorithms towards large-scale online learning tasks.

Cost-sensitive design of error correcting output codes

Many real world applications involve multiclass cost-sensitive learning problems. However, some well-worked binary cost-sensitive learning algorithms cannot be extended into multiclass cost-sensitive learning directly. It is meaningful to decompose the complex multiclass cost-sensitive classification problem into a series of binary cost-sensitive classification problems. So, in this paper we propose an alternative and efficient decomposition framework, using the original error correcting output codes. The main problem in our framework is how to evaluate the binary costs for each binary cost-sensitive base classifier. To solve this problem, we proposed to compute the expected misclassification costs starting from the given multiclass cost matrix. Furthermore, the general formulations to compute the binary costs are given. Experimental results on several synthetic and UCI datasets show that our method can obtain comparable performance in comparison with the state-of-the-art methods.