Cost-sensitive Learning Research Articles

Metric Optimization and Mainstream Bias Mitigation in Recommender Systems

Recommender Systems have drawn extensive attention in recent decades. The most important factor to make a recommender succeed is user satisfaction, which is largely reflected by the recommendation accuracy. In accordance, this thesis dives into recommendation accuracy from two different perspectives that, unfortunately, are at tension with each other: achieving the maximum overall recommendation accuracy, and balancing that accuracy among all users. The first part of this thesis focuses on maximizing the overall recommendation accuracy Li et al. [2021a]. This accuracy is usually evaluated with some user-oriented metrics tailored to the recommendation scenario. However, recommendation models could be trained to maximize some other generic criteria that do not necessarily align with the criteria ultimately captured by the metric above. Recent research usually assumes that the metric used for evaluation is also the metric used for training. We challenge this assumption, mainly because some metrics are more informative than others. Indeed, we show that models trained via the optimization of a loss inspired by Rank-Biased Precision (RBP) tend to yield higher accuracy, even when accuracy is measured with metrics other than RBP. However, the superiority of this RBP-inspired loss stems from further benefiting users who are already well-served, rather than helping those who are not. This observation inspires the second part of this thesis, where our focus turns to helping nonmainstream users, who are difficult to recommend to either because of the lack of data for effective modeling or a niche taste that is hard to match similar users. Our first effort consists in using side data, beyond the user-item interaction matrix, so that users and items are better represented. This will be of benefit especially for the non-mainstream users, for which the user-item matrix alone is ineffective. We propose Neural AutoEncoder Collaborative Filtering (NAECF) Li et al. [2021b], an adversarial learning architecture that, in addition to maximizing the recommendation accuracy, leverages side data to preserve the intrinsic properties of users and items and show that NAECF leads to better recommendations specially for non-mainstream users, while at the same time there is a marginal loss for the mainstream ones. Our second effort Li et al. [2023] consists in explicitly focusing more on non-mainstream users in a recommendation model. In particular, we propose a mechanism based on cost-sensitive learning that weighs users according to their mainstreamness, so that they get more attention during training. The result is a recommendation model tailored to non-mainstream users, that narrows the accuracy gap, and again at negligible cost to the mainstream users. Awarded by : Delft University of Technology, Delft, The Netherlands on 14 November 2023. Supervised by : Alan Hanjalic and Julián Urbano. Available at : https://doi.org/10.4233/uuid:235a5f5c-5e18-4e67-a8e6-84a7e310db12.

Cost-Sensitive Decision Support for Industrial Batch Processes

In this work, cost-sensitive decision support was developed. Using Batch Data Analytics (BDA) methods of the batch data structure and feature accommodation, the batch process property and sensor data can be accommodated. The batch data structure organises the batch processes’ data, and the feature accommodation approach derives statistics from the time series, consequently aligning the time series with the other features. Three machine learning classifiers were implemented for comparison: Logistic Regression (LR), Random Forest Classifier (RFC), and Support Vector Machine (SVM). It is possible to filter out the low-probability predictions by leveraging the classifiers’ probability estimations. Consequently, the decision support has a trade-off between accuracy and coverage. Cost-sensitive learning was used to implement a cost matrix, which further aggregates the accuracy–coverage trade into cost metrics. Also, two scenarios were implemented for accommodating out-of-coverage batches. The batch is discarded in one scenario, and the other is processed. The Random Forest classifier was shown to outperform the other classifiers and, compared to the baseline scenario, had a relative cost of 26%. This synergy of methods provides cost-aware decision support for analysing the intricate workings of a multiprocess batch data system.

Research on the influence factors of accident severity of new energy vehicles based on ensemble learning

With the deepening of the concept of green, low-carbon, and sustainable development, the continuous growth of the ownership of new energy vehicles has led to increasing public concerns about the traffic safety issues of these vehicles. In order to conduct research on the traffic safety of new energy vehicles, three sampling methods, namely, Synthetic Minority Over-sampling Technique (SMOTE), Edited Nearest Neighbours (ENN), and SMOTE-ENN hybrid sampling, were employed, along with cost-sensitive learning, to address the problem of imbalanced data in the UK road traffic accident dataset. Three algorithms, eXtreme Gradient Boosting (XGBoost), Light Gradient Boosting Machine (LightGBM), and Categorical Boosting (CatBoost), were selected for modeling work. Lastly, the evaluation criteria used for model selection were primarily based on G-mean, with AUC and accuracy as secondary measures. The TreeSHAP method was applied to explain the interaction mechanism between accident severity and its influencing factors in the constructed models. The results showed that LightGBM had a more stable overall performance and higher computational efficiency. XGBoost demonstrated a balanced combination of computational efficiency and model performance. CatBoost, however, was more time-consuming and showed less stability with different datasets. Studies have found that people using fewer protective means of transportation (bicycles, motorcycles) and vulnerable groups such as pedestrians are susceptible to serious injury and death.

SynDroid: An adaptive enhanced Android malware classification method based on CTGAN-SVM

Android mobile phones have the highest market share nowadays, bringing a boom of Android application programming as well as malicious software (malware) issues. Traditional machine learning and deep learning methods are widely used in Android malware detection and classification, both require plenty of application samples to train classifiers. However, the collected samples are always imbalanced, because the distribution of malware categories differs hugely in the real environment. For Android malware samples with high dimension features, and Class Imbalance Ratio coming to more than 100:1, traditional methods become weak. To fill the gap, we propose an Android malware classification model named SynDroid. The core step of SynDroid uses CTGAN-SVM to generate qualified high-dimension samples, and adaptively discards bad results. Besides, we propose KS-CIR test to help the model to determine which classes of data needed to be enhanced most. This new proposed test can measure the data in respect of both samples' quality and quantity. Lastly, Random Forest is taken as a classifier to finish the classification task. The performance of SynDroid is evaluated on CCCS-CIC-AndMal2020 on the accuracy, precision, recall and F1-score. Both longitudinal and horizontal comparison experiments have been done with traditional oversampling methods, cost-sensitive learning, and other complicated methods. The result shows that the proposed method gets 12% more accuracy than the method on the same dataset and alleviates imbalanced data problems.

Refining one-class representation: A unified transformer for unsupervised time-series anomaly detection

The deep unsupervised time-series anomaly detector depends on the one-class representation, which is more effective by only formulating the normal samples. However, normal samples are always mixed with anomalies in the unlabeled training dataset. The learned one-class representation may be biased and violates the one-class setting. To address this problem, we refine the one-class representation and propose a unified AMFormer (Active Masked transFormer) framework, which integrates Transformer with the masked operation mechanism and cost-sensitive learning theory. Specifically, we first develop a network-driven masked operation with Hadamard product transformation to damage the initial input samples. The encoder and decoder representations rebuild the processed incomplete samples, which avoids identical shortcuts and further enhances robustness. Secondly, we exploit the active MSE (Mean Squared Error) loss function to purify the training samples. The different weights are dynamically added to different kinds of samples according to their rebuilding-errors-based pseudo labels. The pseudo anomalies with more significant rebuilding errors are removed by putting lower weights. Finally, extensive experiments are conducted on four benchmark datasets. The experimental results demonstrate that our AMFormer outperforms the nine relevant benchmark algorithms, boosting the mean f1-score from 0.851 to 0.937.

Cost-sensitive sparse group online learning for imbalanced data streams

Cost-sensitive sparse group online learning for imbalanced data streams

A machine learning approach to identifying Salmonella stress response genes in isolates from poultry processing

We explored the potential of machine learning to identify significant genes associated with Salmonella stress response during poultry processing using whole genome sequencing (WGS) data. The Salmonella isolates (n = 177) used in this study were obtained from various chicken sources (skin before chiller, chicken carcass before chiller, frozen chicken, and post-chill chicken carcass). Six machine learning algorithms (random forest, neural network, cost-sensitive learning, logit boost, and support vector machine linear and radial kernels) were trained on Salmonella WGS data, and model fit was assessed using standard evaluation metrics such as the area under the receiver operating characteristic (AUROC) curve and confusion matrix statistics. All models achieved high performances based on the AUROC metric, with logit boost showing the best performance with an AUROC score of 0.904, sensitivity of 0.889, and specificity of 0.920. The significant genes identified included ybtX, which encodes a Yersiniabactin-associated zinc transporter, and the transferase-encoding genes yccK and thiS. Additionally, genes coding for cold (cspA, cspD, and cspE) and heat shock (rpoH and rpoE) responses were identified. Other significant genes included those involved in lipopolysaccharide biosynthesis (irp1, waaD, rfc, and rfbX), DNA repair and replication (traI), biofilm formation (ccdA and fyuA), and cellular metabolism (irtA).

Receiver Operating Characteristic Optimization Based on Convex Hull and Evolutionary Algorithm

Classification is basically a Multi-objective problem. The efficiency of classification vastly depends on performance of a classifier which can be evaluated on the basis of Receiver Operating Characteristic (ROC) graph, Area under Curve (AUC), and selection of different threshold values are generally used as a tool. In machine learning, generally, 2-D classifiers are available that deal with bi-objective problems where overlapping of class may occur i.e. sensitivity and specificity may overlap. Recently, multi-class classification in which classes are mutually exclusive is in research trends along with the evolutionary algorithm. The application of ROC graph is extended to evaluate multi-dimensional classification as it is cost sensitive learning. The goal of this paper is to gather recent achievements in the field at one place to analyze the classifier performance for multi-dimensional classification problems using convex hull and evolutionary algorithms. In this paper, we tried to cover all the existing recent advance techniques in maximization of ROC and proposed a convex hull and evolutionary algorithm based new model for ROC maximization.

Cost-Sensitive Online Adaptive Kernel Learning for Large-Scale Imbalanced Classification

Imbalanced classification is a challenging task in the fields of machine learning, data mining and pattern recognition. Cost-sensitive online algorithms are very important methods for large-scale imbalanced classification problems. At present, most of the cost-sensitive classification algorithms focus on the accuracy of the minority class and ignore the accuracy of the majority class. In order to better balance the accuracy between the minority class and the majority class, in this paper, a misclassification cost is presented to ensure that the cost-sensitive online algorithm can better deal with the imbalanced classification problems without signifificantly reducing the accuracy of the majority class. Based on the proposed misclassification cost, a novel cost-sensitive online adaptive kernel learning algorithm is proposed to boost the adaptability of kernel function when data arrives one by one. According to the essential characteristics of the imbalanced binary classification, a cost-sensitive online adaptive kernel learning algorithm is given to handle the large-scale imbalanced multi-class classification problems. Theoretical analysis of the proposed algorithms are provided. Extensive experiments demonstrate that compared with the state-of-the-art imbalanced classification algorithms, the proposed algorithms can significantly improve the classification performances on most of the large-scale imbalanced data sets.

CBDT-Oglyc: Prediction of O-glycosylation sites using ChiMIC-based balanced decision table and feature selection.

O-glycosylation (Oglyc) plays an important role in various biological processes. The key to understanding the mechanisms of Oglyc is identifying the corresponding glycosylation sites. Two critical steps, feature selection and classifier design, greatly affect the accuracy of computational methods for predicting Oglyc sites. Based on an efficient feature selection algorithm and a classifier capable of handling imbalanced datasets, a new computational method, ChiMIC-based balanced decision table O-glycosylation (CBDT-Oglyc), is proposed. ChiMIC-based balanced decision table for O-glycosylation (CBDT-Oglyc), is proposed to predict Oglyc sites in proteins. Sequence characterization is performed by combining amino acid composition (AAC), undirected composition of [Formula: see text]-spaced amino acid pairs (undirected-CKSAAP) and pseudo-position-specific scoring matrix (PsePSSM). Chi-MIC-share algorithm is used for feature selection, which simplifies the model and improves predictive accuracy. For imbalanced classification, a backtracking method based on local chi-square test is designed, and then cost-sensitive learning is incorporated to construct a novel classifier named ChiMIC-based balanced decision table (CBDT). Based on a 1:49 (positives:negatives) training set, the CBDT classifier achieves significantly better prediction performance than traditional classifiers. Moreover, the independent test results on separate human and mouse glycoproteins show that CBDT-Oglyc outperforms previous methods in global accuracy. CBDT-Oglyc shows great promise in predicting Oglyc sites and is expected to facilitate further experimental studies on protein glycosylation.

Cost-Sensitive Learning for Medical Insurance Fraud Detection With Temporal Information

Cost-Sensitive Learning for Medical Insurance Fraud Detection With Temporal Information

A high-precision and transparent step-wise diagnostic framework for hot-rolled strip crown

The strip crown plays a crucial role in determining the quality of products in strip hot rolling. Machine learning (ML) methods have shown promise in crown prediction by effectively capturing the nonlinearities and strong coupling present in hot rolling data, surpassing the capabilities of traditional methods. However, existing ML models ignore the imbalance of strip crown and tend to prioritize learning information from the qualified crown, limiting the precision of diagnosing the faulty crown. To overcome this limitation, a novel high-precision step-wise diagnostic framework is proposed. The framework starts with a crown detection module that promptly detects the faulty crown and enables timely blocking of the faulty strip. To enhance the diagnostic precision for the faulty crown, a novel hybrid data processing strategy that combines resampling method and cost-sensitive learning is introduced within the detection module, and the cost factor is optimized by Chaotic Harris Hawks Optimizer (CHHO). Subsequently, the framework incorporates a crown classification module to accurately recognize the specific fault-type present in the faulty strip. Furthermore, eXplainable Artificial Intelligence (XAI) technique is employed to ensure the transparent decision-making processes of both the detection module and the classification module. The comparative experiment results demonstrate that the proposed framework outperforms other state-of-the-art ML methods. It can achieve an excellent trade-off between precision and efficiency in diagnosing hot-rolled strip crown. Additionally, the feature contributions and decision interpretable analysis based on XAI provide further evidence of the transparency and effectiveness of the proposed framework.

Adaptive Age Estimation towards Imbalanced Datasets

Current age estimation datasets often have a skewed long-tail distribution with significant data imbalance, rather than an ideal uniform distribution for each category. The existing age estimation algorithms that rely on label distribution do not leverage data density information to address the issue of data imbalance. To solve the aforementioned problem, this paper proposes a novel method based on cost-sensitive learning, namely Data-Imbalance Adaptive Age Regression (DIAAR), for age estimation. DIAAR consists of two main modules: the adaptive soft label (ASL) module and the Data Density Smoothing (DDS) module. The ASL module embeds soft labels in the form of probability in the age regression. It assigns different degrees of soft labels adaptively to head and tail data based on their density, which helps balance the dataset. The DDS module further addresses data imbalance by revising data density through kernel smoothing and reweighting the loss function accordingly. Experiments on two benchmark datasets show that DIAAR can effectively deal with data imbalance and improve the accuracy of age estimation, achieving an average improvement of 8% over the baseline models. Moreover, this approach can be applied to various methods based on convolutional neural network models.

ECC + +: An algorithm family based on ensemble of classifier chains for classifying imbalanced multi-label data

Multi-label learning has attracted a great deal of research interests as it has a wide range of real-world applications. Although many multi-label learning methods have been proposed, very few of them have addressed the problem of class imbalance distribution in multi-label data. Moreover, most of the existing class imbalance multi-label learning algorithms only focus on solving the class imbalance problem, without taking into account the correlations among labels. To address these issues simultaneously, we propose to combine the well-known ensemble of classifier chain (ECC) algorithm with various binary-class imbalance learning techniques such as sampling, cost-sensitive learning, and threshold moving. This approach creates a new algorithm family called ECC++, designed specifically for class imbalance multi-label learning. ECC is already an excellent ensemble high-order binary relevance multi-label learning algorithm that is well-suited to exploiting correlations among labels. Combining it with binary-class imbalance learning techniques enables each link in a classifier chain (CC) to overcome the negative effect of skewed data distribution. ECC++ is a dynamic algorithm family that can be extended arbitrarily by applying any new binary-class imbalance learning techniques. To demonstrate the effectiveness and superiority of the proposed ECC++ algorithm family, we developed several ECC++ family members using some popular binary-class imbalance learning techniques. We then compared them with several state-of-the-art class imbalance multi-label learning algorithms on twelve benchmark and four real-world multi-label datasets. Our experimental results showed the effectiveness and superiority of the proposed ECC++ algorithm family over existing class imbalance multi-label learning algorithms. In conclusion, the proposed ECC++ algorithm family combines the strengths of the well-established ECC algorithm and binary-class imbalance learning techniques, resulting in a superior methodology for class imbalance multi-label learning.

Leader learning loss function in neural network classification

Deep learning, based on Empirical Risk Minimization (ERM), typically aims to fit the ideal outputs of all samples due to its large capacity. However, models trained based on empirical losses like cross entropy (CE) or mean square error (MSE), often learn unnecessary information during classification, leading to premature overfitting. On the other hand, the result-focused loss functions, i.e., zero–one loss or hinge loss, are hard to optimize and thus are rarely applied directly in neural network. This paper proposes a novel leader learning in classification, where CE is gradually trained by classification results using sample-dependent cost-sensitive learning. As complementary, the stepwise-changed CE covers the deficiency on classification error while preserving the advantage of fast convergence. In this way, the deviation between CE and classification error can be corrected. Experimental results demonstrate that the proposed leader learning has a more significant convergence trend than the baseline algorithms. Moreover, the loss function learned from a specific dataset has broad generality that can be transferred to other models as prior knowledge.

Realtime gray-box algorithm configuration using cost-sensitive classification

AbstractA solver’s runtime and the quality of the solutions it generates are strongly influenced by its parameter settings. Finding good parameter configurations is a formidable challenge, even for fixed problem instance distributions. However, when the instance distribution can change over time, a once effective configuration may no longer provide adequate performance. Realtime algorithm configuration (RAC) offers assistance in finding high-quality configurations for such distributions by automatically adjusting the configurations it recommends based on instances seen so far. Existing RAC methods treat the solver as a black box, meaning the solver is given a configuration as input, and it outputs either a solution or runtime as an objective function for the configurator. However, analyzing intermediate output from the solver can enable configurators to avoid wasting time on poorly performing configurations. We propose a gray-box approach that utilizes intermediate output during evaluation and implement it within the RAC method Contextual Preselection with Plackett-Luce (CPPL blue). We apply cost-sensitive machine learning with pairwise comparisons to determine whether ongoing evaluations can be terminated to free resources. We compare our approach to a black-box equivalent on several experimental settings and show that our approach reduces the total solving time in several scenarios and improves solution quality in an additional scenario.

Everything has its price: Foundations of cost-sensitive machine learning and its application in psychology.

Psychology has seen an increase in the use of machine learning (ML) methods. In many applications, observations are classified into one of two groups (binary classification). Off-the-shelf classification algorithms assume that the costs of a misclassification (false positive or false negative) are equal. Because this is often not reasonable (e.g., in clinical psychology), cost-sensitive machine learning (CSL) methods can take different cost ratios into account. We present the mathematical foundations and introduce a taxonomy of the most commonly used CSL methods, before demonstrating their application and usefulness on psychological data, that is, the drug consumption data set (N = 1, 885) from the University of California Irvine ML Repository. In our example, all demonstrated CSL methods noticeably reduced mean misclassification costs compared to regular ML algorithms. We discuss the necessity for researchers to perform small benchmarks of CSL methods for their own practical application. Thus, our open materials provide R code, demonstrating how CSL methods can be applied within the mlr3 framework (https://osf.io/cvks7/). (PsycInfo Database Record (c) 2023 APA, all rights reserved).

A landscape metrics-based sample weighting approach for forecasting land cover change with deep learning models

Unaddressed imbalance of multitemporal land cover (LC) data reduces deep learning (DL) model usefulness to forecast changes. To manage geospatial data imbalance, there is a lack of specialized cost-sensitive learning strategies available. Sample weights are typically derived from training instance frequencies which disregard spatial pattern complexities. Therefore, this study proposes a geospatial sample weighting approach underpinned by class-level landscape metrics (LSMs) to assign importance to categories based on relative indicators of spatial form. A case study demonstrates the application and effects of the LSM-based sample weighting approach for projecting LC changes of a region in British Columbia, Canada. Four spatiotemporal DL models are provided weighted training samples including multitemporal LC data and explanatory factors. Sample weights calculated from indicators of patch density, shape irregularity and shape heterogeneity improved figure of merit and related measures over baseline configurations. This study contributes to LC change data imbalance management strategies for DL models.

SMILE: Cost-sensitive multi-task learning for nuclear segmentation and classification with imbalanced annotations.

High throughput nuclear segmentation and classification of whole slide images (WSIs) is crucial to biological analysis, clinical diagnosis and precision medicine. With the advances of CNN algorithms and the continuously growing datasets, considerable progress has been made in nuclear segmentation and classification. However, few works consider how to reasonably deal with nuclear heterogeneity in the following two aspects: imbalanced data distribution and diversified morphology characteristics. The minority classes might be dominated by the majority classes due to the imbalanced data distribution and the diversified morphology characteristics may lead to fragile segmentation results. In this study, a cost-Sensitive MultI-task LEarning (SMILE) framework is conducted to tackle the data heterogeneity problem. Based on the most popular multi-task learning backbone in nuclei segmentation and classification, we propose a multi-task correlation attention (MTCA) to perform feature interaction of multiple high relevant tasks to learn better feature representation. A cost-sensitive learning strategy is proposed to solve the imbalanced data distribution by increasing the penalization for the error classification of the minority classes. Furthermore, we propose a novel post-processing step based on the coarse-to-fine marker-controlled watershed scheme to alleviate fragile segmentation when nuclei are with large size and unclear contour. Extensive experiments show that the proposed method achieves state-of-the-art performances on CoNSeP and MoNuSAC 2020 datasets. The code is available at: https://github.com/panxipeng/nuclear_segandcls.

Multi-view cost-sensitive kernel learning for imbalanced classification problem

Multi-view imbalanced learning concentrates on recognizing valuable patterns from multi-view imbalanced data. There are numerous algorithm-level multi-view imbalanced learning methods including multi-view ensemble learning methods and multi-view cost-sensitive learning methods. However, these approaches have two drawbacks. Firstly, from the multi-view representation perspective, they either ignore the agreement among different views, or fail to fully exploit the complementarity information contained in multi-view data. Secondly, from the class-imbalanced perspective, multi-view ensemble learning first needs to design a specific data pre-processing scheme to generate views and then fuses the local and global information with multi-view ensemble schemes. Such models are closely related to practical problems and lack universality. In contrast, multi-view cost-sensitive learning methods possess concise formulations. But the success of such methods depends on imposing appropriate misclassification cost to the majority and minority classes. Inspired by elegant merits of asymmetric LINEX loss function, we propose a multi-view cost-sensitive kernel learning method with LINEX loss termed as MVCSKL. It not only fully exploits the consensus and complementary information of multiple views, but also adaptively learns and adjusts the misclassification cost of different classes through the asymmetric LINEX loss function. Besides, the introduction of the kernel function enables the MVCSKL model to handle complex nonlinear problem and further guarantees its prediction performance. Furthermore, we adopt the alternating direction method of multipliers and gradient descent algorithm to solve MVCSKL. Based on the Rademacher complexity, we analyze the generalization capability of MVCSKL. Extensive experiments are conducted to verify the effectiveness of the proposed model.