Cost-sensitive Loss Function Research Articles

Prediction of aviation safety incident risk level based on ECSDNN

Prediction of aviation safety incident risk level is an important means of active risk management. Considering the high-dimensional complexity and class imbalance of massive aviation safety incident data, a prediction method for aviation safety incident risk level based on integrated cost-sensitive deep neural network (ECSDNN) is proposed. The feature representation of aviation safety incident data is realized by using the method of embedded coding of categorical attributes and splicing of numerical attributes; the cost-sensitive matrix and cost-sensitive loss function are designed by comprehensively considering the misclassification ratio and fixed cost, and the base classifier model based on cost-sensitive deep neural network (CSDNN) is constructed; the hard voting method is used to integrate multiple base classifiers with different parameters and performances to construct the integrated prediction model ECSDNN for aviation safety incident risk level. The experimental results on the Aviation Safety Incident Reporting System (ASRS) dataset show that compared with the optimal prediction ability of the baseline algorithm, the prediction accuracy of the ECSDNN model is improved 4.51%; compared with the single CSDNN base classifier, the prediction accuracy is improved 3.17%.

Class-imbalanced time series anomaly detection method based on cost-sensitive hybrid network

The time series is a structured data form. In practical engineering, most of the time series is in the form of class-imbalance, in which, abnormal data is rare or underrepresented. In this study, we present a cost sensitive hybrid network (CSHN) model to detect data anomaly in class-imbalanced time series. The proposed model adopts a two-flow structure that consist of a Convolutional Neural Network (CNN) and a Gated Recurrent Unit (GRU) network and combine with a cost-sensitive loss function. This model integrates the advantages of CNN and GRU. The former is provided with strong learning ability for local state features and the latter can extract the sequential features and force the decision logic of the network to pay more attention to the long-term dependence of the time series. The cost-sensitive loss function is used to solve the problem that the detection accuracy of the minority class will be inaccuracy caused by skewed data distribution. The simulation experiments are conducted on the speed and temperature datasets that come from real-world measurement and control engineering, as well as the UCR datasets. Compared to using the cross-entropy loss function, the using of the cost-sensitive loss function can improve G-means and F-measure by 3 %∼5% and 5 %∼9%, respectively, on the speed and temperature datasets. Experimental results demonstrate that our method is efficient for the class-imbalanced data sets, and the numerical evaluation indexes are superior to that of the comparison methods.

A new type of bridge structure reinforcement effect evaluation algorithm based on time series

A supplement method based on reusing the original bridge structure is suggested to expedite construction and reduce costs. Furthermore, theoretical analysis of the single pile bearing capacity is conducted, primarily involving the calculation of the bridge pile foundation settlement, horizontal resistance and axial bearing capacity. At the same time, considering that the time series data generated in the actual process often have characteristics of imbalance between classes, a cost-sensitive hybrid network model is also constructed. Its significance is the analysis of bridge reinforcement performance data. The experimental results show that compared with the cross-entropy loss function, the introduction of a cost-sensitive loss function can significantly improve the monitoring accuracy of abnormal data sets. The ACC– score, G-means and F-measures are higher than those of the specific cross-entropy loss function. The permissible bearing capacity of muddy clay determined using a network model complies with engineering standards in the evaluation of bridge reinforcement performance. As a result, the reliability of this form of reinforcement is improved.

A data-driven inspection method for identifying container bookings with concealed hazardous materials

Hazardous materials (a.k.a. hazmats) are transported by containers with special equipment in transportation engineering but must be declared prior to shipment. Some shippers deliberately conceal hazmat information in their container bookings to gain a higher profit, which has potentially serious consequences. To reduce such risk, the inspection office selects a part of the containers for hazmat concealment inspection, but the efficiency is low. In this study, a data-driven inspection method is proposed for identifying container bookings with concealed hazmats by exploring the relationship between inspection results and items in the container booking information profile filled by shippers. A tailored cost-sensitive loss function is designed to address the class imbalance issue. The application of the proposed method is validated by case studies based on the hazmat inspection database from Ningbo Ocean Shipping Company. The findings provide instructive implications on the identification of subsequent container bookings that are of high interest for inspection.

Machinery fault diagnosis using multi-scale feature focused network and adaptive cost-sensitive loss towards imbalanced dataset

The current deep learning based machinery fault diagnosis models still face challenges in effectively capturing rich multi-scale feature information and dynamically compensating training loss when dealing with imbalanced dataset. This paper presents a novel approach for machinery fault diagnosis using multi-scale feature focused network and adaptive cost-sensitive loss. Firstly, a multi-scale feature focused network is constructed with improved multi-scale CNN and point-wise attention mechanism module, in which the former can synthetically fuse the features at different scales to expand the coverage of the equivalent receptive field, and the latter can further refine fine-grained features and filter out irrelevant feature interference. Then, an adaptive cost-sensitive loss function is designed to adjust the cost matrix in the training process, dynamically assigning more loss weights for small samples that are difficult to distinguish. The experimental results of planetary gearbox fault diagnosis demonstrate that the proposed approach exhibits superior diagnostic performance compared to other existing methods.

Multi-class Cervical Cancer Classification using Transfer Learning-based Optimized SE-ResNet152 model in Pap Smear Whole Slide Images

Among the main factors contributing to death globally is cervical cancer, regardless of whether it can be avoided and treated if the afflicted tissues are removed early. Cervical screening programs must be made accessible to everyone and effectively, which is a difficult task that necessitates, among other things, identifying the population's most vulnerable members. Therefore, we present an effective deep-learning method for classifying the multi-class cervical cancer disease using Pap smear images in this research. The transfer learning-based optimized SE-ResNet152 model is used for effective multi-class Pap smear image classification. The reliable significant image features are accurately extracted by the proposed network model. The network's hyper-parameters are optimized using the Deer Hunting Optimization (DHO) algorithm. Five SIPaKMeD dataset categories and six CRIC dataset categories constitute the 11 classes for cervical cancer diseases. A Pap smear image dataset with 8838 images and various class distributions is used to evaluate the proposed method. The introduction of the cost-sensitive loss function throughout the classifier's learning process rectifies the dataset's imbalance. When compared to prior existing approaches on multi-class Pap smear image classification, 99.68% accuracy, 98.82% precision, 97.86% recall, and 98.64% F1-Score are achieved by the proposed method on the test set. For automated preliminary diagnosis of cervical cancer diseases, the proposed method produces better identification results in hospitals and cervical cancer clinics due to the positive classification results.

Cost-sensitive Tensor-based Dual-stage Attention LSTM with Feature Selection for Data Center Server Power Forecasting

Power forecasting has a guiding effect on power-aware scheduling strategies to reduce unnecessary power consumption in data centers. Many metrics related to power consumption can be collected in physical servers, such as the status of CPU, memory, and other components. However, most existing methods empirically exploit a small number of metrics to forecast power consumption. To this end, this article uses feature selection based on causality to explore the metrics that strongly influence the power consumption of different tasks. Moreover, we propose a tensor-based dual-stage attention LSTM to forecast the non-linear and non-periodic power consumption. In the proposed model, a multi-way delay embedding transform is utilized to convert the time series into tensors along the temporal direction. The LSTM combines with the tensor technique and the attention mechanism to capture the temporal pattern effectively. In addition, we adopt the cost-sensitive loss function to optimize the specific power forecasting problem in data centers. The experimental results demonstrate that our method can achieve up to 1.4% to 4.3% forecasting accuracy improvement compared with the state-of-the-art models.

Fast and Accurate Road Crack Detection Based on Adaptive Cost-Sensitive Loss Function.

Numerous detection problems in computer vision, including road crack detection, suffer from exceedingly foreground-background imbalance. Fortunately, modification of loss function appears to solve this puzzle once and for all. In this article, we propose a pixel-based adaptive weighted cross-entropy (WCE) loss in conjunction with Jaccard distance to facilitate high-quality pixel-level road crack detection. Our work profoundly demonstrates the influence of loss functions on detection outcomes and sheds light on the sophisticated consecutive improvements in the realm of crack detection. Specifically, to verify the effectiveness of the proposed loss, we conduct extensive experiments on four public databases, that is, CrackForest, AigleRN, Crack360, and BJN260. Compared to the vanilla WCE, the proposed loss significantly speeds up the training process while retaining the performance.

Traffic Forecasting via Dilated Temporal Convolution With Peak-Sensitive Loss

Deep learning-based traffic forecasting methods can capture intricate spatiotemporal features in traffic data and environmental factors. However, they have unsatisfactory performance around the minority peaks and are inefficient for modeling wide-range spatial correlations. This article gives a peak-aware deep learning architecture for traffic forecasting by involving a cost-sensitive loss function called peak-sensitive loss. This method can improve the performance since different costs are employed on the prevalent metrics such as mean-square loss and square of mean absolute percentage loss. A spatiotemporal convolutional architecture based on a dilated convolutional network (DCN) and a temporal convolutional network (TCN) is constructed that models the spatial features (both wide and short range) by the DCN and learns the time characteristics by the TCN. The effectiveness of the model is demonstrated with real-world data sets.

Deep Learning for Commodity Procurement: Nonlinear Data-Driven Optimization of Hedging Decisions

As the number of exchange-traded commodity contracts and their volatility increase, risk management through financial hedging gains importance for commodity-purchasing firms. Existing data-driven optimization approaches for hedging decisions include linear regression-based techniques. As such, they assume linear price–feature relationships and, thus, do not automatically detect nonlinear feature effects. We propose an alternative, nonlinear data-driven approach to commodity procurement based on deep learning. The prescriptive algorithm uses artificial neural networks to allow for universal approximation and requires no a priori knowledge regarding underlying price processes. We reformulate the periodic review procurement problem as a multilabel time series classification problem as the optimal bang-bang type procurement policy allows us to treat the hedging decision for each demand period as an individual subproblem that is independent of the other periods. Thereby, we are differentiating between optimal and suboptimal hedging decisions in each period and introduce a novel opportunity cost–sensitive loss function. We train maximum likelihood classifiers based on different deep learning architectures and test their performance in numerical experiments and case studies for natural gas, crude oil, nickel, and copper procurement. We show comparable performance to the state of the art for linear price–feature relationships and considerable advantages in the nonlinear case. Funding: Financial support received through the DFG as part of the AdONE GRK2201 [Grant 277991500] is gratefully acknowledged.

Adaptive cost-sensitive stance classification model for rumor detection in social networks

As online social networks are experiencing extreme popularity growth, determining the veracity of online statements denoted by rumors automatically as earliest as possible is essential to prevent the harmful effects of propagating misinformation. Early detection of rumors is facilitated by considering the wisdom of the crowd through analyzing different attitudes expressed toward a rumor (i.e., users’ stances). Stance detection is an imbalanced problem as the querying and denying stances against a given rumor are significantly less than supportive and commenting stances. However, the success of stance-based rumor detection significantly depends on the efficient detection of “query” and “deny” classes. The imbalance problem has led the previous stance classifier models to bias toward the majority classes and ignore the minority ones. Consequently, the stance and subsequently rumor classifiers have been faced with the problem of low performance. This paper proposes a novel adaptive cost-sensitive loss function for learning imbalanced stance data using deep neural networks, which improves the performance of stance classifiers in rare classes. The proposed loss function is a cost-sensitive form of cross-entropy loss. In contrast to most of the existing cost-sensitive deep neural network models, the utilized cost matrix is not manually set but adaptively tuned during the learning process. Hence, the contributions of the proposed method are both in the formulation of the loss function and the algorithm for calculating adaptive costs. The experimental results of applying the proposed algorithm to stance classification of real Twitter and Reddit data demonstrate its capability in detecting rare classes while improving the overall performance. The proposed method improves the mean F-score of rare classes by about 13% in RumorEval 2017 dataset and about 20% in RumorEval 2019 dataset.

Cost-Sensitive Learning for Anomaly Detection in Imbalanced ECG Data Using Convolutional Neural Networks.

Arrhythmia detection algorithms based on deep learning are attracting considerable interest due to their vital role in the diagnosis of cardiac abnormalities. Despite this interest, deep feature representation for ECG is still challenging and intriguing due to the inter-patient variability of the ECG’s morphological characteristics. The aim of this study was to learn a balanced deep feature representation that incorporates both the short-term and long-term morphological characteristics of ECG beats. For efficient feature extraction, we designed a temporal transition module that uses convolutional layers with different kernel sizes to capture a wide range of morphological patterns. Imbalanced data are a key issue in developing an efficient and generalized model for arrhythmia detection as they cause over-fitting to minority class samples (abnormal beats) of primary interest. To mitigate the imbalanced data issue, we proposed a novel, cost-sensitive loss function that ensures a balanced deep representation of class samples by assigning effective weights to each class. The cost-sensitive loss function dynamically alters class weights for every batch based on class distribution and model performance. The proposed method acquired an overall accuracy of 99.81% for intra-patient classification and 96.36% for the inter-patient classification of heartbeats. The experimental results reveal that the proposed approach learned a balanced representation of ECG beats by mitigating the issue of imbalanced data and achieved an improved classification performance as compared to other studies.

A federated learning based semi-supervised credit prediction approach enhanced by multi-layer label mean

Learning based credit prediction has attracted great interest from academia and industry. Different institutions hold a certain amount of credit data with limited users to build model. An institution has the requirement to obtain data from other institutions for improving model performance. However, due to the privacy protection and subject to legal restrictions, they encounter difficulties in data exchange. This affects the performance of credit prediction. In order to solve the above problem, this paper proposes a federated learning based semi-supervised credit prediction approach enhanced by multi-layer label mean, which can aggregate parameters of each institution via joint training while protecting the data privacy of each institution. Moreover, in actual production and life, there are usually more unlabeled credit data than labeled ones, and the distribution of their feature space presents multiple data-dense divisions. To deal with these, local meanNet model is proposed with a multi-layer label mean based semi-supervised deep learning network. In addition, this paper introduces a cost-sensitive loss function in the supervised part of the local mean model. Conducted on two public credit datasets, experimental results show that our proposed federated learning based approach has achieved promising credit prediction performance in terms of Accuracy and F1 measures. At the same time, the framework design mode that splits data aggregation and keys uniformly can improve the security of data privacy and enhance the flexibility of model training.

A Fault Diagnosis Method for Unbalanced Data Based on a Deep Cost Sensitive Convolutional Neural Network

For the research of industrial machinery intelligent fault diagnosis, traditional methods generally require that the samples share the same distribution or are balanced. In fact, the machines switch working conditions frequently during operation, accordingly resulting in changes in data distributions and the data can be unbalanced. To solve the above, combining transfer learning method, an intelligent diagnosis method for imbalanced data based on Deep Cost Sensitive Convolutional Neural Network is proposed. According to this model, the cost sensitive classification loss function adaptively assigns different error classification costs for all class, and it can mitigate data unbalancing. The domain adaptation is constructed by domain adversary and distance measure to automatically learn domain invariant features. The proposed method is verified by transfer experiments under various working conditions with three published bearing vibration data sets, and compared with other method under different imbalanced ratio. The results show that the proposed method is effectiveness and able to classify the imbalanced data.

Deep hierarchical LSTM networks with attention for video summarization

This paper studies the video summarization task by formulating it as a sequential decision-making process, in which the input is a sequence of video frames and the output is a subset of the original frames. Long Short-Term Memory (LSTM) is a commonly used framework of prior video summarization methods due to great temporal dependencies modeling ability. However, the frame sequence in the video summarization task is relatively long, and LSTM can only handle short video clips of up to 80 frames in length. This paper proposes a novel deep summarization framework named Deep Hierarchical LSTM Networks with Attention for Video Summarization (DHAVS) that consists of delicate feature extraction, temporal dependencies modeling, and video summary generation. Specifically, we employ 3D CNN instead of 2D CNN to extract spatial–temporal features and design an attention-based hierarchical LSTM module to capture the temporal dependencies among video frames. Additionally, we treat video summarization as an imbalanced class distribution problem and design a cost-sensitive loss function. Experimental results show that the proposed method has 0.7% ∼ 21.3% and 4.5% ∼ 12.2% improved to the conventional methods on SumMe and TVSum datasets.

Distributed Sparse Class-Imbalance Learning and Its Applications

In the present work, the study on class imbalance problems in a <i>distributed</i> setting exploiting sparsity structure in the data has been carried out. We formulate the class-imbalance learning problem as a cost-sensitive learning problem with <inline-formula><tex-math notation="LaTeX">$L_1$</tex-math></inline-formula> regularization. The cost-sensitive loss function is a cost-weighted smooth hinge loss. The resultant optimization problem is minimized within the <i>Distributed Alternating Direction Method of Multiplier</i> (DADMM) framework. We partition the data matrix across samples. This operation splits the original problem into a distributed <inline-formula><tex-math notation="LaTeX">$L_2$</tex-math></inline-formula> regularized smooth loss minimization and a <inline-formula><tex-math notation="LaTeX">$L_1$</tex-math></inline-formula> regularized squared loss minimization. <inline-formula><tex-math notation="LaTeX">$L_2$</tex-math></inline-formula> regularized subproblem is solved via Limited-memory Broyden-Fletcher-Goldfarb-Shanno (L-BFGS) and random coordinate descent method in parallel at multiple processing nodes using <i>MPI</i> whereas <inline-formula><tex-math notation="LaTeX">$L_1$</tex-math></inline-formula> regularized problem is just a simple soft-thresholding operation. We show, empirically, that the distributed solution approximates the centralized solution on many benchmark data sets. The centralized solution is obtained via Cost-Sensitive Stochastic Coordinate Descent (CSSCD). Empirical results on small and large-scale benchmark datasets show some promising avenues to further investigate the real-world applications of the proposed algorithms such as anomaly detection, class-imbalance learning, etc. To the best of our knowledge, ours is the first work to study class-imbalance in a <i>distributed</i> environment on large-scale <i>sparse</i> data.

P-035: AI-based models for the identification of critical genetic biomarkers to distinguish MM from MGUS using the WES data

<h3>Background</h3> Multiple Myeloma (MM) is preceded by the premalignant stage of Monoclonal Gammopathy of Undetermined Significance (MGUS) and therefore, it is important to identify the genetic factors responsible for progression of MGUS to MM. We have built machine learning (ML) models to identify pivotal genetic biomarkers that distinguish MM and MGUS. <h3>Methods</h3> Tumor normal matched paired Whole Exome Sequencing (WES) data of 1174 patients of MM and 61 patients of MGUS were analyzed. The data were obtained from dbGaP (phs000748; phs000348), AIIMS, Delhi, India, and EGA (EGAD00001001901). Variants were identified using four variant callers, namely, MuSE, Mutect2, VarScan2, and Somatic-Sniper and; SNVs were annotated using ANNOVAR. Pooled genomic annotations obtained were analyzed to derive significantly mutated genes with ‘dndscv' tool. Union of top ranked 250 significantly mutated genes from each variant caller yielded 1316 genes. For each gene, variant count and (maximum, mean, median, and standard deviation of) VAF and AD were used as features which were reduced by principal component analysis (PCA) and only top-3 principal components were selected for each gene. Next, 5 ML classifiers (random forest, decision tree, logistic regression, XGBoost, and SVM) were used to distinguish MM from MGUS. Imbalance of data (95% MM and 5% MGUS cases) was handled by the cost-sensitive loss function in the classifiers. Permutation based feature importance was carried out on top two performing models to infer the most significant features that were mapped back to genes to obtain the top ranking genes for MM and MGUS. <h3>Results</h3> Cost-sensitive SVM outperformed the rest of the models in balanced accuracy, weighted F1-score, Mathews correlation coefficient (MCC), precision, recall and area under curve (AUC) with values 95.5%, 94.82%, 0.8162, 76.49%, 98.33% and 95.5%, respectively. Top ranking genes identified for MM were: HLA-DQB1 IRF1, MUC6, FGFR3, MUC4, HOXA1, ITPR3, HIST1H1E, MUC12, ITGA2, HLA-DQA2, HUWE1, IGLL5, HLA-DRB5, HLA-DQB2, ILK. Top ranking genes identified for MGUS were: MUC3A, HLA-A, HLA-C, IRF4, JAK1, HDAC2, HLA-DQA1, FRG1, HS6ST1, H2AFV, and HLA-DRB1. HLA-DQB1, IRF1, ITPR3, HOXA1, HIST1H1E, HUWE1, IGLL5, HIPK3, HLA-DQA2, HLA-DRB5, and ILK were found significant for MM; and HLA-A, HLA-C, IRF4, JAK1, HDAC2 HLA-DQA1, HS6ST1, H2AFV, and HLA-DRB1 were found significant for MGUS by the top two ML classifiers. All these genes were found significant in the literature for MM and MGUS. <h3>Conclusion</h3> MGUS and MM share many common features such as genomic biomarkers, structural variants etc. with the difference of having less impact of mutations in MGUS. Thus, it is challenging to distinguish MM from MGUS. Here, we utilized ML classifiers to distinguish MM from MGUS. Our classifiers are able to identify the significant genes that are helpful in MM vs. MGUS classification that can lead to a better understanding of progression from MGUS to MM.

Improving Ponzi Scheme Contract Detection Using Multi-Channel TextCNN and Transformer.

With the development of blockchain technologies, many Ponzi schemes disguise themselves under the veil of smart contracts. The Ponzi scheme contracts cause serious financial losses, which has a bad effect on the blockchain. Existing Ponzi scheme contract detection studies have mainly focused on extracting hand-crafted features and training a machine learning classifier to detect Ponzi scheme contracts. However, the hand-crafted features cannot capture the structural and semantic feature of the source code. Therefore, in this study, we propose a Ponzi scheme contract detection method called MTCformer (Multi-channel Text Convolutional Neural Networks and Transofrmer). In order to reserve the structural information of the source code, the MTCformer first converts the Syntax Tree (AST) of the smart contract code to the specially formatted code token sequence via the Structure-Based Traversal (SBT) method. Then, the MTCformer uses multi-channel TextCNN (Text Convolutional Neural Networks) to learn local structural and semantic features from the code token sequence. Next, the MTCformer employs the Transformer to capture the long-range dependencies of code tokens. Finally, a fully connected neural network with a cost-sensitive loss function in the MTCformer is used for classification. The experimental results show that the MTCformer is superior to the state-of-the-art methods and its variants in Ponzi scheme contract detection.

COVID-19 early detection for imbalanced or low number of data using a regularized cost-sensitive CapsNet

With the presence of novel coronavirus disease at the end of 2019, several approaches were proposed to help physicians detect the disease, such as using deep learning to recognize lung involvement based on the pattern of pneumonia. These approaches rely on analyzing the CT images and exploring the COVID-19 pathologies in the lung. Most of the successful methods are based on the deep learning technique, which is state-of-the-art. Nevertheless, the big drawback of the deep approaches is their need for many samples, which is not always possible. This work proposes a combined deep architecture that benefits both employed architectures of DenseNet and CapsNet. To more generalize the deep model, we propose a regularization term with much fewer parameters. The network convergence significantly improved, especially when the number of training data is small. We also propose a novel Cost-sensitive loss function for imbalanced data that makes our model feasible for the condition with a limited number of positive data. Our novelties make our approach more intelligent and potent in real-world situations with imbalanced data, popular in hospitals. We analyzed our approach on two publicly available datasets, HUST and COVID-CT, with different protocols. In the first protocol of HUST, we followed the original paper setup and outperformed it. With the second protocol of HUST, we show our approach superiority concerning imbalanced data. Finally, with three different validations of the COVID-CT, we provide evaluations in the presence of a low number of data along with a comparison with state-of-the-art.

Risky Driver Recognition with Class Imbalance Data and Automated Machine Learning Framework.

Identifying high-risk drivers before an accident happens is necessary for traffic accident control and prevention. Due to the class-imbalance nature of driving data, high-risk samples as the minority class are usually ill-treated by standard classification algorithms. Instead of applying preset sampling or cost-sensitive learning, this paper proposes a novel automated machine learning framework that simultaneously and automatically searches for the optimal sampling, cost-sensitive loss function, and probability calibration to handle class-imbalance problem in recognition of risky drivers. The hyperparameters that control sampling ratio and class weight, along with other hyperparameters, are optimized by Bayesian optimization. To demonstrate the performance of the proposed automated learning framework, we establish a risky driver recognition model as a case study, using video-extracted vehicle trajectory data of 2427 private cars on a German highway. Based on rear-end collision risk evaluation, only 4.29% of all drivers are labeled as risky drivers. The inputs of the recognition model are the discrete Fourier transform coefficients of target vehicle’s longitudinal speed, lateral speed, and the gap between the target vehicle and its preceding vehicle. Among 12 sampling methods, 2 cost-sensitive loss functions, and 2 probability calibration methods, the result of automated machine learning is consistent with manual searching but much more computation-efficient. We find that the combination of Support Vector Machine-based Synthetic Minority Oversampling TEchnique (SVMSMOTE) sampling, cost-sensitive cross-entropy loss function, and isotonic regression can significantly improve the recognition ability and reduce the error of predicted probability.