Cost-sensitive Neural Network Research Articles

Data augmentation using conditional generative adversarial network (cGAN): applications for sewer condition classification and testing using different machine learning techniques

ABSTRACT The increasing availability of condition assessment data highlights the challenge of managing data imbalance in the asset management of aging infrastructure. Aging sewer pipes pose significant threats to health and the environment, underscoring the importance of proactive management practices to enhance asset maintenance and mitigate associated risks. While machine learning (ML) models are widely employed to model the complex deterioration process of sewer pipes, they face performance limitations when trained on imbalanced condition grade data. This paper addresses this issue by proposing a novel approach using conditional generative adversarial network (cGAN) for data augmentation. By generating synthetic data for minority classes, the skewed distribution of the sewer dataset is balanced, facilitating more robust and accurate predictive models. The utility of the proposed method is evaluated by training different ML classifiers, including neural network (NN), decision tree, quadratic discriminant analysis, Naïve Bayes, support vector machine (SVM), and K-nearest neighbor. Quadratic discriminant, Naïve Bayes, NN, and SVM classifiers demonstrated improvement. The cGAN-based data augmentation method also outperformed two other data imbalance handling techniques, random under-sampling, and cost-sensitive NN. Consequently, data generated by cGAN can effectively aid asset management by developing proactive classifiers that accurately predict pipes at a high risk of failure.

The impact of sport-specific physical fitness change patterns on lower limb non-contact injury risk in youth female basketball players: a pilot study based on field testing and machine learning.

Background: In recent years, identifying players with injury risk through physical fitness assessment has become a hot topic in sports science research. Although practitioners have conducted many studies on the relationship between physical fitness and the likelihood of injury, the relationship between the two remains indeterminate. Consequently, this study utilized machine learning to preliminary investigate the relationship between individual physical fitness tests and injury risk, aiming to identify whether patterns of physical fitness change have an impact on injury risk. Methods: This study conducted a retrospective analysis by extracting the records of 17 young female basketball players from the sport-specific physical fitness monitoring and injury registration database in Fujian Province. Sports-specific physical fitness tests included physical performance, physiological, biochemical, and subjective perceived responses. The data for each player was standardized individually using Z-scores. Synthetic minority over-sampling techniques and edited nearest neighbor algorithms were used to sample the training set to address the negative impact of class imbalance on model performance. Feature extraction was performed on the dataset using linear discriminant analysis, and the prediction model was constructed using the cost-sensitive neural network. Results: The 10 replicate 5-fold stratified cross-validation showed that the lower limb non-contact injury prediction model based on the cost-sensitive neural network had achieved good discrimination and calibration (average Precision: 0.6360; average Recall: 0.8700; average F2-Score: 0.7980; average AUC: 0.8590; average Brier-score: 0.1020), which could be well applied in training practice. According to the attribution analysis, agility and speed were important physical attributes that affect youth female basketball players' non-contact lower limb injury risk. Specifically, there was enhance in the performance of the 1-min double under, accompanied by an increase in urinary ketone and urinary blood levels following the agility test. The 3/4 basketball court sprint performance improved, while urinary protein and RPE levels decreased after the speed test. Conclusion: The sport-specific physical fitness change pattern can impact the lower limb non-contact injury risk of young female basketball players in Fujian Province, specifically in terms of agility and speed. These findings will provide valuable insights for planning athletes' physical training programs, managing fatigue, and preventing injuries.

CDA-SKAG: Predicting circRNA-disease associations using similarity kernel fusion and an attention-enhancing graph autoencoder.

Circular RNAs (circRNAs) constitute a category of circular non-coding RNA molecules whose abnormal expression is closely associated with the development of diseases. As biological data become abundant, a lot of computational prediction models have been used for circRNA-disease association prediction. However, existing prediction models ignore the non-linear information of circRNAs and diseases when fusing multi-source similarities. In addition, these models fail to take full advantage of the vital feature information of high-similarity neighbor nodes when extracting features of circRNAs or diseases. In this paper, we propose a deep learning model, CDA-SKAG, which introduces a similarity kernel fusion algorithm to integrate multi-source similarity matrices to capture the non-linear information of circRNAs or diseases, and construct a circRNA information space and a disease information space. The model embeds an attention-enhancing layer in the graph autoencoder to enhance the associations between nodes with higher similarity. A cost-sensitive neural network is introduced to address the problem of positive and negative sample imbalance, consequently improving our model's generalization capability. The experimental results show that the prediction performance of our model CDA-SKAG outperformed existing circRNA-disease association prediction models. The results of the case studies on lung and cervical cancer suggest that CDA-SKAG can be utilized as an effective tool to assist in predicting circRNA-disease associations.

COMPUTER-AIDED EARLY DIAGNOSIS OF ALZHEIMER’S DISEASE BASED ON COST-SENSITIVE AND DEEP RESIDUAL CONVOLUTIONAL NEURAL NETWORK

Alzheimer’s disease (AD) is a neurodegenerative illness of unclear pathogenic origin that is characterized in its early stages by a steady deterioration in memory and cognitive impairment. Mild cognitive impairment (MCI) is a preclinical stage of AD. If a patient is diagnosed with early MCI, he or she can take preventive actions before permanent brain damage occurs. Improper diagnosis of MCI may cause the patient to miss the best time for treatment and incur heavy cost. Cost-sensitivity refers to the significant difference in cost to the patient between a misdiagnosis of MCI as normal control (NC) or AD and a misdiagnosis of NC or AD as MCI. In order to give consideration to both accuracy and cost, a computer-aided diagnosis algorithm based on cost-sensitive, attention mechanism and deep residual convolutional neural network (CSAResnet) was proposed for AD early diagnosis from MRI images. MRI data were obtained from the open-access AD Neuroimaging Initiative (ADNI) database. The experimental results show that the CSAResnet algorithm can balance the reduction of the total misclassification cost and the improvement of the accuracy to distinguish AD and MCI patients from NC subjects. It can address multiple classes of cost-sensitive problems.

Prediction of diagnosis results of rheumatoid arthritis patients based on autoantibodies and cost-sensitive neural network

To analyze and evaluate the effectiveness of the detection of single autoantibody and combined autoantibodies in patients with rheumatoid arthritis (RA) and related autoimmune diseases and establish a machine learning model to predict the disease of RA. A total of 309 patients with joint pain as the first symptom were retrieved from the database. The effectiveness of single and combined antibodies tests was analyzed and evaluated in patients with RA, a cost-sensitive neural network (CSNN) model was used to integrate multiple autoantibodies and patient symptoms to predict the diagnosis of RA, and the ROC curve was used to analyze the diagnosis performance and calculate the optimal cutoff value. There are differences in the seropositive rate of autoimmune diseases, the sensitivity and specificity of single or multiple autoantibody tests were insufficient, and anti-CCP performed best in RA diagnosis and had high diagnostic value. The cost-sensitive neural network prediction model had a sensitivity of up to 0.90 and specificity of up to 0.86, which was better than a single antibody and combined multiple antibody detection. In-depth analysis of autoantibodies and reliable early diagnosis based on the neural network could guide specialized physicians to develop different treatment plans to prevent deterioration and enable early treatment with antirheumatic drugs for remission. Key Points • There are differences in the seropositive rate of autoimmune diseases. • This is the first study to use a cost-sensitive neural network model to diagnose RA disease in patients. • The diagnosis effect of the cost-sensitive neural network model is better than a single antibody and combined multiple antibody detection.

Heterogeneous graph attention network based on meta-paths for lncRNA–disease association prediction

Discovering long noncoding RNA (lncRNA)-disease associations is a fundamental and critical part in understanding disease etiology and pathogenesis. However, only a few lncRNA-disease associations have been identified because of the time-consuming and expensive biological experiments. As a result, an efficient computational method is of great importance and urgently needed for identifying potential lncRNA-disease associations. With the ability of exploiting node features and relationships in network, graph-based learning models have been commonly utilized by these biomolecular association predictions. However, the capability of these methods in comprehensively fusing node features, heterogeneous topological structures and semantic information is distant from optimal or even satisfactory. Moreover, there are still limitations in modeling complex associations between lncRNAs and diseases. In this paper, we develop a novel heterogeneous graph attention network framework based on meta-paths for predicting lncRNA-disease associations, denoted as HGATLDA. At first, we conduct a heterogeneous network by incorporating lncRNA and disease feature structural graphs, and lncRNA-disease topological structural graph. Then, for the heterogeneous graph, we conduct multiple metapath-based subgraphs and then utilize graph attention network to learn node embeddings from neighbors of these homogeneous and heterogeneous subgraphs. Next, we implement attention mechanism to adaptively assign weights to multiple metapath-based subgraphs and get more semantic information. In addition, we combine neural inductive matrix completion to reconstruct lncRNA-disease associations, which is applied for capturing complicated associations between lncRNAs and diseases. Moreover, we incorporate cost-sensitive neural network into the loss function to tackle the commonly imbalance problem in lncRNA-disease association prediction. Finally, extensive experimental results demonstrate the effectiveness of our proposed framework.

A machine learning approach for building an adaptive, real-time decision support system for emergency response to road traffic injuries

In this paper, historical data about road traffic accidents are utilized to build a decision support system for emergency response to road traffic injuries in real-time. A cost-sensitive artificial neural network with a novel heuristic cost matrix has been used to build a classifier capable of predicting the injury severity of occupants involved in crashes. The proposed system was designed to be used by the medical services dispatchers to better assess the severity of road traffic injuries, and therefore to better decide the most appropriate emergency response. Taking into account that the nature of accidents may change over time due to several reasons, the system enables users to build an updated version of the prediction model based on the historical and newly reported accidents. A dataset of accidents that occurred over a 6-year period (2008-2013) has been used for demonstration purposes throughout this paper. The accuracy of the prediction model was 65%. The Area Under the Curve (AUC) showed that the generated classifier can reasonably predict the severity of road traffic injuries. Importantly, using the cost-sensitive learning technique, the predictor overcame the problem of imbalanced severity distributions which are inherent in traffic accident datasets.

A Novel Method for Credit Scoring Based on Cost-Sensitive Neural Network Ensemble

Most existing studies on credit scoring adapted a concept of classifier ensemble for solving an imbalanced dataset. They apply resampling methods to generate multiple training subsets for constructing multiple base classifiers. However, this approach leads to several problems that degrade the classification performance, such as problems of information loss, model overfitting, and computational cost. Thus, we propose a novel ensemble approach for developing a credit scoring model based on a cost-sensitive neural network, called Cost-sensitive Neural Network Ensemble (CS-NNE). In the proposed approach, multiple class weights are adapted to original training data, enabling the multiple base neural networks to consider imbalanced classes. Following this approach, a high diversity of multiple base classifiers without consequent problems can be achieved. The approach's effectiveness is evaluated on five real-world credit datasets. Among them is a loan-requesting dataset provided by a financial institution in Thailand. The remaining datasets are publicly available and widely used by several existing studies. The experimental results showed that the proposed CS-NNE approach improves the predictive performance over a single neural network based on imbalanced credit datasets, e.g., Thai credit dataset, by achieving 1.36%, 15.67%, and 6.11% Area under the ROC Curve (AUC), Default Detection Rate (DDR), and G-Mean (GM), respectively, and achieving the best Misclassification Cost (MC). The proposed CS-NNE approach can effectively solve a class of imbalance problems and outperform many existing models. The prediction model can well compromise between classes of default (bad credit applicants) and non-default (good credit applicants), whereas existing approaches preferred a class of non-default over default loans (having high specificity and low DDR), resulting in NPL.

Global optimization based on active preference learning with radial basis functions

This paper proposes a method for solving optimization problems in which the decision-maker cannot evaluate the objective function, but rather can only express a preference such as “this is better than that” between two candidate decision vectors. The algorithm described in this paper aims at reaching the global optimizer by iteratively proposing the decision maker a new comparison to make, based on actively learning a surrogate of the latent (unknown and perhaps unquantifiable) objective function from past sampled decision vectors and pairwise preferences. A radial-basis function surrogate is fit via linear or quadratic programming, satisfying if possible the preferences expressed by the decision maker on existing samples. The surrogate is used to propose a new sample of the decision vector for comparison with the current best candidate based on two possible criteria: minimize a combination of the surrogate and an inverse weighting distance function to balance between exploitation of the surrogate and exploration of the decision space, or maximize a function related to the probability that the new candidate will be preferred. Compared to active preference learning based on Bayesian optimization, we show that our approach is competitive in that, within the same number of comparisons, it usually approaches the global optimum more closely and is computationally lighter. Applications of the proposed algorithm to solve a set of benchmark global optimization problems, for multi-objective optimization, and for optimal tuning of a cost-sensitive neural network classifier for object recognition from images are described in the paper. MATLAB and a Python implementations of the algorithms described in the paper are available at http://cse.lab.imtlucca.it/~bemporad/glis.

Effectiveness of Focal Loss for Minority Classification in Network Intrusion Detection Systems

As the rapid development of information and communication technology systems offers limitless access to data, the risk of malicious violations increases. A network intrusion detection system (NIDS) is used to prevent violations, and several algorithms, such as shallow machine learning and deep neural network (DNN), have previously been explored. However, intrusion detection with imbalanced data has usually been neglected. In this paper, a cost-sensitive neural network based on focal loss, called the focal loss network intrusion detection system (FL-NIDS), is proposed to overcome the imbalanced data problem. FL-NIDS was applied using DNN and convolutional neural network (CNN) to evaluate three benchmark intrusion detection datasets that suffer from imbalanced distributions: NSL-KDD, UNSW-NB15, and Bot-IoT. The results showed that the proposed algorithm using FL-NIDS in DNN and CNN architecture increased the detection of intrusions in imbalanced datasets compared to vanilla DNN and CNN in both binary and multiclass classifications.

Inferring LncRNA-disease associations based on graph autoencoder matrix completion

Accumulating studies have indicated that long non-coding RNAs (lncRNAs) play crucial roles in large amount of biological processes. Predicting lncRNA-disease associations can help biologist to understand the molecular mechanism of human disease and benefit for disease diagnosis, treatment and prevention. In this paper, we introduce a computational framework based on graph autoencoder matrix completion (GAMCLDA) to identify lncRNA-disease associations. In our method, the graph convolutional network is utilized to encode local graph structure and features of nodes for learning latent factor vectors of lncRNA and disease. Further, the inner product of lncRNA factor vector and disease factor vector is used as decoder to reconstruct the lncRNA-disease association matrix. In addition, the cost-sensitive neural network is utilized to deal with the imbalance between positive and negative samples. The experimental results show GAMLDA outperforms other state-of-the-art methods in prediction performance which is evaluated by AUC value, AUPR value, PPV and F1-score. Moreover, the case study shows our method is the effectively tool for potential lncRNA-disease prediction.

Software defect prediction via cost-sensitive Siamese parallel fully-connected neural networks

Software defect prediction (SDP) has caused widespread concern among software engineering researchers, which aims to erect a software defect predictor according to historical data. However, it is still difficult to develop an effective SDP model on high-dimensional and limited data. In this study, a novel SDP model for this problem is proposed, called Siamese parallel fully-connected networks (SPFCNN), which combines the advantages of Siamese networks and deep learning into a unified method. And training this model is administered by AdamW algorithm for finding the best weights. The minimum value of a singular formula is the target of training for SPFCNN model. Significantly, we extensively compared SPFCNN method with the state-of-the-art SDP approaches using six openly available datasets from the NASA repository. Six indexes are used to evaluate the performance of the proposed method. Experimental results showed that the SPFCNN method contributes to significantly higher performance compared with benchmarked SDP approaches, indicating that a cost-sensitive neural network could be developed successfully for SDP.

Recognition of Coronary Atherosclerotic Plaque Tissue on Intravascular Ultrasound Images by Using Misclassification Sensitive Training of Discriminative Restricted Boltzmann Machine

Coronary atherosclerotic plaque has been extensively studied in pathological research. Improving the evaluation of vulnerable rupture is important to prevent acute heart failure. Intravascular ultrasound (IVUS) method is one of techniques to acquire information about atherosclerotic plaque, which is backscattered ultrasound signal sensed by IVUS transducer. The vessel structure and tissue components are then characterized in relation to the acquired signals. In this study, eight human coronary vessel sections are involved, and we use discriminative restricted Boltzmann machine (RBM) to classify coronary tissues as a target classifier. The quantization domain of IVUS signals are used to extract binary features for adapting Gaussian model of RBM. In addition, we propose a misclassification sensitive training of disRBM to deal with the class imbalances. The results are compared to the conventional integrated backscattered IVUS method (IB-IVUS) and the cost sensitive neural network for the same tasks.

Automatic diagnosis of imbalanced ophthalmic images using a cost-sensitive deep convolutional neural network

BackgroundOcular images play an essential role in ophthalmological diagnoses. Having an imbalanced dataset is an inevitable issue in automated ocular diseases diagnosis; the scarcity of positive samples always tends to result in the misdiagnosis of severe patients during the classification task. Exploring an effective computer-aided diagnostic method to deal with imbalanced ophthalmological dataset is crucial.MethodsIn this paper, we develop an effective cost-sensitive deep residual convolutional neural network (CS-ResCNN) classifier to diagnose ophthalmic diseases using retro-illumination images. First, the regions of interest (crystalline lens) are automatically identified via twice-applied Canny detection and Hough transformation. Then, the localized zones are fed into the CS-ResCNN to extract high-level features for subsequent use in automatic diagnosis. Second, the impacts of cost factors on the CS-ResCNN are further analyzed using a grid-search procedure to verify that our proposed system is robust and efficient.ResultsQualitative analyses and quantitative experimental results demonstrate that our proposed method outperforms other conventional approaches and offers exceptional mean accuracy (92.24%), specificity (93.19%), sensitivity (89.66%) and AUC (97.11%) results. Moreover, the sensitivity of the CS-ResCNN is enhanced by over 13.6% compared to the native CNN method.ConclusionOur study provides a practical strategy for addressing imbalanced ophthalmological datasets and has the potential to be applied to other medical images. The developed and deployed CS-ResCNN could serve as computer-aided diagnosis software for ophthalmologists in clinical application.

Effects of misclassification costs on mapping mineral prospectivity

Mineral prospectivity mapping is a classification process because in a given study area, a specific region is classified as either a prospective or non-prospective area. The cost of false negative errors differs from the cost of false positive errors because false positive errors lead to wasting much more financial and material resources, whereas false negative errors result in the loss of mineral deposits. Traditional machine learning algorithms using for mapping mineral prospectivity are aimed to minimize classification errors and ignore the cost-sensitive effects. In this study, the effects of misclassification costs on mapping mineral prospectivity are explored. The cost-sensitive neural network (CSNN) for minimizing misclassification costs is applied to map Fe polymetallic prospectivity in China’s southwestern Fujian metalorganic belt (SFMB). A CSNN with a different cost ratio ranging from 1:10 to 10:1 was used to represent various misclassification costs. The cross-validation results indicated a lower misclassification cost compared to traditional neural networks through a threshold-moving based CSNN. The CSNN’s predictive results were compared to those of a traditional neural network, and the results demonstrate that the CSNN method is useful for mapping mineral prospectivity. The targets can be used to further explore undiscovered deposits in the study area.

Hybrid case-based reasoning system by cost-sensitive neural network for classification

Case-based reasoning (CBR) is an artificial intelligent approach to learning and problem-solving, which solves a target problem by relating past similar solved problems. But it faces the challenge of weights assignment to features to measure similarity between cases. There are many methods to overcome this feature weighting problem of CBR. However, neural network’s pruning is one of the powerful and useful methods to overcome this feature weighting problem, which extracts feature weights from trained neural network without losing the generality of training set by four popular mechanisms: sensitivity, activity, saliency and relevance. It is habitually assumed that the training sets used for learning are balanced. However, this hypothesis is not always true in real-world applications, and hence, the tendency is to yield classification models that are biased toward the overrepresented class. Therefore, a hybrid CBR system is proposed in this paper to overcome this problem, which adopts a cost-sensitive back-propagation neural network (BPNN) in network pruning to find feature weights. These weights are used in CBR. A single cost parameter is used by the cost-sensitive BPNN to distinguish the importance of class errors. A balanced decision boundary is generated by the cost parameter using prior information. Thus, the class imbalance problem of network pruning is overcome to improve the accuracy of the hybrid CBR. From the empirical results, it is observed that the performance of the proposed hybrid CBR system is better than the hybrid CBR by standard neural network. The performance of the proposed hybrid system is validated with seven datasets.

Financial Predictions Using Cost Sensitive Neural Networks for Multi-Class Learning

The interest in the localisation of wireless sensor networks has grown in recent years. A variety of machine-learning methods have been proposed in recent years to improve the optimisation of the complex behaviour of wireless networks. Network administrators have found that traditional classification algorithms may be limited with imbalanced datasets. In fact, the problem of imbalanced data learning has received particular interest. The purpose of this study was to examine design modifications to neural networks in order to address the problem of cost optimisation decisions and financial predictions. The goal was to compare four learning-based techniques using cost-sensitive neural network ensemble for multiclass imbalance data learning. The problem is formulated as a combinatorial cost optimisation in terms of minimising the cost using meta-learning classification rules for Naïve Bayes, J48, Multilayer Perceptions, and Radial Basis Function models. With these models, optimisation faults and cost evaluations for network training are considered.

Software defect prediction using cost-sensitive neural network

Software defect prediction model was built by Artificial Neural Network (ANN).ANN connection weights were optimized by Artificial Bee Colony (ABC).Parametric cost-sensitivity feature was added to ANN by using a new error function.Model was applied to five publicly available datasets from the NASA repository.Results were compared with other cost-sensitive and non-cost-sensitive studies. The software development life cycle generally includes analysis, design, implementation, test and release phases. The testing phase should be operated effectively in order to release bug-free software to end users. In the last two decades, academicians have taken an increasing interest in the software defect prediction problem, several machine learning techniques have been applied for more robust prediction. A different classification approach for this problem is proposed in this paper. A combination of traditional Artificial Neural Network (ANN) and the novel Artificial Bee Colony (ABC) algorithm are used in this study. Training the neural network is performed by ABC algorithm in order to find optimal weights. The False Positive Rate (FPR) and False Negative Rate (FNR) multiplied by parametric cost coefficients are the optimization task of the ABC algorithm. Software defect data in nature have a class imbalance because of the skewed distribution of defective and non-defective modules, so that conventional error functions of the neural network produce unbalanced FPR and FNR results. The proposed approach was applied to five publicly available datasets from the NASA Metrics Data Program repository. Accuracy, probability of detection, probability of false alarm, balance, Area Under Curve (AUC), and Normalized Expected Cost of Misclassification (NECM) are the main performance indicators of our classification approach. In order to prevent random results, the dataset was shuffled and the algorithm was executed 10 times with the use of n-fold cross-validation in each iteration. Our experimental results showed that a cost-sensitive neural network can be created successfully by using the ABC optimization algorithm for the purpose of software defect prediction.

A neural network algorithm for semi-supervised node label learning from unbalanced data

Given a weighted graph and a partial node labeling, the graph classification problem consists in predicting the labels of all the nodes. In several application domains, from gene to social network analysis, the labeling is unbalanced: for instance positive labels may be much less than negatives. In this paper we present COSNet (COst Sensitive neural Network), a neural algorithm for predicting node labels in graphs with unbalanced labels. COSNet is based on a 2-parameter family of Hopfield networks, and consists of two main steps: (1) the network parameters are learned through a cost-sensitive optimization procedure; (2) a suitable Hopfield network restricted to the unlabeled nodes is considered and simulated. The reached equilibrium point induces the classification of the unlabeled nodes. The restriction of the dynamics leads to a significant reduction in time complexity and allows the algorithm to nicely scale with large networks. An experimental analysis on real-world unbalanced data, in the context of the genome-wide prediction of gene functions, shows the effectiveness of the proposed approach.

A Normalized Probabilistic Expectation-Maximization Neural Network for Minimizing Bayesian Misclassification Cost Risk

In this paper, we propose a normalized semi-supervised probabilistic expectation-maximization neural network (PEMNN) that minimizes Bayesian misclassification cost risk. Using simulated and real-world datasets, we compare the proposed PEMNN with supervised cost sensitive probabilistic neural network (PNN), discriminant analysis (DA), mathematical integer programming (MIP) model and support vector machines (SVM) for different misclassification cost asymmetries and class biases. The results of our experiments indicate that the PEMNN performs better when class data distributions are normal or uniform. However, when class data distribution is exponential the performance of PEMNN deteriorates giving slight advantage to competing MIP, DA, PNN and SVM techniques. For real-world data with non-parametric distributions and mixed decision-making attributes (continuous and categorical), the PEMNN outperforms the PNN.