Multi-label Research Articles

Conditional entropy-based hybrid DDoS detection model for IoT networks

In a distributed denial-of-service (DDoS) attack, an attacker channelizes the resources of a botnet to launch denial of service attack on the victim. The increased use of IoT devices and dependence of users on e-services like online shopping and online payments have elevated the liability risks. The entropy provides a significant measure of randomness. The variation in entropy of traffic features determines the presence of abrupt traffic. This paper uses entropy and conditional entropy to achieve insights on data and feeds it to the proposed 2-stage detection approach for multi-class classification. The proposed model employs four classifiers for first hand classification. Further, stacking generalization-based second stage achieves the final detection process. The recently launched CIC IoT 2023 dataset is used to illustrate the findings of the study. The proposed approach produces an accuracy of 99.86%. Further, this paper utilizes relative entropy for the determination of deflection of traffic behavior between the attack and legitimate samples. Comparisons have been made among symmetric versions of information divergence, ϕ-divergence and Kullback–Leibler divergence along with, Hellinger distance and total variation distance. It is found that the information distance gives a better differentiation between the entropy of legitimate traffic and attack traffic. Significance Statement Entropy has been manipulated to define the nature of incoming traffic for any rule-based detection. This work explores the significance of conditional entropy for the ML-based detection of DDoS attacks in a recently launched IoT-based dataset. Additionally, the effectiveness of KL-divergence, information divergence, ϕ-divergence, Hellinger distance and total variation distance is compared for differentiating between legitimate traffic and attack traffic.

HAIS-IDS:A Hybrid Artificial Immune System Model for Intrusion Detection in IoT

The application of the Internet of Things(IoT) is increasing exponentially, the dynamic data flow and distributive operation over low resource devices possesses huge threat to sensitive human data. This paper introduces an artificial immune system (AIS) based approach to intrusion detection in IoT network ecosystems, the proposed approach implements dual-layered AIS; which is robust to zero-day attacks and designed to adapt new types of attack classes in the form of antibodies.In this paper, a Hybrid method has been presented which uses Hybrid of Clonal Selection using Variation auto-encoders as Innate Immune Layer and Apaptive Dentritic Model for identifying intrusions over IoT Specific Datasets.Moreover we present extensive empirical analysis over six IoT network benchmark datasets for semi-supervised multi-class classification task and obtain superior performance compared to five state-of-the-art baselines. Finally, VC-ADIS achieves 99.83% accuracy over MQTT-set dataset.

Quantum neural networks under depolarization noise: exploring white-box attacks and defenses

AbstractLeveraging the unique properties of quantum mechanics, quantum machine learning (QML) promises computational breakthroughs and enriched perspectives where traditional systems reach their boundaries. However, similarly to classical machine learning, QML is not immune to adversarial attacks. Quantum adversarial machine learning has become instrumental in highlighting the weak points of QML models when faced with adversarial crafted feature vectors. Diving deep into this domain, our exploration shines a light on the interplay between depolarization noise and adversarial robustness. While previous results enhanced robustness from adversarial threats through depolarization noise, our findings paint a different picture. Interestingly, adding depolarization noise discontinued the effect of providing further robustness for a multi-class classification scenario. Consolidating our findings, we conducted experiments with a multi-class classifier adversarially trained on gate-based quantum simulators, further elucidating this unexpected behavior.

Electromagnetic side-channel attack risk assessment on a practical quantum-key-distribution receiver based on multi-class classification

While quantum key distribution (QKD) is a theoretically secure way of growing quantum-safe encryption keys, many practical implementations are challenged due to various open attack vectors, resulting in many variations of QKD protocols. Side channels are one such vector that allows a passive or active eavesdropper to obtain QKD information leaked through practical devices. This paper assesses the feasibility and implications of extracting the raw secret key from far-field radiated emissions from the single-photon avalanche diodes used in a BB84 QKD quad-detector receiver. Enhancement of the attack was also demonstrated through the use of deep-learning model to distinguish radiated emissions due to the four polarized encoding states. To evaluate the severity of such side-channel attack, multi-class classification based on raw-data and pre-processed data is implemented and assessed. Results show that classifiers based on both raw-data and pre-processed features can discern variations of the electromagnetic emissions caused by specific orientations of the detectors within the receiver with an accuracy higher than 90%. This research proposes machine learning models as a technique to assess EM information leakage risk of QKD and highlights the feasibility of side-channel attacks in the far-field region, further emphasizing the need to utilise mechanisms to avoid electromagnetic radiation information leaks and measurement-device-independent QKD protocols.

Predictive risk models for COVID-19 patients using the multi-thresholding meta-algorithm

This study aims to develop a Machine Learning model to assess the risks faced by COVID-19 patients in a hospital setting, focusing specifically on predicting the complications leading to Intensive Care Unit (ICU) admission or mortality, which are minority classes compared to the majority class of discharged patients. We operate within a multiclass framework comprising three distinct classes, and address the challenge of dataset imbalance, a common source of model bias. To effectively manage this, we introduce the Multi-Thresholding meta-algorithm (MTh), an innovative output-level methodology that extends traditional thresholding from binary to multiclass classification. This methodology dynamically adjusts class probabilities using misclassification costs, making it highly effective in imbalanced datasets. Our approach is further enhanced by integrating the simplicity, transparency, and effectiveness of Bayesian networks to create a robust predictive model. Using patient admission data, the model accurately identifies key risk and protective factors for COVID-19 outcomes. Our findings indicate that certain patient characteristics, such as high Charlson Index and pre-existing conditions, significantly influence the risk of ICU admission and mortality. Moreover, we introduce an explanatory model that elucidates the interrelationships among these factors, demonstrating the influence of therapeutic limits on the overall risk assessment of COVID-19 patients. Overall, our research provides a significant contribution to the field of Machine Learning by offering a novel solution for multiclass classification in the context of imbalanced datasets. This model not only enhances predictive accuracy but also supports critical decision-making processes in healthcare, potentially improving patient outcomes and optimizing clinical resource allocation.

Possibility of Using the Attention Mechanism in Multimodal Recognition of Cardiovascular Diseases

The paper is about studying the possibility of using the attention mechanism in diagnosing various cardiovascular diseases. Biomedical data were presented in different modalities (text, images, and time series). A comparison of the efficiency of 5 transformers based on the attention mechanism (Dosovitsky transformer, compact convolutional trans­former, transformer with external attention, transformer based on tokenization with patch shift and local self-attention, transformer based on multiple deep attention) was carried out with the Exception convolutional neural network, three fully connected neural networks (MLP-Mixer, Fnet, and gMLP), and the YOLO architecture on the problem of multi-class classification (16 classes of dangerous arrhythmias). High efficiency of transformers in diagnosing cardiac diseases was shown. The transformer based on tokenization with patch shift and local self-attention showed the greatest efficiency.

Neyman-Pearson Multi-Class Classification via Cost-Sensitive Learning

Most existing classification methods aim to minimize the overall misclassification error rate. However, in applications such as loan default prediction, different types of errors can have varying consequences. To address this asymmetry issue, two popular paradigms have been developed: the Neyman-Pearson (NP) paradigm and the cost-sensitive (CS) paradigm. Previous studies on the NP paradigm have primarily focused on the binary case, while the multi-class NP problem poses a greater challenge due to its unknown feasibility. In this work, we tackle the multi-class NP problem by establishing a connection with the CS problem via strong duality and propose two algorithms. We extend the concept of NP oracle inequalities, crucial in binary classifications, to NP oracle properties in the multi-class context. Our algorithms satisfy these NP oracle properties under certain conditions. Furthermore, we develop practical algorithms to assess the feasibility and strong duality in multi-class NP problems, which can offer practitioners the landscape of a multi-class NP problem with various target error levels. Simulations and real data studies validate the effectiveness of our algorithms. To our knowledge, this is the first study to address the multi-class NP problem with theoretical guarantees. The proposed algorithms have been implemented in the R package npcs, which is available on CRAN. Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work.

DMVMLC-VT: Deep incomplete multi-view multi-label image classification with view translation and pseudo-label enhancement

DMVMLC-VT: Deep incomplete multi-view multi-label image classification with view translation and pseudo-label enhancement

A novel random fast multi-label deep forest classification algorithm

With the development of big data, a large amount of multi-label data has emerged in various fields. The use of deep classification models to solve multi-label classification problems is the current research hotspot. However, the existing deep classification algorithms still face two great challenges in solving the multi-label classification problem: the poor multi-label classification accuracy and the high time cost. In order to solve the above challenges, this paper proposes a novel random fast multi-label deep forest classification algorithm (RFDF). Firstly, the label-by-label approach is adopted for the training and prediction of base classifiers, while the multi-label cascade model is constructed, so as to obtain better multi-label classification results. Then, an adaptive method is proposed to determine the number of base classifiers, and a random features partitioning strategy is provided to generate the initial base classifier, achieving parallel training of the base classifier. Finally, the proposed RFDF is compared with the existing advanced algorithms, and the experimental results show that the RFDF achieves better results on most of the metrics of the ten datasets, and obtains lower time cost compared to VDSDF and better stability.

BCDNet: A Deep Learning Model with Improved Convolutional Neural Network for Efficient Detection of Bone Cancer Using Histology Images

Among the several types of cancer, bone cancer is the most lethal prevailing in the world. Its prevention is better than cure. Besides early detection of bone cancer has potential to have medical intervention to prevent spread of malignant cells and help patients to recover from the disease. Many medical imaging modalities such as histology, histopathology, radiology, X-rays, MRIs, CT scans, phototherapy, PET and ultrasounds are being used in bone cancer detection research. However, hematoxylin and eosin stained histology images are found crucial for early diagnosis of bone cancer. Existing Convolutional Neural Network (CNN) based deep learning techniques are found suitable for medical image analytics. However, the models are prone to mediocre performance unless configured properly with empirical study. Within this article, we suggested a framework centered on deep learning for automatic bone cancer detection. We also proposed a CNN variant known as Bone Cancer Detection Network (BCDNet) which is configured and optimized for detection of a common kind of bone cancer named Osteosarcoma. An algorithm known as Learning based Osteosarcoma Detection (LbOD). It exploits BCDNet model for both binomial and multi-class classification. Osteosarcoma-Tumor-Assessment is the histology dataset used for our empirical study. Our the outcomes of the trial showed that BCDNet outperforms baseline models with 96.29% accuracy in binary classification and 94.69% accuracy in multi-class classification.

Deep active learning for multi label text classification

Given a set of labels, multi-label text classification (MLTC) aims to assign multiple relevant labels for a text. Recently, deep learning models get inspiring results in MLTC. Training a high-quality deep MLTC model typically demands large-scale labeled data. And comparing with annotations for single-label data samples, annotations for multi-label samples are typically more time-consuming and expensive. Active learning can enable a classification model to achieve optimal prediction performance using fewer labeled samples. Although active learning has been considered for deep learning models, there are few studies on active learning for deep multi-label classification models. In this work, for the deep MLTC model, we propose a deep Active Learning method based on Bayesian deep learning and Expected confidence (BEAL). It adopts Bayesian deep learning to derive the deep model’s posterior predictive distribution and defines a new expected confidence-based acquisition function to select uncertain samples for deep MLTC model training. Moreover, we perform experiments with a BERT-based MLTC model, where BERT can achieve satisfactory performance by fine-tuning in various classification tasks. The results on benchmark datasets demonstrate that BEAL enables more efficient model training, allowing the deep model to achieve training convergence with fewer labeled samples.

Sentiment Analysis of Hate Speech on Women in Social Media Platform Using Multi Label Classification

Sentiment Analysis of Hate Speech on Women in Social Media Platform Using Multi Label Classification

Label correlated contrastive learning for medical report generation

Background and Objective:Automatic generation of medical reports reduces both the burden on radiologists and the possibility of errors due to the inexperience of radiologists. The model that utilizes attention mechanism and contrastive learning can generate medical reports by capturing both general and specific semantics. However, existing contrastive learning methods ignore the specificity of medical data, that is, a patient may suffer from multiple diseases at the same time. This means that the lack of fine-grained relationships for contrastive learning will lead to the problem of insufficient specificity. Methods:To address the above problem, a label correlated contrastive learning method is proposed to encourage the model to generate higher-quality reports. Firstly, the refined similarity description matrix of the contrastive relationship between the reports is obtained by calculating the similarities between the multi-label classification of the reports. Secondly, the representations of image features and the embeddings containing semantic information from the decoder are projected into a hidden space. Thirdly, label correlated contrastive learning is performed with the hidden representations of the image, the embeddings of the text, and the similarity matrix. Through contrastive learning, the “hard” negative samples that share more labels with the target sample are being assigned more weights. Finally, label correlated contrastive learning and attention mechanism are combined to generate reports. Results:Comprehensive experiments are conducted on widely used datasets, IU X-ray and MIMIC-CXR. Specifically, on IU X-ray dataset, our method achieves METEOR and ROUGE-L scores of 0.198 and 0.392, respectively. On MIMIC-CXR dataset, our method achieves precision, recall, and F-1 scores of 0.384, 0.376, and 0.304, respectively. The results indicate that proposed method outperforms previous state-of-the-art models. Conclusions:This work improves the performance of automatically generating medical reports, making their application in computer-aided diagnosis feasible.

Automatic localization and deep convolutional generative adversarial network-based classification of focal liver lesions in computed tomography images: A preliminary study.

Computed tomography of the abdomen exhibits subtle and complex features of liver lesions, subjectively interpreted by physicians. We developed a deep learning-based localization and classification (DLLC) system for focal liver lesions (FLLs) in computed tomography imaging that could assist physicians in more robust clinical decision-making. We conducted a retrospective study (approval no. EMRP-109-058) on 1589 patients with 17335 slices with 3195 FLLs using data from January 2004 to December 2020. The training set included 1272 patients (male: 776, mean age 62±10.9), and the test set included 317 patients (male: 228, mean age 57±11.8). The slices were annotated by annotators with different experience levels, and the DLLC system was developed using generative adversarial networks for data augmentation. A comparative analysis was performed for the DLLC system versus physicians using external data. Our DLLC system demonstrated mean average precision at 0.81 for localization. The system's overall accuracy for multiclass classifications was 0.97 (95% confidence interval [CI]: 0.95-0.99). Considering FLLs≤3cm, the system achieved an accuracy of 0.83 (95% CI: 0.68-0.98), and for size>3cm, the accuracy was 0.87 (95% CI: 0.77-0.97) for localization. Furthermore, during classification, the accuracy was 0.95 (95% CI: 0.92-0.98) for FLLs≤3cm and 0.97 (95% CI: 0.94-1.00) for FLLs>3cm. This system can provide an accurate and non-invasive method for diagnosing liver conditions, making it a valuable tool for hepatologists and radiologists.

Enhanced Multi-Class Skin Lesion Classification of Dermoscopic Images Using an Ensemble of Deep Learning Models

This study presents an advanced approach to multi-class skin lesion classification by leveraging an ensemble model comprising the Inception-V3, ResNet-50, and VGG16 architectures. The classification task focuses on categorizing skin lesions into distinct classes, including Melanoma, basal cell carcinoma (BCC), and squamous cell carcinoma (SCC), using the ISIC dataset, a comprehensive collection of dermoscopic images. In order to properly balance the dataset, the oversampling strategy is utilized, as some lesion types are underrepresented due to inherent imbalances in the dataset. By ensuring that the model is trained on a more representative dataset, this balancing improves the algorithm's capacity to categorize all lesion types properly and impartially. By combining the complementary features of ResNet-50, Inception-V3, and VGG16, the ensemble technique improves the overall classification performance. ResNet-50 is chosen for its deep feature extraction capabilities, which help capture fine details in lesion patterns. Inception-V3 is selected for its multi-scale processing, allowing it to effectively analyze lesions at varying resolutions and sizes. VGG16 is included due to its simple yet highly effective architecture for image classification tasks. The ensemble model with data augmentation significantly outperforms individual models in skin lesion classification for both the original and balanced ISIC datasets regarding accuracy, precision, recall, and F1-score. This method offers a robust solution for skin lesion classification, contributing to more accurate and reliable diagnostic tools in dermatology.

Robust Multi-Class Classification for Real-Time Agricultural Applications Using Efficient and Adaptive Deep Learning

Plant diseases severely affect agricultural productivity, necessitating accurate and rapid detection methods. This research presents a robust, multi-class plant disease classification framework using adaptive deep learning. We utilize pre-trained convolutional neural networks (CNNs), specifically Xception, InceptionResNetV2, InceptionV3, ResNet50, and the proposed EfficientNetB3-based Adaptive Augmented Deep Learning (EfficientNetB3-AADL) model. Our approach leverages transfer learning combined with extensive data augmentation and trimming techniques to enhance model performance and mitigate overfitting. The EfficientNetB3-AADL architecture incorporates convolutional and max pooling layers, regularization strategies, and a dense feature learning layer, optimized to classify 52 disease categories from a publicly available leaf image dataset. The model’s performance is extensively evaluated using metrics such as accuracy, precision, recall, and F1 score. Notably, EfficientNetB3-AADL achieves superior accuracy over 98%, outperforming other CNN models. The proposed methodology highlights the efficacy of compound scaling and adaptive data augmentation in ensuring robust and efficient disease classification, suitable for real-time agricultural applications. This advancement supports sustainable farming by offering a scalable, computationally efficient solution for early and accurate disease detection in diverse crop species.

IResNetDM: An interpretable deep learning approach for four types of DNA methylation modification prediction

MotivationAlthough several computational methods for predicting DNA methylation modifications have been developed, two main limitations persist: 1) All of the models are currently confined to binary predictors, which merely determine the presence or absence of DNA methylation modifications and thus prevent comprehensive analyses of the interrelations among varied modification types. Multi-class classification models for RNA modifications have been developed, and a comparable approach for DNA is essential. 2) Few previous studies offer adequate explanations of how models make decisions, instead relying on the extraction and visualization of attention matrices, which have identified few motifs and do not provide sufficient insights into the model decision-making process. ResultIn this study, we introduce the task of DNA methylation modification prediction as a multi-class classification problem for the first time. We present iResNetDM, a deep learning model that integrates Residual Networks (ResNet) with self-attention mechanisms. To the best of our knowledge, iResNetDM is the first model capable of distinguishing between four types of DNA methylation modifications. Our model not only demonstrates good performance across various DNA methylation modifications but can also capture relationships between different types of modifications. We used the integrated gradients technique to enhance the interpretability of the iResNetDM. This method can effectively elucidate the model’s decision-making process, thus enabling the successful identification of multiple motifs. Notably, our model displays remarkable robustness, and can effectively identify unique motifs across different methylation modifications. We also compared the motifs discovered in various modifications and found that some had notable sequence similarities, suggesting that they may be subject to different types of modifications. This finding highlights the potential importance of these motifs in gene regulation.

Trust my IDS: An explainable AI integrated deep learning-based transparent threat detection system for industrial networks

Industrial networks are vulnerable to various cyber threats that can compromise their Confidentiality, Integrity, and Availability (CIA). To counter the increasing frequency of such threats, we designed and developed an Explainable Artificial Intelligence (XAI) integrated Deep Learning (DL)-based threat detection system (XDLTDS). We first employ a Long-Short Term Memory-AutoEncoder (LSTM-AE) to encode IIoT data and mitigate inference attacks. Then, we introduce an Attention-based Gated Recurrent Unit (AGRU) with softmax for multiclass threat classification in IIoT networks. To address the black-box nature of DL-based IDS, we use the Shapley Additive Explanations (SHAP) mechanism to provide transparency and trust for the system’s decisions. This interpretation helps SOC analysts understand why specific events are flagged as malicious by the XDLTDS framework. Our approach reduces the risk of sensitive data and reputation loss. We also present a Software-Defined Networking (SDN)-based deployment architecture for the XDLTDS framework. Extensive experiments with the N-BaIoT, Edge-IIoTset, and CIC-IDS2017 datasets confirm the effectiveness of XDLTDS against existing frameworks in addressing modern cybersecurity challenges and protecting industrial networks.

Deep learning-based multilabel compound-fault diagnosis in centrifugal pumps

In this study, the issue of centrifugal pumps used in marine engineering is addressed, as they are susceptible to malfunction owing to long-term operation in highly corrosive seawater and extreme weather conditions, resulting in operational interruptions and safety risks. We propose a high-precision intelligent fault-diagnosis method for multiple fault types based on deep learning. In this method, continuous wavelet transform is firstly employed to extract signal time–frequency domain features. Subsequently, the Swin transformer model is used to process the wavelet time–frequency images converted from signals. Finally, multilabel classification methods are combined to diagnose various complex faults. The effectiveness of the proposed method is validated using a dataset obtained from simulation experiments pertaining to centrifugal-pump faults. The results show that the proposed method achieves 100% accuracy in diagnosing 27 types of faults and provides excellent diagnosis even under limited compound-fault samples, thus offering an efficient and practical method for fault diagnosis in centrifugal pumps used in marine engineering.

HELMET: A Hybrid Machine Learning Framework for Real-Time Prediction of Edema Trajectory in Large Middle Cerebral Artery Stroke.

Malignant cerebral edema occurs when brain swelling displaces and compresses vital midline structures within the first week of a large middle cerebral artery stroke. Early interventions such as hyperosmolar therapy or surgical decompression may reverse secondary injury but must be administered judiciously. To optimize treatment and reduce secondary damage, clinicians need strategies to frequently and quantitatively assess the trajectory of edema using updated, relevant information. However, existing risk assessment tools are limited by the absence of structured records capturing the evolution of edema and typically estimate risk at a single time point early in the admission, therefore failing to account for changes in variables over the following hours or days. To address this, we developed and validated dynamic machine learning models capable of accurately predicting the severity of midline structure displacement, an established indicator of malignant edema, in real-time. Our models can provide updated estimations as frequently as every hour, using data from structured time-varying patient records, radiographic text, and human-curated neurological characteristics. Our work resulted in two novel multi-class classification models, collectively named Hybrid Ensemble Learning Models for Edema Trajectory (HELMET), predicting the progression of midline shift over 8-hour (HELMET-8) and 24-hour windows (HELMET-24), respectively. HELMET combines transformer-based large language models with supervised ensemble learning, demonstrating the value of merging human expertise and multimodal health records in developing clinical risk scores. Both models were trained on a retrospective cohort of 15,696 observations from 623 patients hospitalized with large middle cerebral artery ischemic stroke and were externally validated using 3,713 observations from 60 patients at a separate hospital system. Our HELMET models are accurate and generalize effectively to diverse populations, achieving a cross-validated mean area under the receiver operating characteristic score of 96.6% in the derivation cohort and 92.5% in the external validation cohort. Moreover, our approach provides a framework for developing hybrid risk prediction models that integrate both human-extracted and algorithm-derived multi-modal inputs. Our work enables accurate estimation of complex, dynamic, and highly specific clinical targets, such as midline shift, in real-time, even when relevant structured information is limited in electronic health record databases.