Multi-label Learning Research Articles

Abstract 4136032: Digital Biomarkers Associated With Coronary Artery Calcium And Traditional Risk Factors Extracted From Facial Photos Through Multi-Label Deep Learning For Detecting Coronary Artery Disease

Background: Biomarkers like coronary artery calcium (CAC) and traditional risk factors are well-validated for coronary artery disease (CAD) assessment but not always available, and with limited workup efficiency and potential radiation risk. Facial features contain biological information related to atherosclerosis. Aims: We aim to extract CAC and traditional risk factor-associated digital biomarkers from facial images and evaluate their value in predicting CAD status compared with conventional clinical approaches. Methods: Suspected individuals referred for confirmatory CAD evaluation across nine centers were included. Participants in one center constituted the derivation set. External validation included one dataset of participants from a different time period in the derivation center and the other dataset from the other eight centers. We developed a multi-label deep-learning model to extract digital biomarkers from facial photos based on a multi-dimensional label of CAC score and eight traditional risk factors to comprehensively represent coronary atherosclerosis risk profile. The extracted digital biomarkers were evaluated for both effectiveness in reflecting components of the multi-dimensional label and clinical value in predicting obstructive CAD. Results: A total of 13248 facial photos from 3312 eligible participants (mean age, 58.5 years; 517 [25.9%] female) were included. In external validation, a set of digital biomarkers (FacialCAD) were extracted, effectively reflecting components of the multi-dimensional label, especially for CAC stratification (CAC>0, AUC 0.919 [0.885 – 0.949]; CAC≥100, AUC 0.906 [0.876 – 0.933]) and nearly perfect prediction for the age and sex. The performance of the FacialCAD in predicting obstructive CAD (AUC 0.721 [0.694–0.748]) significantly outperformed two guideline-recommended CAD models (0.721 vs. 0.653, P<0.001; 0.721 vs. 0.686, P=0.032), with further significant improvement when combined with these models (△AUC= +0.085, P<0.001; △AUC= +0.068, P<0.001). Conclusion: FacialCAD, a set of CAC and risk factors-associated novel digital biomarkers extracted from facial images, exhibited superior and incremental value to conventional clinical approaches in predicting CAD status.

Multi-label domain adversarial reinforcement learning for unsupervised compound fault recognition

Multi-label domain adversarial reinforcement learning for unsupervised compound fault recognition

Boosting Adaptive Weighted Broad Learning System for Multi-Label Learning

Boosting Adaptive Weighted Broad Learning System for Multi-Label Learning

Regularized Multi-Label Learning Empowered Joint Activity Recognition and Indoor Localization With CSI Fingerprints

Regularized Multi-Label Learning Empowered Joint Activity Recognition and Indoor Localization With CSI Fingerprints

A multilabel deep learning model for the detection of conduction and rhythm disorders from PDF outputs of a widely available portable ECG Device

Abstract Background Wearable and portable devices with ECG capabilities can provide broad access to screening for cardiovascular disorders. However, most are approved for the detection of only atrial fibrillation. Most algorithms for detecting rhythm and conduction disorders are trained and tested on 12-lead ECGs or the lead I of clinical ECGs, with an unclear role for extending the diagnostic range of wearable and portable devices. Moreover, no current algorithms enable cardiovascular diagnosis on data available to end users, who largely have access to the PDF outputs of their data. Purpose To validate a novel artificial intelligence (AI) model trained to identify conduction disorders and arrhythmias on synthetic ECG outputs from lead I data of clinical ECGs, to detect the same disorders on cardiologist-annotated PDF outputs from the Alivecor KardiaMobile single-lead ECGs (Kardia ECGs). Methods We extracted lead I data from 3,076,352 clinical ECGs across 5 hospitals of a large U.S.-based hospital system from 2000-2024 and simulated wearable PDF outputs from commercial devices using a novel plotting approach that replicated the variations in plotted formats. Gold-standard cardiologist written diagnosis statements were processed to generate 8 clinical labels spanning conduction and rhythm disorders, and we developed a multilabel model to predict these labels using an EfficientNet-B3 convolutional neural network. We examined the performance of the model on 24,808 Kardia ECGs annotated by cardiologists. Results The model achieved a high performance on clinical ECGs in a held-out subset of the synthetic lead I ECG printouts resembling wearable outputs (AUC > 0.9 for all 8 labels). In the independent 24,808 Kardia ECGs, annotated by experts, the model achieved high performance across all labels. For the rhythm disorders atrial fibrillation or atrial flutter, premature atrial contraction, and premature ventricular contraction, the model had AUROCs of 0.99, 0.91, and 0.95, respectively. For the conduction disorders 1st-degree AV block and 2nd or 3rd-degree AV blocks, the model had AUROCs of 0.94 and 0.99, respectively. Finally, for supraventricular tachycardias, wide QRS, and paced ECGs, the model had AUROCs of 0.998, 0.98, and 0.98, respectively. Conclusions We demonstrate that a deep learning model trained to identify rhythm and conduction disorders based on synthetic ECGs where lead I data from clinical 12 lead ECGs are used to replicate real-world PDF outputs of wearable ECGs, generalizes to real-world wearable and handheld single-lead devices.

Partial Multi-Label Learning via Exploiting Instance and Label Correlations

The goal of partial multi-label learning is to induce a multi-label classifier from partial multi-label data where each instance is annotated with a number of candidate labels but only a subset of them are valid. Many of the existing studies either fail to fully utilize instance and label correlations to eliminate noisy labels or build an oversimplified multi-label classifier, both of which are unfavorable for the improvement of generalization performance. In this article, we put forward a novel model named Pml-ilc to learn a multi-label classifier from partial multi-label data. Specifically, Pml-ilc first encodes instances and labels into a compact semantic space and takes full advantage of instance and label correlations to eliminate noisy labels. Then, it induces a linear mapping from the feature space to the label space while exploiting label-specific features and instance correlations to facilitate the multi-label classifier learning process. Finally, the above two steps are combined into a joint optimization problem and an efficient alternating optimization procedure is developed to find a satisfactory solution. Extensive experiments show that Pml-ilc achieves superior performance on both real-world and synthetic partial multi-label datasets in terms of different evaluation metrics.

MMLmiRLocNet: miRNA Subcellular Localization Prediction based on Multi-view Multi-label Learning for Drug Design.

Identifying subcellular localization of microRNAs (miRNAs) is essential for comprehensive understanding of cellular function and has significant implications for drug design. In the past, several computational methods for miRNA subcellular localization is being used for uncovering multiple facets of RNA function to facilitate the biological applications. Unfortunately, most existing classification methods rely on a single sequencebased view, making the effective fusion of data from multiple heterogeneous networks a primary challenge. Inspired by multi-view multi-label learning strategy, we propose a computational method, named MMLmiRLocNet, for predicting the subcellular localizations of miRNAs. The MMLmiRLocNet predictor extracts multi-perspective sequence representations by analyzing lexical, syntactic, and semantic aspects of biological sequences. Specifically, it integrates lexical attributes derived from k-mer physicochemical profiles, syntactic characteristics obtained via word2vec embeddings, and semantic representations generated by pre-trained feature embeddings. Finally, module for extracting multi-view consensus-level features and specific-level features was constructed to capture consensus and specific features from various perspectives. The full connection networks are utilized as the output module to predict the miRNA subcellular localization. Experimental results suggest that MMLmiRLocNet outperforms existing methods in terms of F1, subACC, and Accuracy, and achieves best performance with the help of multi-view consensus features and specific features extract network.

External validation of an artificial intelligence multi-label deep learning model capable of ankle fracture classification

BackgroundAdvances in medical imaging have made it possible to classify ankle fractures using Artificial Intelligence (AI). Recent studies have demonstrated good internal validity for machine learning algorithms using the AO/OTA 2018 classification. This study aimed to externally validate one such model for ankle fracture classification and ways to improve external validity.MethodsIn this retrospective observation study, we trained a deep-learning neural network (7,500 ankle studies) to classify traumatic malleolar fractures according to the AO/OTA classification. Our internal validation dataset (IVD) contained 409 studies collected from Danderyd Hospital in Stockholm, Sweden, between 2002 and 2016. The external validation dataset (EVD) contained 399 studies collected from Flinders Medical Centre, Adelaide, Australia, between 2016 and 2020. Our primary outcome measures were the area under the receiver operating characteristic (AUC) and the area under the precision-recall curve (AUPR) for fracture classification of AO/OTA malleolar (44) fractures. Secondary outcomes were performance on other fractures visible on ankle radiographs and inter-observer reliability of reviewers.ResultsCompared to the weighted mean AUC (wAUC) 0.86 (95%CI 0.82–0.89) for fracture detection in the EVD, the network attained wAUC 0.95 (95%CI 0.94–0.97) for the IVD. The area under the precision-recall curve (AUPR) was 0.93 vs. 0.96. The wAUC for individual outcomes (type 44A-C, group 44A1-C3, and subgroup 44A1.1-C3.3) was 0.82 for the EVD and 0.93 for the IVD. The weighted mean AUPR (wAUPR) was 0.59 vs 0.63. Throughout, the performance was superior to that of a random classifier for the EVD.ConclusionAlthough the two datasets had considerable differences, the model transferred well to the EVD and the alternative clinical scenario it represents. The direct clinical implications of this study are that algorithms developed elsewhere need local validation and that discrepancies can be rectified using targeted training. In a wider sense, we believe this opens up possibilities for building advanced treatment recommendations based on exact fracture types that are more objective than current clinical decisions, often influenced by who is present during rounds.

Study on Methods Using Multi-Label Learning for the Classification of Compound Faults in Auxiliary Equipment Pumps of Marine Engine Systems

Study on Methods Using Multi-Label Learning for the Classification of Compound Faults in Auxiliary Equipment Pumps of Marine Engine Systems

Parallel–serial architecture with instance correlation label-specific features for multi-label learning

Feature extraction plays a crucial role in capturing data correlations, thereby improving the performance of multi-label learning models. Popular approaches mainly include feature space manipulation techniques, such as recursive feature elimination, and feature alternative techniques, such as label-specific feature extraction. However, the former does not utilize label information, while the latter does not consider correlation among instances. In this study, we propose a label-specific feature extraction approach embedding instance correlation by a joint loss function under a parallel–serial architecture (LSIC-PS). Our approach incorporates three main techniques. First, we employ a parallel isomorphic network to extract label-specific features, which are directly integrated into a serial network to enhance label correlation. Second, we introduce instance correlation to guide feature extraction in parallel networks, leveraging label information from other instances to improve generalization. Third, we design a parameter-setting strategy to control a new joint loss function, adapting its instance correlation proportion to different datasets. We conduct experiments on sixteen widely used datasets and compare the results of our approach with those of twelve popular algorithms. Across eight evaluation metrics, LSIC-PS demonstrates state-of-art performance in multi-label learning. The source code is available at github.com/fansmale/lsic-ps.

Predicting Multiple Outcomes Associated with Frailty based on Imbalanced Multi-label Classification.

Frailty syndrome is prevalent among the elderly, often linked to chronic diseases and resulting in various adverse health outcomes. Existing research has predominantly focused on predicting individual frailty-related outcomes. However, this paper takes a novel approach by framing frailty as a multi-label learning problem, aiming to predict multiple adverse outcomes simultaneously. In the context of multi-label classification, dealing with imbalanced label distribution poses inherent challenges to multi-label prediction. To address this issue, our study proposes a hybrid resampling approach tailored for handling imbalance problems in the multi-label scenario. The proposed resampling technique and prediction tasks were applied to a high-dimensional real-life medical dataset comprising individuals aged 65years and above. Several multi-label algorithms were employed in the experiment, and their performance was evaluated using multi-label metrics. The results obtained through our proposed approach revealed that the best-performing prediction model achieved an average precision score of 83%. These findings underscore the effectiveness of our method in predicting multiple frailty outcomes from a complex and imbalanced multi-label dataset.

Breaking the gap between label correlation and instance similarity via new multi-label contrastive learning

Multi-label text classification (MLTC) is a fundamental yet challenging task in natural language processing. Existing MLTC models mostly learn text representations and label correlations, separately; while the instance-level correlation, which is crucial for the classification is ignored. To rectify this, we propose a new multi-label contrastive learning model, that captures instance-level correlations, for the MLTC task. Specifically, we first learn label representations by using Graph Convolutional Network (GCN) on label co-occurrence graphs. We next learn text representations by taking label correlations into consideration. Through an attention mechanism, instance-level correlation can be established. To better utilize label correlations, we propose a new contrastive learning model, whose learning is guided by a new learning objective, to further refine label representations. We finally implement a k-NN mechanism, that identifies k nearest neighbors of a given text for final prediction. Intensive experimental studies over benchmark multi-label datasets demonstrate the effectiveness of our approach.

Cross-feature fusion speech emotion recognition based on attention mask residual network and Wav2vec 2.0

Speech Emotion Recognition (SER) has received widespread attention as a crucial way for understanding human emotional states. However, the impact of irrelevant information on speech signals and data sparsity limit the development of SER system. To address these issues, this paper proposes a framework that incorporates the Attentive Mask Residual Network (AM-ResNet) and the self-supervised learning model Wav2vec 2.0 to obtain AM-ResNet features and Wav2vec 2.0 features respectively, together with a cross-attention module to interact and fuse these two features. The AM-ResNet branch mainly consists of maximum amplitude difference detection, mask residual block, and an attention mechanism. Among them, the maximum amplitude difference detection and the mask residual block act on the pre-processing and the network, respectively, to reduce the impact of silent frames, and the attention mechanism assigns different weights to unvoiced and voiced speech to reduce redundant emotional information caused by unvoiced speech. In the Wav2vec 2.0 branch, this model is introduced as a feature extractor to obtain general speech features (Wav2vec 2.0 features) through pre-training with a large amount of unlabeled speech data, which can assist the SER task and cope with data sparsity problems. In the cross-attention module, AM-ResNet features and Wav2vec 2.0 features are interacted with and fused to obtain the cross-fused features, which are used to predict the final emotion. Furthermore, multi-label learning is also used to add ambiguous emotion utterances to deal with data limitations. Finally, experimental results illustrate the usefulness and superiority of our proposed framework over existing state-of-the-art approaches.

Deep learning-based gamma spectroscopic analysis considering multiple variables for in situ applications

Gamma spectroscopy in environments with fluctuating temperatures, magnetic fields, and radiation doses can lead to inaccurate material analysis and increased radiation exposure for workers. In this manuscript, we propose a deep learning model that reduces human intervention and remains robust against variables in uncontrolled environments, thereby enhancing the applicability of gamma spectroscopy in the field. The process of establishing a dataset to train, validate, and test the deep learning model involved simultaneous consideration of multiple variables, including gain shifts, detector energy resolutions, the number of mixed radioisotopes (RIs), mixed RI ratios, and statistical fluctuations of the spectrum. The model was trained using multi-label learning to perform RI identification and quantification simultaneously. Specifically, the RI quantification output of our model structure was designed to predict full-energy peak areas and mixed ratios for each RI. Highlighting the importance of a high-performance training dataset, the training and validation results of the gamma spectroscopy model were analyzed in depth. In the test results, organized by RI type and number of mixed RIs, the model demonstrated that following: even under complex conditions with multiple variables, all outcomes for RI identification and quantification were predicted with high accuracy and precision, exceeding 98%.

Pedestrian Re-Identification Based on Weakly Supervised Multi-Feature Fusion

This article proposes a weakly supervised multi-feature fusion pedestrian re-identification method, which introduces a multi-feature fusion mechanism to extract feature information from different layers into the same feature space and fuse them into the deep and shallow joint features. The goal is to fully utilize the rich information in the image and improve the performance and robustness of the pedestrian re-identification model. Secondly, by matching the target character with unprocessed surveillance videos, one only needs to know that the identity of a person appears in the video, without annotating the identity of a person in any of the frames of the video during the training process. This simplifies the annotation of training images by replacing accurate annotations with broad annotations; that is, it puts the pedestrian identities that appeared in the video in one package and assigns a video-level label to each package. This greatly reduces the annotation work and transforms this weakly supervised pedestrian re-identification challenge into a multi-instance and multi-label learning problem. The experimental results show that the method proposed in this paper is effective and can significantly improve mAP.

A novel multi-label classification deep learning method for hybrid fault diagnosis in complex industrial processes

Hybrid Fault Detection and Diagnosis, encompassing both individual and simultaneous faults, is an important solution needed in chemical process safety practice. We systematically examine the two perspectives of simultaneous faults in previous studies: multi-class and multi-label classification, and highlight the limitations of the former while demonstrating the efficacy of the latter. Then, a novel multi-label classification Hybrid Fault Transformer (mcHFT) model was put forward to address hybrid faults. Our model is capable of learning not only intrinsic features of individual faults but also their coupled relationships. Importantly, this work constitutes the first comprehensive evaluation of Hybrid FDD on the Tennessee Eastman (TE) process to our knowledge. The mcHFT model significantly enhances key performance indicators over existing models and introduces an adaptive strategy to reduce false positives. The dataset developed for this research is made available under an MIT license, contributing a valuable resource for future exploration in this field.

Multi-scale locality preserving projection for partial multi-view incomplete multi-label learning

Amidst advancements in feature extraction techniques, research on multi-view multi-label classifications has attracted widespread interest in recent years. However, real-world scenarios often pose a challenge where the completeness of multiple views and labels cannot be ensured. At present, only a handful of techniques have attempted to address the complex issue of partial multi-view incomplete multi-label classification, and the majority of these approaches overlook the significance of manifold structures between instances. To tackle these challenges, we propose a novel partial multi-view incomplete multi-label learning model, termed MSLPP. Differing from existing studies, MSLPP emphasizes retaining the effective inherent structure of data during the feature extraction process, thereby facilitating a richer semantic information extraction. Specifically, MSLPP captures and integrates four types of information: the distance and similarity information in the original feature space, and the distance and similarity information in the extracted feature space. Further, by adopting the graph embedding technique, it simultaneously preserves the intrinsic structure with multi-scale information through a constraint term. Moreover, taking into account the negative impact of the missing views on the model and the possible impact of missing views on the data inherent structure, we further propose a shielding strategy for missing views, which not only eliminates the negative effects of missing views on the model but also more accurately captures the inherent data structure. The experimental results on five widely recognized datasets indicate that the model performs better than many excellent methods.

Enhancing Urban Sustainability: Developing an Open-Source AI Framework for Smart Cities

To address the growing need for advanced tools that enable urban policymakers to conduct comprehensive cost-benefit analyses of traffic management changes, the Urbanite H2020 project has developed innovative artificial intelligence methods. Among them is a robust decision support system that assists policymakers in evaluating and selecting optimal urban mobility planning modifications by combining objective and subjective criteria. Utilising open-source microscopic traffic simulation tools, accurate digital models (or “digital twins”) of four pilot cities—Bilbao, Amsterdam, Helsinki, and Messina—were created, each addressing unique mobility challenges. These challenges include reducing private vehicle access in Bilbao’s city center, analysing the impact of increased bicycle traffic and population growth in Amsterdam, constructing a mobility-enhancing tunnel in Helsinki, and improving public transport connectivity in Messina. The research introduces five key innovations: the application of a consistent open-source simulation platform across diverse urban environments, addressing integration and consistency challenges; the pioneering use of Dexi for advanced decision support in smart cities; the implementation of advanced visualisations; and the integration of the machine learning tool, Orange, with a user-friendly GUI interface. These innovations collectively make complex data analysis accessible to non-technical users. By applying multi-label machine learning techniques, the decision-making process is accelerated by three orders of magnitude, significantly enhancing urban planning efficiency. The Urbanite project’s findings offer valuable insights into both anticipated and unexpected outcomes of mobility interventions, presenting a scalable, open-source AI-based framework for urban decision-makers worldwide.

A cross-domain person re-identification algorithm based on distribution-consistency and multi-label collaborative learning

A cross-domain person re-identification algorithm based on distribution-consistency and multi-label collaborative learning

Label distribution learning for compound facial expression recognition in‐the‐wild: A comparative study

AbstractHuman emotional states encompass both basic and compound facial expressions. However, current works primarily focus on basic expressions, consequently neglecting the broad spectrum of human emotions encountered in practical scenarios. Compound facial expressions involve the simultaneous manifestation of multiple emotions on an individual's face. This phenomenon reflects the complexity and richness of human states, where facial features dynamically convey a combination of feelings. This study embarks on a pioneering exploration of Compound Facial Expression Recognition (CFER), with a distinctive emphasis on leveraging the Label Distribution Learning (LDL) paradigm. This strategic application of LDL aims to address the ambiguity and complexity inherent in compound expressions, marking a significant departure from the dominant Single Label Learning (SLL) and Multi‐Label Learning (MLL) paradigms. Within this framework, we rigorously investigate the potential of LDL for a critical challenge in Facial Expression Recognition (FER): recognizing compound facial expressions in uncontrolled environments. We utilize the recently introduced RAF‐CE dataset, meticulously designed for compound expression assessment. By conducting a comprehensive comparative analysis pitting LDL against conventional SLL and MLL approaches on RAF‐CE, we aim to definitively establish LDL's superiority in handling this complex task. Furthermore, we assess the generalizability of LDL models trained on RAF‐CE by evaluating their performance on the EmotioNet and RAF‐DB Compound datasets. This demonstrates their effectiveness without domain adaptation. To solidify these findings, we conduct a comprehensive comparative analysis of 12 cutting‐edge LDL algorithms on RAF‐CE, S‐BU3DFE, and S‐JAFFE datasets, providing valuable insights into the most effective LDL techniques for FER in‐the‐wild.