Multi-label Classification Task Research Articles

Feature selection technique on convolutional neural network – multilabel classification task

Automated text-based recommendation, an artificial intelligence development, finds application in document analysis like job resumes. The classification of job resumes poses challenges due to the ambiguity in categorizing multiple potential jobs in a single application file, termed multi-label classification, deep learning, particularly convolutional neural networks (CNN), offers flexibility in enhancing feature representations. Despite its robust learning capabilities, the black-box design of deep learning lacks interpretability and demands a substantial number of parameters, requiring significant computational resources. The primary challenge in multilabel learning is the ambiguity of labels not fully explained by traditional equivalence relations. To address this, the research employs feature selection techniques, specifically the Chi-square method. The goal is to reduce features in deep learning models while considering label relevance in multi-label text classification, easing computational workload while preserving model performance. Experimental tests, both with and without the Chi-square feature selection technique on the dataset, underscore its substantial impact on the classification model's ability. The conclusion emphasizes the influence of the Chi-square feature selection technique on performance and computational time. In summary, the research underscores the importance of balancing computational efficiency and model interpretability, especially in complex multi-label classification tasks like job applications.

ICDXML: enhancing ICD coding with probabilistic label trees and dynamic semantic representations

Accurately assigning standardized diagnosis and procedure codes from clinical text is crucial for healthcare applications. However, this remains challenging due to the complexity of medical language. This paper proposes a novel model that incorporates extreme multi-label classification tasks to enhance International Classification of Diseases (ICD) coding. The model utilizes deformable convolutional neural networks to fuse representations from hidden layer outputs of pre-trained language models and external medical knowledge embeddings fused using a multimodal approach to provide rich semantic encodings for each code. A probabilistic label tree is constructed based on the hierarchical structure existing in ICD labels to incorporate ontological relationships between ICD codes and enable structured output prediction. Experiments on medical code prediction on the MIMIC-III database demonstrate competitive performance, highlighting the benefits of this technique for robust clinical code assignment.

Deep Double Incomplete Multi-View Multi-Label Learning With Incomplete Labels and Missing Views.

View missing and label missing are two challenging problems in the applications of multi-view multi-label classification scenery. In the past years, many efforts have been made to address the incomplete multi-view learning or incomplete multi-label learning problem. However, few works can simultaneously handle the challenging case with both the incomplete issues. In this article, we propose a new incomplete multi-view multi-label learning network to address this challenging issue. The proposed method is composed of four major parts: view-specific deep feature extraction network, weighted representation fusion module, classification module, and view-specific deep decoder network. By, respectively, integrating the view missing information and label missing information into the weighted fusion module and classification module, the proposed method can effectively reduce the negative influence caused by two such incomplete issues and sufficiently explore the available data and label information to obtain the most discriminative feature extractor and classifier. Furthermore, our method can be trained in both supervised and semi-supervised manners, which has important implications for flexible deployment. Experimental results on five benchmarks in supervised and semi-supervised cases demonstrate that the proposed method can greatly enhance the classification performance on the difficult incomplete multi-view multi-label classification tasks with missing labels and missing views.

Multi-source partial multi-label learning via tensor decomposition and nonconvex regularization

Multi-source partial multi-label learning (MSPMLL) aims to learn a multi-label classifier from training examples with multi-source features and multiple labels. Most of the existing MSPMLL models tackle each source independently to derive classifiers. Those methods cannot efficiently reveal the intrinsic structure and relationships among instances across different sources, nor deal with inconsistent labels caused by noise. To address those challenges, this paper proposes a tensor decomposition and nonconvex regularization approach to multi-source partial multi-label learning, nominated as MSPML-TLR. In this model, a hypergraph based affinity graph is constructed from each of multi-source data in the feature space. A tensor representation of multi-source multi-label data is proposed based on the affinity graphs. A multi-linear projection is learned to map the feature affinity space to a latent ground-truth label space via tensor robust principal component analysis. Furthermore, a notion called the parameterized tensorial logarithmic rank is designed to approximate the rank of the multi-linear projection and the latent ground-truth label tensor. Moreover, a hypergraph-based Laplacian regularization is imposed on each frontal slice of the latent ground-truth label tensor to preserve the topological structure of instances across multi-source feature spaces. An efficient algorithm is designed to implement MSPML-TLR and the convergence of the algorithm is rigorously proven. Extensive experiments on various real-world benchmark datasets demonstrate that the proposed model outperforms state-of-the-art methods in multi-label classification tasks.

Federated Multi-Label Learning (FMLL): Innovative Method for Classification Tasks in Animal Science.

Federated learning is a collaborative machine learning paradigm where multiple parties jointly train a predictive model while keeping their data. On the other hand, multi-label learning deals with classification tasks where instances may simultaneously belong to multiple classes. This study introduces the concept of Federated Multi-Label Learning (FMLL), combining these two important approaches. The proposed approach leverages federated learning principles to address multi-label classification tasks. Specifically, it adopts the Binary Relevance (BR) strategy to handle the multi-label nature of the data and employs the Reduced-Error Pruning Tree (REPTree) as the base classifier. The effectiveness of the FMLL method was demonstrated by experiments carried out on three diverse datasets within the context of animal science: Amphibians, Anuran-Calls-(MFCCs), and HackerEarth-Adopt-A-Buddy. The accuracy rates achieved across these animal datasets were 73.24%, 94.50%, and 86.12%, respectively. Compared to state-of-the-art methods, FMLL exhibited remarkable improvements (above 10%) in average accuracy, precision, recall, and F-score metrics.

A multi-label dataset and its evaluation for automated scoring system for cleanliness assessment in video capsule endoscopy.

An automated scoring system for cleanliness assessment during video capsule endoscopy (VCE) is presently lacking. The present study focused on developing an approach to automatically assess the cleanliness in VCE frames as per the latest scoring i.e., Korea-Canada (KODA). Initially, an easy-to-use mobile application called artificial intelligence-KODA (AI-KODA) score was developed to collect a multi-label image dataset of twenty-eight patient capsule videos. Three readers (gastroenterology fellows), who had been trained in reading VCE, rated this dataset in a duplicate manner. The labels were saved automatically in real-time. Inter-rater and intra-rater reliability were checked. The developed dataset was then randomly split into train:validate:test ratio of 70:20:10 and 60:20:20. It was followed by a comprehensive benchmarking and evaluation of three multi-label classification tasks using ten machine learning and two deep learning algorithms. Reliability estimation was found to be overall good among the three readers. Overall, random forest classifier achieved the best evaluation metrics, followed by Adaboost, KNeighbours, and Gaussian naive bayes in the machine learning-based classification tasks. Deep learning algorithms outperformed the machine learning-based classification tasks for only VM labels. Thorough analysis indicates that the proposed approach has the potential to save time in cleanliness assessment and is user-friendly for research and clinical use. Further research is required for the improvement of intra-rater reliability of KODA, and the development of automated multi-task classification in this field.

Leadwise clustering multi-branch network for multi-label ECG classification

The 12-lead electrocardiogram (ECG) is widely used for diagnosing cardiovascular diseases in clinical practice. Recently, deep learning methods have become increasingly effective for automatically classifying ECG signals. However, most current research simply combines the 12-lead ECG signals into a matrix without fully considering the intrinsic relationships between the leads and the heart's structure. To better utilize medical domain knowledge, we propose a multi-branch network for multi-label ECG classification and introduce an intuitive and effective lead grouping strategy. Correspondingly, we design multi-branch networks where each branch employs a multi-scale convolutional network structure to extract more comprehensive features, with each branch corresponding to a lead combination. To better integrate features from different leads, we propose a feature weighting fusion module. We evaluate our method on the PTB-XL dataset for classifying 4 arrhythmia types and normal rhythm, and on the China Physiological Signal Challenge 2018 (CPSC2018) database for classifying 8 arrhythmia types and normal rhythm. Experimental results on multiple multi-label datasets demonstrate that our proposed multi-branch network outperforms state-of-the-art networks in multi-label classification tasks

Imbalance-Robust Multi-Label Self-Adjusting kNN

In the task of multi-label classification in data streams, instances arriving in real time need to be associated with multiple labels simultaneously. Various methods based on the k Nearest Neighbors algorithm have been proposed to address this task. However, these methods face limitations when dealing with imbalanced data streams, a problem that has received limited attention in existing works. To approach this gap, this paper introduces the Imbalance-Robust Multi-Label Self-Adjusting kNN (IRMLSAkNN), designed to tackle multi-label imbalanced data streams. IRMLSAkNN’s strength relies on maintaining relevant instances with imbalance labels by using a discarding mechanism that considers the imbalance ratio per label. On the other hand, it evaluates subwindows with an imbalance-aware measure to discard older instances that are lacking performance. We conducted statistical experiments on 32 benchmark data streams, evaluating IRMLSAkNN against eight multi-label classification algorithms using common accuracy-aware and imbalance-aware measures. The obtained results demonstrate that IRMLSAkNN consistently outperforms these algorithms in terms of predictive capacity and time cost across various levels of imbalance.

Comparison between parameter-efficient techniques and full fine-tuning: A case study on multilingual news article classification.

Adapters and Low-Rank Adaptation (LoRA) are parameter-efficient fine-tuning techniques designed to make the training of language models more efficient. Previous results demonstrated that these methods can even improve performance on some classification tasks. This paper complements existing research by investigating how these techniques influence classification performance and computation costs compared to full fine-tuning. We focus specifically on multilingual text classification tasks (genre, framing, and persuasion techniques detection; with different input lengths, number of predicted classes and classification difficulty), some of which have limited training data. In addition, we conduct in-depth analyses of their efficacy across different training scenarios (training on the original multilingual data; on the translations into English; and on a subset of English-only data) and different languages. Our findings provide valuable insights into the applicability of parameter-efficient fine-tuning techniques, particularly for multilabel classification and non-parallel multilingual tasks which are aimed at analysing input texts of varying length.

Business text classification with imbalanced data and moderately large label spaces for digital transformation

Digital transformation refers to an organization’s use of digital technology to improve its products, services, and operations, aligning them with evolving business requirements. To demonstrate this transformative process, we present a real-life case study where a company seeks to automate the classification of their textual data rather than relying on manual methods. Transitioning to automated classification involves deploying machine learning models, which rely on pre-labeled datasets for training and making predictions on new data. However, upon receiving the dataset from the company, we faced challenges due to the imbalanced distribution of labels and moderately large label spaces. To tackle text classification with such a business dataset, we evaluated four distinct methods for multi-label text classification: fine-tuned Bidirectional Encoder Representations from Transformers (BERT), Binary Relevance, Classifier Chains, and Label Powerset. The results revealed that fine-tuned BERT significantly outperformed the other methods across key metrics like Accuracy, F1-score, Precision, and Recall. Binary Relevance also displayed competence in handling the dataset effectively, while Classifier Chains and Label Powerset exhibited comparatively less impressive performance. These findings highlight the remarkable effectiveness of fine-tuned BERT model and the Binary Relevance classifier in multi-label text classification tasks, particularly when dealing with imbalanced training datasets and moderately large label spaces. This positions them as valuable assets for businesses aiming to automate data classification in the digital transformation era.

Mutual Balancing in State-Object Components for Compositional Zero-Shot Learning

Compositional Zero-Shot Learning (CZSL) aims to recognize unseen compositions from seen states and objects. The disparity between the manually labeled semantic information and its actual visual features causes a significant imbalance of visual deviation in the distribution of various object classes and state classes, which is ignored by existing methods. To ameliorate these issues, we consider the CZSL task as an unbalanced multi-label classification task and propose a novel method called MUtual balancing in STate-object components (MUST) for CZSL, which provides a balancing inductive bias for the model. In particular, we split the classification of the composition classes into two consecutive processes to analyze the entanglement of the two components to get additional knowledge in advance, which reflects the degree of visual deviation between the two components. We use the knowledge gained to modify the model’s training process in order to generate more distinct class borders for classes with significant visual deviations. Extensive experiments demonstrate that our approach significantly outperforms the state-of-the-art on MIT-States, UT-Zappos, and C-GQA when combined with the basic CZSL frameworks, and it can improve various CZSL frameworks. Our code is available at https://github.com/LanchJL/MUST.

Taming the Sigmoid Bottleneck: Provably Argmaxable Sparse Multi-Label Classification

Sigmoid output layers are widely used in multi-label classification (MLC) tasks, in which multiple labels can be assigned to any input. In many practical MLC tasks, the number of possible labels is in the thousands, often exceeding the number of input features and resulting in a low-rank output layer. In multi-class classification, it is known that such a low-rank output layer is a bottleneck that can result in unargmaxable classes: classes which cannot be predicted for any input. In this paper, we show that for MLC tasks, the analogous sigmoid bottleneck results in exponentially many unargmaxable label combinations. We explain how to detect these unargmaxable outputs and demonstrate their presence in three widely used MLC datasets. We then show that they can be prevented in practice by introducing a Discrete Fourier Transform (DFT) output layer, which guarantees that all sparse label combinations with up to k active labels are argmaxable. Our DFT layer trains faster and is more parameter efficient, matching the F1@k score of a sigmoid layer while using up to 50% fewer trainable parameters. Our code is publicly available at https://github.com/andreasgrv/sigmoid-bottleneck.

Attention-Induced Embedding Imputation for Incomplete Multi-View Partial Multi-Label Classification

As a combination of emerging multi-view learning methods and traditional multi-label classification tasks, multi-view multi-label classification has shown broad application prospects. The diverse semantic information contained in heterogeneous data effectively enables the further development of multi-label classification. However, the widespread incompleteness problem on multi-view features and labels greatly hinders the practical application of multi-view multi-label classification. Therefore, in this paper, we propose an attention-induced missing instances imputation technique to enhance the generalization ability of the model. Different from existing incomplete multi-view completion methods, we attempt to approximate the latent features of missing instances in embedding space according to cross-view joint attention, instead of recovering missing views in kernel space or original feature space. Accordingly, multi-view completed features are dynamically weighted by the confidence derived from joint attention in the late fusion phase. In addition, we propose a multi-view multi-label classification framework based on label-semantic feature learning, utilizing the statistical weak label correlation matrix and graph attention network to guide the learning process of label-specific features. Finally, our model is compatible with missing multi-view and partial multi-label data simultaneously and extensive experiments on five datasets confirm the advancement and effectiveness of our embedding imputation method and multi-view multi-label classification model.

Covering variable precision fuzzy rough sets based on overlap functions and the application to multi-label classification

Fuzzy rough sets (FRSs) play a significant role in the field of data analysis, and one of the common methods for constructing FRSs is the use of the fuzzy logic operators. To further extend FRSs theory to more diverse information backgrounds, this article proposes a covering variable precision fuzzy rough set model based on overlap functions and fuzzy β-neighbourhood operators (OCVPFRS). Some necessary properties of OCVPFRS have also been studied in this work. Furthermore, multi-label classification is a prevalent task in the realm of machine learning. Each object (sample or instance) in multi-label data is associated with various labels (classes), and there are numerous features or attributes that need to be taken into account within the attribute space. To enhance various performance metrics in the multi-label classification task, attribute reduction is an essential pre-processing step. Therefore, according to overlap functions and fuzzy rough sets’ excellent work on applications: such as image processing and multi-criteria decision-making, we establish an attribute reduction method suitable for multi-label data based on OCVPFRS. Through a series of experiments and comparative analysis with existing multi-label attribute reduction methods, the effectiveness and superiority of the proposed method have been verified.

MMDL-Net: Multi-Band Multi-Label Remote Sensing Image Classification Model

High-resolution remote sensing imagery comprises spatial structure features of multispectral bands varying in scale, color, and shape. These heterogeneous geographical features introduce grave challenges to the fine segmentation required for classification applications in remote sensing imagery, where direct application of traditional image classification models fails to deliver optimal results. To overcome these challenges, a multispectral, multi-label model, MMDL-Net, has been developed. This model is integrated with the multi-label BigEarthNet dataset, primarily employed for land cover classification research in remote sensing imagery, with each image composed of 13 spectral bands and spatial resolutions of 10 m, 20 m, and 60 m. To effectively utilize the information across these bands, a multispectral stacking module has been introduced to concatenate this spectral information. To proficiently process three distinct large-scale remote sensing image datasets, a multi-label classification module has been incorporated for training and inference. To better learn and represent the intricate features within the images, a twin-number residual structure has been proposed. The results demonstrate that the MMDL-Net model achieves a top accuracy of 83.52% and an F1 score of 77.97%, surpassing other deep learning models and conventional methods, thereby exhibiting exceptional performance in the task of multispectral multi-label classification of remote sensing imagery.

Combining convolutional neural networks and spatial-channel “squeeze and excitation” block for multiple-label image classification

<span lang="EN-US">In emergency rooms and intensive care units, catheters and tubes are used to keep critically ill patients alive. Appropriate catheter or tube insertion is crucial to avoiding serious complications. Such issues can be rectified if they are identified early. Chest X-rays are commonly used to assess catheter placement. Convolutional neural networks (</span><span lang="IN">CNN</span><span lang="EN-US">)</span><span lang="EN-US"> have recently been found to enhance multi-label classification tasks on chest X-rays images. Furthermore, attention modules have shown the effect of enhancing spatial encoding on network feature maps. This research analyzed the experiments of each CNN model with different attention blocks. Resnet200D with batch normalization and spatial-channel squeeze and excitation block (BN+scSE) is the best architecture for multiple-label image classification on a chest X-rays dataset from National Institutes of Health Clinical Center (NIH) with multiple catheters and tubes. Then came EfficientNetB5 with BN+scSE and Inception_v3 with spatial squeeze and channel excitation block, respectively.</span>

Correlation concept-cognitive learning model for multi-label classification

As a cognitive process, concept-cognitive learning (CCL) emphasizes the structured expression of data through systematic cognition and understanding, to obtain valuable information in the data. Although concept-cognitive learning has achieved good results in single-label classification tasks, it has not yet been applied to multi-label learning. The difficulty is that the existing concept-cognitive learning fails to effectively associate and utilize the structural relationships in the feature concepts and the multi-label concepts. In addition, in the face of complex multi-label datasets, it is difficult to obtain effective concepts for classification tasks. To solve these problems, this paper proposes a correlation concept-cognitive learning method and applies it to the multi-label classification (MLC) task. Moreover, the relationship between the feature concepts and the multi-label concepts is established by extent. On this basis, we comprehensively consider the extent relevancy and intent relevancy, and learn correlation concepts. In order to improve the classification precision, we construct correlation concept spaces to obtain the representation of effective concepts. Finally, we conducted experimental evaluations on ten datasets to illustrate the effectivity and advantages of the proposed approach.

MULTI-LABEL CLASSIFICATION OF THORACIC DISEASES USING STRUCTURED DEEP LEARNING FRAMEWORK

The current medical scenario indicates that thoracic diseases are the primary cause of illnesses of human beings worldwide. COVID-19 outbreak results in the generation of tremendous Chest X-Ray (CXR) and Computed Tomography (CT) image data archives because CXR and CT imaging are the foremost effective screening tools that characterize patterns and features of pathologies present in the lungs. After careful observation, these chest radiographs show that images are often labeled with more than one pathology, that extends disease diagnosis problem to tedious multi-label classification task. So, this paper proposes a modified mish activation function and compound loss-based Separable Convolution Neural Network (SCNN) model that accomplishes image-level detection of multiple pathologies and COVID-19 from multi-modality chest radiographs. We investigate the power of separable convolution in the CNN network that interprets spatial and depth-wise dimensions of each pixel in CXR and CT images. A modified mish called as Swmish, a non-monotonic activation function, is introduced that preserves negative gradient flow of input and capture fine-grained detail of multiple regions. Moreover, a compound loss function of dice coefficient and binary cross entropy is applied that strongly optimizes the model during training. The generality of the proposed SCNN model is confirmed after conducting comprehensive experiments on multi-label CXR and COVID19-CT datasets. The proposed model gains an AUC score of 0.91 and loss of 0.13 on the multi-label CXR dataset and achieves a testing accuracy of 0.97, AUC score of 0.99 and a loss of 0.11 on the COVID19-CT dataset. These results show that the proposed model achieves highest evaluation metrics in classifying binary and multi-label thoracic abnormalities from multi-modality chest radiographs.

A Bayesian network learning method for sparse and unbalanced data with GNN-based multilabel classification application

Creating a well-defined Bayesian network (BN) is helpful for developing effective graph neural networks (GNNs) that exploit annotated labels in multilabel classification (MLC) tasks. Obtaining correct BNs can be challenging when the labels are highly unbalanced and sparse. This study proposes a novel scoring method to address data imbalance and sparsity by introducing an imbalanced weight term. Comparison results show that the BNs learned from the proposed scoring method are simpler and more accurate than the well-known methods. The obtained BNs based on the proposed method are used to construct GNNs for MLC tasks based on three datasets. The findings suggest that the adjacency matrices based on moral graphs of BNs exhibit more excellent stability than those derived from mathematical approaches. By selecting the parameter thoughtfully, GNNs based on the proposed BDsp scores showcase up to 12% classification performance improvement compared to GNNs based on existing Bayesian scores.

Gridless Underdetermined DOA Estimation for Mobile Agents with Limited Snapshots Based on Deep Convolutional Generative Adversarial Network

Deep learning techniques have made certain breakthroughs in direction-of-arrival (DOA) estimation in recent years. However, most of the current deep-learning-based DOA estimation methods view the direction finding problem as a grid-based multi-label classification task and require multiple samplings with a uniform linear array (ULA), which leads to grid mismatch issues and difficulty in ensuring accurate DOA estimation with insufficient sampling and in underdetermined scenarios. In order to solve these challenges, we propose a new DOA estimation method based on a deep convolutional generative adversarial network (DCGAN) with a coprime array. By employing virtual interpolation, the difference co-array derived from the coprime array is extended to a virtual ULA with more degrees of freedom (DOFs). Then, combining with the Hermitian and Toeplitz prior knowledge, the covariance matrix is retrieved by the DCGAN. A backtracking method is employed to ensure that the reconstructed covariance matrix has a low-rank characteristic. We performed DOA estimation using the MUSIC algorithm. Simulation results demonstrate that the proposed method can not only distinguish more sources than the number of physical sensors but can also quickly and accurately solve DOA, especially with limited snapshots, which is suitable for fast estimation in mobile agent localization.