Multi-label Classification Methods Research Articles

Multi-class multi-label classification of social media texts for typhoon damage assessment: a two-stage model fully integrating the outputs of the hidden layers of BERT

ABSTRACT With the development of social media, it has become increasingly important to quickly and accurately identify social media texts related to disasters (e.g. typhoon) to aid in rescue and recovery efforts. Currently, multi-class classification and pre-trained language model Bidirectional Encoder Representations from Transformers (BERT) are widely used for text classification. However, most studies on typhoon damage classification are multi-class single-label, which contradicts to the reality that a social media text may correspond to multiple types of damage. Moreover, the outputs of the hidden layers of BERT are not fully utilized. This paper proposes a two-stage multi-class multi-label classification method for typhoon damage assessment by fully integrating the outputs of the hidden layers of BERT. In the first stage, sentence vectors are adopted to identify typhoon damage-related texts. In the second stage, word matrices are applied for multi-class multi-label classification to further classify the texts into five damage categories (i.e. transportation, public, electricity, forestry, and waterlogging). The two stages are trained end-to-end to identify typhoon damage from social media texts. Experiments on SinaWeibo texts during typhoon landfall in Chinese coastal regions demonstrate that the proposed method can effectively improve the accuracy of text classification and comprehensively assess typhoon damage.

Analysis of the retraining strategies for multi-label text message classification in call/contact center systems

Today, in many areas of technology, we can come across applications of various artificial intelligence methods. They usually involve models trained on some specific pool of learning data. Sometimes, however, the data analyzed by these solutions can change its nature over time. This usually results in a decrease in classification efficiency. In such a case, the use of techniques to retrain the originally trained reference models should be considered. One of the industries where the nature of data changes quite dynamically over time is the broadly defined call/contact center systems. An example of a module that is often found in this type of system and that, due to frequently changing marketing campaigns, requires the use of learning techniques is the automatic classification of text data. The paper describes the process of retraining the original reference models used in a multi-label text message classification method dedicated directly to call/contact center systems applications. In order to carry out the retraining process, Polish-language data from the actual archives of a large commercial contact center system and English-language data extracted from a publicly available database were used. The study was conducted for models based on artificial neural networks and bidirectional encoder representations from transformer type models. In addition, two different retraining strategies were studied, the results of which were compared with data obtained from the operation of reference models. As a result of the research work, an improvement of up to 5% in classification efficiency, as described by the metric Emotica was obtained, which means that proper integration of the retraining process brings tangible benefits to the solution tested in the article. Thus, it can also benefit the solutions used in business.

Multi-Label Noise Robust Collaborative Learning for Remote Sensing Image Classification.

The development of accurate methods for multi-label classification (MLC) of remote sensing (RS) images is one of the most important research topics in RS. The MLC methods based on convolutional neural networks (CNNs) have shown strong performance gains in RS. However, they usually require a high number of reliable training images annotated with multiple land-cover class labels. Collecting such data is time-consuming and costly. To address this problem, the publicly available thematic products, which can include noisy labels, can be used to annotate RS images with zero-labeling cost. However, multi-label noise (which can be associated with wrong and missing label annotations) can distort the learning process of the MLC methods. To address this problem, we propose a novel multi-label noise robust collaborative learning (RCML) method to alleviate the negative effects of multi-label noise during the training phase of a CNN model. RCML identifies, ranks, and excludes noisy multi-labels in RS images based on three main modules: 1) the discrepancy module; 2) the group lasso module; and 3) the swap module. The discrepancy module ensures that the two networks learn diverse features, while producing the same predictions. The task of the group lasso module is to detect the potentially noisy labels assigned to multi-labeled training images, while the swap module is devoted to exchange the ranking information between two networks. Unlike the existing methods that make assumptions about noise distribution, our proposed RCML does not make any prior assumption about the type of noise in the training set. The experiments conducted on two multi-label RS image archives confirm the robustness of the proposed RCML under extreme multi-label noise rates. Our code is publicly available at: https://www.noisy-labels-in-rs.org.

Feature selection for multilabel classification with missing labels via multi-scale fusion fuzzy uncertainty measures

Numerous high-dimension multilabel data are generated, posing a challenge for multilabel learning. Building effective learning models with discriminative features is essential to improve the performance of multilabel learning. Multilabel feature selection can filter out the discriminative features according to their contribution to classification. However, ambiguity, uncertainty, and missing labels coexist in real-life multilabel data, which brings adverse effects to multilabel feature selection. The multi-scale fuzzy rough set gives an effective way to mine intrinsic knowledge hidden in uncertain data. This paper first extends the multi-scale learning to multilabel data with missing labels and proposes a feature selection method for multilabel classification with missing labels via multi-scale fusion fuzzy uncertainty measures called FSMML. The missing label space construction and feature evaluation metric are carefully investigated in the framework of multi-scale learning. A multilabel multi-scale learning strategy is formalized with the fuzzy granularity cognitive mechanism as the core, and the multi-scale fusion fuzzy label learning is given to reconstruct the missing label space. Then, a novel multilabel multi-scale fuzzy rough sets with missing labels is developed, and the significance of each scale is quantified. Moreover, some multi-scale fusion fuzzy uncertainty measures are defined by capturing the sample fuzzy similarity in the feature and reconstructed label spaces. Accordingly, the relevance between features and label set and the interactivity and redundancy between features in feature evaluation are discussed. Finally, FSMML chooses high-quality features to maximize relevance and interactivity and minimize redundancy. Extensive experiments demonstrate the effectiveness of FSMML on fifteen datasets with missing labels.

Business text classification with imbalanced data and moderately large label spaces for digital transformation

Digital transformation refers to an organization’s use of digital technology to improve its products, services, and operations, aligning them with evolving business requirements. To demonstrate this transformative process, we present a real-life case study where a company seeks to automate the classification of their textual data rather than relying on manual methods. Transitioning to automated classification involves deploying machine learning models, which rely on pre-labeled datasets for training and making predictions on new data. However, upon receiving the dataset from the company, we faced challenges due to the imbalanced distribution of labels and moderately large label spaces. To tackle text classification with such a business dataset, we evaluated four distinct methods for multi-label text classification: fine-tuned Bidirectional Encoder Representations from Transformers (BERT), Binary Relevance, Classifier Chains, and Label Powerset. The results revealed that fine-tuned BERT significantly outperformed the other methods across key metrics like Accuracy, F1-score, Precision, and Recall. Binary Relevance also displayed competence in handling the dataset effectively, while Classifier Chains and Label Powerset exhibited comparatively less impressive performance. These findings highlight the remarkable effectiveness of fine-tuned BERT model and the Binary Relevance classifier in multi-label text classification tasks, particularly when dealing with imbalanced training datasets and moderately large label spaces. This positions them as valuable assets for businesses aiming to automate data classification in the digital transformation era.

Multi-label text classification of cardiovascular drug attributes based on BERT and BiGRU

Extracting and digitizing drug attributes from medical literature is the first step to build a knowledge computing system for precision disease treatment. In order to build a cardiovascular drug knowledge base, this paper proposes a multi-label text classification method for cardiovascular drug attributes from the Chinese drug guideline. The drug attributes are characterized by a BERT pre-trained model, and a dual-feature extraction structure is proposed based on the BiGRU neural network to capture high-level semantic information. Label categorization of cardiovascular drug attributes, such as indications and mode of administration, is accomplished. The F1 score of 0.8431 was obtained using 5-fold cross-validation. Comparing KNN and Naïve bayes, and conducting CNN and BiGRU control experiments on the basis of Word2Vec characterization of medication guidelines, the proposed multi-label text classification method is effective and the F1 value is significantly improved. Proved by analysis of ablation and crossover experiments, the proposed method can achieve a high accuracy rate averaged at 0.8339.

Semi-Supervised Predictive Clustering Trees for (Hierarchical) Multi-Label Classification

Semi-supervised learning (SSL) is a common approach to learning predictive models using not only labeled, but also unlabeled examples. While SSL for the simple tasks of classification and regression has received much attention from the research community, this is not the case for complex prediction tasks with structurally dependent variables, such as multi-label classification and hierarchical multi-label classification. These tasks may require additional information, possibly coming from the underlying distribution in the descriptive space provided by unlabeled examples, to better face the challenging task of simultaneously predicting multiple class labels. In this paper, we investigate this aspect and propose a (hierarchical) multi-label classification method based on semi-supervised learning of predictive clustering trees, which we also extend towards ensemble learning. Extensive experimental evaluation conducted on 24 datasets shows significant advantages of the proposed method and its extension with respect to their supervised counterparts. Moreover, the method preserves interpretability of classical tree-based models.

A multi-label image classification method combining multi-stage image semantic information and label relevance

A multi-label image classification method combining multi-stage image semantic information and label relevance

Performance evaluation of seven multi-label classification methods on real-world patent and publication datasets

Performance evaluation of seven multi-label classification methods on real-world patent and publication datasets

Composite Harmonic Source Detection with Multi-Label Approach Using Advanced Fusion Method

With the integration of clean energy and new power electronic devices into the power grid, the superposition of harmonic sources has become increasingly apparent and common. There is an urgent need to effectively identify composite harmonic sources in the new energy grid. This article proposes a multi-label composite harmonic source classification method that integrates knowledge representation with the transformer model. First, triplets from harmonic monitoring data are extracted and TransR models are used to train time-frequency feature representation vectors. Then, the transformer model is trained to learn the data features of different harmonic sources. Finally, based on the multi-label classification method, composite harmonic sources are identified. This article integrates the semantic information of time-frequency features into the data samples, increasing the interpretability of the model while expanding the inter-class features, which is conducive to the classification and recognition of the model. Compared with other deep learning recognition methods, verification based on simulation data and measured data shows that this method has low training complexity and higher recognition accuracy.

Multi-Label Supervised Contrastive Learning

Multi-label classification is an arduous problem given the complication in label correlation. Whilst sharing a common goal with contrastive learning in utilizing correlations for representation learning, how to better leverage label information remains challenging. Previous endeavors include extracting label-level presentations or mapping labels to an embedding space, overlooking the correlation between multiple labels. It exhibits a great ambiguity in determining positive samples with different extent of label overlap between samples and integrating such relations in loss functions. In our work, we propose Multi-Label Supervised Contrastive learning (MulSupCon) with a novel contrastive loss function to adjust weights based on how much overlap one sample shares with the anchor. By analyzing gradients, we explain why our method performs better under multi-label circumstances. To evaluate, we conduct direct classification and transfer learning on several multi-label datasets, including widely-used image datasets such as MS-COCO and NUS-WIDE. Validation indicates that our method outperforms the traditional multi-label classification method and shows a competitive performance when comparing to other existing approaches.

Graph Neural Network Based Multi-Label Hierarchical Classification for Disease Predictions in General Practice.

General practitioners are supposed to be better diagnostics to detect patients with serious diseases earlier, and conduct early interventions and appropriate referrals of patients. However, in the current general practice, primary general practitioners lack sufficient clinical experiences, and the correct rate of general disease diagnosis is low. To assist general practitioners in diagnosis, this paper proposes a multi-label hierarchical classification method based on graph neural network, which integrates medical knowledge and electronic health record (EHR) data to build a disease prediction model. The experimental results based on data consist of 231,783 visits from EHR show that the proposed model outperforms all baseline models in the general disease prediction task with a top-3 recall of 0.865. The interpretable results of the model can effectively help clinicians understand the basis of the model's decision-making.

Structure Mapping Generative Adversarial Network for Multi-View Information Mapping Pattern Mining.

Multi-view learning is dedicated to integrating information from different views and improving the generalization performance of models. However, in most current works, learning under different views has significant independency, overlooking common information mapping patterns that exist between these views. This paper proposes a Structure Mapping Generative adversarial network (SM-GAN) framework, which utilizes the consistency and complementarity of multi-view data from the innovative perspective of information mapping. Specifically, based on network-structured multi-view data, a structural information mapping model is proposed to capture hierarchical interaction patterns among views. Subsequently, three different types of graph convolutional operations are designed in SM-GAN based on the model. Compared with regular GAN, we add a structural information mapping module between the encoder and decoder wthin the generator, completing the structural information mapping from the micro-view to the macro-view. This paper conducted sufficient validation experiments using public imaging genetics data in Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset. It is shown that SM-GAN outperforms baseline and advanced methods in multi-label classification and evolution prediction tasks.

An adaptive ensemble framework using multi-source data for day-ahead photovoltaic power forecasting

Day-ahead photovoltaic (PV) power forecasting plays a crucial role in power market trading and grid dispatching. It has been empirically demonstrated in various fields that combining forecasts yields better results than using individual models. In this work, a novel adaptive ensemble framework is proposed based on multi-source data. First, incorporating prior information from physical models, three types of high-performance component models are constructed based on different types of data. Second, a multi-label classification method is utilized to select better performing models, allowing for switching between different model combinations depending on the weather conditions. Finally, a dynamic ensemble method is used to update the weights of the component forecasts based on its cumulative errors observed in the recent past. The proposed method was evaluated on a four-year PV multi-source dataset from 2019 to 2022. The forecasting skill (FS) in the test year (2022) reaches 50.61%. The results show that FS is improved by 4.75% compared to the optimal component model. Compared with other state-of-the-art methods, our method has achieved the best performance by improving FS at least 3.94%. The proposed framework in this study can be widely applied to other energy forecasting fields, such as wind/load forecasting.

BoostXML: Gradient Boosting for Extreme Multilabel Text Classification With Tail Labels.

Multilabel learning involving hundreds of thousands or even millions of labels is referred to as extreme multilabel learning (XML), in which the labels often follow a power-law distribution with the majority occurring in very few data points as tail labels. Recent years have witnessed the intensive use of deep-learning methods for high-performance XML, but they are typically optimized for the head labels with abundant training instances and less consider the performance on tail labels, which, however, like the needles in haystacks, are often the focus of attention in real-life applications. In light of this, we present BoostXML, a deep learning-based XML method for extreme multilabel text classification, enhanced greatly by gradient boosting. In BoostXML, we pay more attention to tail labels in each Boosting Step by optimizing the residual mostly from unfitted training instances with tail labels. A Corrective Step is further proposed to avoid the mismatching between the text encoder and weak learners during optimization, which reduces the risk of falling into local optima and improves model performance. A Pretraining Step is also introduced in the initial stage of BoostXML to avoid exorbitant bias to tail labels. Extensive experiments on five benchmark datasets with state-of-the-art baselines demonstrate the advantage of BoostXML in tail-label prediction.

Prompt Consistency for Multi-Label Textual Emotion Detection

Textual emotion detection is playing an important role in the human-computer interaction domain. The mainstream methods of textual emotion detection are extracting semantic features and fine-tuning by language models. Due to the information redundancy in semantics, it is difficult for these methods to accurately detect all the emotions implied in the text. The prompting method has been shown to make the language models more purposeful in prediction by filling the cloze or prefix prompts defined. Therefore, we design a prompting method for multi-label classification. To stabilize the output, we design two consistency training strategies. We experiment on two multi-label emotion classification datasets: Ren-CECps and NLPCC2018. Our proposed prompting method with consistency training strategies for multi-label textual emotion detection (PC-MTED) model achieves state-of-the-art Macro F1 scores of 0.5432 and 0.5269, respectively. The experimental results indicate that our proposed method is effective in the multi-label textual emotion detection task.

Multi-Label Diagnosis of Arrhythmias Based on a Modified Two-Category Cross-Entropy Loss Function

The 12-lead resting electrocardiogram (ECG) is commonly used in hospitals to assess heart health. The ECG can reflect a variety of cardiac abnormalities, requiring multi-label classification. However, the diagnosis results in previous studies have been imprecise. For example, in some previous studies, some cardiac abnormalities that cannot coexist often appeared in the diagnostic results. In this work, we explore how to realize the effective multi-label diagnosis of ECG signals and prevent the prediction of cardiac arrhythmias that cannot coexist. In this work, a multi-label classification method based on a convolutional neural network (CNN), long short-term memory (LSTM), and an attention mechanism is presented for the multi-label diagnosis of cardiac arrhythmia using resting ECGs. In addition, this work proposes a modified two-category cross-entropy loss function by introducing a regularization term to avoid the existence of arrhythmias that cannot coexist. The effectiveness of the modified cross-entropy loss function is validated using a 12-lead resting ECG database collected by our team. Using traditional and modified cross-entropy loss functions, three deep learning methods are employed to classify six types of ECG signals. Experimental results show the modified cross-entropy loss function greatly reduces the number of non-coexisting label pairs while maintaining prediction accuracy. Deep learning methods are effective in the multi-label diagnosis of ECG signals, and diagnostic efficiency can be improved by using the modified cross-entropy loss function. In addition, the modified cross-entropy loss function helps prevent diagnostic models from outputting two arrhythmias that cannot coexist, further reducing the false positive rate of non-coexisting arrhythmic diseases, thereby demonstrating the potential value of the modified loss function in clinical applications.

Online Passive-Aggressive Multilabel Classification Algorithms.

Most existing multilabel classification methods are batch learning methods, which may suffer from expensive retraining costs when dealing with new incoming data. In order to overcome the drawbacks of batch learning, we develop a family of online multilabel classification algorithms, which can update the model instantly and efficiently, and make a timely online prediction when new data arrive. Our algorithms all take a closed-form update, which is obtained by solving a constrained optimization problem in each round of online learning. Label correlation is explicitly modeled in our optimization problem. The label thresholding function, an important component of our online classifier, can also be learned online. Our algorithms can be easily generalized to the nonlinear prediction cases using Mercer kernels. The worst case loss bounds for our algorithms are provided. The bounds are relative to the cumulative loss suffered by the best fixed predictive model that can be attained in hindsight. Finally, we corroborate the merits of our algorithms in both linear and nonlinear predictions on nine open multilabel benchmark datasets.

Multi-label learning based on instance correlation and feature redundancy

Multi-label classification is a machine-learning task that simultaneously processes instances associated with multiple labels. Label-specific feature learning selects each label's most discriminative feature subset, effectively reducing the feature dimension and improving the classification performance. However, most methods only consider label correlation, ignoring the correlation between instances and feature redundancy. To solve this problem, a multi-label classification method based on instance correlation and feature redundancy is proposed. The proposed method merges instance correlation by updating the data set and removes redundant features by calculating mutual information. By jointly considering label correlation, instance correlation, and feature redundancy, our method promotes effective multi-label feature selection. The experimental results on ten data sets demonstrate the effectiveness of the proposed method.

Robust Label and Feature Space Co-Learning for Multi-Label Classification

Multi-label classification remains a challenging task for high-dimensional data samples and their labels both increase the complexity of training models. In this paper, we propose a Robust Label and Feature Space Co-Learning method, referred to as RLFSCL, for multi-label classification. Different from traditional multi-label classification methods which focus on feature space learning through regression directly between data samples and labels, our proposed method can further learn robust low rank label space from this traditional regression method. Therefore, our RLFSCL can learn better low rank feature and label representations simultaneously in original noisy and high dimensional spaces. Experimental comparison on five benchmark datasets, including Rcv1s5, Cal500, and Corel16k4 shows that the proposed RLFSCL algorithm outperforms state-of-the-art multi-label classification methods. The code of RLFSCL is made available on <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/JingChuanTang/RLFSCL.</uri>