Multi-label Classification Methods Research Articles

ISOUP-SymRF: Symbolic feature ranking with random forests in online multi-target regression and multi-label classification

The task of feature ranking has received considerable attention across various predictive modelling tasks in the batch learning scenario, but not in the online learning setting. Available methods that estimate feature importances on data streams have so far predominantly focused on ranking the features for the tasks of classification and occasionally multi-label classification. We propose a novel online feature ranking method for online multi-target regression iSOUP-SymRF, which estimates feature importance scores based on the positions at which a feature appears in the trees of a random forest of iSOUP-Trees, and additionally extend it to task of online feature ranking for multi-label classification. By utilizing iSOUP-Trees, which can address multiple structured output prediction tasks on data streams, iSOUP-SymRF promises feature ranking across a variety of online structured output prediction tasks. We examine the ranking convergence of iSOUP-SymRF in terms of the methods’ parameters, the size of the ensemble and the number of selected features, as well as their stability under different random seeds. Furthermore, to show the utility of iSOUP-SymRF and its rankings we use them in conjunction with two state-of-the-art online multi-target regression and multi-label classification methods, iSOUP-Tree and AMRules, and analyze the impact of adding features according to the rankings obtained from iSOUP-SymRF.

Aspect-based Multilabel Classification of E-commerce Reviews Using Fine-tuned IndoBERT

In recent years, e-commerce has experienced rapid growth. A significant change in consumer behavior is marked by the ease of access and time flexibility offered by e-commerce platforms, as well as the existence of the review feature to assess products and services. However, with the ever-increasing number of reviews, consumers and store owners face challenges in sorting out relevant information. This research focuses on the multilabel classification of Indonesian e-commerce reviews. This research was undertaken because the application of multilabel classification, especially for e-commerce reviews in Indonesia, has received little attention. This research compares three classification models: IndoBERT, IndoBERT-CNN, and IndoBERT-LSTM, to determine the most effective model for multilabel aspect classification of customer reviews. The multilabel classification method was applied to determine the aspect categories of the reviews, such as product, customer service, and delivery, using different thresholds for evaluation. Results show that 0.6 threshold is optimal, with the IndoBERT-LSTM model as the best-performing model for the multilabel aspect classification of these e-commerce reviews. Optimal classification of the model enables more precise information extraction from customer reviews. This can be useful for e-commerce businesses to gain insight from the reviews they get from customers. This insight can be used to find out which aspects need to be improved from the e-commerce business which leads to increased customer satisfaction and trust.

Multi-Label Classification of Complaint Texts: Civil Aviation Service Quality Case Study

Customer complaints play an important role in the adjustment of business operations and improvement of services, particularly in the aviation industry. However, extracting adequate textual features to perform a multi-label classification of complaints remains a difficult problem. Current multi-label classification methods applied to complaint texts have not been able to fully utilize complaint information, and little research has been performed on complaint classification in the aviation industry. Therefore, to solve the problems of insufficient text feature extraction and the insufficient learning of inter-feature relationships, we constructed a multi-label classification model (MAG, or multi-feature attention gradient boosting decision tree classifier) for civil aviation service quality complaint texts. This model incorporates multiple features and attention mechanisms to improve the classification accuracy. First, the BERT (Bidirectional Encoder Representations from Transformers) model and attention mechanisms are used to represent the semantic and label features of the text. Then, the Text-CNN (a convolutional neural network) and BiLSTM (bidirectional long short-term memory) multi-channel feature extraction networks are used to extract the local and global features of the complaint text, respectively. Subsequently, a co-attention mechanism is used to learn the relationship between the local and global features. Finally, the travelers’ complaint texts are accurately classified by integrating the base classifiers. The results show that our proposed model improves the multi-label classification accuracy, outperforming other modern algorithms. We demonstrate how the label feature representation based on association rules and the multi-channel feature extraction network can enrich textual information and more fully extract features. Overall, the co-attention mechanism can effectively learn the relationships between text features, thereby improving the classification accuracy of the model and enabling better identification of travelers’ complaints. This study not only effectively extracted text features by integrating multiple features and attention mechanisms, but also constructed a targeted feature word set for complaint texts based on the domain-specific characteristics of the civil aviation industry. Furthermore, by iterating the basic classifier using a multi-label classification model, a classifier with higher accuracy was successfully obtained, providing strong technical support and new practical paths for improving the civil aviation service quality and complaint management.

BioElectra-BiLSTM-Dual Attention classifier for optimizing multilabel scientific literature classification

Abstract Scientific literature is growing in volume with time. The number of papers published each year by 28 100 journals is 2.5 million. The citation indexes and search engines are used extensively to find these publications. An individual receives many documents in response to a query, but only a few are relevant. The final documents lack structure due to inadequate indexing. Many systems index research papers using keywords instead of subject hierarchies. In the scientific literature classification paradigm, various multilabel classification methods have been proposed based on metadata features. The existing metadata-driven statistical measures use bag of words and traditional embedding techniques, like Word2Vec and BERT, which cannot quantify textual properties effectively. In this paper, we try to solve the limitations of existing classification techniques by unveiling the semantic context of the words using an advanced transformer-based recurrent neural networks (RNN) approach incorporating Dual Attention and layer-wise learning rate to enhance the classification performance. We propose a novel model, BioElectra-BiLSTM-Dual Attention that extracts the semantic features from the titles and abstracts of the research articles using BioElectra-encoder and then BILSTM layer along with Dual Attention label embeddings their correlation matrix and layer-wise learning rate strategy employed for performance enhancement. We evaluated the performance of the proposed model on the multilabel scientific literature LitCovid dataset and the results suggest that it significantly improves the macro-F1 and micro-F1 score as compared to the state-of-the-art baselines (ML-Net, Binary Bert, and LitMCBert).

Deep learning-based multilabel compound-fault diagnosis in centrifugal pumps

In this study, the issue of centrifugal pumps used in marine engineering is addressed, as they are susceptible to malfunction owing to long-term operation in highly corrosive seawater and extreme weather conditions, resulting in operational interruptions and safety risks. We propose a high-precision intelligent fault-diagnosis method for multiple fault types based on deep learning. In this method, continuous wavelet transform is firstly employed to extract signal time–frequency domain features. Subsequently, the Swin transformer model is used to process the wavelet time–frequency images converted from signals. Finally, multilabel classification methods are combined to diagnose various complex faults. The effectiveness of the proposed method is validated using a dataset obtained from simulation experiments pertaining to centrifugal-pump faults. The results show that the proposed method achieves 100% accuracy in diagnosing 27 types of faults and provides excellent diagnosis even under limited compound-fault samples, thus offering an efficient and practical method for fault diagnosis in centrifugal pumps used in marine engineering.

BoostXML: Gradient Boosting for Extreme Multilabel Text Classification With Tail Labels.

Multilabel learning involving hundreds of thousands or even millions of labels is referred to as extreme multilabel learning (XML), in which the labels often follow a power-law distribution with the majority occurring in very few data points as tail labels. Recent years have witnessed the intensive use of deep-learning methods for high-performance XML, but they are typically optimized for the head labels with abundant training instances and less consider the performance on tail labels, which, however, like the needles in haystacks, are often the focus of attention in real-life applications. In light of this, we present BoostXML, a deep learning-based XML method for extreme multilabel text classification, enhanced greatly by gradient boosting. In BoostXML, we pay more attention to tail labels in each Boosting Step by optimizing the residual mostly from unfitted training instances with tail labels. A Corrective Step is further proposed to avoid the mismatching between the text encoder and weak learners during optimization, which reduces the risk of falling into local optima and improves model performance. A Pretraining Step is also introduced in the initial stage of BoostXML to avoid exorbitant bias to tail labels. Extensive experiments on five benchmark datasets with state-of-the-art baselines demonstrate the advantage of BoostXML in tail-label prediction.

Automated Hematoma Detection and Outcome Prediction in Patients With Traumatic Brain Injury.

To develop a tool for automated subtype classification and segmentation of intracranial hemorrhages (ICH) on CT scans of patients with traumatic brain injury (TBI). Furthermore, outcome prediction for patients can effectively facilitate patient management. This study presents a cascade framework for two-stage segmentation and multi-label classification. The hematoma region of interest (ROI) is localized, and then the ROI is cropped and resized to the original pixel size before being input into the model again to obtain the segmentation results. In multilabel classification, the mask obtained from automatic segmentation is superimposed onto the corresponding ROI and CT slices, respectively, to constitute the input image. Subsequently, the ROI image is employed as the local network input to obtain local features. Third, the CT image is utilized to construct a feature extraction network to obtain global features. Ultimately, the local and global features are fused dimensions in the pooling layer,and calculated to generate the final retrieval results. For the prediction of 14-day in-hospital mortality, automatically extracted hematoma subtype and volume features were integrated to enhance the widely used CRASH model. The proposed segmentation method achieves the best estimates on the Dice similarity coefficient and Jaccard Similarity Index. The proposed multilabel classification method achieved an average accuracy of 95.91%. For mortality prediction, the best model achieved an average area under the receiver operating characteristic curve (AUC) of 0.91 by 5-fold cross-validation. The proposed method enhances the precision of hematoma segmentation and subtype classification. In clinical settings, the method can streamline the evaluation of ICH for radiologists, and the automatically extracted features are anticipated to facilitate prognosis assessment.

Application of Label Correlation in Multi-Label Classification: A Survey

Multi-Label Classification refers to the classification task where a data sample is associated with multiple labels simultaneously, which is widely used in text classification, image classification, and other fields. Different from the traditional single-label classification, each instance in Multi-Label Classification corresponds to multiple labels, and there is a correlation between these labels, which contains a wealth of information. Therefore, the ability to effectively mine and utilize the complex correlations between labels has become a key factor in Multi-Label Classification methods. In recent years, research on label correlations has shown a significant growth trend internationally, reflecting its importance. Given that, this paper presents a survey on the label correlations in Multi-Label Classification to provide valuable references and insights for future researchers. The paper introduces multi-label datasets across various fields, elucidates and categorizes the concept of label correlations, emphasizes their utilization in Multi-Label Classification and associated subproblems, and provides a prospect for future work on label correlations.

Enhancing power equipment defect identification through multi-label classification methods

Accurate identification and classification of equipment defects are essential for assessing the health of power equipment and making informed maintenance decisions. Traditional defect classification methods, which rely on subjective manual records and intricate defect descriptions, have proven to be inefficient. Existing approaches often evaluate equipment status solely based on defect grade. To enhance the precision of power equipment defect identification, we developed a multi-label classification dataset by compiling historical defect records. Furthermore, we assessed the performance of 11 established multi-label classification methods on this dataset, encompassing both traditional machine learning and deep learning methods. Experimental results reveal that methods considering label correlations exhibit significant performance advantages. By employing balanced loss functions, we effectively address the challenge of sample imbalance across various categories, thereby enhancing classification accuracy. Additionally, segmenting the power equipment defect classification task, which involves numerous labels, into label recall and ranking stages can substantially improve classification performance. The dataset we have created is available for further research by other scholars in the field.

Granular correlation-based label-specific feature augmentation for multi-label classification

Multi-label classification is an extension of single-label classification with generations of multi-output for unseen instances. Label correlation is an essential component in constructing multi-label classifiers. How to optimize the representation of label correlation while preserving the semantics of label-specific remains an uncertain issue. Instead of estimating label correlation by a holistic feature representation, we present an augmented label correlation model by generating multi-granularity label-specific features. Firstly, we devise a mixture distance measure to characterize the closeness of an instance by weighing the Pearson correlation coefficient with cosine similarity. Secondly, we explore the local label-specific relative discrimination by leveraging from both the instance-level and class-level correlation distribution within k nearest neighborhood. Finally, we conduct an information fusion strategy to comprehensively integrate the positive and the negative tendencies at the neighborhood level. Instances with salient positive tendency and compact neighborhood structure receive larger values while receiving smaller values with salient negative tendency and sparse neighborhood structure. With the concatenation of original features and augmented features, we examine the classification performance of the proposed granule correlation-based feature augmentation (GOFA) on well-established second-order multi-label classification methods. Extensive comparisons on thirteen benchmarks demonstrate the statistical superiority of GOFA over state-of-the-art multi-label classifications.

Text Classification Based on Natural Language Processing and Machine Learning in Multi-Label Corpus

The rapid development of the Internet has led to a geometric expansion of text information resources online. Among them, corpus, as the basic data source of natural language processing based on statistical language model, its construction and application have become a hot issue in current language processing research. After consulting a large number of relevant literature and materials, it was found that many researchers have provided new ideas for multi label corpus text classification methods. However, this article adds its own understanding and takes this as the direction and basis. In the introduction, the research significance of text classification was introduced, and then academic research and analysis were carried out on the two key sentences of corpus text classification and natural language processing in multi-tag corpus text classification. This article then utilizes an algorithm model to provide a theoretical basis for the study of multi-label corpus text classification methods; At the end of this article, a simulation comparative experiment is conducted, and the experiment is summarized and discussed; In the Enterprise L corpus, the difference in recall rates before and after the use of Entrance 1 was 5.5%, the difference in recall rates before and after the use of Entrance 2 was 7.8%, the difference in recall rates before and after the use of Entrance 3 was 3.3%, and the difference in recall rates before and after the use of Entrance 4 was 4.5%. At the same time, with the continuous research of natural language processing and machine learning, the research on text classification methods of multi tag corpus is also facing new opportunities and challenges.

A deep low-rank semantic factorization method for micro-video multi-label classification

A deep low-rank semantic factorization method for micro-video multi-label classification

A Lightweight Multi-Label Classification Method for Urban Green Space in High-Resolution Remote Sensing Imagery

Urban green spaces are an indispensable part of the ecology of cities, serving as the city’s “purifier” and playing a crucial role in promoting sustainable urban development. Therefore, the refined classification of urban green spaces is an important task in urban planning and management. Traditional methods for the refined classification of urban green spaces heavily rely on expert knowledge, often requiring substantial time and cost. Hence, our study presents a multi-label image classification model based on MobileViT. This model integrates the Triplet Attention module, along with the LSTM module, to enhance its label prediction capabilities while maintaining its lightweight characteristic for standalone operation on mobile devices. Trial outcomes in our UGS dataset in this study demonstrate that the approach we used outperforms the baseline by 1.64%, 3.25%, 3.67%, and 2.71% in mAP,F1,precision, and recall, respectively. This indicates that the model can uncover the latent dependencies among labels to enhance the multi-label image classification device’s performance. This study provides a practical solution for the intelligent and detailed classification of urban green spaces, which holds significant importance for the management and planning of urban green spaces.

Updating Correlation-Enhanced Feature Learning for Multi-Label Classification

In the domain of multi-label classification, label correlations play a crucial role in enhancing prediction precision. However, traditional methods heavily depend on ground-truth label sets, which can be incompletely tagged due to the diverse backgrounds of annotators and the significant cost associated with procuring extensive labeled datasets. To address these challenges, this paper introduces a novel multi-label classification method called updating Correlation-enhanced Feature Learning (uCeFL), which extracts label correlations directly from the data instances, circumventing the dependency on potentially incomplete label sets. uCeFL initially computes a revised label matrix by multiplying the incomplete label matrix with the label correlations extracted from the data matrix. This revised matrix is then utilized to enrich the original data features, enabling a neural network to learn correlation-enhanced representations that capture intricate relationships between data features, labels, and their interactions. Notably, label correlations are not static; they are dynamically updated during the neural network’s training process. Extensive experiments carried out on various datasets emphasize the effectiveness of the proposed approach. By leveraging label correlations within data instances, along with the hierarchical learning capabilities of neural networks, it offers a significant improvement in multi-label classification, even in scenarios with incomplete labels.

Influence of patient characteristics on microbial composition in surgical-site infections: insights from national surveillance study.

Although the impact of surgery- and patient-dependent factors on surgical-site infections (SSIs) have been studied extensively, their influence on the microbial composition of SSI remains unexplored. The aim of this study was to identify patient-dependent predictors of the microbial composition of SSIs across different types of surgery. This retrospective cohort study included 538 893 patients from the Swiss national infection surveillance programme. Multilabel classification methods, adaptive boosting and Gaussian Naive Bayes were employed to identify predictors of the microbial composition of SSIs using 20 features, including sex, age, BMI, duration of surgery, type of surgery, and surgical antimicrobial prophylaxis. Overall, SSIs were recorded in 18 642 patients (3.8%) and, of these, 10 632 had microbiological wound swabs available. The most common pathogens identified in SSIs were Enterobacterales (57%), Staphylococcus spp. (31%), and Enterococcus spp. (28%). Age (mean feature importance 0.260, 95% c.i. 0.209 to 0.309), BMI (0.224, 0.177 to 0.271), and duration of surgery (0.221, 0.180 to 0.269) were strong and independent predictors of the microbial composition of SSIs. Increasing age and duration of surgical procedure as well as decreasing BMI were associated with a shift from Staphylococcus spp. to Enterobacterales and Enterococcus spp. An online application of the machine learning model is available for validation in other healthcare systems. Age, BMI, and duration of surgery were key predictors of the microbial composition of SSI, irrespective of the type of surgery, demonstrating the relevance of patient-dependent factors to the pathogenesis of SSIs.

A data decomposition-based hierarchical classification method for multi-label classification of contractual obligations for the purpose of their governance

Contract governance ensures that the agreed outcomes between customers and vendors are fulfilled. Information Technology (IT) outsourcing organizations enter thousands of contractual relationships each month leading to a high volume of business-critical contractual text that must be reviewed and deciphered for effective governance. The key to effective governance of contracts is a model that facilitates assigning ownership of the obligations to the right departments in an organization and allocating their accountability to the right stakeholders. For this, the contractual obligations need to be identified and classified so that details such as actions to be taken by departments in an organization and their ownership as per a given clause are brought out for the purpose of governance. In this paper, we present our work on automated extraction and classification of obligations present in Software Engineering (SE) contracts for the purpose of contracts governance. We propose a novel data decomposition-based hierarchical classification method for a multi-label classification of contractual obligations. We conducted experiments for a fine-grained automated classification of more than 55,000 statements from 50 large real-life SE contract documents received from a large vendor organization into 152 governance-specific classes. The results indicate that the proposed method can bring about a 7–8% improvement in accuracies when compared to state-of-the-art classification baselines such as BERT, RoBERTa, and generative models such as GPT-2.

Feature selection for multilabel classification with missing labels via multi-scale fusion fuzzy uncertainty measures

Numerous high-dimension multilabel data are generated, posing a challenge for multilabel learning. Building effective learning models with discriminative features is essential to improve the performance of multilabel learning. Multilabel feature selection can filter out the discriminative features according to their contribution to classification. However, ambiguity, uncertainty, and missing labels coexist in real-life multilabel data, which brings adverse effects to multilabel feature selection. The multi-scale fuzzy rough set gives an effective way to mine intrinsic knowledge hidden in uncertain data. This paper first extends the multi-scale learning to multilabel data with missing labels and proposes a feature selection method for multilabel classification with missing labels via multi-scale fusion fuzzy uncertainty measures called FSMML. The missing label space construction and feature evaluation metric are carefully investigated in the framework of multi-scale learning. A multilabel multi-scale learning strategy is formalized with the fuzzy granularity cognitive mechanism as the core, and the multi-scale fusion fuzzy label learning is given to reconstruct the missing label space. Then, a novel multilabel multi-scale fuzzy rough sets with missing labels is developed, and the significance of each scale is quantified. Moreover, some multi-scale fusion fuzzy uncertainty measures are defined by capturing the sample fuzzy similarity in the feature and reconstructed label spaces. Accordingly, the relevance between features and label set and the interactivity and redundancy between features in feature evaluation are discussed. Finally, FSMML chooses high-quality features to maximize relevance and interactivity and minimize redundancy. Extensive experiments demonstrate the effectiveness of FSMML on fifteen datasets with missing labels.

Multi-class multi-label classification of social media texts for typhoon damage assessment: a two-stage model fully integrating the outputs of the hidden layers of BERT

ABSTRACT With the development of social media, it has become increasingly important to quickly and accurately identify social media texts related to disasters (e.g. typhoon) to aid in rescue and recovery efforts. Currently, multi-class classification and pre-trained language model Bidirectional Encoder Representations from Transformers (BERT) are widely used for text classification. However, most studies on typhoon damage classification are multi-class single-label, which contradicts to the reality that a social media text may correspond to multiple types of damage. Moreover, the outputs of the hidden layers of BERT are not fully utilized. This paper proposes a two-stage multi-class multi-label classification method for typhoon damage assessment by fully integrating the outputs of the hidden layers of BERT. In the first stage, sentence vectors are adopted to identify typhoon damage-related texts. In the second stage, word matrices are applied for multi-class multi-label classification to further classify the texts into five damage categories (i.e. transportation, public, electricity, forestry, and waterlogging). The two stages are trained end-to-end to identify typhoon damage from social media texts. Experiments on SinaWeibo texts during typhoon landfall in Chinese coastal regions demonstrate that the proposed method can effectively improve the accuracy of text classification and comprehensively assess typhoon damage.

Analysis of the retraining strategies for multi-label text message classification in call/contact center systems

Today, in many areas of technology, we can come across applications of various artificial intelligence methods. They usually involve models trained on some specific pool of learning data. Sometimes, however, the data analyzed by these solutions can change its nature over time. This usually results in a decrease in classification efficiency. In such a case, the use of techniques to retrain the originally trained reference models should be considered. One of the industries where the nature of data changes quite dynamically over time is the broadly defined call/contact center systems. An example of a module that is often found in this type of system and that, due to frequently changing marketing campaigns, requires the use of learning techniques is the automatic classification of text data. The paper describes the process of retraining the original reference models used in a multi-label text message classification method dedicated directly to call/contact center systems applications. In order to carry out the retraining process, Polish-language data from the actual archives of a large commercial contact center system and English-language data extracted from a publicly available database were used. The study was conducted for models based on artificial neural networks and bidirectional encoder representations from transformer type models. In addition, two different retraining strategies were studied, the results of which were compared with data obtained from the operation of reference models. As a result of the research work, an improvement of up to 5% in classification efficiency, as described by the metric Emotica was obtained, which means that proper integration of the retraining process brings tangible benefits to the solution tested in the article. Thus, it can also benefit the solutions used in business.

Multi-Label Noise Robust Collaborative Learning for Remote Sensing Image Classification.

The development of accurate methods for multi-label classification (MLC) of remote sensing (RS) images is one of the most important research topics in RS. The MLC methods based on convolutional neural networks (CNNs) have shown strong performance gains in RS. However, they usually require a high number of reliable training images annotated with multiple land-cover class labels. Collecting such data is time-consuming and costly. To address this problem, the publicly available thematic products, which can include noisy labels, can be used to annotate RS images with zero-labeling cost. However, multi-label noise (which can be associated with wrong and missing label annotations) can distort the learning process of the MLC methods. To address this problem, we propose a novel multi-label noise robust collaborative learning (RCML) method to alleviate the negative effects of multi-label noise during the training phase of a CNN model. RCML identifies, ranks, and excludes noisy multi-labels in RS images based on three main modules: 1) the discrepancy module; 2) the group lasso module; and 3) the swap module. The discrepancy module ensures that the two networks learn diverse features, while producing the same predictions. The task of the group lasso module is to detect the potentially noisy labels assigned to multi-labeled training images, while the swap module is devoted to exchange the ranking information between two networks. Unlike the existing methods that make assumptions about noise distribution, our proposed RCML does not make any prior assumption about the type of noise in the training set. The experiments conducted on two multi-label RS image archives confirm the robustness of the proposed RCML under extreme multi-label noise rates. Our code is publicly available at: https://www.noisy-labels-in-rs.org.