Multi-label Classification Performance Research Articles

CorLabelNet: a comprehensive framework for multi-label chest X-ray image classification with correlation guided discriminant feature learning and oversampling.

Recent advancements in deep learning techniques have significantly improved multi-label chest X-ray (CXR) image classification for clinical diagnosis. However, most previous studies neither effectively learn label correlations nor take full advantage of them to improve multi-label classification performance. In addition, different labels of CXR images are usually severely imbalanced, resulting in the model exhibiting a bias towards the majority class. To address these challenges, we introduce a framework that not only learns label correlations but also utilizes them to guide the learning of features and the process of oversampling. In this paper, our approach incorporates self-attention to capture high-order label correlations and considers label correlations from both global and local perspectives. Then, we propose a consistency constraint and a multi-label contrastive loss to enhance feature learning. To alleviate the imbalance issue, we further propose an oversampling approach that exploits the learned label correlation to identify crucial seed samples for oversampling. Our approach repeats 5-fold cross-validation process experiments three times and achieves the best performance on both the CheXpert and ChestX-Ray14 datasets. Learning accurate label correlation is significant for multi-label classification and taking full advantage of label correlations is beneficial for discriminative feature learning and oversampling. A comparative analysis with the state-of-the-art approaches highlights the effectiveness of our proposed methods.

HydraViT: Adaptive multi-branch transformer for multi-label disease classification from Chest X-ray images

Chest X-ray is an essential diagnostic tool in the identification of chest diseases given its high sensitivity to pathological abnormalities in the lungs. However, image-driven diagnosis is still challenging due to heterogeneity in size and location of pathology, as well as visual similarities and co-occurrence of separate pathology. Since disease-related regions often occupy a relatively small portion of diagnostic images, classification models based on traditional convolutional neural networks (CNNs) are adversely affected given their locality bias. While CNNs were previously augmented with attention maps or spatial masks to guide focus on potentially critical regions, learning localization guidance under heterogeneity in the spatial distribution of pathology is challenging. To improve multi-label classification performance, here we propose a novel method, HydraViT, that synergistically combines a transformer backbone with a multi-branch output module with learned weighting. The transformer backbone enhances sensitivity to long-range context in X-ray images, while using the self-attention mechanism to adaptively focus on task-critical regions. The multi-branch output module dedicates an independent branch to each disease label to attain robust learning across separate disease classes, along with an aggregated branch across labels to maintain sensitivity to co-occurrence relationships among pathology. Experiments demonstrate that, on average, HydraViT outperforms competing attention-guided methods by 1.9% AUC and 5.3% MAE, region-guided methods by 2.1% AUC and 8.3% MAE, and semantic-guided methods by 2.0% AUC and 6.5% MAE in multi-label classification performance.

ADVANCED TECHNIQUES IN MULTI-LABEL TEXT CLASSIFICATION: INTEGRATION OF BETA ANT COLONY AND DEEP LEARNING APPROACHES

The rapid growth of unstructured textual data in various domains has necessitated the development of sophisticated techniques for multi-label text classification. Traditional methods often struggle with handling the complexity and interdependence of multiple labels, leading to suboptimal performance. This paper presents an advanced approach that integrates the Beta Ant Colony Optimization (BACO) algorithm with deep learning techniques for multi-label text classification. The BACO algorithm effectively explores the feature space and selects relevant features by leveraging pheromone trails, while the deep learning model captures intricate patterns and relationships within the textual data. The integration of these two methodologies aims to enhance the efficiency and accuracy of multi-label classification tasks, particularly in domains where label dependency is prominent. Empirical evaluations on benchmark datasets demonstrate that the proposed hybrid approach outperforms existing state-of-the-art techniques in terms of precision, recall, F1-score, and computational efficiency. The findings suggest that combining heuristic optimization algorithms with deep learning can significantly improve multi-label text classification performance, providing a robust solution for real-world applications.

Feature selection based on chi-square and ant colony optimization for multi-label classification

Text classification is widely used in organizations with large databases and digital documents. In text classification, there are many features, most of which are redundant. High-dimensional features impact multi-label classification performance. Feature selection is a data processing technique that can overcome this problem. Feature selection techniques have two major approaches: filter and wrapper. This paper proposes a hybrid filter-wrapper technique combining two algorithms: Chi-square (CS) and ant colony optimization (ACO). In the first stage, CS is used to reduce the number of irrelevant features. The ACO method is in the second stage. The ACO is applied to select the efficient features and improve classifier performance. The experiment results show that CS-ACO, CS-grey wolf optimizer (GWO), CS, and without feature selection (FS) have a micro F1-score based multinomial naïve Bayes classifier including 80%, 79.75%, 79.64% and 77.78%. The result indicates that the CS-ACO algorithm is suitable for solving multi-label classification problems.

TagCLIP: A Local-to-Global Framework to Enhance Open-Vocabulary Multi-Label Classification of CLIP without Training

Contrastive Language-Image Pre-training (CLIP) has demonstrated impressive capabilities in open-vocabulary classification. The class token in the image encoder is trained to capture the global features to distinguish different text descriptions supervised by contrastive loss, making it highly effective for single-label classification. However, it shows poor performance on multi-label datasets because the global feature tends to be dominated by the most prominent class and the contrastive nature of softmax operation aggravates it. In this study, we observe that the multi-label classification results heavily rely on discriminative local features but are overlooked by CLIP. As a result, we dissect the preservation of patch-wise spatial information in CLIP and proposed a local-to-global framework to obtain image tags. It comprises three steps: (1) patch-level classification to obtain coarse scores; (2) dual-masking attention refinement (DMAR) module to refine the coarse scores; (3) class-wise reidentification (CWR) module to remedy predictions from a global perspective. This framework is solely based on frozen CLIP and significantly enhances its multi-label classification performance on various benchmarks without dataset-specific training. Besides, to comprehensively assess the quality and practicality of generated tags, we extend their application to the downstream task, i.e., weakly supervised semantic segmentation (WSSS) with generated tags as image-level pseudo labels. Experiments demonstrate that this classify-then-segment paradigm dramatically outperforms other annotation-free segmentation methods and validates the effectiveness of generated tags. Our code is available at https://github.com/linyq2117/TagCLIP.

The Development of a Feature-Driven Analytical Approach for Gamma-Ray Spectral Analysis

Gamma-ray spectroscopy is an essential tool in nuclear science, nuclear security, and environmental monitoring. However, challenges arise in interpreting spectral data due to the presence of low counts, multiple sources, and dynamic backgrounds. To address these issues, a novel feature-driven analytical approach for gamma-ray spectral analysis using machine-learning techniques is developed. The method utilizes a series of random forest models for in-distribution (ID) multi-label classification, and the model-derived feature importance values to guide the out-of-distribution (OOD) detection task. The performance of this approach is quantitatively evaluated across various spectral parameters, including acquisition time, number of sources, energy of an OOD source, and background composition. Increasing the acquisition time from 1 s to 100 s leads to improved performance for multi-label classification, with 22 sources achieving F1-scores ≥ 0.9 after 50 s acquisitions for a CLLBC handheld detector and a standoff distance of 30 cm. The feature-driven analytical approach also demonstrates robustness when handling complex source mixtures. Furthermore, it provides contextual energetic information for OOD detection. The results presented here highlight the interpretability of the approach, establishing clear links between the spectral features and underlying physics. Moreover, the approach effectively distinguishes overlapping spectral signatures of different ID gamma-ray sources, enhancing human reliability in machine learning-based gamma-ray spectral analysis. The feature-driven analytical approach offers a promising solution to automate gamma-ray spectral analysis by addressing existing limitations and providing insights into performance across diverse spectral parameters.

Multiple marine algae identification based on three-dimensional fluorescence spectroscopy and multi-label convolutional neural network

Accurate identification of algal populations plays a pivotal role in monitoring seawater quality. Fluorescence-based techniques are effective tools for quickly identifying different algae. However, multiple coexisting algae and their similar photosynthetic pigments can constrain the efficacy of fluorescence methods. This study introduces a multi-label classification model that combines a specific Excitation-Emission matric convolutional neural network (EEM-CNN) with three-dimensional (3D) fluorescence spectroscopy to detect single and mixed algal samples. Spectral data can be input directly into the model without transforming into images. Rectangular convolutional kernels and double convolutional layers are applied to enhance the extraction of balanced and comprehensive spectral features for accurate classification. A dataset comprising 3D fluorescence spectra from eight distinct algae species representing six different algal classes was obtained, preprocessed, and augmented to create input data for the classification model. The classification model was trained and validated using 4448 sets of test samples and 60 sets of test samples, resulting in an accuracy of 0.883 and an F1 score of 0.925. This model exhibited the highest recognition accuracy in both single and mixed algae samples, outperforming comparative methods such as ML-kNN and N-PLS-DA. Furthermore, the classification results were extended to three different algae species and mixed samples of skeletonema costatum to assess the impact of spectral similarity on multi-label classification performance. The developed classification models demonstrated robust performance across samples with varying concentrations and growth stages, highlighting CNN's potential as a promising tool for the precise identification of marine algae.

Boosting Multi-Label Classification Performance Through Meta-Model

Multi-label classification problem, where each instance can be associated with multiple labels, has received considerable attention from machine learning community. To address the inherent challenges of multi-label classification including data imbalance, label dependence, and high dimensionality, ensemble approaches have been developed, gaining popularity across various real-world applications. This paper proposes a novel ensemble method called ConfBoost that addresses these challenges and enhances the generalization ability of learning systems. ConfBoost which is a meta-model based on a weighted stacking paradigm using local confidence, combines heterogeneous and complementary ensembles of multi-label classifiers. The proposed approach achieves two main objectives: Firstly, by focusing on label weights based on their confidence scores, the model can generate more relevant predictions and enhance the accuracy at the base-level by mitigating the impact of irrelevant labels during the stacking process. Moreover, assigning higher weights to certain labels exhibits better discrimination and adaptability to capture complex label relationships. Second, applying adjusted thresholds enables the model to generate predictions adapted to the specific characteristics of each label, effectively addressing imbalanced label distributions. Extensive experiments on publicly available datasets demonstrate that ConfBoost outperforms conventional combination methods and consistently surpasses related state-of-the-art methods. These findings highlight the effectiveness and potential of ConfBoost as an advanced ensemble method for multi-label classification tasks.

GA-based weighted ensemble learning for multi-label aerial image classification using convolutional neural networks and vision transformers

Multi-label classification (MLC) of aerial images is a crucial task in remote sensing image analysis. Traditional image classification methods have limitations in image feature extraction, leading to an increasing use of deep learning models, such as convolutional neural networks (CNN) and vision transformers (ViT). However, the standalone use of these models may have limitations when dealing with MLC. To enhance the generalization performance of MLC of aerial images, this paper combines two CNN and two ViT models, comparing four single deep learning models, a manually weighted ensemble learning method, and a GA-based weighted ensemble method. The experimental results using two public multi-label aerial image datasets show that the classification performance of ViT models is better than CNN models, the traditional weighted ensemble learning model performs better than a single deep learning model, and the GA-based weighted ensemble method performs better than the manually weighted ensemble learning method. The GA-based weighted ensemble method proposed in this study can achieve better MLC performance of aerial images than previous results.

Multi-Abnormality Attention Diagnosis Model Using One-vs-Rest Classifier in a Nuclear Power Plant

Multi-abnormal events, referring to the simultaneous occurrence of multiple single abnormal events in a nuclear power plant, have not been subject to consideration because multi-abnormal events are extremely unlikely to occur and indeed have not yet occurred. Such events, though, would be more challenging to diagnose than general single abnormal events, exacerbating the human error issue. This study introduces an efficient abnormality diagnosis model that covers multi-abnormality diagnosis using a one-vs-rest classifier and compares it with other artificial intelligence models. The multi-abnormality attention diagnosis model deals with multi-label classification problems, for which two methods are proposed. First, a method to effectively cluster single and multi-abnormal events is introduced based on the predicted probability distribution of each abnormal event. Second, a one-vs-rest classifier with high accuracy is employed as an efficient way to obtain knowledge on which particular multi-abnormal events are the most difficult to diagnose and therefore require the most attention to improve the multi-label classification performance in terms of data usage. The developed multi-abnormality attention diagnosis model can reduce human errors of operators due to excessive information and limited time when unexpected multi-abnormal events occur by providing diagnosis results as part of an operator support system.

Label correlation transformer for automated chest X-ray diagnosis with reliable interpretability.

Computer-aided diagnosis of chest X-ray (CXR) images can help reduce the huge workload of radiologists and avoid the inter-observer variability in large-scale early disease screening. Recently, most state-of-the-art studies employ deep learning techniques to address this problem through multi-label classification. However, existing methods still suffer from low classification accuracy and poor interpretability for each diagnostic task. This study aims to propose a novel transformer-based deep learning model for automated CXR diagnosis with high performance and reliable interpretability. We introduce a novel transformer architecture into this problem and utilize the unique query structure of transformer to capture the global and local information of the images and the correlation between labels. In addition, we propose a new loss function to help the model find correlations between the labels in CXR images. To achieve accurate and reliable interpretability, we generate heatmaps using the proposed transformer model and compare with the true pathogenic regions labeled by the physicians. The proposed model achieves a mean AUC of 0.831 on chest X-ray 14 and 0.875 on PadChest dataset, which outperforms existing state-of-the-art methods. The attention heatmaps show that our model could focus on the exact corresponding areas of related truly labeled pathogenic regions. The proposed model effectively improves the performance of CXR multi-label classification and the interpretability of label correlations, thus providing new evidence and methods for automated clinical diagnosis.

Multi-label classification of chest X-ray images with pre-trained vision Transformer model

目的 基于计算机的胸腔X线影像疾病检测和分类目前存在误诊率高，准确率低的问题。本文在视觉Transformer（vision Transformer，ViT）预训练模型的基础上，通过迁移学习方法，实现胸腔X线影像辅助诊断，提高诊断准确率和效率。方法 选用带有卷积神经网络（convolutional neural network，CNN）的ViT模型，其在超大规模自然图像数据集中进行了预训练；通过微调模型结构，使用预训练的ViT模型参数初始化主干网络，并迁移至胸腔X线影像数据集中再次训练，实现疾病多标签分类。结果 在IU X-Ray数据集中对ViT迁移学习前、后模型平均AUC（area under ROC curve）得分进行对比分析实验。结果表明，预训练ViT模型平均AUC得分为0.774，与不使用迁移学习相比提升了0.208。并针对模型结构和数据预处理进行了消融实验，对ViT中的注意力机制进行可视化，进一步验证了模型有效性。最后使用Chest X-Ray14和CheXpert数据集训练微调后的ViT模型，平均AUC得分为0.839和0.806，与对比方法相比分别有0.014～0.031的提升。结论 与其他方法相比，ViT模型胸腔X线影像的多标签分类精确度更高，且迁移学习可以在降低训练成本的同时提升ViT模型的分类性能和泛化性。消融实验与模型可视化表明，包含CNN结构的ViT模型能重点关注有意义的区域，高效获取胸腔X线影像的视觉特征。;Objective The chest X-ray-relevant screening and diagnostic method is essential for radiology nowadays. Most of chest X-ray images interpretation is still restricted by clinical experience and challenged for misdiagnose and missed diagnoses. To detect and identify one or more potential diseases in images automatically，it is beneficial for improving diagnostic efficiency and accuracy using computer-based technique. Compared to natural images，multiple lesions are challenged to be detected and distinguished accurately in a single image because abnormal areas have a small proportion and complex representations in chest X-ray images. Current convolutional neural network（CNN）based deep learning models have been widely used in the context of medical imaging. The structure of the CNN convolution kernel has sensitive to local detail information，and it is possible to extract richer image features. However，the convolution kernel cannot be used to get global information，and the features-extracted are restricted of redundant information like its relevance of background， muscles，and bones. The model’s performance in multi-label classification tasks are affected to a certain extent. At present，the vision Transformer（ViT）model has achieved its priorities in computer vision-related tasks. The ViT can be used to capture information simultaneously and effectively for multiple regions of the entire image. However，it is required to use large-scale dataset training to achieve good performance. Due to some factors like patient privacy and manual annotate costs，the size of the chest X-ray image data set has been limited. To reduce the model's dependence on data scale and improve the performance of multi-label classification，we develop the CNN-based ViT pre-training model in terms of the transfer learning method for diagnosis-assisted of chest X-ray image and multi-label classification. Method The CNN-based ViT model is pre-trained on a huge scale ground truth dataset，and it is used to obtain the initial parameters of the model. The model structure is fine-tuned according to the features of chest X-ray dataset. A 1×1 convolution layer is used to convert the chest X-ray images channels between 1 to 3. The number of output nodes of the linear layer in the classifier is balanced from 1 000 to the number of chest X-ray classification labels，and the Sigmoid is used as an activation function. The parameters of the backbone network are initialized in terms of the pre-trained ViT model parameters，and it is trained in the chest X-ray dataset after that to complete multi-label classification. The experiment is configured of Python3. 7 and PyTorch1. 8 to construct the model and RTX3090 GPU for training. Stochastic gradient descent（SGD）optimizer，binary cross-entropy（BCE）loss function，an initial learning rate of 1E-3，the cosine annealing learning rate decay are used. For training，each image is scaled to a size of 512×512 pixels，and a 224×224 pixels area and it is then cropped in random as the model input，and data augmentation is performed randomly by some of the flipping，perspective transformation， shearing，translation，zooming，and changing brightness. For testing，the chest X-ray image is scaled to 256×256 pixels and center crop a 224×224 area to input the trained model. Result The experiment is performed on the IU X-Ray，which is a small-scale chest X-ray dataset. This model is evaluated in quantitative using the average of area under ROC curve （AUC）scores across all classification labels. The results show that the average AUC score of the pre-trained ViT model is 0. 774. The accuracy and training efficiency of the non-pre-trained ViT model is dropped significantly. The average AUC score is reached to 0. 566 only，which is 0. 208 lower. In addition，the attention mechanism heat map is generated based on the ViT model，which can strengthen the interpretability of the model. A series of ablation experiments are carried out for data augmentation，model structure，and batch size design. The fine-tuned ViT model is trained on the Chest-Ray14 and CheXpert dataset as well. The average AUC score is reached to 0. 839 and 0. 806，which is optimized by 0. 014 and 0. 031. Conclusion A pre-trained ViT model is used for the multi-label classification of chest X-ray images via transfer learning. The experimental results illustrate that the ViT has its stronger multi-label classification performance in chest Xray images，and its attention mechanism is beneficial for lesions precision-focused like the interior of the chest cavity and the heart. Transfer learning is potential to improve the classification performance and model generalization of the ViT in small-scale datasets，and the training cost is reduced greatly. Ablation experiments demonstrate that the incorporated model of CNN and Transformer has its priority beyond single-structure model. Data enhancement and the batch size cutting can improve the performance of the model，but smaller scale of batch is still interlinked to longer training span. To improve the model's ability，we predict that future research direction can be focused on the extraction for complex disease and highlevel semantic information，such as their small lesions，disease location，and severity.

SHO-CNN: A Metaheuristic Optimization of a Convolutional Neural Network for Multi-Label News Classification

News media always pursue informing the public at large. It is impossible to overestimate the significance of understanding the semantics of news coverage. Traditionally, a news text is assigned to a single category; however, a piece of news may contain information from more than one domain. A multi-label text classification model for news is proposed in this paper. The proposed model is an automated expert system designed to optimize CNN’s classification of multi-label news items. The performance of a CNN is highly dependent on its hyperparameters, and manually tweaking their values is a cumbersome and inefficient task. A high-level metaheuristic optimization algorithm, spotted hyena optimizer (SHO), has higher advanced exploration and exploitation capabilities. SHO generates a collection of solutions as a group of hyperparameters to be optimized, and the process is repeated until the desired optimal solution is achieved. SHO is integrated to automate the tuning of the hyperparameters of a CNN, including learning rate, momentum, number of epochs, batch size, dropout, number of nodes, and activation function. Four publicly available news datasets are used to evaluate the proposed model. The tuned hyperparameters and higher convergence rate of the proposed model result in higher performance for multi-label news classification compared to a baseline CNN and other optimizations of CNNs. The resulting accuracies are 93.6%, 90.8%, 68.7%, and 95.4% for RCV1-v2, Reuters-21578, Slashdot, and NELA-GT-2019, respectively.

Improved Multi-Label Image Classification Performance using Supervised CNN-LSTM Deep Neural Network

Improved Multi-Label Image Classification Performance using Supervised CNN-LSTM Deep Neural Network

A label distribution manifold learning algorithm

In this paper, we propose a novel label distribution manifold learning (LDML) method for solving the multilabel distribution learning problem. First, using manifold learning, we extract the accurate and reduced-dimension features of the training data. Second, we estimate the unknown label distributions associated with the extracted reduced-dimension features based on multi-output kernel regression. Third, we use the extracted reduced-dimension features and their associated estimated label distributions to form an enhanced maximum entropy model, which enables us to accurately and efficiently estimate the unknown true label distributions for the training data. We refer to this algorithm as the LDML. We also propose to apply the tangent space alignment regression in the second stage, and the resulting algorithm is called the LDML-R. The LDML-R has better label distribution learning performance than the LDML but imposes higher complexity than the latter. We evaluate the proposed LDML and LDML-R algorithms on 15 real-world data sets with ground-truth label distributions, and the experimental results obtained show that our method has advantages in terms of learning accuracy compared to the latest multi-label distribution learning approaches. We also use another 10 real-world multi-class data sets, which do not have the ground-truth label distributions, to demonstrate the superior multilabel classification performance of our LDML-R algorithm over the existing state-of-the-art multi-label classification algorithms.

Enhancement of DNN-based multilabel classification by grouping labels based on data imbalance and label correlation

Multilabel classification (MLC) is a challenging task in real-world applications, such as project document classification which led us to conduct this research. In the past decade, deep neural networks (DNNs) have been explored in MLC due to their flexibility in dealing with annotated data. However, DNN-based MLC still suffers many problems. Two critical problems are data imbalance and label correlation. These two problems will become more prominent when a training dataset is limited and with a large label set. In this study, special neural network configurations were developed to enhance the performance of DNN-based MLC based on data imbalance and label correlation. The classification accuracy of minority labels and users-preferred labels was increased using customized label groups. The proposed method was evaluated using river restoration project documents and other fifteen datasets. The results show that the proposed method generally increases f1-score for minority labels up to 10%. Adding label dependence into label groups improves the f1-score of user-preferred majority labels up to 5%. The accuracy increase varies in different datasets.

Robust Multi-Label Relief Feature Selection Based on Fuzzy Margin Co-Optimization

Feature extraction is one of the most important tasks in multi-label learning. The performance of multi-label classification can be effectively improved by reducing the dimension of multi-label datasets. Although research on multi-label feature extraction has received extensive attention and made significant progress, further improvement is still necessary. One of the concerns is robustness, where existing multi-label feature extraction algorithms are usually sensitive to noise and outliers. To address this issue, a robust multi-label relief feature selection algorithm based on fuzzy margin co-optimization, called ML-FS-FM, is proposed in this article. Under the multi-label learning framework, the classical fuzzy relief feature weighting algorithm is introduced to ML-FS-FM, which involves the mechanisms of fuzzy feature weighting, fuzzy nearest neighbor and fuzzy instance force coefficient, so as to effectively reduce the influence of noise and outliers, and to extract feature subsets that are beneficial to the classification task. The effectiveness of the proposed algorithm is verified by a large number of experiments on multi-label datasets.

Unified Graph-Based Missing Label Propagation Method for Multilabel Text Classification

In multilabel classification, each sample can be allocated to multiple class labels at the same time. However, one of the prominent problems of multilabel classification is missing labels (incomplete labels) in multilabel text. The multilabel classification performance is reduced significantly with the presence of missing labels. In order to address the incomplete or missing label problem, this study proposes two methods: an aggregated feature and label graph-based missing label handling method (GB-AS), and a unified graph-based missing label propagation method (UG-MLP). GB-AS is used to obtain an initial label matrix based on the similarity of both document levels: feature-based weighting representation and label-based weighting representation. On the other hand, UG-MLP is introduced to construct a mixed graph that combines GB-AS and label correlations into a single groundwork. A high-order label correlation is learned from the incomplete training data and applied to supplement the missing label matrix, which guides the creation of multilabel classification models. The combination of the mixed graphs by UG-MLP is aimed to obtain the benefits of both graphs to increase the classification performance. To evaluate UG-MLP, the metrics of precision, recall and F-measure were used on three benchmark datasets, namely, the Reuters-21578, Bibtex and Enron datasets. The experimental results show that UG-MLP outperformed GB-AS as well as other state-of-the-art approaches. Therefore, we can infer from the findings that by plotting a unified graph based on joining aggregated feature and label weightings together with the label correlation, the performance of multilabel classification can be improved.

Multi-label Text Categorization using Error-correcting Output Coding with Weighted Probability

In several real-world categorization problems, labeled data is generally hard to acquire when there is a huge number of unlabeled data. Hence, it is very important to devise a novel approaches to solve these problems, thereby choosing the most valuable instances for labeling and creating a superior classifier. Several existing techniques are devised for the binary categorization issues, only a limited number of algorithms are designed for handling the multi-label cases. The multi-label classification problem turns out to be more complex when the sample belongs to multiple labels from the group of accessible classes. In World Wide Web, text data is generally present nowadays, and is an obvious example for such type of tasks. This paper develops a novel technique to perform the multi-label text categorization by modifying the Error-Correcting Output Coding (ECOC) approach. Here, a cluster of binary complimentary classifiers are employed to facilitate the ECOC more effective for the multi-class problems. In addition, a weighted posterior probability is computed to enhance the multi-label text classification performance more effectively. Moreover, the performance of the proposed ECOC with weighted probability is analyzed using the performance metrics, like precision, recall, and f-measure with maximal precision of 0.897, higher recall value of 0.896, and maximum f-measure of 0.895.

PatentNet: multi-label classification of patent documents using deep learning based language understanding

Patent classification is an expensive and time-consuming task that has conventionally been performed by domain experts. However, the increase in the number of filed patents and the complexity of the documents make the classification task challenging. The text used in patent documents is not always written in a way to efficiently convey knowledge. Moreover, patent classification is a multi-label classification task with a large number of labels, which makes the problem even more complicated. Hence, automating this expensive and laborious task is essential for assisting domain experts in managing patent documents, facilitating reliable search, retrieval, and further patent analysis tasks. Transfer learning and pre-trained language models have recently achieved state-of-the-art results in many Natural Language Processing tasks. In this work, we focus on investigating the effect of fine-tuning the pre-trained language models, namely, BERT, XLNet, RoBERTa, and ELECTRA, for the essential task of multi-label patent classification. We compare these models with the baseline deep-learning approaches used for patent classification. We use various word embeddings to enhance the performance of the baseline models. The publicly available USPTO-2M patent classification benchmark and M-patent datasets are used for conducting experiments. We conclude that fine-tuning the pre-trained language models on the patent text improves the multi-label patent classification performance. Our findings indicate that XLNet performs the best and achieves a new state-of-the-art classification performance with respect to precision, recall, F1 measure, as well as coverage error, and LRAP.