Multi-label Classification Algorithms Research Articles

A novel random fast multi-label deep forest classification algorithm

A novel random fast multi-label deep forest classification algorithm

An efficient class-dependent learning label approach using feature selection to improve multi-label classification algorithms

An efficient class-dependent learning label approach using feature selection to improve multi-label classification algorithms

Soft-label recover based label-specific features learning

Presently, multi-label classification algorithms are mainly based on positive and negative logical labels, which have achieved good results. However, logical labeling inevitably leads to the label misclassification problem. In addition, missing labels are common in multi-label datasets. Recovering missing labels and constructing soft labels that reflect the mapping relationship between instances and labels is a difficult task. Most existing algorithms can only solve one of these problems. Based on this, this paper proposes a soft-label recover based label-specific features learning (SLR-LSF) to solve the above problems simultaneously. Firstly, the information entropy is used to calculate the confidence matrix between labels, and the membership degree of soft labels is obtained by combining the label density information. Secondly, the membership degree and confidence matrix are combined to construct soft labels, and this process not only solves the problem of missing labels but also obtains soft labels with richer semantic information. Finally, in the process of learning specific label features for soft labels. The local smoothness of the labels learned through stream regularization is complemented by the global label correlation, thus improving the classification performance of the algorithm. To demonstrate the effectiveness of the proposed algorithm, we conduct comprehensive experiments on several datasets.

Toward Multilabel Classification for Multiple Disease Prediction Using Gut Microbiota Profiles.

Advancements in high-throughput technologies have yielded large-scale human gut microbiota profiles, sparking considerable interest in exploring the relationship between the gut microbiome and complex human diseases. Through extracting and integrating knowledge from complex microbiome data, existing machine learning (ML)-based studies have demonstrated their effectiveness in the precise identification of high-risk individuals. However, these approaches struggle to address the heterogeneity and sparsity of microbial features and explore the intrinsic relatedness among human diseases. In this work, we reframe human gut microbiome-based disease detection as a multilabel classification (MLC) problem and integrate a range of innovative techniques within the proposed MLC framework, aptly named GutMLC. Specifically, the entity semantic similarity as priori knowledge is incorporated into multilabel feature selection and loss functions by capturing the shared attributes and inherent associations among diseases and microbes. To tackle the issue of label imbalance, both within and between labels, we adapt the focal loss (FL) function for MLC using debiased inverse weighting. Extensive experiment results consistently demonstrate the competitive performance of GutMLC in comparison with commonly used MLC and single-label classification (SLC) algorithms. This work seeks to unlock the potential of gut microbiota as robust biomarkers for multiple disease prediction.

A semi-supervised learning algorithm for multi-label classification and multi-assignment clustering problems based on a Multivariate Data Analysis

Classically, in classification and clustering problems, an individual is exclusively assigned to a class (labeled data) or a cluster (unlabeled data). However, the assignment to a single class or cluster may be too strict in several real-world domains. In many cases, an individual may belong to multiple classes or clusters at the same time. On the other hand, we are starting to see machine learning algorithms that solve the problem of multi-label classification and multiple cluster assignment, but there are no algorithms that solve both problems simultaneously. Classic semi-supervised algorithms can work with labeled and unlabeled data simultaneously, but these types of algorithms assign individuals to a single class or a single cluster. Today, artificial intelligence faces the challenge of developing semi-supervised learning algorithms to work with semi-labeled data that can be assigned to different classes or clusters at the same time. In particular, this paper proposes a semi-supervised learning algorithm to fill this gap, which can solve the problems of multi-label classification and multiple cluster assignment simultaneously, on a semi-labeled dataset. This proposal is based on the LAMDA algorithm (Learning Algorithm for Multivariate Data Analysis), which calculates the degree of membership of a data to a group/class. In this work, a membership threshold is defined, which allows individuals to be assigned to classes or clusters that have a membership greater than this membership threshold. Thus, the main contribution of this work is the development of a semi-supervised algorithm that can process semi-labeled datasets to assign them to multiple classes and/or clusters. Furthermore, the work defines a metric to evaluate its efficiency in a semi-supervised context, called Multi Label-Cluster Index (MULCI). This proposal is tested on several datasets from the domains of multi-label classification or multi-assignment clustering, or a combination of both, showing very encouraging results. Very good quality metrics results are achieved in multiclass and multicluster tasks.

FDIA localization and classification detection in smart grids using multi-modal data and deep learning technique

False data injection attacks (FDIA) exploit the vulnerabilities of bad data detection in energy management systems to maliciously tamper with state estimation results, seriously jeopardizing the safe and reliable operation of power systems. In order to promptly detect FDIA, recent studies have used machine learning techniques to extract attack characteristics and detect FDIA based on changes in these characteristics. Current research predominantly focuses on detecting a single type of FDIA attack model and topology. However, the increasing diversification of FDIA construction methods and the variability of topological structures in power systems can reduce or even invalidate detection effectiveness. To cope with the abovementioned difficulties, this study introduces a multimodal deep learning detection model based on variational graph auto-encoders (VGAE), temporal convolutional networks (TCN), and gated recurrent units (GRU). The topological features of power systems are obtained through VGAE to adapt to the detection needs of various topological structures and performance enhancement. These features are then integrated with preprocessed measurement data via BDD to form multimodal data. Furthermore, this multimodal data is used to locate and classify FDIA with complete information, FDIA with incomplete information, and topology attack after applying a multi-label classification algorithm based on TCN-GRU for temporal feature extraction. Experiments carried out on the IEEE 14 and IEEE 118 bus systems show that the proposed method is robust and has high detection performance. The results indicate that the proposed detection method achieves AUC values that are, on average, 0.085, 0.145, and 0.04 higher than those of CNN, LSTM, and CNN-LSTM, respectively. Moreover, under various noise and attack intensities, recall, precision, F1 score, and RACC remain above 0.859, 0.877, 0.867, and 0.818, respectively, with a classification accuracy greater than 0.912. This study provides a unique perspective on detecting FDIA across various attack models and power system topologies.

An Underwater Multi-Label Classification Algorithm Based on a Bilayer Graph Convolution Learning Network with Constrained Codec

An Underwater Multi-Label Classification Algorithm Based on a Bilayer Graph Convolution Learning Network with Constrained Codec

Application of CNN for multiple phase corrosion identification and region detection

Corrosion is a significant issue that contributes negatively to the degradation of materials most especially metals. To ensure proper maintenance, enhance reliability and prevent breakdown, it is very essential to not only effectively detect corrosion but to also understand its locations and distributions on the materials. A Multiple phase Convolutional Neural Network (CNN) model is created for this purpose. The Multiple phase CNN model consists of custom designed deep learning algorithms at various stages. This created the opportunity to make use of binary classification, multi-label classification and patch distribution algorithm to detect and identify corrosion regions on metallic materials. Six (6) different labels of corrosion were modelled to represent different levels of degradation using 600 anonymized images. The images were used in the various stages of the framework for training the respective models. Results at the binary level shows 94.87 % of corrosion detection. The multiclass stage of the Multiple phase CNN records the highest accuracy of 92.1 %. The patch distribution stage recorded a highest accuracy of 96.5 % and 94.6 % for the Average Image and Average Pixel ROCAUC (Region of Concentration Area Under Cover). It also shows a region segment average accuracy detection of 91.5 % (image level) and 89.2 %(pixel level) for 9 distinct regions. The research provides a comprehensive and detailed reliability and maintenance information for Aerospace, Transport and Manufacturing Materials experts and non-experts. The framework shows a robust approach to detecting corrosion which is essential for critical and safety applications as well as preventing economic loss due to corrosion. This can also be extended to other domains beyond the corrosion case study.

A novel bayesian network-based ensemble classifier chains for multi-label classification

In this paper, we address the challenges of random label ordering and limited interpretability associated with Ensemble Classifier Chains (ECC) by introducing a novel ECC method, ECC-MOO&BN, which integrates Bayesian Networks (BN) and Multi-Objective Optimization (MOO). This approach is designed to concurrently overcome these ECC limitations. The ECC-MOO&BN method focuses on extracting diverse and interpretable label orderings for the ECC classifier. We initiated this process by employing mutual information to investigate label relationships and establish the initial structures of the BN. Subsequently, an enhanced NSGA-II algorithm was applied to develop a series of Directed Acyclic Graphs (DAGs) that effectively balance the likelihood and complexity of the BN structure. The rationale behind using the MOO method lies in its ability to optimize both complexity and likelihood simultaneously, which not only diversifies DAG generation but also helps avoid overfitting during the production of label orderings. The DAGs, once sorted topologically, yielded a series of label orderings, which were then seamlessly integrated into the ECC framework for addressing multi-label classification (MLC) problems. Experimental results show that when benchmarked against eleven leading-edge MLC algorithms, our proposed method achieves the highest average ranking across seven evaluation criteria on nine out of thirteen MLC datasets. The results of the Friedman test and Nemenyi test also indicate that the performance of the proposed method has a significant advantage compared to other algorithms.

Compressed sensing: a discrete optimization approach

We study the Compressed Sensing (CS) problem, which is the problem of finding the most sparse vector that satisfies a set of linear measurements up to some numerical tolerance. CS is a central problem in Statistics, Operations Research and Machine Learning which arises in applications such as signal processing, data compression, image reconstruction, and multi-label learning. We introduce an ℓ2\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\ell _2$$\\end{document} regularized formulation of CS which we reformulate as a mixed integer second order cone program. We derive a second order cone relaxation of this problem and show that under mild conditions on the regularization parameter, the resulting relaxation is equivalent to the well studied basis pursuit denoising problem. We present a semidefinite relaxation that strengthens the second order cone relaxation and develop a custom branch-and-bound algorithm that leverages our second order cone relaxation to solve small-scale instances of CS to certifiable optimality. When compared against solutions produced by three state of the art benchmark methods on synthetic data, our numerical results show that our approach produces solutions that are on average 6.22%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$6.22\\%$$\\end{document} more sparse. When compared only against the experiment-wise best performing benchmark method on synthetic data, our approach produces solutions that are on average 3.10%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$3.10\\%$$\\end{document} more sparse. On real world ECG data, for a given ℓ2\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\ell _2$$\\end{document} reconstruction error our approach produces solutions that are on average 9.95%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$9.95\\%$$\\end{document} more sparse than benchmark methods (3.88%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$3.88\\%$$\\end{document} more sparse if only compared against the best performing benchmark), while for a given sparsity level our approach produces solutions that have on average 10.77%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$10.77\\%$$\\end{document} lower reconstruction error than benchmark methods (1.42%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$1.42\\%$$\\end{document} lower error if only compared against the best performing benchmark). When used as a component of a multi-label classification algorithm, our approach achieves greater classification accuracy than benchmark compressed sensing methods. This improved accuracy comes at the cost of an increase in computation time by several orders of magnitude. Thus, for applications where runtime is not of critical importance, leveraging integer optimization can yield sparser and lower error solutions to CS than existing benchmarks.

A principal label space transformation and ridge regression-based hybrid gorilla troops optimization and jellyfish search algorithm for multi-label classification

A principal label space transformation and ridge regression-based hybrid gorilla troops optimization and jellyfish search algorithm for multi-label classification

Lazy Multi-Label Classification algorithms based on Non-Parametric Predictive Inference

Multi-Label Classification (MLC) extends standard classification in the sense that an instance might belong to multiple labels simultaneously. Many lazy approaches to MLC have been proposed so far. The majority of them, to classify an instance, use statistical estimators from the neighboring instances based on classical probability theory. In this work, we propose lazy algorithms for MLC that employ the Non-Parametric Predictive Inference Model (NPI-M) for the statistical estimators based on the neighboring instances. It is shown that our proposed lazy MLC algorithms are more suitable to tackle the class-imbalance problem that usually arises in MLC, especially when data contain label noise. This issue is corroborated via an exhaustive experimental analysis.

Imbalance-Robust Multi-Label Self-Adjusting kNN

In the task of multi-label classification in data streams, instances arriving in real time need to be associated with multiple labels simultaneously. Various methods based on the k Nearest Neighbors algorithm have been proposed to address this task. However, these methods face limitations when dealing with imbalanced data streams, a problem that has received limited attention in existing works. To approach this gap, this paper introduces the Imbalance-Robust Multi-Label Self-Adjusting kNN (IRMLSAkNN), designed to tackle multi-label imbalanced data streams. IRMLSAkNN’s strength relies on maintaining relevant instances with imbalance labels by using a discarding mechanism that considers the imbalance ratio per label. On the other hand, it evaluates subwindows with an imbalance-aware measure to discard older instances that are lacking performance. We conducted statistical experiments on 32 benchmark data streams, evaluating IRMLSAkNN against eight multi-label classification algorithms using common accuracy-aware and imbalance-aware measures. The obtained results demonstrate that IRMLSAkNN consistently outperforms these algorithms in terms of predictive capacity and time cost across various levels of imbalance.

Multi-Label Sampling based on Label Imbalance Rate and Neighborhood Distribution

Existing multi-label classification algorithms often assume that label distribution in the training set is balanced, but practical datasets frequently exhibit significant label imbalance. This imbalance affects the learning and generalization performance of the classifiers. To address the problem of label imbalance in multi-label classification, this paper proposes a new synthetic oversampling algorithm, named Multi-Label Synthetic Oversampling based on Label Imbalance Rate and Neighborhood Distribution (MLSIN). This algorithm synthesizes new samples by considering both the imbalance rate of labels and the distribution of samples in their neighborhood, aiming to improve the classifiers performance on minority labels. The rest of this chapter first introduces the evaluation metrics for multi-label classification effectiveness. Then it defines and computes the degree of label imbalance, describes the calculation of imbalance weights, and proposes a sample type correction penalty strategy, detailing the algorithm's process for selecting base and auxiliary samples., and validates the proposed method on public datasets and summarizes the experimental results.

Semantic features analysis for biomedical lexical answer type prediction using ensemble learning approach

Lexical answer type prediction is integral to biomedical question–answering systems. LAT prediction aims to predict the expected answer’s semantic type of a factoid or list-type biomedical question. It also aids in the answer processing stage of a QA system to assign a high score to the most relevant answers. Although considerable research efforts exist for LAT prediction in diverse domains, it remains a challenging biomedical problem. LAT prediction for the biomedical field is a multi-label classification problem, as one biomedical question might have more than one expected answer type. Achieving high performance on this task is challenging as biomedical questions have limited lexical features. One biomedical question must be assigned multiple labels given these limited lexical features. In this paper, we develop a novel feature set (lexical, noun concepts, verb concepts, protein–protein interactions, and biomedical entities) from these lexical features. Using ensemble learning with bagging, we use the label power set transformation technique to classify multi-label. We evaluate the integrity of our proposed methodology on the publicly available multi-label biomedical questions dataset (MLBioMedLAT) and compare it with twelve state-of-the-art multi-label classification algorithms. Our proposed method attains a micro-F1 score of 77%, outperforming the baseline model by 25.5%.

A Hybrid Principal Label Space Transformation-Based Binary Relevance Support Vector Machine and Q-Learning Algorithm for Multi-label Classification

A Hybrid Principal Label Space Transformation-Based Binary Relevance Support Vector Machine and Q-Learning Algorithm for Multi-label Classification

Learning Label Correlations for Multi-Label Online Passive Aggressive Classification Algorithm

Label correlations are an essential technique for data mining that solves the possible correlation problem between different labels in multi-label classification. Although this technique is widely used in multi-label classification problems, batch learning deals with most issues, which consumes a lot of time and space resources. Unlike traditional batch learning methods, online learning represents a promising family of efficient and scalable machine learning algorithms for large-scale datasets. However, existing online learning research has done little to consider correlations between labels. On the basis of existing research, this paper proposes a multi-label online learning algorithm based on label correlations by maximizing the interval between related labels and unrelated labels in multi-label samples. We evaluate the performance of the proposed algorithm on several public datasets. Experiments show the effectiveness of our algorithm.

Cost-sensitive budget adaptive label thresholding algorithms for large-scale online multi-label classification

Cost-sensitive budget adaptive label thresholding algorithms for large-scale online multi-label classification

Online Passive-Aggressive Multilabel Classification Algorithms.

Most existing multilabel classification methods are batch learning methods, which may suffer from expensive retraining costs when dealing with new incoming data. In order to overcome the drawbacks of batch learning, we develop a family of online multilabel classification algorithms, which can update the model instantly and efficiently, and make a timely online prediction when new data arrive. Our algorithms all take a closed-form update, which is obtained by solving a constrained optimization problem in each round of online learning. Label correlation is explicitly modeled in our optimization problem. The label thresholding function, an important component of our online classifier, can also be learned online. Our algorithms can be easily generalized to the nonlinear prediction cases using Mercer kernels. The worst case loss bounds for our algorithms are provided. The bounds are relative to the cumulative loss suffered by the best fixed predictive model that can be attained in hindsight. Finally, we corroborate the merits of our algorithms in both linear and nonlinear predictions on nine open multilabel benchmark datasets.

Exploiting local label correlation from sample perspective for multi-label classification via three-way decision theory

In multi-label classification, the expansion of output dimension seriously interferes learning performance, and even fails to build a joint prediction model. In order to restrain the proliferation of multi-label classifier’s hypothesis space, the current works focus on the application of global positive label correlation. However, the “black or white” mechanism ignore other possible forms of label correlation, such as negative or neutral correlation. By introducing the doctrine of the mean, three-way decision (3WD) theory provides a solution for in-depth research on local label correlation, and aims to handle the uncertainty of multi-label learning tasks. In this paper, a novel learning algorithm for multi-label joint classification, namely ML-3WD, is proposed by considering the 3WD label correlation from the perspective of samples. According to the weights of different features on any label, the comprehensive loss of each sample to three action strategies can be measured. Obviously, the 3WD rules for any label variable in multi-label output space is obtained. By aggregating the cutting thresholds between different labels, the division principles of 3WD label correlation are further established. Given any multi-label sample, the local fuzzy membership to co-occurrence or mutual state for label pair is examined based on kernelized fuzzy rough sets. The 3WD local label relevance of each sample is confirmed, that is, positive, negative or neutral. The global application strategy for multi-label classification is utilized to avoid over-fitting induced by local mining strategy. Based on the integral mean of the distribution of 3WD local label relevance in multi-label sample space, two different versions of empirical label relevance are constructed. By constraining the relative position between sub-separation hyperplanes, the 3WD label correlation distribution-based model for multi-label joint classification is designed. The experiment results on fifteen real world multi-label datasets reflect that our algorithm achieves good classification ability and versatility. The impact of core parameters on learning performance is also dissected.