Multi-label Learning Methods Research Articles

Fuzzy bifocal disambiguation for partial multi-label learning.

In partial multi-label learning (PML), each instance is associated with multiple candidate labels, but only a subset is the ground-truth label. Due to the ambiguous label information, PML is more challenging than traditional multi-label learning. Conventional PML mainly focuses on learning a desired feature space or label space for disambiguation, ignoring the tight correlation between two spaces. To this end, a novel Fuzzy Bifocal Disambiguation for Partial Multi-Label Learning (FBD-PML) method is proposed by employing the fuzzy label confidence to connect label and feature spaces so that these double spaces alternatively refine the fuzzy label confidence for connecting themselves better. FBD-PML learns the predictive label in the label space while using the fuzzy label confidence learned from the feature space to improve the discriminative ability of the classifier. Moreover, the manifold embedding is used to preserve the structure consistency between the original data and the fuzzy label confidence, encouraging a more accurate structure for the predictive label. Extensive experiments are conducted on a number of different data sets with different evaluation metrics and consistently demonstrate the superiority of FBD-PML.

Improving the risk profile of Indonesian enterprise taxpayers using multilabel classification

Optimizing tax revenues is difficult in Indonesia due to obstacles such as tax evasion and tax avoidance. It is closely related to an organization's compliance with tax regulations, known as the taxpayers risk profile. However, this mechanism does not accurately detect tax avoidance and tax evasion risks. To overcome this limitation, we use a multilabel classification machine learning method in this study, which classifies a single observation into one or more labels at once. The approach involves problem transformation (binary relevance and label powerset), algorithm adaptation (multilabel k-nearest neighbor (ML-kNN) and multilabel-adaptive resonance associative map (ML ARAM)), and ensemble (label space partitioning and random k-label sets with disjoint (RAkELd)). Based on the model performance comparisons, we discovered that the ML-ARAM method based on deep learning is the best, with an average F1-score of 95.5% and a hamming loss of 7.4%. We also examine the feature importance of the best model to reduce the dimensions of features so that we can identify the dominant factors that encourage a taxpayer entity to engage in tax avoidance or tax evasion. The findings of this study improve the accuracy of tax avoidance risk detection and tax evasion risk profiles using machine learning methods, ensuring maximum tax revenues in Indonesia.

Incorporating view location information for multi-view multi-label learning

In multi-view multi-label learning (MVML), subspace learning provides an effective solution for integrating multi-view data. However, the current learning concentrates on the shared subspace construction process and neglects the exploration of location information, i.e., the view source of the feature, which is inherent to the feature itself. This has somewhat limited the progress of research on MVML. On this basis, we propose a multi-view multi-label learning method based on incorporating view location information (MMVLI) for the first time with reference to the Vision Transformer architecture. Firstly, the common features and special features between views are obtained through the feature extraction. The above two are combined according to the feature dimension and added into the learnable location information encoding matrix named the collaborative matrix. Then, the collaborative matrix is put into Transformer Encoder for self-attention learning to get the final feature matrix for multi-label learning. Extensive comparative experiments with existing state-of-the-art methods on six publicly available multi-view multi-label datasets demonstrate the effectiveness of MMVLI.

A novel multi-label classification deep learning method for hybrid fault diagnosis in complex industrial processes

Hybrid Fault Detection and Diagnosis, encompassing both individual and simultaneous faults, is an important solution needed in chemical process safety practice. We systematically examine the two perspectives of simultaneous faults in previous studies: multi-class and multi-label classification, and highlight the limitations of the former while demonstrating the efficacy of the latter. Then, a novel multi-label classification Hybrid Fault Transformer (mcHFT) model was put forward to address hybrid faults. Our model is capable of learning not only intrinsic features of individual faults but also their coupled relationships. Importantly, this work constitutes the first comprehensive evaluation of Hybrid FDD on the Tennessee Eastman (TE) process to our knowledge. The mcHFT model significantly enhances key performance indicators over existing models and introduces an adaptive strategy to reduce false positives. The dataset developed for this research is made available under an MIT license, contributing a valuable resource for future exploration in this field.

Machine learning based stellar classification with highly sparse photometry data.

Identifying stars belonging to different classes is vital in order to build up statistical samples of different phases and pathways of stellar evolution. In the era of surveys covering billions of stars, an automated method of identifying these classes becomes necessary. Many classes of stars are identified based on their emitted spectra. In this paper, we use a combination of the multi-class multi-label Machine Learning (ML) method XGBoost and the PySSED spectral-energy-distribution fitting algorithm to classify stars into nine different classes, based on their photometric data. The classifier is trained on subsets of the SIMBAD database. Particular challenges are the very high sparsity (large fraction of missing values) of the underlying data as well as the high class imbalance. We discuss the different variables available, such as photometric measurements on the one hand, and indirect predictors such as Galactic position on the other hand. We show the difference in performance when excluding certain variables, and discuss in which contexts which of the variables should be used. Finally, we show that increasing the number of samples of a particular type of star significantly increases the performance of the model for that particular type, while having little to no impact on other types. The accuracy of the main classifier is ∼0.7 with a macro F1 score of 0.61. While the current accuracy of the classifier is not high enough to be reliably used in stellar classification, this work is an initial proof of feasibility for using ML to classify stars based on photometry.

Towards reliable domain generalization: Insights from the PF2HC benchmark and dynamic evaluations

Deep neural networks (DNNs) are easily biased towards the training set, which causes substantial performance degradation for out-of-distribution data. Many methods are studied to generalize under various distribution shifts in the literature of domain generalization (DG). To facilitate practical DG research, we construct a large-scale non-independent and identically distributed Chinese characters dataset called PaHCC (Printed and Handwritten Chinese Characters) for a real application scenario (generalization from Printed Fonts to Handwritten Characters, PF2HC) of DG methods. We evaluate eighteen DG methods on the proposed PaHCC dataset and demonstrate that the performance of the current algorithms on this dataset remains inadequate. To improve the performance, we propose a radical-based multi-label learning method by integrating structure learning into statistical methods. Moreover, in the dynamic evaluation settings, we discover additional properties of DG methods and demonstrate that many algorithms suffer from unstable performances. We advocate that researchers in the DG community pay attention not only to accuracy under the fixed leave-one-domain-out protocol but also to algorithmic stability across variable training domains in future studies. Our dataset, method, and evaluations bring valuable insights to the DG community and could promote the development of realistic and stable algorithms.

Multi-Label Noise Robust Collaborative Learning for Remote Sensing Image Classification.

The development of accurate methods for multi-label classification (MLC) of remote sensing (RS) images is one of the most important research topics in RS. The MLC methods based on convolutional neural networks (CNNs) have shown strong performance gains in RS. However, they usually require a high number of reliable training images annotated with multiple land-cover class labels. Collecting such data is time-consuming and costly. To address this problem, the publicly available thematic products, which can include noisy labels, can be used to annotate RS images with zero-labeling cost. However, multi-label noise (which can be associated with wrong and missing label annotations) can distort the learning process of the MLC methods. To address this problem, we propose a novel multi-label noise robust collaborative learning (RCML) method to alleviate the negative effects of multi-label noise during the training phase of a CNN model. RCML identifies, ranks, and excludes noisy multi-labels in RS images based on three main modules: 1) the discrepancy module; 2) the group lasso module; and 3) the swap module. The discrepancy module ensures that the two networks learn diverse features, while producing the same predictions. The task of the group lasso module is to detect the potentially noisy labels assigned to multi-labeled training images, while the swap module is devoted to exchange the ranking information between two networks. Unlike the existing methods that make assumptions about noise distribution, our proposed RCML does not make any prior assumption about the type of noise in the training set. The experiments conducted on two multi-label RS image archives confirm the robustness of the proposed RCML under extreme multi-label noise rates. Our code is publicly available at: https://www.noisy-labels-in-rs.org.

An artificial immune system algorithm for classification tasks. An electronic nose case study

This study explores the application of an electronic nose (e-nose), employing an Artificial Immune System (AIS) as a competitive alternative to the typical machine learning techniques. The proposed algorithm's main function is to modify its dimensions and adjust the values of its parameters during the training or mutation process, with no complex operations and minimal parameter tuning. This analytical tool has been applied to the qualitative analysis of chocolate bars, classifying varieties in single and multi-label scenarios. The classification results demonstrate promising results in accurately distinguishing individual chocolate types, obtaining an average accuracy of 97%. Subsequently, multi-label classification (several labels associated with a single chocolate) was executed based on shared ingredient content, achieving an average accuracy of 94%. The evaluation and validation of the proposed AIS were also verified; the obtained performance metrics were compared with those of typical classification techniques, k-Nearest Neighbors (KNN), Support Vector Machine (SVM), and Artificial Neural Network (ANN). Satisfactory results show the competitive performance of AIS in single-label classification tasks compared to named techniques. Promisingly, a novel method for multi-label learning in e-noses is obtained to accurately identify different types of chocolates, outperforming methods like KNN and SVM. These results demonstrate the AIS potential as an effective classification method for e-nose applications.

Multi-label deep transfer learning method for coupling fault diagnosis

With the development of complexity and integration of machines, the multiple components of the system are prone to simultaneous failures and the multi-fault signals may be measured in limited sensors. This fault mode is defined as coupling fault in this paper and becomes more common and attracts great attention. Current coupling fault diagnosis approaches rely on labeled data under consistent working conditions. However, in practical varying working conditions, the data is difficult to label and the traditional methods perform poorly. To address the challenge problem, we utilize some related annotated data under other working conditions for auxiliary learning and abstract this problem as a multi-label transfer learning problem which is a new research topic in fault diagnosis. We proposed a novel deep transfer coupling fault diagnosis method from the global and local views to achieve the effect of transfer, which is the first attempt to address this practical problem as far as we know. At the global level, we used Maximum Mean Discrepancy (MMD) at multiple stages of the network to reduce the hypothesis space of the model. Furthermore, we design a special manifold framework with multi-level similarity to constrain the hierarchy of the distance of the sample and the hierarchy of the labels being consistent between the two domains, to further improve the effect of local alignment. The proposed method has high accuracy in multiple transfer tasks for both public and laboratory datasets, powerfully demonstrating its effectiveness.

Depicting Risk Profile over Time: A Novel Multiperiod Loan Default Prediction Approach

With the rapid development of fintech, the need for dynamic credit risk evaluation is becoming increasingly important. While previous studies on credit scoring have mostly focused on single-period loan default prediction, we call for a new avenue—multiperiod default prediction (MPDP)—to depict risk profiles over time. To address the challenges raised by MPDP, such as monotonic default probability prediction and complex relationship accommodation, we propose a novel approach, hybrid and collective scoring (HACS). We design a hybrid modeling strategy to predict whether and when a borrower will default separately through a default discrimination model and a default time estimation model, respectively, and synthesize them through a probabilistic framework. To accommodate various possible patterns of default time and measure the distribution of default probability over successive time intervals, we propose a joint default modeling method to train the default time estimation model. Empirical evaluations at the model (time-to-default prediction performance and discrimination performance) and mechanism (identifiability and discriminability) levels, as well as impact analyses at the application (granting performance and profitability performance) level, show that HACS outperforms the benchmarked survival analysis and multilabel learning methods on all fronts. It can more accurately predict time-to-default and provide financial institutions and investors better decision-support in granting loans and selecting loan portfolios.

A partition-based problem transformation algorithm for classifying imbalanced multi-label data

Multi-label learning has garnered much research interest due to its wide range of real-world applications. Many multi-label learning methods have been proposed; however, few have addressed the class imbalance problem existing in multi-label data. Even though some studies have taken this issue into account, most of them have ignored the label correlations or only considered random correlations between them. In this study, we propose a novel partition-based imbalanced multi-label learning algorithm, named Multi-label Learning based on Hierarchical Clustering (MLHC), to tackle this problem. MLHC first carries out hierarchical clustering on the original label space to divide it into several disconnected subspaces, each of which contains several labels that are strongly correlated with each other. Then, for each label subspace, we use the problem transformation strategy to convert it into a multi-class problem by binary coding. Any multi-class imbalance learning algorithm can be applied to the transformed multi-class data. Finally, the classification results will be decoded to retrieve the corresponding label subspace, and all label subspace results are combined to show the predicted label vector in the original label space. We conducted experiments not only on thirteen benchmark multi-label datasets but also carried out them on XJTU-SY which is a multi-label engineering application dataset, and the results indicated that our proposed MLHC learning algorithm outperforms several state-of-the-art class imbalance multi-label learning algorithms, demonstrating the effectiveness and necessity of discovering label correlations and transforming the original imbalanced multi-label learning problem into multiple strongly correlated multi-class imbalanced learning problems.

A new multi-view multi-label model with privileged information learning

In multi-view multi-label learning (MVML), the data is described by multiple feature views and annotated by a number of categorical labels. At present, most of the existing MVML methods are proposed based on subspace learning, neural networks and so on. There is little work done on support vector machine (SVM)-based MVML. In this paper, we propose a novel SVM-based multi-view multi-label learning method with privileged information learning (MVMLP). By introducing the idea of privileged information learning, MVMLP implements both the consensus principle and complementarity principle in MVML. Specifically, we enforce similarity between the outputs of different views to explore the consensus information. Moreover, we allow different views to serve as privileged information for each other, such that the complementary information among distinct views can be introduced into the training process. MVMLP constructs an SVM-based model for each view and trains the models of multiple views jointly. The derived learning problem can be efficiently solved by adapting the Franke-Wolfe algorithm. Experimental results on real-life datasets show that MVMLP delivers explicitly better classification performance than state-of-the-art MVML methods.

Missing multi-label learning based on the fusion of two-level nonlinear mappings

The relationship between features and labels is an important factor affecting multi-label learning. Many existing multi-label learning approaches decompose label space into a low-rank latent label space and construct linear mappings between the feature space and the latent label space. This type of models may not depict the true data distribution well in high-dimensional spaces. In this work, a model called missing multi-label learning based on the fusion of two-level nonlinear mappings (MML-TLNM) is proposed. Specifically, in the second-level mapping, the label matrix is nonlinearly decomposed into a low-rank latent label matrix and a correlation coefficient matrix to realize the nonlinear transformation from the latent label space to the actual label space. In the first-level mapping, a nonlinear relationship is constructed from the sample matrix to the latent label matrix to realize the nonlinear transformation between the original feature space and the latent label space. The two levels of nonlinear mappings are fused with each other in a nested manner and better depict the true distribution of multi-label data. In addition, the F-norm regularization constraints are imposed on the variables in the two-level nonlinear mappings. Extensive comparative experiments showed that the proposed learning model has good robustness and is superior to the state-of-the-art multi-label learning methods.

Learning Common and Task-Specific Radiomic Features via Graph Regularized NMF for the Joint Prediction of Multiple Clinical Indicators in Breast Cancer.

Assessments of multiple clinical indicators based on radiomic analysis of magnetic resonance imaging (MRI) are beneficial to the diagnosis, prognosis and treatment of breast cancer patients. Many machine learning methods have been designed to jointly predict multiple indicators for more accurate assessments while using original clinical labels directly without considering the noisy and redundant information among them. To this end, we propose a multilabel learning method based on label space dimensionality reduction (LSDR), which learns common and task-specific features via graph regularized nonnegative matrix factorization (CTFGNMF) for the joint prediction of multiple indicators in breast cancer. A nonnegative matrix factorization (NMF) is adopted to map original clinical labels to a low-dimensional latent space. The latent labels are employed to exploit task correlations by using a least square loss function with [Formula: see text]-norm regularization to identify common features, which help to improve the generalization performance of correlated tasks. Furthermore, task-specific features were retained by a multitask regression formulation to increase the discrimination power for different tasks. Common and task-specific features are incorporated by dynamic graph Laplacian regularization into a unified model to learn complementary features. Then, a multilabel classification is built to predict multiple clinical indicators including human epidermal growth factor receptor 2 (HER2), Ki-67, and histological grade. Experimental results show that CTFGNMF achieves AUCs of 0.823, 0.691 and 0.776 in the three indicator predictions, outperforming other counterparts that consider only task-independent features or common features. It indicates CTFGNMF is a promising application for multiple classification tasks in breast cancer.

Graph-Based Class-Imbalance Learning With Label Enhancement.

Class imbalance is a common issue in the community of machine learning and data mining. The class-imbalance distribution can make most classical classification algorithms neglect the significance of the minority class and tend toward the majority class. In this article, we propose a label enhancement method to solve the class-imbalance problem in a graph manner, which estimates the numerical label and trains the inductive model simultaneously. It gives a new perspective on the class-imbalance learning based on the numerical label rather than the original logical label. We also present an iterative optimization algorithm and analyze the computation complexity and its convergence. To demonstrate the superiority of the proposed method, several single-label and multilabel datasets are applied in the experiments. The experimental results show that the proposed method achieves a promising performance and outperforms some state-of-the-art single-label and multilabel class-imbalance learning methods.

ECC + +: An algorithm family based on ensemble of classifier chains for classifying imbalanced multi-label data

Multi-label learning has attracted a great deal of research interests as it has a wide range of real-world applications. Although many multi-label learning methods have been proposed, very few of them have addressed the problem of class imbalance distribution in multi-label data. Moreover, most of the existing class imbalance multi-label learning algorithms only focus on solving the class imbalance problem, without taking into account the correlations among labels. To address these issues simultaneously, we propose to combine the well-known ensemble of classifier chain (ECC) algorithm with various binary-class imbalance learning techniques such as sampling, cost-sensitive learning, and threshold moving. This approach creates a new algorithm family called ECC++, designed specifically for class imbalance multi-label learning. ECC is already an excellent ensemble high-order binary relevance multi-label learning algorithm that is well-suited to exploiting correlations among labels. Combining it with binary-class imbalance learning techniques enables each link in a classifier chain (CC) to overcome the negative effect of skewed data distribution. ECC++ is a dynamic algorithm family that can be extended arbitrarily by applying any new binary-class imbalance learning techniques. To demonstrate the effectiveness and superiority of the proposed ECC++ algorithm family, we developed several ECC++ family members using some popular binary-class imbalance learning techniques. We then compared them with several state-of-the-art class imbalance multi-label learning algorithms on twelve benchmark and four real-world multi-label datasets. Our experimental results showed the effectiveness and superiority of the proposed ECC++ algorithm family over existing class imbalance multi-label learning algorithms. In conclusion, the proposed ECC++ algorithm family combines the strengths of the well-established ECC algorithm and binary-class imbalance learning techniques, resulting in a superior methodology for class imbalance multi-label learning.

A nonlinear multi-label learning model based on Tanh mapping

The relationship between features and labels plays an important role in multi-label learning. The purpose of multi-label learning is to learn a mapping from the feature space to the label space. Many of the existing methods decompose the observed label matrix into the product of a low-rank latent label matrix and a projection matrix, and construct a linear mapping between the feature space and low-rank latent label space. However, in practice, the relationship between two high-dimensional spaces is often nonlinear. Consequently, these linear models may not sufficiently capture the data distribution and the true dependencies between features and labels. Herein, a novel classification model for multi-label learning is proposed, which considers nonlinear mapping from the feature space to the latent label space. Because of correlation between labels, the label matrix is usually of a low rank. To explore the low-rank structure, a latent label matrix is obtained using the decomposition of the original label matrix. To characterize the nonlinear relationship between features and latent labels, the tanh function is employed to learn a nonlinear classifier. The tanh function describes the distribution of labels by constraining the domain of the latent label space to the interval [−1,1]. In addition, the Frobenius norm regularization constraint is imposed on the variables in the model for optimization. The experimental results demonstrate that the proposed method is robust and comparable to state-of-the-art multi-label learning methods.

Progressive Enhancement of Label Distributions for Partial Multilabel Learning.

Partial multi-label learning (PML) aims to learn a multilabel predictive model from the PML training examples, each of which is associated with a set of candidate labels where only a subset is valid. The common strategy to induce a predictive model is identifying the valid labels in each candidate label set. Nonetheless, this strategy ignores considering the essential label distribution corresponding to each instance as label distributions are not explicitly available in the training dataset. In this article, a novel partial multilabel learning method is proposed to recover the latent label distribution and progressively enhance it for predictive model induction. Specifically, the label distribution is recovered by considering the observation model for logical labels and the sharing topological structure from feature space to label distribution space. Besides, the latent label distribution is progressively enhanced by recovering latent labeling information and supervising predictive model training alternatively to make the label distribution appropriate for the induced predictive model. Experimental results on PML datasets clearly validate the effectiveness of the proposed method for solving partial multilabel learning problems. In addition, further experiments show the high quality of the recovered label distributions and the effectiveness of adopting label distributions for partial multilabel learning.

Generative Multi-Label Correlation Learning

In real-world applications, a single instance could have more than one label. To solve this task, multi-label learning methods emerged in recent years. It is a more challenging problem for many reasons, such as complex label correlation, long-tail label distribution, and data shortage. In general, overcoming these challenges and bettering learning performance could be achieved by utilizing more training samples and including label correlations. However, these solutions are expensive and inflexible. Large-scale, well-labeled datasets are difficult to obtain, and building label correlation maps requires task-specific semantic information as prior knowledge. To address these limitations, we propose a general and compact Multi-Label Correlation Learning (MUCO) framework. MUCO explicitly and effectively learns the latent label correlations by updating a label correlation tensor, which provides highly accurate and interpretable prediction results. In addition, a multi-label generative strategy is deployed to handle the long-tail label distribution challenge. It borrows the visual clues from limited samples and synthesizes more diverse samples. All networks in our model are optimized simultaneously. Extensive experiments illustrate the effectiveness and efficiency of MUCO. Ablation studies further prove the effectiveness of all the modules.

Влияние проблемы многозначности меток классов системных журналов на защищенность компьютерных сетей

The security of information circulating in a computer network is related to the security of the supporting infrastructure. An important problem in the intelligent processing of syslog data is the existence of multi-label datasets. Among the Russian-language scientific publications, the problem under consideration in the context of information security of computer networks is not presented. Purpose: increase the security of computer networks by using multi-label learning methods when solving the problem of classifying system logs class labels. Results: A comparative analysis of single-valued and multi-label classifiers was carried out in a computational experiment on the Mean accuracy metric. A non-linear relationship was found between the proportion of experimental data sections containing multi-label class labels and the overall accuracy of data classification. Despite the fact that multilabel plots in the studied experimental data are only 3%, the gain in accuracy reaches 23% according to the specified metric. According to the results of the analysis, 80% of unambiguous classifiers were inferior in classification accuracy according to the Mean accuracy multi-label metric to their analogues, which may signal a strong influence of multi-label class labels on the models under consideration. It is shown that the considered structure of experimental data in a tabular form is affected by the multi-label problem much more strongly than it can be estimated by a standard frequency check, which actualizes further research in this direction. Practical relevance: The practical significance of the results obtained lies in increasing the security of computer networks through the use of a multi-label approach in the classification problem. The tasks of information security solved by multi-label classification may include: the area of monitoring, detection or prevention of violations and computer attacks in computer networks. Discussion: Since the predictive power of frequency testing of the influence of multi-label class label results on the classification results of unambiguous classifiers is low, further research on this topic is planned. It is planned to expand the list of classification quality assessment metrics in future experiments.