Learning For Multi-label Classification Research Articles

Deep Learning for Multilabel Classification of Coral Reef Conditions in the Indo‐Pacific Using Underwater Photo Transect Method

ABSTRACTBecause coral reef ecosystems face threats from human activities and climate change, coral reef conservation programmes are implemented worldwide. Monitoring coral health provides references for guiding conservation activities. However, current labour‐intensive methods result in a backlog of unsorted images, highlighting the need for automated classification. Few studies have simultaneously utilized accurate labels along with updated algorithms and datasets. This study aimed to create a dataset representing common coral reef conditions and associated stressors in the Indo‐Pacific. Concurrently, it assessed existing classification algorithms and proposed a new multilabel method for automatically detecting coral reef conditions and extracting ecological information. A dataset containing over 20,000 high‐resolution coral images of different health conditions and stressors was constructed based on the field survey. Seven representative deep learning architectures were tested on this dataset, and their performance was quantitatively evaluated using the F1 metric and the match ratio. Based on this evaluation, a new method utilizing the ensemble learning approach was proposed. The proposed method accurately classified coral reef conditions as healthy, compromised, dead and rubble; it also identified corresponding stressors, including competition, disease, predation and physical issues. This method can help develop the coral image archive, guide conservation activities and provide references for decision‐making for reef managers and conservationists. The proposed ensemble learning approach outperforms others on the dataset, showing state‐of‐the‐art (SOTA) performance. Future research should improve its generalizability and accuracy to support global coral reef conservation efforts.

Inverse design of nonlinear phononic crystal configurations based on Multi-label classification learning neural networks

Abstract Phononic crystals, as artificial composite materials, have sparked significant interest due to their novel characteristics that emerge upon the introduction of nonlinearity. Among these properties, second-harmonic features exhibit potential applications in acoustic frequency conversion, non-reciprocal wave propagation, and non-destructive testing. Precisely manipulating the harmonic band structure presents a major challenge in the design of nonlinear phononic crystals. Traditional design approaches based on parameter adjustments to meet specific application requirements are inefficient and often yield suboptimal performance. Therefore, this paper develops a design methodology using Softmax logistic regression and Multi-label classification learning to inversely design the material distribution of nonlinear phononic crystals by exploiting information from harmonic transmission spectra. The results demonstrate that the neural network-based inverse design method can effectively tailor nonlinear phononic crystals with desired functionalities. This work establishes a mapping relationship between the band structure and the material distribution within phononic crystals, providing valuable insights into the inverse design of metamaterials.

Updating Correlation-Enhanced Feature Learning for Multi-Label Classification

In the domain of multi-label classification, label correlations play a crucial role in enhancing prediction precision. However, traditional methods heavily depend on ground-truth label sets, which can be incompletely tagged due to the diverse backgrounds of annotators and the significant cost associated with procuring extensive labeled datasets. To address these challenges, this paper introduces a novel multi-label classification method called updating Correlation-enhanced Feature Learning (uCeFL), which extracts label correlations directly from the data instances, circumventing the dependency on potentially incomplete label sets. uCeFL initially computes a revised label matrix by multiplying the incomplete label matrix with the label correlations extracted from the data matrix. This revised matrix is then utilized to enrich the original data features, enabling a neural network to learn correlation-enhanced representations that capture intricate relationships between data features, labels, and their interactions. Notably, label correlations are not static; they are dynamically updated during the neural network’s training process. Extensive experiments carried out on various datasets emphasize the effectiveness of the proposed approach. By leveraging label correlations within data instances, along with the hierarchical learning capabilities of neural networks, it offers a significant improvement in multi-label classification, even in scenarios with incomplete labels.

Dual-channel graph contrastive learning for multi-label classification with label-specific features and label correlations

Dual-channel graph contrastive learning for multi-label classification with label-specific features and label correlations

Enhancing Multi-Label Classification via Dynamic Label-Order Learning

Generative methods tackle Multi-Label Classification (MLC) by autoregressively generating label sequences. These methods excel at modeling label correlations and have achieved outstanding performance. However, a key challenge is determining the order of labels, as empirical findings indicate the significant impact of different orders on model learning and inference. Previous works adopt static label-ordering methods, assigning a unified label order for all samples based on label frequencies or co-occurrences. Nonetheless, such static methods neglect the unique semantics of each sample. More critically, these methods can cause the model to rigidly memorize training order, resulting in missing labels during inference. In light of these limitations, this paper proposes a dynamic label-order learning approach that adaptively learns a label order for each sample. Specifically, our approach adopts a difficulty-prioritized principle and iteratively constructs the label sequence based on the sample s semantics. To reduce the additional cost incurred by label-order learning, we use the same SEQ2SEQ model for label-order learning and MLC learning and introduce a unified loss function for joint optimization. Extensive experiments on public datasets reveal that our approach greatly outperforms previous methods. We will release our code at https: //github.com/KagamiBaka/DLOL.

Ensemble Learning for Multi-Label Classification with Unbalanced Classes: A Case Study of a Curing Oven in Glass Wool Production

The Industrial Internet of Things (IIoT), which integrates sensors into the manufacturing system, provides new paradigms and technologies to industry. The massive acquisition of data, in an industrial context, brings with it a number of challenges to guarantee its quality and reliability, and to ensure that the results of data analysis and modelling are accurate, reliable, and reflect the real phenomena being studied. Common problems encountered with real industrial databases are missing data, outliers, anomalies, unbalanced classes, and non-exhaustive historical data. Unlike papers present in the literature that respond to those problems in a dissociated way, the work performed in this article aims to address all these problems at once. A comprehensive framework for data flow encompassing data acquisition, preprocessing, and machine class classification is proposed. The challenges of missing data, outliers, and anomalies are addressed with critical and novel class outliers distinguished. The study also tackles unbalanced class classification and evaluates the impact of missing data on classification accuracy. Several machine learning models for the operating state classification are implemented. The study also compares the performance of the proposed framework with two existing methods: the Histogram Gradient Boosting Classifier and the Extreme Gradient Boosting classifier. It is shown that using “hard voting” ensemble learning methods to combine several classifiers makes the final classifier more robust to missing data. An application is carried out on data from a real industrial dataset. This research contributes to narrowing the theory–practice gap in leveraging IIoT technologies, offering practical insights into data analytics implementation in real industrial scenarios.

Multi-kernel learning for multi-label classification with local Rademacher complexity

Multi-label classification aims to construct prediction models from input space to output space for multi-label datesets. However, the feature space of multi-label dataset and the hypothesis space of classifier are always complex. In order to obtain high performance multi-kernel learning algorithms, the core problem is to design algorithms that compress both hypothesis space and feature space. In this paper, by combining the local Rademacher complexity and Hilbert-Schmidt independence criterion (HSIC), an effective multi-kernel learning algorithm for multi-label classification is proposed to compress the feature space and hypothesis space simultaneously. Based on the tail sum of eigenvalues of the integral operator corresponding to kernels, the upper bound of local Rademacher complexity of linear functions class in the feature space is determined. The Hilbert-Schmidt independence criterion is applied to maximize the correlation between the convex combination of the input kernel matrices and the ideal kernel matrix in the label space. Finally, the Laplace regularization model is built by controlling the tail sum of eigenvalues of the integral operator corresponding to kernels of this criterion. Therefore, the above process obtains a combined kernel, simultaneously by sorting the coefficients of the combined kernel, a feature selection algorithm can be constructed. On the basis, the combined kernel can also be used to design a new binary classifier for multi-label classification. The experimental results verify that our proposed multi-kernel learning algorithms can effectively compress the hypothesis space, while the feature selection algorithms can effectively compress the feature space.

Geometrically Preserved Dual Projections Learning for Multi-label Classification

Geometrically Preserved Dual Projections Learning for Multi-label Classification

Deep Active Learning for Multi-Label Classification of Remote Sensing Images

Deep Active Learning for Multi-Label Classification of Remote Sensing Images

Dynamic ensemble learning for multi-label classification

Ensemble learning has been shown to be an effective approach to solve multi-label classification problem. However, most existing ensemble learning methods do not consider the difference between unseen instances, and existing methods that consider such difference fail to effectively explore label correlation, which limits their performance. To address these issues, we propose a novel method named MLDE (Multi-Label classification with Dynamic Ensemble learning). In MLDE, the most competent ensemble of base classifiers is selected and combined to predict each unseen instance. To make dynamic selection specific to multi-label problem and achieve better performance, we combine classification accuracy and ranking loss to serve as the competence measurement for the base classifiers. Specifically, classification accuracy is decomposable to multiple labels and distinguishes the ability difference of a classifier with respect to different labels, while ranking loss focuses on the overall performance of a classifier on the label set and thus fully considers the correlation between multiple labels. Extensive experiments on 24 publicly available datasets demonstrate that MLDE outperforms the state-of-the-art methods.

Multi-Label Annotation and Classification of Arabic Texts Based on Extracted Seed Keyphrases and Bi-Gram Alphabet Feed Forward Neural Networks Model

In natural language processing, text classification is a fundamental problem. Multi-label classification of textual data is a challenging topic in text classification where an instance can be associated with more than one label. This paper presents a multi-label annotation and classification methodology for Arabic text data that is not currently classified as multi-label, aiming to analyze and compare the performance of various multi-label learning approaches. The current work includes two phases: The first involves automatic annotation of hotel reviews with more than one label based on the aspects found in the reviews. In this phase, review data instances were automatically annotated as multi-label based on the extracted seed keyphrases clusters. The second phase involves experiments to compare the performance of various multi-label classification learning methods. In this phase, we introduced different models including a feed-forward networks model that learns a vector representation based on the bi-gram alphabet rather than the commonly used bag-of-words model. The bi-gram alphabet vector representation model has the advantage of having reduced feature dimensions and not requiring natural language processing tools. The results indicated that employing the bi-gram alphabet vector representation feed forward neural network is a competitive solution for the multi-label text classification problem. It has achieved an accuracy of about 75.2%, and standard deviation (0.062).

End-to-End Probabilistic Label-Specific Feature Learning for Multi-Label Classification

Label-specific features serve as an effective strategy to learn from multi-label data with tailored features accounting for the distinct discriminative properties of each class label. Existing prototype-based label-specific feature transformation approaches work in a three-stage framework, where prototype acquisition, label-specific feature generation and classification model induction are performed independently. Intuitively, this separate framework is suboptimal due to its decoupling nature. In this paper, we make a first attempt towards a unified framework for prototype-based label-specific feature transformation, where the prototypes and the label-specific features are directly optimized for classification. To instantiate it, we propose modelling the prototypes probabilistically by the normalizing flows, which possess adaptive prototypical complexity to fully capture the underlying properties of each class label and allow for scalable stochastic optimization. Then, a label correlation regularized probabilistic latent metric space is constructed via jointly learning the prototypes and the metric-based label-specific features for classification. Comprehensive experiments on 14 benchmark data sets show that our approach outperforms the state-of-the-art counterparts.

Extension of Multi-Objective Fuzzy Genetics-Based Machine Learning for Multi-Label Classification to Many-Objective Optimization

Extension of Multi-Objective Fuzzy Genetics-Based Machine Learning for Multi-Label Classification to Many-Objective Optimization

Compact learning for multi-label classification

Multi-label classification (MLC) studies the problem where each instance is associated with multiple relevant labels, which leads to the exponential growth of output space. It confronts with the great challenge for the exploration of the latent label relationship and the intrinsic correlation between feature and label spaces. MLC gave rise to a framework named label compression (LC) to obtain a compact space for efficient learning. Nevertheless, most existing LC methods failed to consider the influence of the feature space or misguided by original problematic features, which may result in performance degradation instead. In this paper, we present a compact learning (CL) framework to embed the features and labels simultaneously and with mutual guidance. The proposal is a versatile concept that does not rigidly adhere to some specific embedding methods, and is independent of the subsequent learning process. Following its spirit, a simple yet effective implementation called compact multi-label learning (CMLL) is proposed to learn a compact low-dimensional representation for both spaces. CMLL maximizes the dependence between the embedded spaces of the labels and features, and minimizes the loss of label space recovery concurrently. Theoretically, we provide a general analysis for different embedding methods. Practically, we conduct extensive experiments to validate the effectiveness of the proposed method.

Semi-Supervised Dual Relation Learning for Multi-Label Classification.

In a real-world scenario, an object could contain multiple tags instead of a single categorical label. To this end, multi-label learning (MLL) emerged. In MLL, the feature distributions are long-tailed and the complex semantic label relation and the long-tailed training samples are the main challenges. Semi-supervised learning is a potential solution. While, existing methods are mainly designed for single class scenario while ignoring the latent label relations. In addition, they cannot well handle the distribution shift commonly existing across source and target domains. To this end, a Semi-supervised Dual Relation Learning (SDRL) framework for multi-label classification is proposed. SDRL utilizes a few labeled samples as well as large scale unlabeled samples in the training stage. It jointly explores the inter-instance feature-level relation and the intra-instance label-level relation even from the unlabeled samples. In our model, a dual-classifier structure is deployed to obtain domain invariant representations. The prediction results from the classifiers are further compared and the most confident predictions are extracted as pseudo labels. A trainable label relation tensor is designed to explicitly explore the pairwise latent label relations and refine the predicted labels. SDRL is able to effectively and efficiently explore the feature-label relation as well as the label-label relation knowledge without any extra semantic knowledge. We evaluated SDRL in general and zero-shot multi-label classification tasks and we concluded that SDRL is superior to other SOTA baselines. Furthermore, extensive ablation studies have been done which reveal the effectiveness of each component in our framework.

Compositional metric learning for multi-label classification

Multi-label classification aims to assign a set of proper labels for each instance, where distance metric learning can help improve the generalization ability of instance-based multi-label classification models. Existing multi-label metric learning techniques work by utilizing pairwise constraints to enforce that examples with similar label assignments should have close distance in the embedded feature space. In this paper, a novel distance metric learning approach for multi-label classification is proposed by modeling structural interactions between instance space and label space. On one hand, compositional distance metric is employed which adopts the representation of a weighted sum of rank-1 PSD matrices based on component bases. On the other hand, compositional weights are optimized by exploiting triplet similarity constraints derived from both instance and label spaces. Due to the compositional nature of employed distance metric, the resulting problem admits quadratic programming formulation with linear optimization complexity w.r.t. the number of training examples. We also derive the generalization bound for the proposed approach based on algorithmic robustness analysis of the compositional metric. Extensive experiments on sixteen benchmark data sets clearly validate the usefulness of compositional metric in yielding effective distance metric for multi-label classification.

Distributed Information-Theoretic Semisupervised Learning for Multilabel Classification.

Multilabel classification (MLC) has received much attention recently. The existing MLC algorithms usually learn multiple classifiers simultaneously by exploiting the correlations among different labels. However, it is difficult and/or expensive to collect a large amount of multilabeled data in practice. The lack of labeled data significantly deteriorates the performance of classification. Moreover, the existing algorithms belong to centralized learning, that is, all the data with their labels must be transmitted to a fusion center for processing. But in many real applications, data may be dispersedly collected/stored in distributed nodes of networks. Due to the concerns of communication cost, processing ability, and data privacy, it is impossible to transmit and/or process the data centrally at one node. Considering this, the problem of distributed MLC over networks is studied, and two distributed information-theoretic semisupervised multilabel learning (dITS2ML2) algorithms are proposed, which are, respectively, used for solving linear and nonlinear MLC problems. In the proposed algorithms, a cost-sensitive objective function is designed, in which a new label correlation term defined on some anchor data is suggested. Besides, to decentralize the global objective function, a distributed matrix completion algorithm is developed to distributively complete the label matrix of the anchor data. Then, by exchanging and combining a few intermediate quantities instead of the original data for both linear and nonlinear cases, the model parameters can be adaptively estimated. The convergence of the proposed dITS2ML2 algorithms is analyzed, and their effectiveness in MLC is verified by simulations on various real datasets.

Online Metric Learning for Multi-Label Classification

Existing research into online multi-label classification, such as online sequential multi-label extreme learning machine (OSML-ELM) and stochastic gradient descent (SGD), has achieved promising performance. However, these works lack an analysis of loss function and do not consider label dependency. Accordingly, to fill the current research gap, we propose a novel online metric learning paradigm for multi-label classification. More specifically, we first project instances and labels into a lower dimension for comparison, then leverage the large margin principle to learn a metric with an efficient optimization algorithm. Moreover, we provide theoretical analysis on the upper bound of the cumulative loss for our method. Comprehensive experiments on a number of benchmark multi-label datasets validate our theoretical approach and illustrate that our proposed online metric learning (OML) algorithm outperforms state-of-the-art methods.

Joint Ranking SVM and Binary Relevance with robust Low-rank learning for multi-label classification

Multi-label classification studies the task where each example belongs to multiple labels simultaneously. As a representative method, Ranking Support Vector Machine (Rank-SVM) aims to minimize the Ranking Loss and can also mitigate the negative influence of the class-imbalance issue. However, due to its stacking-style way for thresholding, it may suffer error accumulation and thus reduces the final classification performance. Binary Relevance (BR) is another typical method, which aims to minimize the Hamming Loss and only needs one-step learning. Nevertheless, it might have the class-imbalance issue and does not take into account label correlations. To address the above issues, we propose a novel multi-label classification model, which joints Ranking support vector machine and Binary Relevance with robust Low-rank learning (RBRL). RBRL inherits the ranking loss minimization advantages of Rank-SVM, and thus overcomes the disadvantages of BR suffering the class-imbalance issue and ignoring the label correlations. Meanwhile, it utilizes the hamming loss minimization and one-step learning advantages of BR, and thus tackles the disadvantages of Rank-SVM including another thresholding learning step. Besides, a low-rank constraint is utilized to further exploit high-order label correlations under the assumption of low dimensional label space. Furthermore, to achieve nonlinear multi-label classifiers, we derive the kernelization RBRL. Two accelerated proximal gradient methods (APG) are used to solve the optimization problems efficiently. Extensive comparative experiments with several state-of-the-art methods illustrate a highly competitive or superior performance of our method RBRL.

Supervised representation learning for multi-label classification

Representation learning is one of the most important aspects of multi-label learning because of the intricate nature of multi-label data. Current research on representation learning either fails to consider label knowledge or is affected by the lack of labeled data. Moreover, most of them learn the representations and incorporate the label information in a two-step manner. In this paper, due to the success of representation learning by deep learning we propose a novel framework based on neural networks named SERL to learn global feature representation by jointly considering all labels in an effective supervised manner. At its core, a two-encoding-layer autoencoder, which can utilize labeled and unlabeled data, is adopted to learn feature representation in the supervision of softmax regression. Specifically, the softmax regression incorporates label knowledge to improve the performance of both representation learning and multi-label learning by being jointly optimized with the autoencoder. Moreover, the autoencoder is expanded into two encoding layers to share knowledge with the softmax regression by sharing the second encoding weight matrix. We conduct extensive experiments on five real-world datasets to demonstrate the superiority of SERL over other state-of-the-art multi-label learning approaches.