Multi-label Data Research Articles

Feature Selection With Missing Labels Using Multilabel Fuzzy Neighborhood Rough Sets and Maximum Relevance Minimum Redundancy

Recently, multilabel classification has generated considerable research interest. However, the high dimensionality of multilabel data incurs high costs; moreover, in many real applications, a number of labels of training samples are randomly missed. Thus, multilabel classification can have great complexity and ambiguity, which means some feature selection methods exhibit poor robustness and yield low prediction accuracy. To solve these issues, this article presents a novel feature selection method based on multilabel fuzzy neighborhood rough sets (MFNRS) and maximum relevance minimum redundancy (MRMR) that can be used on multilabel data with missing labels. First, to handle multilabel data with missing labels, a relation coefficient of samples, label complement matrix, and label-specific feature matrix are constructed and implemented in a linear regression model to recover missing labels. Second, the margin-based fuzzy neighborhood radius, fuzzy neighborhood similarity relationship, and fuzzy neighborhood information granule are developed. The MFNRS model is built based on multilabel neighborhood rough sets combined with fuzzy neighborhood rough sets. Based on algebra and information views, certain fuzzy neighborhood entropy-based uncertainty measures are proposed for MFNRS. The fuzzy neighborhood mutual information-based MRMR model with label correlation is improved to evaluate the performance of candidate features. Finally, a feature selection algorithm is designed to improve the performance for multilabel data with missing labels. Experiments on 20 datasets verify that our method is effective not only for recovering missing labels but also for selecting significant features with better classification performance.

Two-step multi-view and multi-label learning with missing label via subspace learning

In multi-view and multi-label learning, each example can be represented by multiple data view features and annotated with a set of discrete non-exclusive labels. Missing label learning is an important branch of multi-label learning, which can handle incomplete labels with annotations. Previous work on multi-label learning with missing labels mainly considered data in a single view representation. Based on intuitive understanding, we propose a Two-step Multi-view and Multi-label Missing Label learning optimization solution(TM3L). The first step is to solve the multi-view learning problem by finding the data representation of the common low-dimensional space of all views through subspace learning. While fully considering the complementary information between multiple views, the different degrees of contribution combined with different views are weighted differently. The second step is to solve the multi-label missing label learning problem by using the label matrix completion method in combination with the kernel extreme learning machine classifier. The kernel extreme learning machine can effectively enhance the robustness of the algorithm to missing labels. The experimental results and analysis on multiple benchmark multi-view and multi-label data sets verify the effectiveness of TM3L compared with the state-of-the-art solutions.

Comparing Different Machine Learning Techniques for Classifying Multi Label Data

In recent years, multi-label classifications have become common. Multi label classification is a classification in which a collection of labels is associated with a single instance, which may be a variation of the classification of a single label. The problem of huge data is the classification in which each instance is of different kind which further can be identified with more than one class. The various machine learning strategies for classifying multi-label data are discussed in this paper. Many researches have been carried out that specify the grouping of multiple labels. Here we will compare various classification machine learning techniques that involve two approaches: the adapted algorithm approach and the method of problem transformation. Here we are using naive multinomial bayes and logistic regression. We use certain evaluation metrics to predict the differences as well. Better classification methods are discussed in this paper.

Multi-Label Classification With Label-Specific Feature Generation: A Wrapped Approach.

Label-specific features serve as an effective strategy to learn from multi-label data, where a set of features encoding specific characteristics of each label are generated to help induce multi-label classification model. Existing approaches work by taking the two-stage strategy, where the procedure of label-specific feature generation is independent of the follow-up procedure of classification model induction. Intuitively, the performance of resulting classification model may be suboptimal due to the decoupling nature of the two-stage strategy. In this paper, a wrapped learning approach is proposed which aims to jointly perform label-specific feature generation and classification model induction. Specifically, one (kernelized) linear model is learned for each label where label-specific features are simultaneously generated within an embedded feature space via empirical loss minimization and pairwise label correlation regularization. Comparative studies over a total of sixteen benchmark data sets clearly validate the effectiveness of the wrapped strategy in exploiting label-specific features for multi-label classification.

Multi-Label Data Fusion to Support Agricultural Vulnerability Assessments

Identifying crop species and varieties adaptable to climate change impacts is one of the main aspects of climate vulnerability assessments. This estimation involves processing, integrating, and analyzing many information sources to provide accurate and timely responses. However, designing this evaluation, examine the information gathered, and reaching agreements among all stakeholders and experts, often requires considerable effort in time, money, and people. In this study, we propose a data fusion strategy to support climate vulnerability assessments by identifying the adaptability of crops in a territory in the short term. This strategy follows the Joint Directors of Laboratories’ data fusion model guidelines. It was evaluated and validated through a case study in Colombia’s upper Cauca river basin. For this purpose, we identified Climate, Soil, Water Quality, Productive Alliances, and Production as the most relevant data sources to be integrated, and using metrics such as Mean IR, SCUMBLE, TCS, among others, we evaluated the combined datasets according to their theoretical complexity. The adaptability of crops in a territory was addressed as a multi-label learning problem, assessing the performance of different multi-label classification and multi-view multi-label classification models with both test and actual data. Comparing the predicted crops with the actual ones, we obtained a 98% similarity without considering crop ranking using the Binary Relevance approach and the Random Forest and XGBoost algorithms. Using a more exhaustive test involving order, we obtained a maximum similarity of 67% applying Binary Relevance and Random Forest.

Learning Individual Class Representation From Biased Multi-Label Data

Image recognition is a popular and important research field of computer vision. Recently with the development of deep learning technology, image recognition performance has been improved significantly. However with multi-label images, recognizing individual category is a challenging task. In order to address the problem, we propose a Feature Disintegrator (FD) that decomposes co-occurred instance features of multi-label into individual categories. In experimental evaluation, proposed method achieves the gains of mean average precision (mAP) up to 18.67% and 29.05% over baseline networks in ML-MNIST and ML-CIFAR, respectively. It shows 5.76% and 6.65% higher mAP than baseline on PASCAL VOC-2007 and MS-COCO data set respectively.

Representation Learning With Dual Autoencoder for Multi-Label Classification

Multi-label classification aims to deal with the problem that an object may be associated with one or more labels, which is a more difficult task due to the complex nature of multi-label data. The crucial problem of multi-label classification is the more robust and higher-level feature representation learning, which can reduce non-helpful feature attributes from the input space prior to training. In recent years, deep learning methods based on autoencoders have achieved excellent performance in multi-label classification for the advantages of powerful representations learning ability and fast convergence speed. However, most existing autoencoder-based methods only rely on the single autoencoder model, which pose challenges for multi-label feature representations learning and fail to measure similarities between data spaces. To address this problem, in this paper, we propose a novel representation learning method with dual autoencoder for multi-label classification. Compared to the existing autoencoder-based methods, our proposed method can capture different characteristics and more abstract features from data by the serially connection of two different types of autoencoders. More specifically, firstly, the algorithm of Reconstruction Independent Component Analysis (RICA) in sparse autoencoder is trained on patches on all training and test dataset for robust global feature representations learning. Secondly, with the output of RICA, stacked autoencoder with manifold regularization (SAMR) is introduced to ameliorate the quality of multi-label features learning. Comprehensive experiments on several real-world data sets demonstrate the effectiveness of our proposed approach compared with several competing state-of-the-art methods.

A ranking-based feature selection for multi-label classification with fuzzy relative discernibility

Feature selection is a crucial pre-processing step for learning tasks to mitigate the “curse of dimensionality”, which is caused by irrelevant and redundant features in high-dimensional feature space. The fuzzy rough set is an effective tool in feature selection, and the fuzzy discernibility matrix, one of the generalized model of the fuzzy rough set, has attracted significant attention recently. However, with the complicated discrimination relation of multiple labels, the traditional fuzzy discernibility matrix does not fit well for multi-label data. To address this problem, in this paper, the fuzzy label discernibility relation and the fuzzy relative discernibility relation are defined firstly, which are derived from fuzzy discernibility relation. Subsequently, we present the feature discernibility significance which measures the discernibility ability of the conditional feature and selects the most relevant features using the value of discernibility significance. On this basis, we propose a ranking-based feature selection algorithm for multi-label classification with fuzzy relative discernibility. Finally, in terms of six widely-accepted multi-label evaluation metrics, a series of experiments is conducted based on the multi-label classifier with ten different multi-label datasets from Mulan library and MLL resource, to compare the performance of the proposed algorithm with four state-of-the-art multi-label feature selection algorithms. The experimental results demonstrate the superiority and effectiveness of the proposed algorithm in multi-label classification.

Multi-label feature selection based on the division of label topics

Multi-label feature selection has attracted much attention from researchers and can reduce the high dimensionality of multi-label data. Previous multi-label methods consider the importance of labels equal, as a result, they choose the discriminative features based on the entire label set. In fact, there exists a latent semantic structure in the label set. Specifically, labels can be sorted into some central topics and some subordinate topics. Features regarding central topics should be chosen first and the number of them should be chosen more. To this end, we first explore the latent semantic structure according to spectral clustering. The labels are abstracted into several clusters named central clusters and subordinate clusters. Second, the importance of features with respect to the labels in each cluster is scored. Finally, we obtain the feature subset based on both the scores of features and the type of clusters. Comprehensive experiments demonstrate the superiority of the proposed method against seven state-of-the-art multi-label feature selection methods on fourteen benchmark multi-label data sets.

Improved Ensemble Classification for Evolving Data Streams

A major challenge for evolving data stream classification is feature evolution where features of stream instances are dynamically changing as they progress. Existing classification methods considered feature evolution either for fixed-size data or of limited degree with presumed dependence to history, making them unable to work effectively on evolving data streams of unbounded size and arbitrary feature evolution. Particularly, for evolving data streams containing instances of multiple labels, classification coping with feature evolution faces significant challenges. In this article, we present efficient ensemble methods for classifying evolving data streams of both single label and multiple labels through effective model coupling. For single-label classification, we present an improved unsupervised classification algorithm that applies multi-cluster feature selection (MCFS), which was originally proposed for static data classification, in the DXMiner framework to handle each window of instances in a dynamic stream. Our method generates an optimal feature subset and achieves a high classification accuracy. We further improve the time complexity of the feature selection process in MCFS by applying the Ball-tree searching technique. For multi-label classification, we propose an effective fixed-size ensemble classifier based on multi-label KNN, which works only for static multi-label data classification, by incorporating a weight adaptation strategy among the classifiers in the ensemble to dynamically update the model and cope with arbitrary feature evolution of stream instances as the stream progresses. Extensive experiment results on real-life data streams show that our algorithms outperform the existing results for single-label and multi-label classification in classification accuracy and efficiency.

A novel reasoning mechanism for multi-label text classification

The aim in multi-label text classification is to assign a set of labels to a given document. Previous classifier-chain and sequence-to-sequence models have been shown to have a powerful ability to capture label correlations. However, they rely heavily on the label order, while labels in multi-label data are essentially an unordered set. The performance of these approaches is therefore highly variable depending on the order in which the labels are arranged. To avoid being dependent on label order, we design a reasoning-based algorithm named Multi-Label Reasoner (ML-Reasoner) for multi-label classification. ML-Reasoner employs a binary classifier to predict all labels simultaneously and applies a novel iterative reasoning mechanism to effectively utilize the inter-label information, where each instance of reasoning takes the previously predicted likelihoods for all labels as additional input. This approach is able to utilize information between labels, while avoiding the issue of label-order sensitivity. Extensive experiments demonstrate that our method outperforms state-of-the art approaches on the challenging AAPD dataset. We also apply our reasoning module to a variety of strong neural-based base models and show that it is able to boost performance significantly in each case.

Multi-label feature selection with constraint regression and adaptive spectral graph

Like single-label learning, multi-label learning also suffers from the curse of dimensionality. Due to the existence of high-dimensional data, feature selection as a preprocessing tool always plays a key position in the multi-label learning. Although a variety of multi-label feature selection methods have been developed, they neglect to consider the redundancy of features, thereby degrading learning performance. To address this problem, we present a novel multi-label feature selection approach with uncorrelated regression and adaptive spectral graph. Specifically, we first construct a manifold framework with uncorrelated regression model to hunt for uncorrelated yet discriminative features, which also utilize the low-dimensional representation based on feature space to fit the label distribution. Then, a spectral graph term based on information entropy is incorporated into the manifold framework, so as to ensure the local geometric structure of data in subsequent learning process. Following the above principles, we design an objective function to achieve multi-label feature selection, and propose a detailed optimization method. Comprehensive experiments are conducted on multiple public multi-label data sets, the results show that our method is superior to other compared methods.

Towards Class-Imbalance Aware Multi-Label Learning.

Multi-label learning deals with training examples each represented by a single instance while associated with multiple class labels. Due to the exponential number of possible label sets to be considered by the predictive model, it is commonly assumed that label correlations should be well exploited to design an effective multi-label learning approach. On the other hand, class-imbalance stands as an intrinsic property of multi-label data which significantly affects the generalization performance of the multi-label predictive model. For each class label, the number of training examples with positive labeling assignment is generally much less than those with negative labeling assignment. To deal with the class-imbalance issue for multi-label learning, a simple yet effective class-imbalance aware learning strategy called cross-coupling aggregation (COCOA) is proposed in this article. Specifically, COCOA works by leveraging the exploitation of label correlations as well as the exploration of class-imbalance simultaneously. For each class label, a number of multiclass imbalance learners are induced by randomly coupling with other labels, whose predictions on the unseen instance are aggregated to determine the corresponding labeling relevancy. Extensive experiments on 18 benchmark datasets clearly validate the effectiveness of COCOA against state-of-the-art multi-label learning approaches especially in terms of imbalance-specific evaluation metrics.

Multi-View Multi-Label Learning With Sparse Feature Selection for Image Annotation

In image analysis, image samples are always represented by multiple view features and associated with multiple class labels for better interpretation. However, multiple view data may include noisy, irrelevant and redundant features, while multiple class labels can be noisy and incomplete. Due to the special data characteristic, it is hard to perform feature selection on multi-view multi-label data. To address these challenges, in this paper, we propose a novel multi-view multi-label sparse feature selection (MSFS) method, which exploits both view relations and label correlations to select discriminative features for further learning. Specifically, the multi-labeled information is decomposed into a reduced latent label representation to capture higher level concepts and correlations among multiple labels. Multiple local geometric structures are constructed to exploit visual similarities and relations for different views. By taking full advantage of the latent label representation and multiple local geometric structures, the sparse regression model with an $l_{2,1}$ -norm and an Frobenius norm ( $F$ -norm) penalty terms is utilized to perform hierarchical feature selection, where the $F$ -norm penalty performs high-level (i.e., view-wise) feature selection to preserve the informative views and the $l_{2,1}$ -norm penalty conducts low-level (i.e., row-wise) feature selection to remove noisy features. To solve the proposed formulation, we also devise a simple yet efficient iterative algorithm. Experiments and comparisons on real-world image datasets demonstrate the effectiveness and potential of MSFS.

Cost-sensitive feature selection on multi-label data via neighborhood granularity and label enhancement

Multi-label feature selection, which is an efficient and effective pre-processing step in machine learning and data mining, can select a feature subset that contains more contributions for multi-label classification while improving the performance of the classifiers. In real-world applications, an instance may be associated with multiple related labels with different relative importances, and the process of obtaining different features usually requires different costs, containing money, and time, etc. However, most existing works with regard to multi-label feature selection do not take into consideration the above two critical issues simultaneously. Therefore, in this paper, we exploit the idea of neighborhood granularity to enhance the traditional logical labels into label distribution forms to excavate the deeper supervised information hidden in multi-label data, and further consider the effect of the test cost under three different distributions, simultaneously. Motivated by these issues, a novel test cost multi-label feature selection algorithm with label enhancement and neighborhood granularity is designed. Moreover, the proposed algorithm is tested upon ten publicly available benchmark multi-label datasets with six widely-used metrics from two different aspects. Finally, two groups of experimental results demonstrate that the proposed algorithm achieves the satisfactory and superior performance over other four state-of-the-art comparing algorithms, and it is effective for improving the learning performance and decreasing the total test costs of the selected feature subset.

Multilabel Distribution Learning Based on Multioutput Regression and Manifold Learning.

Real-world multilabel data are high dimensional, and directly using them for label distribution learning (LDL) will incur extensive computational costs. We propose a multilabel distribution learning algorithm based on multioutput regression through manifold learning, referred to as MDLRML. By exploiting smooth, similar spaces' information provided by the samples' manifold learning and LDL, we link the two spaces' manifolds. This facilitates using the topological relationship of the manifolds in the feature space to guide the manifold construction of the label space. The smoothest regression function is used to fit the manifold data, and a locally constrained multioutput regression is designed to improve the data's local fitting. Based on the regression results, we enhance the logical labels into the label distributions, thereby mining and revealing the label's hidden information regarding importance or significance. Extensive experimental results using real-world multilabel datasets show that the proposed MDLRML algorithm significantly improves the multilabel distribution learning accuracy and efficiency over several existing state-of-the-art schemes.

Label distribution feature selection for multi-label classification with rough set

Multi-label learning deals with cases where every instance corresponds to multiple labels. The objective is to learn mapping from an instance to a relevant label set. Existing multi-label learning approaches assume that the significance for all related labels is same for every instance. Several problems of label ambiguity can be dealt with using multi-label learning, but some practical applications with significance among related labels for every instance cannot be effectively processed. To achieve superior results by conducting different significance of labels, label distribution learning is used for such applications. First, the probability model and rough set are embedded in the labeling significance, thus more supervised information can be obtained from original multi-label data. Subsequently, to resolve the feature selection problem of label distribution data, according to the feature dependency and the rough set, a novel feature selection algorithm for multi-label classification is designed. Finally, to verify the effectiveness of the proposed algorithms, an extensive experiment is conducted on 15 real-world multiple label data sets. The performance of the proposed algorithm through the multi-label classifier is compared with seven state-of-the-art approaches, thereby indicating the applicability and effectiveness of label distribution feature selection.

Partially disentangled latent relations for multi-label deep learning

For multi-label learning, the specific features are extracted from the instances under the supervised of class label is meaningful, and the "purified" feature representation can also be shared with other features during learning process. Besides, it is essential to distinguish the inter-instance relations in input space and inter-label correlation relations in the output space on the multi-label datasets, which is conducive to improve the performance of the multi-label algorithm. However, most current multi-label algorithms aim to capture the mapping between instances and labels, while ignoring the information about instance relations and label correlations in the multi-label data structure. Motivated by these issues, we leverage the deep network to learn the special feature representations for multi-label components without abandoning overlapped features which may belong to other multi-label components. Meanwhile, the Euclidean matrices are leveraged to construct the diagonal matrix for the diffusion function, obtaining the new class latent representation by graph-based diffusion method preserve the inter-instance relations; it ensures that similar features have similar label sets. Further, considering that the contributions of these feature representation are different and have distinct influences on the final multi-label prediction results, the self-attention mechanism is introduced to fusion the other label-specific instance features to build the new joint feature representation, which derives dynamic weights for multi-label prediction. Finally, experimental results on the real data sets show promising wide availability for our approach.

Adaptive Decision Threshold-Based Extreme Learning Machine for Classifying Imbalanced Multi-label Data

Multi-label learning is a popular area of machine learning research as it is widely applicable to many real-world scenarios. In comparison with traditional binary and multi-classification tasks, the multi-label data are more easily impacted or destroyed by an imbalanced data distribution. This paper describes an adaptive decision threshold-based extreme learning machine algorithm (ADT-ELM) that addresses the imbalanced multi-label data classification problem. Specifically, the macro and micro F-measure metrics are adopted as the optimization functions for ADT-ELM, and the particle swarm optimization algorithm is employed to determine the optimal decision threshold combination. We use the optimized thresholds to make decision for future multi-label instances. Twelve baseline multi-label data sets are used in a series of experiments o verify the effectiveness and superiority of the proposed algorithm. The experimental results indicate that the proposed ADT-ELM algorithm is significantly superior to many state-of-the-art multi-label imbalance learning algorithms, and it generally requires less training time than more sophisticated algorithms.

Novel multi-label feature selection via label symmetric uncertainty correlation learning and feature redundancy evaluation

Multi-label data with high dimensionality, widely existed in the real world, bring many challenges to the applications of machine learning, pattern recognition and other fields. Scholars have proposed some multi-label feature selection methods from various aspects. However, there are few studies on the feature selection for multi-label data based on fuzzy mutual information, and most existing methods neglect the correlation between labels. In this study, we propose two novel multi-label feature selection approaches via label symmetric uncertainty correlation and feature redundancy evaluation. Firstly, we propose the concept of symmetric uncertainty correlation between labels via fuzzy mutual information, and design a label importance weight based on label symmetric uncertainty correlation learning. Further, we define a label similarity relation matrix on multi-label space via the label importance weight. Secondly, we define the symmetric uncertainty correlation between features and labels, and propose the first multi-label feature selection approach. Thirdly, considering the above-proposed method can only get a feature sequence and does not remove the redundancy features, we further propose an improved multi-label removing-redundancy feature selection approach through introducing feature redundancy evaluation. Finally, comprehensive experiments are executed to demonstrate the performance of our methods. The results illustrate that our study is better than other representative feature selection methods.