Feature Selection For Multi-label Learning Research Articles

Multiple reference points-based multi-objective feature selection for multi-label learning

Multiple reference points-based multi-objective feature selection for multi-label learning

Feature selection for multi-label learning based on variable-degree multi-granulation decision-theoretic rough sets

Multi-label learning (MLL) suffers from the high-dimensional feature space teeming with irrelevant and redundant features. To tackle this, several multi-label feature selection (MLFS) algorithms have emerged as vital preprocessing steps. Nonetheless, existing MLFS methods have their shortcomings. Primarily, while they excel at harnessing label-feature relationships, they often struggle to leverage inter-feature information effectively. Secondly, numerous MLFS approaches overlook the uncertainty in the boundary domain, despite its critical role in identifying high-quality features. To address these issues, this paper introduces a novel MLFS algorithm, named VMFS. It innovatively integrates multi-granulation rough sets with three-way decision, leveraging multi-granularity decision-theoretic rough sets (MGDRS) with variable degrees for optimal performance. Initially, we construct coarse decision (RDC), fine decision (RDF), and uncertainty decision (RDU) functions for each object based on MGDRS with variable degrees. These decision functions then quantify the dependence of attribute subsets, considering both deterministic and uncertain aspects. Finally, we employ the dependency to assess attribute importance and rank them accordingly. Our proposed method has undergone rigorous evaluation on various standard multi-label datasets, demonstrating its superiority. Experimental results consistently show that VMFS significantly outperforms other algorithms on most datasets, underscoring its effectiveness and reliability in multi-label learning tasks.

Dynamic mutual information-based feature selection for multi-label learning

In classification problems, feature selection is used to identify important input features to reduce the dimensionality of the input space while improving or maintaining classification performance. Traditional feature selection algorithms are designed to handle single-label learning, but classification problems have recently emerged in multi-label domain. In this study, we propose a novel feature selection algorithm for classifying multi-label data. This proposed method is based on dynamic mutual information, which can handle redundancy among features controlling the input space. We compare the proposed method with some existing problem transformation and algorithm adaptation methods applied to real multi-label datasets using the metrics of multi-label accuracy and hamming loss. The results show that the proposed method demonstrates more stable and better performance for nearly all multi-label datasets.

Integrating Global and Local Feature Selection for Multi-Label Learning

Multi-label learning deals with the problem where an instance is associated with multiple labels simultaneously. Multi-label data is often of high dimensionality and has many noisy, irrelevant, and redundant features. As an important machine learning task, multi-label feature selection has received considerable attention in recent years due to its promising performance in dealing with high-dimensional multi-label data. Existing multi-label feature selection methods typically select the global features which are shared by all instances in a dataset. However, these multi-label feature selection methods may be suboptimal since they do not consider the specific characteristics of instances. In this paper, we propose a novel algorithm that integrates Global and Local Feature Selection (GLFS) to exploit both the global features and a subset of discriminative features shared only locally by a subgroup of instances in a multi-label dataset. Specifically, GLFS employs linear regression and ℓ 2,1 -norm on the regression parameters to achieve simultaneous global and local feature selection. Moreover, the proposed algorithm has an effective mechanism for utilizing label correlations to improve the feature selection. Experiments on real-world multi-label datasets show the superiority of GLFS over the state-of-the-art multi-label feature selection methods.

Group-preserving label-specific feature selection for multi-label learning

In many real-world application domains, e.g., text categorization and image annotation, objects naturally belong to more than one class label, giving rise to the multi-label learning problem. The performance of multi-label learning greatly relies on the quality of available features, whereas the data generally involve a lot of irrelevant, redundant, even noisy features. This fact has led to that a surge of research on feature selection methods that select significant features for multi-label learning. Nevertheless, most of the previous approaches suffer from the deficiency that label-specific features are not taken into account, and they are also inefficient in exploiting labeling information such as local label correlations. Moreover, these methods lack interpretability, which can only find a feature subset for all labels, however, cannot show how features are related to different labels. Based on this, we present a new group-preserving label-specific feature selection (GLFS) framework for multi-label learning, which simultaneously considers the features special to the labels in the same group and specific features owned by each label to execute feature selection. In addition, we further consider to learn label-group and instance-group correlations for the exploitation of labeling information, and make a collaborative use of them to improve the model generalization. Extensive experiments validate the advantages of the proposed GLFS method.

A bipartite matching-based feature selection for multi-label learning

Many real-world data have multiple class labels known as multi-label data, where the labels are correlated with each other, and as such, they are not independent. Since these data are usually high-dimensional, and the current multi-label feature selection methods have not been precise enough, then a new feature selection method is necessary. In this paper, for the first time, we have modeled the problem of multi-label feature selection to a bipartite graph matching process. The proposed method constructs a bipartite graph of features (as the left vertices) and labels (as the right vertices), called Feature-Label Graph (FLG), where each feature is connected to the set of labels, where the weight of the edge between each feature and label is equal to their correlation. Then, the Hungarian algorithm estimates the best matching in FLG. The selected features in each matching are sorted by weighted correlation distance and added to the ranking vector. To select the discriminative features, the proposed method considers both the redundancy of features and the relevancy of each feature to the class labels. The results indicate the superiority of the proposed method against the other methods in classification measures.

Mutual information-based label distribution feature selection for multi-label learning

Feature selection used for dimensionality reduction of the feature space plays an important role in multi-label learning where high-dimensional data are involved. Although most existing multi-label feature selection approaches can deal with the problem of label ambiguity which mainly focuses on the assumption of uniform distribution with logical labels, it cannot be applied to many practical applications where the significance of related label for every instance tends to be different. To deal with this issue, in this study, label distribution learning covered with a certain real number of labels is introduced to design a model for the labeling-significance. Nevertheless, multi-label feature selection is limited to handling only labels consisting of logical relations. In order to solve this problem, combining the random variable distribution with granular computing, we first propose a label enhancement algorithm to transform logical labels in multi-label data into label distribution with more supervised information, which can mine the hidden label significance from every instance. On this basis, to remove some redundant or irrelevant features in multi-label data, a label distribution feature selection algorithm using mutual information and label enhancement is developed. Finally, the experimental results show that the performance of the proposed method is superior to the other state-of-the-art approaches when dealing with multi-label data.

Feature selection for multi-label learning with streaming label

Multi-label feature selection has drawn wide attention in recent years. The existing multi-label feature selection algorithms mainly assume that the labels of the training data are obtained before feature selection takes place. However, this assumption does not always founded because the acquisition of labeling data is costly. In real-world applications, the available labels usually arrive one by one over time. To address this problem, we develop a novel multi-label feature selection method under the circumstance of streaming label to select a set of the most relevant and discriminative features. Specifically, we firstly select label-specific features for each newly-arrived label by designing inter-class discrimination and intra-class neighbor recognition. Then, a feature conversion is created to fuse the generated label-specific feature sets. Comprehensive experiments on a series of benchmark data sets clearly demonstrate the superiority of the proposed method against other state-of-the-art multi-label feature selection methods.

Feature Selection for Multi-Label Learning Based on F-Neighborhood Rough Sets

Multi-label learning is often applied to handle complex decision tasks, and feature selection is its essential part. The relation of labels is always ignored or not enough to consider for both multi-label learning and its feature selection. To deal with the problem, F-neighborhood rough sets are employed. Different from other methods, the original approximate space is not changed, but the relation of labels is sufficient to consider. To be specific, a multi-label decision system is discomposed into a family of single-label decision tables with the label set(first-order strategy) at first. Secondly, calculate attribute significance in the family of single-label decision tables. Third, construct an attribute significance matrix and improved attribute significance matrices to evaluate the quality of the features, then a parallel reduct is obtained with information fusion. These processes construct F-neighborhood parallel reduction algorithm for a multi-label decision system(FNPRMS). Compared with the state-of-the-arts, experimental results show that FNPRMS is effective and efficient on 9 publicly available data sets.

Manifold regularized discriminative feature selection for multi-label learning

In multi-label learning, objects are essentially related to multiple semantic meanings, and the type of data is confronted with the impact of high feature dimensionality simultaneously, such as the bioinformatics and text mining applications. To tackle the learning problem, the key technology, i.e., feature selection, is developed to reduce dimensionality, whereas most of the previous methods for multi-label feature selection are either directly transformed from traditional single-label feature selection methods or half-baked in the label information exploitation, and thus causing the redundant or irrelevant features involved in the selected feature subset. Aimed to seek discriminative features across multiple class labels, we propose an embedded multi-label feature selection method with manifold regularization. To be specific, a low-dimensional embedding is constructed based on the original feature space to fit the label distribution for capturing the label correlations locally, which is also constrained using the label information in consideration of the co-occurrence relationships of label pairs. Following this principle, we design an optimization objective function involving l2,1-norm regularization to achieve multi-label feature selection, and the convergence is guaranteed. Empirical studies on various multi-label data sets reveal that the proposed method can obtain highly competitive performance against some state-of-the-art multi-label feature selection methods.

Alignment Based Feature Selection for Multi-label Learning

Multi-label learning deals with data sets in which each example is assigned with a set of labels, and the goal is to construct a learning model to predict the label set for unseen examples. Multi-label data sets share the same problems with single-label data sets that usually possess high-dimensional features and may exist redundancy features, which will influence the performance of the algorithm. Thus it is obviously necessary to address feature selection in multi-label learning. Meanwhile, information among labels play an important role in multi-label learning, thereby it is significance to measure information among labels in order to improve the performance of learning algorithms. In this paper, we introduce kernel alignment into multi-label learning to measure the consistency between feature space and label space by which features are ranked and selected. Firstly we define an ideal kernel in label space as a convex combination of ideal kernels defined by each label, and a linear combination of kernels where each kernel corresponds to a feature. Secondly, through maximizing the kernel alignment value between linear combination kernel and ideal kernel, both weights in the two defined kernels are learned in this process simultaneously, and the learned weights of labels can be employed as the degree of labeling importance regarded as a kind of information among labels. Finally, features are ranked according to their weights in linear combined kernel, and a proper feature subset consisting of top ranking features is selected. Thus a novel method of feature selection for multi-label learning is developed which can learn and address importance degree of labels automatically, and effectiveness of this method is demonstrated by experimental comparisons.

Feature selection for multi-label learning with missing labels

In multi-label learning, feature selection is a non-ignorable preprocessing step which can alleviate the negative effect of high-dimensionality. To address this problem, a number of effective information theory based feature selection algorithms for multi-label learning are proposed. However, these existing algorithms assume that the label space of multi-label training data is complete. In practice, the standpoint does not always hold true, due to the ambiguity among class labels or the cost effort to fully annotate instances. In this paper, we first define the new concepts of multi-label information entropy and multi-label mutual information. Then, feature redundancy, feature independence, and feature interaction are defined, respectively. In which, feature interaction is used to select more valuable features which may be ignored due to the incomplete label space. Moreover, a multi-label feature selection method with missing labels is proposed. Finally, extensive experiments conducted on eight publicly available data sets verify the effectiveness of the proposed algorithm via comparing it with state-of-the-art methods.

Feature selection for multi-label learning based on kernelized fuzzy rough sets

Abstract Feature selection is an essential pre-processing part in multi-label learning. Multi-label learning is usually used to deal with many complicated tasks, in which each sample is associated with multiple labels simultaneously. Fuzzy rough set model is one of the most effective ways for multi-label learning. However, it treats feature space and label space separately, and only uses features to describe sample structure information. In this paper, we fully consider the internal correlation between feature space and label space while fusing kernelized information from respective spaces. Moreover, we integrate fuzzy rough set with multiple kernel learning to finally realize feature selection. To be specific, firstly, we leverage one kind of kernel function to reveal the similarity between samples in feature space, and another one to assess the degree of label overlap between samples in label space. Secondly, we combine the kernelized information from the two spaces through linear combination to achieve precisely the lower approximation and construct a robust multi-label kernelized fuzzy rough set model, called RMFRS in this paper. Meanwhile, we discuss its properties and give theoretical analysis. Finally, we define a measurement criterion for selecting optimal features to evaluate the performance of the proposed algorithm. As many as 10 publicly available data sets are used to validate the effectiveness of our methods, and the result shows a distinct advantage over the state-of-the-art.

Online Multi-label Group Feature Selection

Feature selection for multi-label learning has received intensive interest in recent years. However, traditional multi-label feature selection are incapable of considering intrinsic group structures of features and handling streaming features simultaneously. To solve this problem, we develop an algorithm called Online Multi-label Group Feature Selection (OMGFS). Our proposed method consists of two-phase: online group selection and online inter-group selection. In the group selection, we design a criterion to select feature groups which is important to label set. In the inter-group selection, we consider feature interaction and feature redundancy to select an optimal feature subset. This two-phase procedure continues until there are no more features arriving. An empirical study using a series of benchmark data sets demonstrates that the proposed method outperforms other state-of-the-art multi-label feature selection methods.

Streaming Feature Selection for Multilabel Learning Based on Fuzzy Mutual Information

Due to complex semantics, a sample may be associated with multiple labels in various classification and recognition tasks. Multilabel learning generates training models to map feature vectors to multiple labels. There are several significant challenges in multilabel learning. Samples in multilabel learning are usually described with high-dimensional features and some features may be sequentially extracted. Thus, we do not know the full feature set at the beginning of learning, referred to as streaming features. In this paper, we introduce fuzzy mutual information to evaluate the quality of features in multilabel learning, and design efficient algorithms to conduct multilabel feature selection when the feature space is completely known or partially known in advance. These algorithms are called multilabel feature selection with label correlation (MUCO) and multilabel streaming feature selection (MSFS), respectively. MSFS consists of two key steps: online relevance analysis and online redundancy analysis. In addition, we design a metric to measure the correlation between the label sets, and both MUCO and MSFS take label correlation to consideration. The proposed algorithms are not only able to select features from streaming features, but also able to select features for ordinal multilabel learning. However streaming feature selection is more efficient. The proposed algorithms are tested with a collection of multilabel learning tasks. The experimental results illustrate the effectiveness of the proposed algorithms.

Joint Feature Selection and Classification for Multilabel Learning.

Multilabel learning deals with examples having multiple class labels simultaneously. It has been applied to a variety of applications, such as text categorization and image annotation. A large number of algorithms have been proposed for multilabel learning, most of which concentrate on multilabel classification problems and only a few of them are feature selection algorithms. Current multilabel classification models are mainly built on a single data representation composed of all the features which are shared by all the class labels. Since each class label might be decided by some specific features of its own, and the problems of classification and feature selection are often addressed independently, in this paper, we propose a novel method which can perform joint feature selection and classification for multilabel learning, named JFSC. Different from many existing methods, JFSC learns both shared features and label-specific features by considering pairwise label correlations, and builds the multilabel classifier on the learned low-dimensional data representations simultaneously. A comparative study with state-of-the-art approaches manifests a competitive performance of our proposed method both in classification and feature selection for multilabel learning.

Feature Selection for Multi-label Learning: A Systematic Literature Review and Some Experimental Evaluations

AbstractFeature selection can remove non-important features from the data and promote better classifiers. This task, when applied to multi-label data where each instance is associated with a set of labels, supports emerging applications. Although multi-label data usually exhibit label relations, label dependence has been little studied in feature selection. We proposed two multi-label feature selection algorithms that consider label relations. These methods were experimentally competitive with traditional approaches. Moreover, this work conducted a systematic literature review, summarizing 74 related papers.

Symptom selection for multi-label data of inquiry diagnosis in traditional Chinese medicine

In traditional Chinese medicine (TCM) diagnosis, a patient may be associated with more than one syndrome tags, and its computer-aided diagnosis is a typical application in the domain of multi-label learning of high-dimensional data. It is common that a great deal of symptoms can occur in traditional Chinese medical diagnosis, which affects the modeling of diagnostic algorithm. Feature selection entails choosing the smallest feature subset of relevant symptoms, and maximizing the generalization performance of the model. At present there are rare researches on feature selection on multi-label data. A hybrid optimization technique is introduced to symptom selection for multi-label data in TCM diagnosis in this paper, and modeling is made by means of four multi-label learning algorithms like k nearest neighbors, etc. We compare the performance of the algorithm with the current popular dimension reduction algorithms like MEFS (embedded feature selection for multi-Label learning), MDDM (multi-label dimensionality reduction via dependence maximization) on the UCI Yeast gene functional data set and an inquiry diagnosis dataset of coronary heart disease (CHD). Experimental results show that the algorithm we present has significantly improved the performance. In particular, the improvement on the average precision for the classifier is up to 10.62% and 14.54%. Syndrome inquiry modeling of CHD in TCM is realized in this paper, providing effective reference for the diagnosis of CHD and analysis of other multi-label data.