Multi-label Feature Selection Methods Research Articles

Discriminative label correlation based robust structure learning for multi-label feature selection

Feature selection is a key technique to tackle the curse of dimensionality in multi-label learning. Lots of embedded multi-label feature selection methods have been developed. However, they face challenges in identifying and excluding redundant features. To address these issues, this paper proposes a multi-label feature selection method that combines robust structural learning and discriminative label regularization. The proposed method starts from the feature space rather than data space, motivated by the principle that redundant features have high similarity or strong correlation. To exclude redundant features, a regularization on the feature selection matrix is designed by combining ℓ2,1-norm penalty with inner products of feature weight vectors. This regularization can help to learn a robust structure in the feature selection matrix. Meanwhile, both of the similarity and dissimilarity of labels of instances are involved in exploring discriminative label correlations. Extensive experiments verified the effectiveness of the proposed model for feature selection.

Partial multi-label feature selection via low-rank and sparse factorization with manifold learning

Feature selection is a commonly utilized methodology in multi-label learning (MLL) for tackling the challenge of high-dimensional data. Accurate annotation of relevant labels is crucial for successful multi-label feature selection (MFS). Nevertheless, multi-label datasets frequently consist of ground-truth and noisy labels in real-world applications, giving rise to the partial multi-label learning (PML) problem. The inclusion of noisy labels complicates the task of conventional MFS methods in accurately identifying the optimal features subset in such datasets. To tackle this issue, we propose a novel partial multi-label feature selection method with low-rank sparse factorization and manifold learning, called PMFS-LRS. Specifically, we first decompose the candidate label matrix into two distinct components: a low-rank matrix referring to ground-truth labels and a sparse matrix referring to noisy labels. This decomposition allows PMFS-LRS to effectively distinguish noise labels from ground-truth labels, thereby mitigating the impact of noisy data. Then, the local label correlations are explored using a manifold learning framework to improve the label disambiguation performance. Finally, a l2,1-norm regularization is integrated into the objective function to facilitate effective feature selection. Comprehensive experiments conducted on both real-world and synthetic PML datasets demonstrate that PMFS-LRS is superior to several existing state-of-the-art MFS methods.

Label relaxation and shared information for multi-label feature selection

Due to the rapid growth of labels and high-dimensional data, multi-label feature selection has attracted increasing attention. However, two common issues are ignored by existing multi-label feature selection methods: 1) The pseudo-label matrix is constructed by using logical label matrix directly. 2) There is an imbalance of label density in multi-label data. To tackle the mentioned issues, we innovatively propose a new method named Label relaxation and Shared information for Multi-label Feature Selection (LSMFS). Specifically, LSMFS combines a logical label matrix with a non-negative label relaxation matrix to fit a pseudo-label matrix, which is used for learning the correlations of class labels. LSMFS uses the feature weight matrix to capture a shared information of different related labels, which mitigates the influence of low-density labels on feature selection. The above principles are transformed into the objective function of LSMFS, and an alternate iterative optimization algorithm is developed to solve the objective function. Experiments on multi-label datasets from different domains have demonstrated the effectiveness of the proposed method LSMFS. Code is released at https://github.com/HQUF/LSMFS.

Semi-supervised multi-label feature selection based on dual dynamic graph

Traditional multi-label feature selection is usually performed under the condition of given label information, but nowadays, labeling multi-label data is a huge project, which is both time-consuming and labor-intensive, but if there is no label information condition, it will lead to poor feature selection, in order to solve this problem, this paper proposes a new semi-supervised multi-label feature selection method, i.e., semi-supervised multi-label feature selection algorithm based on dual dynamic graph. In this paper, a semi-supervised multi-label feature selection algorithm is proposed by constructing a dual dynamic graph. First, the method selects the most discriminative features for dimensionality reduction through the feature selection method of least squares regression, combined with the redundancy penalty of highly correlated features. Second, the label information is added to the construction of sample matrix similarity to learn the similarity. A semi-supervised multi-label feature selection framework is constructed by designing iterative updates of dual dynamic graphs to learn more accurate pseudo-label matrices to guide feature selection. Finally, the paper validates the above model using the alternating iteration optimization algorithm and verifies the effectiveness of the algorithm through experiments.

Multi-label feature selection via latent representation learning and dynamic graph constraints

As an effective method to deal with the curse of dimensionality, multi-label feature selection aims to select the most representative subset of features by eliminating unfavorable features. Although great progress has been made in this field, how to mine adequate supervisory information from multi-label data remains a key challenge. Compared to the latent information of instances, the latent information of instance relevance contains both the basic information of instances and the latent relevance between instances. Base on this knowledge, we propose a novel multi-label feature selection method named LRDG that explores latent representation learning and dynamic graph constraints. Specifically, we introduce the latent representation of instance relevance as supervisory information for pseudo-label learning, and minimize information loss during pseudo-label learning by means of the label manifold, the non-negative constraints, and the minimization of the Frobenius norm between pseudo-labels and ground-truth labels. In addition, considering the shortcomings brought by traditional graph regularization, we propose to use the dynamic graph constructed from low-dimensional pseudo-labels to constrain feature weights. Extensive experiments on various multi-label datasets demonstrate the effectiveness of the proposed method. The code is available at https://github.com/yunbao520/LRDG.

Multi-label Feature selection with adaptive graph learning and label information enhancement

The high dimensionality and complexity of multi-label data make obtaining accurate label sets in practical applications difficult. Noisy data in the labels will affect the model’s classification performance. Existing methods only reconstruct logical labels into semantic labels to mitigate the effect of inherent noise and do not specifically address the noise. In order to solve this problem and eliminate the effect of noise, this paper proposes a multi-label feature selection method with adaptive graph learning and label information enhancement (AGLE). First, each label is represented as a combination of other labels using global label correlation, which can reconstruct the original label matrix to obtain the self-expression label matrix. Based on this, similar labels in two matrices can be considered reliable labels, and not similar may be noise. Therefore, constructing a predictive label matrix to weigh the learning of original and self-expression labels can enhance reliable information. Second, an adaptive graph is utilized to learn global label correlation to ensure that the self-expression label matrix can be continuously optimized. Then, the local label manifold structure is utilized to constrain the adaptive graph for more stable performance. Finally, the adaptive graph and ℓ2,1-norm are combined to guide learning the feature weight matrix. For the proposed method, an iterative algorithm based on alternating optimization is used to compute the weight matrix. Comparing experimental results with several state-of-the-art algorithms demonstrates the effectiveness of the AGLE method for the noise problem.

Multi-label feature selection via adaptive dual-graph optimization

As in single-label learning, multi-label learning (MLL) also suffers from the problem of “the curse of dimensionality” due to the redundancy of features in the original data. To address this problem, extensive research has been conducted to explore the various feature selection algorithms aiming to eliminate irrelevant and redundant features. Among these, the manifold learning based methods have gained significant attention for their ability to capture local label correlations. However, previous approaches have mainly focused on extracting information from data space, overlooking valuable information in feature space. In addition, these methods conduct manifold learning and feature selection independently, thereby degrading the learning performance. To tackle this problem, we propose a novel multi-label feature selection method based on adaptive dual-graph optimization (MFS-ADGO) to learn the geometric structure of the data and select discriminative feature subsets simultaneously. Specifically, a dual-graph manifold regularization is constructed first, to locally capture label correlations in the data and feature spaces for discriminative features selection. A constraint is then imposed on the similarity matrices to adaptively learn the local geometric structure of the data during the feature selection, thereby capturing more reliable information from the original data. To facilitate feature selection, an objective function incorporating the l2,1−2-norm is proposed, along with an iterative updating algorithm. Extensive experiments unequivocally demonstrated the efficacy of MFS-ADGO in enhancing state-of-the-art feature selection methods in the realm of MLL.

A Robust Multilabel Feature Selection Approach Based on Graph Structure Considering Fuzzy Dependency and Feature Interaction

The performance of multilabel learning depends heavily on the quality of the input features. A mass of irrelevant and redundant features may seriously affect the performance of multilabel learning, and feature selection is an effective technique to solve this problem. However, most multilabel feature selection methods mainly emphasize removing these useless features, and the exploration of feature interaction is ignored. Moreover, the widespread existence of real-world data with uncertainty, ambiguity, and noise limits the performance of feature selection. To this end, our work is dedicated to designing an efficient and robust multilabel feature selection scheme. First, the distribution character of multilabel data is analyzed to generate robust fuzzy multi-neighborhood granules. By exploring the classification information implied in the data under the granularity structure, a robust multilabel <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$k$</tex-math></inline-formula> -nearest neighbor fuzzy rough set model is constructed, and the concept of fuzzy dependency is studied. Second, a series of fuzzy multi-neighborhood uncertainty measures in <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$k$</tex-math></inline-formula> -nearest neighbor fuzzy rough approximation spaces are studied to analyze the correlations of feature pairs, including interactivity. Third, by investigating the uncertainty measure between feature and label, between features, multilabel data is modeled as a complete weighted graph. Then, these vertices are assessed iteratively to guide the assignment of feature weights. Finally, a graph structure-based robust multilabel feature selection algorithm (GRMFS) is designed. The experiments are conducted on fifteen multilabel datasets. The results verify the superior performance of GRMFS as compared with nine representative feature selection methods.

Multi-label feature selection by strongly relevant label gain and label mutual aid

Multi-label feature selection, which addresses the challenge of high dimensionality in multi-label learning, has wide applicability in pattern recognition, machine learning, and related domains. Most existing studies on multi-label feature selection assume that all labels have the same importance with respect to features, however, they overlook the differences between labels and candidate features relative to selected features and the internal influence of the label space. To address this issue, we propose a novel method for multi-label feature selection that accounts for both the strongly relevant label gain and the label mutual aid. Firstly, we advance two new potential relationships between labels and candidate features relative to selected features, and the label discriminant function is introduced. Secondly, the mutual aid information between labels is presented to describe the internal correlation of the label space. Thirdly, the concept of strongly relevant label gain is defined based on the label discriminant function, which allows better exploration of positive correlation between features. Finally, the experimental results on sixteen multi-label benchmark datasets indicate that the proposed method outperforms other compared representative multi-label feature selection methods.

Learning correlation information for multi-label feature selection

In many real-world multi-label applications, the content of multi-label data is usually characterized by high dimensional features, which contains complex correlation information, i.e., label correlations and redundant features. To alleviate the problem, we present a novel scheme, called learning correlation information for multi-label feature selection (LCIFS) method, by jointly digging up label correlations and controlling feature redundancy. To be specific, the regression model via manifold framework is presented to fit the relationship between feature space and label distribution, during which adaptive spectral graph is leveraged to learn more precise structural correlations of labels simultaneously. Besides, we utilize the relevance of features to constrain the redundancy of the generated feature subset, and a general ℓ2,p-norm regularized model is employed to fulfill more robust feature selection. The proposed method is transformed into an explicit optimization function, which is conquered by an efficient iterative optimization algorithm. Finally, we conduct comprehensive experiments on twelve realistic multi-label datasets, including text domain, image domain, and audio domain. The statistic results demonstrate the effectiveness and superiority of the proposed method among nine competition methods.

Multi-Label Feature Selection Via Adaptive Label Correlation Estimation

In multi-label learning, each instance is associated with multiple labels simultaneously. Multi-label data often have noisy, irrelevant, and redundant features of high dimensionality. Multi-label feature selection has received considerable attention as an effective means for dealing with high-dimensional multi-label data. Many multi-label feature selection methods exploit label correlations to help select features. However, finding label correlations and selecting features in existing multi-label feature selection methods are often two separate processes, the existence of noises and outliers in training data makes the label correlations exploited from label space less reliable. Therefore, the learned label correlations may mislead the feature selection process and result in the selection of less informative features. This article proposes a novel algorithm named ROAD, i.e., multi-label featuRe selectiOn via ADaptive label correlation estimation. ROAD jointly performs adaptive label correlation exploration and feature selection with alternating optimization to obtain reliable estimation of label correlations, which can more effectively reveal the intrinsic manifold structure among labels and lead to the selection of a more proper feature subset. Comprehensive experiments on several frequently used datasets validate the superiority of ROAD against the state-of-the-art multi-label feature selection algorithms.

Multilabel Feature Selection With Constrained Latent Structure Shared Term.

High-dimensional multilabel data have increasingly emerged in many application areas, suffering from two noteworthy issues: instances with high-dimensional features and large-scale labels. Multilabel feature selection methods are widely studied to address the issues. Previous multilabel feature selection methods focus on exploring label correlations to guide the feature selection process, ignoring the impact of latent feature structure on label correlations. In addition, one encouraging property regarding correlations between features and labels is that similar features intend to share similar labels. To this end, a latent structure shared (LSS) term is designed, which shares and preserves both latent feature structure and latent label structure. Furthermore, we employ the graph regularization technique to guarantee the consistency between original feature space and latent feature structure space. Finally, we derive the shared latent feature and label structure feature selection (SSFS) method based on the constrained LSS term, and then, an effective optimization scheme with provable convergence is proposed to solve the SSFS method. Better experimental results on benchmark datasets are achieved in terms of multiple evaluation criteria.

Integrating Global and Local Feature Selection for Multi-Label Learning

Multi-label learning deals with the problem where an instance is associated with multiple labels simultaneously. Multi-label data is often of high dimensionality and has many noisy, irrelevant, and redundant features. As an important machine learning task, multi-label feature selection has received considerable attention in recent years due to its promising performance in dealing with high-dimensional multi-label data. Existing multi-label feature selection methods typically select the global features which are shared by all instances in a dataset. However, these multi-label feature selection methods may be suboptimal since they do not consider the specific characteristics of instances. In this paper, we propose a novel algorithm that integrates Global and Local Feature Selection (GLFS) to exploit both the global features and a subset of discriminative features shared only locally by a subgroup of instances in a multi-label dataset. Specifically, GLFS employs linear regression and ℓ 2,1 -norm on the regression parameters to achieve simultaneous global and local feature selection. Moreover, the proposed algorithm has an effective mechanism for utilizing label correlations to improve the feature selection. Experiments on real-world multi-label datasets show the superiority of GLFS over the state-of-the-art multi-label feature selection methods.

MFSJMI: Multi-label feature selection considering join mutual information and interaction weight

Multi-label feature selection captures a reliable and informative feature subset from high-dimensional multi-label data, which plays an important role in pattern recognition. In conventional information-theoretical based multi-label feature selection methods, the high-order feature relevance between feature and label set is evaluated using low-order mutual information. However, existing methods do not establish the theoretical basis for the low-order approximation. To fill this gap, we first identify two underlying assumptions based on high-order label distribution: Label Independence Assumption (LIA) and Paired-label Independence Assumption (PIA). Second, we systematically analyze the strengths and weaknesses of two assumptions and introduce joint mutual information to satisfy more realistic label distribution. Furthermore, by decomposing joint mutual information, an interaction weight is proposed to consider multiple label correlations. Finally, a new method considering join mutual information and interaction weight is proposed. Comprehensive experiments demonstrate the effectiveness of the proposed method on various evaluation metrics.

Robust feature selection using label enhancement and β-precision fuzzy rough sets for multilabel fuzzy decision system

High-dimensionality is the most noticeable characteristic of multilabel data. In practice, multilabel data typically contain complex noises. Ignoring these noises in the feature selection process tends to cause an inaccurate prediction. Besides, many existing multilabel feature selection methods assume that the relation between samples and relevant labels is crisp, and the difference information hiding in the label space is lost. Given all these, in this paper, soft labels are investigated by label enhancement in multilabel feature selection. A robust feature selection algorithm is proposed using label enhancement and multilabel β-precision fuzzy rough sets. With the perspective of data distribution from samples in the same and different class, the margin-based robust fuzzy neighborhood is first defined to construct the robust fuzzy granular space. Second, label enhancement strategy is given in the robust fuzzy granular space considering multilabel data distribution. To investigate the noise-tolerant model, the underlying structure of label space after label enhancement is employed to encode the score vector of samples, which is used to search the pseudo-different class's samples of target sample. Then, the multilabel β-precision fuzzy rough set model is built to deal with multilabel data. Moreover, the fuzzy approximation degree of knowledge and the fuzzy dependency of decision classes with respect to conditional features are fused to measure the significance of features. Finally, a robust heuristic multilabel feature selection algorithm is proposed. Extensive experiments on classification performance and anti-noise ability are conducted, which verify that the proposed algorithm is effective and robust.

Multi-Label Feature Selection Based on Min-Relevance Label

Multi-label feature selection has been widely adopted to address multi-label data with high-dimension features. It is critical to calculate label correlations for multi-label feature selection. Existing methods adopt different schemes to calculate label correlations, which obtain different importance of labels. However, there exist two issues regarding these schemes calculating label importance: first, previous schemes cannot predict the whole labels well because they only focus on the most important labels; second, most of important labels have similar classification information corresponding to redundant features. To this end, we use the mutual information metric to obtain different cores regarding label set rather than calculating the importance of labels. Afterwards, we capture features with respect to each core label, finally, obtaining an optimal feature subset. To verify the effectiveness, our method is compared to state-of-the-art multi-label methods on 16 real-world data sets with several evaluating metrics. The results of experiment proves that the proposed method has achieves the best classification performance among all multi-label feature selection methods.

Multi-label feature selection based on HSIC and sparrow search algorithm.

Feature selection has always been an important topic in machine learning and data mining. In multi-label learning tasks, each sample in the dataset is associated with multiple labels, and labels are usually related to each other. At the same time, multi-label learning has the problem of "curse of dimensionality". Feature selection therefore becomes a difficult task. To solve this problem, this paper proposes a multi-label feature selection method based on the Hilbert-Schmidt independence criterion (HSIC) and sparrow search algorithm (SSA). It uses SSA for feature search and HSIC as feature selection criterion to describe the dependence between features and all labels, so as to select the optimal feature subset. Experimental results demonstrate the effectiveness of the proposed method.

A novel multi-label feature selection method with association rules and rough set

In multi-label learning, each instance is associated with a set of labels. To improve the accuracy and efficiency of multi-label learning tasks, label correlations have been explored. However, existing conventional algorithms obtain label correlation directly from the raw label space, ignoring the significance of the label to the instance. In this study, a novel feature selection method was proposed to select a more relevant and compact feature subset by considering the label distribution and inter-label correlations. First, the concept of label distribution was defined to reflect the significance of each label. Second, a new algorithm for mining association rules was designed to obtain the correlation between labels by improving the existing association rules algorithm. Thus, a new information system was designed by combining the label distribution and correlation between labels. Subsequently, a novel feature selection algorithm was designed in this information system which can effectively remove irrelevant and redundant features in the feature space. Finally, the experimental results demonstrated that the proposed algorithm effectively improves the classification performance and perform better than some state-of-the-art methods.

A Multi-Objective online streaming Multi-Label feature selection using mutual information

Multi-label classification methods aim at assigning more than one label to each instance. In many real-world classification problems such as image multi-label classification tasks such as cancer detection, and text classification, we faced with thousands of thousand features. The performance of machine learning methods will be reduced while faced with high dimensional problems. To tackle this issue, feature selection methods are introduced to choose a small set of prominent features which best describe the data. Traditional multi-label feature selection methods are required to access to whole feature space, while in online platforms such as Facebook and Twitter, we faced with streams of data added by the users of these platforms over the time. Traditional multi-label feature selection methods are failed while applied on data streams. To solve this issue, online methods are introduced to deal with data streams. Existing streaming multi-label feature selection methods consider the task as a single optimization process while there are several contradictory objectives that need to be optimize simultaneously. To solve this issue, this paper uses a multi-objective search strategy to choose streaming features by using the mutual information and Pareto optimal set theories. There are several objectives such as minimizing the redundancy of features, and maximizing the relevancy of features to a set of labels that are need to be optimized during the feature selection process. Here, we used the Pareto set theory to identify a set of no-dominant solutions which best describe the problem. The proposed method has compared with a set of state-of-the-art online feature selection methods and the obtained results demonstrate the effectiveness of the proposed strategy.

Robust sparse and low-redundancy multi-label feature selection with dynamic local and global structure preservation

Recent years, joint feature selection and multi-label learning have received extensive attention as an open problem. However, there exist three general issues in previous multi-label feature selection methods. First of all, existing methods either consider local label correlations or global label correlations when they design multi-label feature selection methods, in fact, both two types of label correlations are significant for feature selection; second, previous methods use the low-quality graph to excavate local label correlations so that the results of these methods are under-performing; third, feature redundancy is ignored by most of the sparse learning methods. To overcome these challenges, we preserve global label correlations and dynamic local label correlations by preserving graph structure. Additionally, the l2,1-norm and an inner product regularization term are imposed onto the objective function to preserve robust high row-sparsity and to select low redundant features. All the above terms are integrated into one learning framework, and then we utilize a simple yet effective scheme to optimize the framework. Experimental results demonstrate the classification superiority of the proposed method.