Multi-label Feature Selection Research Articles

Multi-label feature selection with global and local label correlation

In various application domains, high-dimensional multi-label data has become more prevalent, presenting two significant challenges: instances with high-dimensional features and a large number of labels. In the context of multi-label feature selection, the objective is to choose a subset of features from a given set that is highly pertinent for predicting multiple labels or categories associated with each instance. However, certain characteristics of multi-label classification, such as label dependencies and imbalanced label distribution, have often been overlooked although they hold valuable insights for designing effective multi-label feature selection algorithms. In this paper, we propose a feature selection model which exploits explicit global and local label correlations to select discriminative features across multiple labels. In addition, by representing the feature matrix and label matrix in a shared latent space, the model aims to capture the underlying correlations between features and labels. The shared representation can reveal common patterns or relationships that exist across multiple labels and features. An objective function involving L2,1-norm regularization is formulated, and an alternating optimization-based iterative algorithm is designed to obtain the sparse coefficients for multi-label feature selection. The proposed method was evaluated on 14 real-world multi-label datasets using six evaluation metrics, through comprehensive experiments. The results indicate its effectiveness, surpassing that of several representative methods.

Multi-label feature selection with high-sparse personalized and low-redundancy shared common features

Prevalent multi-label feature selection (MLFS) approaches to obtain the most suitable feature subset by dealing with two issues, namely sparsity and redundancy. In this paper, we design an efficient Elastic net based high Sparse personalized and low Redundancy Feature Selection approach for multi-label data named ESRFS to address the two obstacles, i.e., low-sparse LASSO-norm leads to personalized features for each label while high redundancy l2,1-norm explore shared common features for all labels in multi-label learning. These two problems impede the selection of high-quality features for classification. In comparison with previous MLFS approaches, ESRFS has two main superiority. First, ESRFS achieves higher sparse personalized features than LASSO-norm. Second, ESRFS can identify low redundancy shared common features with strong discrimination by introducing a novel regularization term. To effectively and efficiently identify the most optimal feature subset, an alternating-multiplier-based rule is introduced to optimize ESRFS. Experimental results on fifteen multi-label data sets show that ESRFS can achieve obvious superior performance compared to eight state-of-the-art MLFS approaches in 80%, 80%, 73.3%, 80%, 86.7%, 80% cases based on Hamming Loss, Zero-One Loss using MLkNN, Micro-F1 and Macro-F1 using SVM as well as Micro-F1 and Macro-F1 using 3NN perspectives.

Multi-label Feature selection with adaptive graph learning and label information enhancement

The high dimensionality and complexity of multi-label data make obtaining accurate label sets in practical applications difficult. Noisy data in the labels will affect the model’s classification performance. Existing methods only reconstruct logical labels into semantic labels to mitigate the effect of inherent noise and do not specifically address the noise. In order to solve this problem and eliminate the effect of noise, this paper proposes a multi-label feature selection method with adaptive graph learning and label information enhancement (AGLE). First, each label is represented as a combination of other labels using global label correlation, which can reconstruct the original label matrix to obtain the self-expression label matrix. Based on this, similar labels in two matrices can be considered reliable labels, and not similar may be noise. Therefore, constructing a predictive label matrix to weigh the learning of original and self-expression labels can enhance reliable information. Second, an adaptive graph is utilized to learn global label correlation to ensure that the self-expression label matrix can be continuously optimized. Then, the local label manifold structure is utilized to constrain the adaptive graph for more stable performance. Finally, the adaptive graph and ℓ2,1-norm are combined to guide learning the feature weight matrix. For the proposed method, an iterative algorithm based on alternating optimization is used to compute the weight matrix. Comparing experimental results with several state-of-the-art algorithms demonstrates the effectiveness of the AGLE method for the noise problem.

Multi-label feature selection via constraint mapping space regularization

<abstract><p>Multi-label feature selection, an essential means of data dimension reduction in multi-label learning, has become one of the research hotspots in the field of machine learning. Because the linear assumption of sample space and label space is not suitable in most cases, many scholars use pseudo-label space. However, the use of pseudo-label space will increase the number of model variables and may lead to the loss of sample or label information. A multi-label feature selection scheme based on constraint mapping space regularization is proposed to solve this problem. The model first maps the sample space to the label space through the use of linear mapping. Second, given that the sample cannot be perfectly mapped to the label space, the mapping space should be closest to the label space and still retain the space of the basic manifold structure of the sample space, so combining the Hilbert-Schmidt independence criterion with the sample manifold, basic properties of constraint mapping space. Finally, the proposed algorithm is compared with MRDM, SSFS, and other algorithms on multiple classical multi-label data sets; the results show that the proposed algorithm is effective on multiple indicators.</p></abstract>

Related Factors Mining of Diabetes Complications Based on Manifold-Constrained Multi-Label Feature Selection.

The primary cause of mortality among individuals with diabetes stems from complications. Identifying related factors for these complications holds immense potential for early prevention. Previous research predominantly employed traditional machine-learning techniques to establish prediction models utilizing medical indicators for related factor selection. However, uncovering the intricate correlations among complication labels and identifying similar characteristics among medical indicators has been challenging. We propose a novel embedded multi-label feature selection approach called LCFSM(Label Cosine and Feature Similar Manifold) to address the issue. LCFSM introduces manifold constraints into the objective function to uncover risk factors associated with diabetes complications. Label cosine similarity is set to optimize feature weights, forming label manifold constraints. Similarly, feature manifold constraints are established to utilize feature kernel similarity in optimizing feature weights. LCFSM formulates an objective function based on the l2,1 regularized Least Squares and previous manifolds constraints, employing the Sylvester equationfor convergence assurance. The experimental evaluation compares LCFSM against eight baselines, demonstrating superior performance in top-10 feature selection and feature stacking.LCFSM is applied to identify primary risk factors for diabetes complications. Related factors involve Electromyogram, Urine Routine Protein Positive, etc, offering valuable insights for early treatment.

Multilabel Feature Selection via Shared Latent Sublabel Structure and Simultaneous Orthogonal Basis Clustering.

Multilabel feature selection solves the dimension distress of high-dimensional multilabel data by selecting the optimal subset of features. Noisy and incomplete labels of raw multilabel data hinder the acquisition of label-guided information. In existing approaches, mapping the label space to a low-dimensional latent space by semantic decomposition to mitigate label noise is considered an effective strategy. However, the decomposed latent label space contains redundant label information, which misleads the capture of potential label relevance. To eliminate the effect of redundant information on the extraction of latent label correlations, a novel method named SLOFS via shared latent sublabel structure and simultaneous orthogonal basis clustering for multilabel feature selection is proposed. First, a latent orthogonal base structure shared (LOBSS) term is engineered to guide the construction of a redundancy-free latent sublabel space via the separated latent clustering center structure. The LOBSS term simultaneously retains latent sublabel information and latent clustering center structure. Moreover, the structure and relevance information of nonredundant latent sublabels are fully explored. The introduction of graph regularization ensures structural consistency in the data space and latent sublabels, thus helping the feature selection process. SLOFS employs a dynamic sublabel graph to obtain a high-quality sublabel space and uses regularization to constrain label correlations on dynamic sublabel projections. Finally, an effective convergence provable optimization scheme is proposed to solve the SLOFS method. The experimental studies on the 18 datasets demonstrate that the presented method performs consistently better than previous feature selection methods.

Multi-Label Feature Selection Based on Improved Ant Colony Optimization Algorithm with Dynamic Redundancy and Label Dependence

Multi-Label Feature Selection Based on Improved Ant Colony Optimization Algorithm with Dynamic Redundancy and Label Dependence

Feature relevance and redundancy coefficients for multi-view multi-label feature selection

Feature relevance and redundancy coefficients for multi-view multi-label feature selection

Joint subspace reconstruction and label correlation for multi-label feature selection

Joint subspace reconstruction and label correlation for multi-label feature selection

Multi-label feature selection via adaptive dual-graph optimization

As in single-label learning, multi-label learning (MLL) also suffers from the problem of “the curse of dimensionality” due to the redundancy of features in the original data. To address this problem, extensive research has been conducted to explore the various feature selection algorithms aiming to eliminate irrelevant and redundant features. Among these, the manifold learning based methods have gained significant attention for their ability to capture local label correlations. However, previous approaches have mainly focused on extracting information from data space, overlooking valuable information in feature space. In addition, these methods conduct manifold learning and feature selection independently, thereby degrading the learning performance. To tackle this problem, we propose a novel multi-label feature selection method based on adaptive dual-graph optimization (MFS-ADGO) to learn the geometric structure of the data and select discriminative feature subsets simultaneously. Specifically, a dual-graph manifold regularization is constructed first, to locally capture label correlations in the data and feature spaces for discriminative features selection. A constraint is then imposed on the similarity matrices to adaptively learn the local geometric structure of the data during the feature selection, thereby capturing more reliable information from the original data. To facilitate feature selection, an objective function incorporating the l2,1−2-norm is proposed, along with an iterative updating algorithm. Extensive experiments unequivocally demonstrated the efficacy of MFS-ADGO in enhancing state-of-the-art feature selection methods in the realm of MLL.

A Robust Multilabel Feature Selection Approach Based on Graph Structure Considering Fuzzy Dependency and Feature Interaction

The performance of multilabel learning depends heavily on the quality of the input features. A mass of irrelevant and redundant features may seriously affect the performance of multilabel learning, and feature selection is an effective technique to solve this problem. However, most multilabel feature selection methods mainly emphasize removing these useless features, and the exploration of feature interaction is ignored. Moreover, the widespread existence of real-world data with uncertainty, ambiguity, and noise limits the performance of feature selection. To this end, our work is dedicated to designing an efficient and robust multilabel feature selection scheme. First, the distribution character of multilabel data is analyzed to generate robust fuzzy multi-neighborhood granules. By exploring the classification information implied in the data under the granularity structure, a robust multilabel <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$k$</tex-math></inline-formula> -nearest neighbor fuzzy rough set model is constructed, and the concept of fuzzy dependency is studied. Second, a series of fuzzy multi-neighborhood uncertainty measures in <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$k$</tex-math></inline-formula> -nearest neighbor fuzzy rough approximation spaces are studied to analyze the correlations of feature pairs, including interactivity. Third, by investigating the uncertainty measure between feature and label, between features, multilabel data is modeled as a complete weighted graph. Then, these vertices are assessed iteratively to guide the assignment of feature weights. Finally, a graph structure-based robust multilabel feature selection algorithm (GRMFS) is designed. The experiments are conducted on fifteen multilabel datasets. The results verify the superior performance of GRMFS as compared with nine representative feature selection methods.

Multi-label feature selection based on rough granular-ball and label distribution

Multi-label feature selection based on rough granular-ball and label distribution

Sparse low-redundancy multilabel feature selection based on dynamic local structure preservation and triple graphs exploration

Much semantic information is involved in multilabel data due to more than one label associated with each instance. The redundant features and noise challenge knowledge mining in multilabel data. Constructing a learning model with discriminative features is essential for multilabel learning. Sparse graph-based methods simultaneously consider the topological structure, complex relations between features and labels, and the significance of features. However, three challenges exist. First, they either consider local label correlation or local label relevance and are complementary in the feature selection process. Second, existing methods use low-quality static graphs to explore local label correlations that result in degraded performance. Finally, only some methods deal with redundant features. A ridge regression-based sparse multilabel learning is proposed in this study to address these problems. The global and local label correlation are explored by preserving the instance-level graph structure to obtain a robust low-dimensional pseudo-label matrix to construct a high-quality dynamic label-level graph. Meanwhile, it preserves the feature-level graph structure to select low-redundant features. In addition, a new ℓ2,1/2−2-norm is designed to maintain the high-row sparsity of the model. The above items are embedded into a unified multilabel learning framework. A simple and effective optimization solution is finally designed and compared with eight relevant algorithms on twelve public benchmark data sets. The results demonstrate that the algorithm can improve classification performance.

Discriminative multi-label feature selection with adaptive graph diffusion

Feature selection can alleviate the problem of the curse of dimensionality by selecting more discriminative features, which plays an important role in multi-label learning. Recently, embedded feature selection methods have received increasing attentions. However, most existing methods learn the low-dimensional embeddings under the guidance of the local structure between the original instance pairs, thereby ignoring the high-order structure between instances and being sensitive to noise in the original features. To address these issues, we propose a feature selection method named discriminative multi-label feature selection with adaptive graph diffusion (MFS-AGD). Specifically, we first construct a graph embedding learning framework equipped with adaptive graph diffusion to uncover a latent subspace that preserves the higher-order structure information between four tuples. Then, the Hilbert–Schmidt independence criterion (HSIC) is incorporated into the embedding learning framework to ensure the maximum dependency between the latent representation and labels. Benefiting from the interactive optimization of the feature selection matrix, latent representation and similarity graph, the selected features can accurately explore the higher-order structural and supervised information of data. By further considering the correlation between labels, MFS-AG is extended to a more discriminative version,i.e., LMFS-AG. Extensive experimental results on various benchmark data sets validate the advantages of the proposed MFS-AGD and LMFS-AGD methods.

Exploiting feature multi-correlations for multilabel feature selection in robust multi-neighborhood fuzzy [formula omitted] covering space

Multilabel data contains rich label semantic information, and its data structure conforms to the cognitive laws of the actual world. However, these data usually involve many irrelevant, redundant, and noisy features, challenging constructing effective learning models. It is necessary to design a feature selection strategy to select valid information in multilabel data. Generally, constructing a robust granular space is essential for building learning methods that can capture the intrinsic information contained in data. In addition, carefully investigating the complex relationship between features facilitates the evaluation of features. However, the interactivity and complementarity between features are rarely studied for multilabel data. Fuzzy β covering constructs granular space flexibly and excavates potential uncertainty information. The concept of fuzzy β covering theory is extended to multilabel data to reasonably construct the granular space of multilabel data and effectively quantify the uncertainty of multilabel data. Concretely, a parameterized multilabel fuzzy β covering relation is proposed to resist the negative effect of ambiguity, uncertainty, and noise in multilabel data, and a robust multi-neighborhood fuzzy β covering granular space is constructed. Besides, the concept of multilabel fuzzy β covering decision is explored to improve the accuracy and robustness of multilabel learning. The above concepts build the basis of multilabel fuzzy β covering uncertainty measures, and a series of fuzzy β covering entropy measures are then defined. Consequently, feature multi-correlations are presented, including relevance, redundancy, interactivity, and complementarity between features. Finally, following the principle of correlation maximization, a robust multilabel feature selection considering feature multi-correlations (RMSMC) is designed. Extensive experiments illustrate the superiority of RMSMC over nine representative algorithms.

A Filter-Based Improved Multi-Objective Equilibrium Optimizer for Single-Label and Multi-Label Feature Selection Problem

Effectively reducing the dimensionality of big data and retaining its key information has been a research challenge. As an important step in data pre-processing, feature selection plays a critical role in reducing data size and increasing the overall value of the data. Many previous studies have focused on single-label feature selection, however, with the increasing variety of data types, the need for feature selection on multi-label data types has also arisen. Unlike single-labeled data, multi-labeled data with more combinations of classifications place higher demands on the capabilities of feature selection algorithms. In this paper, we propose a filter-based Multi-Objective Equilibrium Optimizer algorithm (MOEO-Smp) to solve the feature selection problem for both single-label and multi-label data. MOEO-Smp rates the optimization results of solutions and features based on four pairs of optimization principles, and builds three equilibrium pools to guide exploration and exploitation based on the total scores of solutions and features and the ranking of objective fitness values, respectively. Seven UCI single-label datasets and two Mulan multi-label datasets and one COVID-19 multi-label dataset are used to test the feature selection capability of MOEO-Smp, and the feature selection results are compared with 10 other state-of-the-art algorithms and evaluated using three and seven different metrics, respectively. Feature selection experiments and comparisons with the results in other literatures show that MOEO-Smp not only has the highest classification accuracy and excellent dimensionality reduction on single-labeled data, but also performs better on multi-label data in terms of Hamming loss, accuracy, dimensionality reduction, and so on.

Multi-label learning based on instance correlation and feature redundancy

Multi-label classification is a machine-learning task that simultaneously processes instances associated with multiple labels. Label-specific feature learning selects each label's most discriminative feature subset, effectively reducing the feature dimension and improving the classification performance. However, most methods only consider label correlation, ignoring the correlation between instances and feature redundancy. To solve this problem, a multi-label classification method based on instance correlation and feature redundancy is proposed. The proposed method merges instance correlation by updating the data set and removes redundant features by calculating mutual information. By jointly considering label correlation, instance correlation, and feature redundancy, our method promotes effective multi-label feature selection. The experimental results on ten data sets demonstrate the effectiveness of the proposed method.

Learning implicit labeling-importance and label correlation for multi-label feature selection with streaming labels

Multi-label feature selection plays an increasingly important role in alleviating the high dimensionality of multi-label learning tasks. Most extant methods posit that the learning task is performed in an environment where the label space is statically known. In reality, however, the environment is open and the labels may arrive dynamically, which is coined as streaming labels. Streaming labels-based multi-label feature selection suffers from many challenges derived from label space: (1) The label space expands dynamically; (2) Newly arrived labels exhibit complex relationships, often involving label correlation and labeling-importance. To cope with this challenge, in this paper, an intuitive yet effective algorithm named LLSL, i.e. learning implicit labeling-importance and label correlation for multi-label feature selection with streaming labels, is proposed. To be specific, the implicit labeling-importance with respect to streaming labels is firstly formalized by conducting the nearest neighbor reconstruction on feature space. Secondly, label correlation is seamlessly integrated into the objective function of feature relevance by designing the feature relevance influence factor. Based on the above, we build a feature conversion, which can realize the fusion of label-specific features for each streaming label. Finally, extensive experiments conducted on fifteen benchmark datasets provide clear evidence that LLSL has superior performance compared to three established streaming label-based MFS algorithms and seven static label space-based MFS algorithms.

Label correlations-based multi-label feature selection with label enhancement

Feature selection, as an important pre-processing technique, can efficiently mitigate the issue of “the curse of dimensionality” by selecting discriminative features especially for multi-label learning, a discriminative feature subset can improve the classification accuracy. The existing feature selection methods for multi-label classification address the problem of label ambiguity by with logical labels. However, the significance of each label is often different in many practical applications. Using logical label to train the model may result in unsatisfactory performance due to not considering the importance of related labels with each sample. To address this issue, a novel multi-label feature selection algorithm is proposed with two-step: label enhancement and label correlations-based feature selection with label enhancement. In the step of label enhancement, a framework of label enhancement based on deep forest is utilized to transform the logical label to label distribution, which contains rich semantic information and then guides a more correct exploration of semantic correlations. In the step of feature selection, a novel multi-label feature selection algorithm is proposed based on label distribution data. Firstly, the samples are divided into multiple different clusters by using spectral clustering in the label space. Then, the label correlations can be reflected by multiple different clusters. Finally, the l2,1-norm is used to construct an objective function to achieve multi-label feature selection. Experimental results demonstrate that competitiveness of the proposed algorithm over six state-of-the-art multi-label feature selection algorithms on eighteen benchmark datasets in terms of six widely accepted evaluation metrics.

Multi-label feature selection via maximum dynamic correlation change and minimum label redundancy

Multi-label feature selection via maximum dynamic correlation change and minimum label redundancy