Multi-label Feature Selection Algorithm Research Articles

Feature selection for multi-labeled data based on label enhancement technique and mutual information

Feature selection for multi-labeled data based on label enhancement technique and mutual information

Semi-supervised multi-label feature selection based on dual dynamic graph

Traditional multi-label feature selection is usually performed under the condition of given label information, but nowadays, labeling multi-label data is a huge project, which is both time-consuming and labor-intensive, but if there is no label information condition, it will lead to poor feature selection, in order to solve this problem, this paper proposes a new semi-supervised multi-label feature selection method, i.e., semi-supervised multi-label feature selection algorithm based on dual dynamic graph. In this paper, a semi-supervised multi-label feature selection algorithm is proposed by constructing a dual dynamic graph. First, the method selects the most discriminative features for dimensionality reduction through the feature selection method of least squares regression, combined with the redundancy penalty of highly correlated features. Second, the label information is added to the construction of sample matrix similarity to learn the similarity. A semi-supervised multi-label feature selection framework is constructed by designing iterative updates of dual dynamic graphs to learn more accurate pseudo-label matrices to guide feature selection. Finally, the paper validates the above model using the alternating iteration optimization algorithm and verifies the effectiveness of the algorithm through experiments.

Feature selection for multi-label learning based on variable-degree multi-granulation decision-theoretic rough sets

Multi-label learning (MLL) suffers from the high-dimensional feature space teeming with irrelevant and redundant features. To tackle this, several multi-label feature selection (MLFS) algorithms have emerged as vital preprocessing steps. Nonetheless, existing MLFS methods have their shortcomings. Primarily, while they excel at harnessing label-feature relationships, they often struggle to leverage inter-feature information effectively. Secondly, numerous MLFS approaches overlook the uncertainty in the boundary domain, despite its critical role in identifying high-quality features. To address these issues, this paper introduces a novel MLFS algorithm, named VMFS. It innovatively integrates multi-granulation rough sets with three-way decision, leveraging multi-granularity decision-theoretic rough sets (MGDRS) with variable degrees for optimal performance. Initially, we construct coarse decision (RDC), fine decision (RDF), and uncertainty decision (RDU) functions for each object based on MGDRS with variable degrees. These decision functions then quantify the dependence of attribute subsets, considering both deterministic and uncertain aspects. Finally, we employ the dependency to assess attribute importance and rank them accordingly. Our proposed method has undergone rigorous evaluation on various standard multi-label datasets, demonstrating its superiority. Experimental results consistently show that VMFS significantly outperforms other algorithms on most datasets, underscoring its effectiveness and reliability in multi-label learning tasks.

Multi-label Feature Selection based on Label-specific features and Manifold Learning

Each instance in multi-label data is associated with multiple labels, and there are irrelevant or redundant features in its feature space, which leads to the performance degradation of multi-label learning algorithms. Multi-label feature selection selects representative features from the feature space to improve the accuracy of the model. Due to the high cost of labels and the difficulty of data collection, there will be some missing labels in the data set, which affects the accuracy of feature selection. To solve this problem, a multi-label feature selection algorithm based on label-specific features and manifold learning is proposed. The algorithm uses the linear relationship between the features and labels in known label samples to build a linear regression model for learning label-specific features. By using the nonlinear relation between instances and the nonlinear relation between features, we can precisely learn the label-specific features. We use the Laplacian feature mapping method to construct the instance manifold model and the feature manifold model, which are also used as the regular term constraint weight matrix. The final model can not only complete the missing labels, but also select sparse and representative features. The feature selection is carried out by analyzing the weight of the feature given by the final model. Experiments were conducted to verify the effectiveness of the proposed algorithm under different label deletion rates on four evaluation indexes.

Dynamic multi-label feature selection algorithm based on label importance and label correlation

Multi-label distribution is a popular direction in current machine learning research and is relevant to many practical problems. In multi-label learning, samples are usually described by high-dimensional features, many of which are redundant or invalid. This paper proposes a multi-label static feature selection algorithm to solve the problems caused by high-dimensional features of multi-label learning samples. This algorithm is based on label importance and label relevance, and improves the neighborhood rough set model. One reason for using neighborhood rough sets is that feature selection using neighborhood rough sets does not require any prior knowledge of the feature space structure. Another reason is that it does not destroy the neighborhood and order structure of the data when processing multi-label data. The method of mutual information is used to achieve the extension from single labels to multiple labels in the multi-label neighborhood; through this method, the label importance and label relevance of multi-label data are connected. In addition, in the multi-label task scenario, features may be interdependent and interrelated, and features often arrive incrementally or can be extracted continuously; we call these flow features. Traditional static feature selection algorithms do not handle flow features well. Therefore, this paper proposes a dynamic feature selection algorithm for flow features, which is based on previous static feature selection algorithms. The proposed static and dynamic algorithms have been tested on a multi-label learning task set and the experimental results show the effectiveness of both algorithms.

Multi-label feature selection based on fuzzy rough sets with metric learning and label enhancement

Multi-label feature selection based on fuzzy rough sets, as a key step of multi-label data preprocessing, has been widely concerned by scholars in recent years. Most of the existing multi-label feature selection algorithms directly treat labels as logical labels and use a single distance metric to describe similarity. However, the variability of label descriptions and the limitations of a single distance measure should not be overlooked. In this paper, we propose a fuzzy rough set model with metric learning and label enhancement. Specifically, we use a kernel membership label enhancement algorithm based on JS divergence to convert logical labels into numerical labels, which not only reflects the importance of different labels, but also takes into account the differences in label distribution. In addition, a multi-metric learning algorithm is proposed for multi-label learning, in which the metric distance function under the label space and feature space can be learned autonomously. Then, based on the proposed model, we propose a novel multi-label feature selection algorithm based on metric learning and fuzzy rough sets. On this basis, a fast multi-label feature selection algorithm is further designed to improve the computational efficiency. In the experiments, compared with other nine algorithms on real-world datasets, the results show the superiority of the proposed algorithm.

Multi-label feature selection with global and local label correlation

In various application domains, high-dimensional multi-label data has become more prevalent, presenting two significant challenges: instances with high-dimensional features and a large number of labels. In the context of multi-label feature selection, the objective is to choose a subset of features from a given set that is highly pertinent for predicting multiple labels or categories associated with each instance. However, certain characteristics of multi-label classification, such as label dependencies and imbalanced label distribution, have often been overlooked although they hold valuable insights for designing effective multi-label feature selection algorithms. In this paper, we propose a feature selection model which exploits explicit global and local label correlations to select discriminative features across multiple labels. In addition, by representing the feature matrix and label matrix in a shared latent space, the model aims to capture the underlying correlations between features and labels. The shared representation can reveal common patterns or relationships that exist across multiple labels and features. An objective function involving L2,1-norm regularization is formulated, and an alternating optimization-based iterative algorithm is designed to obtain the sparse coefficients for multi-label feature selection. The proposed method was evaluated on 14 real-world multi-label datasets using six evaluation metrics, through comprehensive experiments. The results indicate its effectiveness, surpassing that of several representative methods.

A Robust Multilabel Feature Selection Approach Based on Graph Structure Considering Fuzzy Dependency and Feature Interaction

The performance of multilabel learning depends heavily on the quality of the input features. A mass of irrelevant and redundant features may seriously affect the performance of multilabel learning, and feature selection is an effective technique to solve this problem. However, most multilabel feature selection methods mainly emphasize removing these useless features, and the exploration of feature interaction is ignored. Moreover, the widespread existence of real-world data with uncertainty, ambiguity, and noise limits the performance of feature selection. To this end, our work is dedicated to designing an efficient and robust multilabel feature selection scheme. First, the distribution character of multilabel data is analyzed to generate robust fuzzy multi-neighborhood granules. By exploring the classification information implied in the data under the granularity structure, a robust multilabel <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$k$</tex-math></inline-formula> -nearest neighbor fuzzy rough set model is constructed, and the concept of fuzzy dependency is studied. Second, a series of fuzzy multi-neighborhood uncertainty measures in <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$k$</tex-math></inline-formula> -nearest neighbor fuzzy rough approximation spaces are studied to analyze the correlations of feature pairs, including interactivity. Third, by investigating the uncertainty measure between feature and label, between features, multilabel data is modeled as a complete weighted graph. Then, these vertices are assessed iteratively to guide the assignment of feature weights. Finally, a graph structure-based robust multilabel feature selection algorithm (GRMFS) is designed. The experiments are conducted on fifteen multilabel datasets. The results verify the superior performance of GRMFS as compared with nine representative feature selection methods.

Label correlations-based multi-label feature selection with label enhancement

Feature selection, as an important pre-processing technique, can efficiently mitigate the issue of “the curse of dimensionality” by selecting discriminative features especially for multi-label learning, a discriminative feature subset can improve the classification accuracy. The existing feature selection methods for multi-label classification address the problem of label ambiguity by with logical labels. However, the significance of each label is often different in many practical applications. Using logical label to train the model may result in unsatisfactory performance due to not considering the importance of related labels with each sample. To address this issue, a novel multi-label feature selection algorithm is proposed with two-step: label enhancement and label correlations-based feature selection with label enhancement. In the step of label enhancement, a framework of label enhancement based on deep forest is utilized to transform the logical label to label distribution, which contains rich semantic information and then guides a more correct exploration of semantic correlations. In the step of feature selection, a novel multi-label feature selection algorithm is proposed based on label distribution data. Firstly, the samples are divided into multiple different clusters by using spectral clustering in the label space. Then, the label correlations can be reflected by multiple different clusters. Finally, the l2,1-norm is used to construct an objective function to achieve multi-label feature selection. Experimental results demonstrate that competitiveness of the proposed algorithm over six state-of-the-art multi-label feature selection algorithms on eighteen benchmark datasets in terms of six widely accepted evaluation metrics.

Multi-Label Feature Selection via Label Enhancement and Analytic Hierarchy Process

Multi-label feature selection can effectively resolve the challenges of high or even ultra-high dimensionality in multi-label data. However, most existing multi-label feature selection algorithms can only handle a single data type, assume all labels are equally significant and utilize heuristic search strategies, which results in inefficient and relatively unsatisfactory classification accuracy. In view of the above shortcomings, this paper proposes a new multi-label feature selection algorithm that effectively resolves existing algorithms' issues through three innovative procedures. First, a new similarity relation metric is proposed to deal with hybrid data types effectively. Second, a label enhancement algorithm is designed to enhance and transform the logical labels into a label distribution by fully considering the analytic hierarchy process (AHP) embedded with label correlation, which can automatically identify the significance of different labels. Third, a feature weighting evaluation is redesigned in the feature selection process to obtain the optimal feature subset through feature ranking directly. Under these proposed procedures, multi-label feature selection can effectively utilize the abundant semantic information of the label significance and can significantly improve the operating accuracy and efficiency simultaneously. Comparative experiments are conducted on 20 real multi-label datasets with seven state-of-the-art multi-label feature selection algorithms. Experimental results show that the proposed multi-label feature selection algorithm in this paper is about 5–10% better than the algorithms in 80% of the compared datasets.

Multi-label feature selection based on stable label relevance and label-specific features

Multi-label feature selection can efficiently handle large amounts of multi-label data. However, two pressing issues remain in sparse learning for multi-label data. First, many methods explore label relevance through the original label matrix, which may introduce redundant and useless information, and they either consider local label relevance or global label relevance, which are complementary to obtaining potential label information. Second, most existing multi-label feature selection (MLFS) algorithms do not consider the characteristics of the labels themselves, i.e., label-specific features. To solve the above problems, an MLFS method based on stable label relevance and label-specific features are proposed in this study, which combines global and local label relevance to learn the label-specific features. A stable global label relevance strategy is designed for the self-representation model to avoid the effect of noise and outliers, and a manifold regularization term is adopted to explore the local label relevance. Moreover, ℓ1-norm is applied to the feature weight matrix and self-representation matrix to select the label-specific features. Then, a solution for optimizing the objective function is designed. Compared with eight related algorithms on sixteen public datasets. The results show that the proposed method can improve the classification performance of multi-label data.

Multi-label feature selection based on correlation label enhancement

Feature selection is an effective data preprocessing technique that can effectively alleviate the curse of dimensionality in multi-label learning. The technique selects a subset of features with high discriminative power to maintain or improve the classifier's classification performance. In many practical applications, it is not possible to use multi-label data to express the relative importance of each label versus each sample. Therefore, all relevant or irrelevant labels are measured with the same importance, neglecting the potential relationship between labels and samples. To better account for the relationship between samples and labels, traditional logical labels are enhanced to label distributions. Aiming at the application of latent sample correlation in label enhancement, a new label enhancement algorithm is proposed. First, to extract potential correlation from the sample space, an objective function is proposed and solved. Second, in the process of label enhancement, the correlation is embedded into label distributions. Finally, label distributions are applied to the sparse linear regression model instead of logical labels. Experiments show that proposed method is superior to 5 advanced multi-label feature selection algorithms on 11 multi-label datasets.

Multi-Label Feature Selection Via Adaptive Label Correlation Estimation

In multi-label learning, each instance is associated with multiple labels simultaneously. Multi-label data often have noisy, irrelevant, and redundant features of high dimensionality. Multi-label feature selection has received considerable attention as an effective means for dealing with high-dimensional multi-label data. Many multi-label feature selection methods exploit label correlations to help select features. However, finding label correlations and selecting features in existing multi-label feature selection methods are often two separate processes, the existence of noises and outliers in training data makes the label correlations exploited from label space less reliable. Therefore, the learned label correlations may mislead the feature selection process and result in the selection of less informative features. This article proposes a novel algorithm named ROAD, i.e., multi-label featuRe selectiOn via ADaptive label correlation estimation. ROAD jointly performs adaptive label correlation exploration and feature selection with alternating optimization to obtain reliable estimation of label correlations, which can more effectively reveal the intrinsic manifold structure among labels and lead to the selection of a more proper feature subset. Comprehensive experiments on several frequently used datasets validate the superiority of ROAD against the state-of-the-art multi-label feature selection algorithms.

Semi-supervised multi-label feature selection algorithm for online monitoring of laser metal deposition manufacturing quality

Laser metal deposition (LMD) manufacturing, as a technique of laser metal additive manufacturing, has the advantages of short production periods, good economy and unrestricted forming shapes in the direct manufacturing of metal parts. However, the poor accuracy of single-object monitoring and the lack of labelled data make quality monitoring a huge challenge. Accordingly, this study extracts multi-features from the multi-object, including melt pool dimensions, spatters and temperature field, and proposes a semi-supervised multi-label feature selection (SSMLFS) algorithm. For SSMLFS algorithm, a multiple regression quality model is firstly proposed to produce different quality levels for unlabelled data based on microstructure. Then, to achieve feature dimension reduction and filter out highly relevant features, a quality correlation evaluation function is developed to calculate the contribution and ranking of the various features. Moreover, a local search algorithm based on quality is designed to improve the search speed of feature subsets and speed up the convergence of the SSMLFS algorithm. Experimental validation is conducted on BeAM Magic 800 machine and uses several commonly monitoring algorithms based on machine learning. Related experimental results prove that SSMLFS can significantly improve the accuracy and efficiency of quality monitoring compared to single-feature and full-feature monitoring. The proposed SSMLFS algorithm provides a semi-supervised feature selection framework to handle the low accuracy of single-feature monitoring and the complexity of multi-feature monitoring in the quality monitoring of LMD manufacturing.

Multilabel Feature Selection: A Local Causal Structure Learning Approach.

Multilabel feature selection plays an essential role in high-dimensional multilabel learning tasks. Existing multilabel feature selection approaches mainly either explore the feature-label and feature-feature correlations or the label-label and feature-feature correlations. A few of them are able to deal with all three types of correlations simultaneously. To address this problem, in this article, we formulate multilabel feature selection as a local causal structure learning problem and propose a novel algorithm, M2LC. By learning the local causal structure of each class label, M2LC considers three types of feature relationships simultaneously and is scalable to high-dimensional datasets as well. To tackle false discoveries caused by the label-label correlations, M2LC consists of two novel error-correction subroutines to correct those false discoveries. Through local causal structure learning, M2LC learns the causal mechanism behind data, and thus, it can select causally informative features and visualize common features shared by class labels and specific features owned by an individual class label using the learned causal structures. Extensive experiments have been conducted to evaluate M2LC in comparison with the state-of-the-art multilabel feature selection algorithms.

Parallel dual-channel multi-label feature selection

In the process of multi-label learning, feature selection methods are often adopted to solve the high-dimensionality problem in feature spaces. Most existing multi-label feature selection algorithms focus on exploring the correlation between features and labels and then obtain the target feature subset by importance ranking. These algorithms commonly use single-channel structure to obtain important features, which induces the excessive reliance on the ranking results and causes the loss of important features. However, the correlation between label-specific feature and label-instance is ignored. Therefore, this paper proposes Parallel Dual-channel Multi-label Feature Selection algorithm (PDMFS). We first introduce the concept of dual channel and design the algorithm model as two independent modules. The algorithm obtained different feature correlation sequences, thus avoided relevant feature loss. And then, the proposed algorithm uses the subspace model to select the feature subset with the maximum correlation and minimum redundancy for each sequence, thus obtaining feature subsets under respective correlations. Finally, the subsets are cross-merged to reduce the important feature loss caused by the serial structure processing single feature correlation. The experimental results on eight datasets and statistical hypothesis testing indicate that the proposed algorithm is effective.

Robust feature selection using label enhancement and β-precision fuzzy rough sets for multilabel fuzzy decision system

High-dimensionality is the most noticeable characteristic of multilabel data. In practice, multilabel data typically contain complex noises. Ignoring these noises in the feature selection process tends to cause an inaccurate prediction. Besides, many existing multilabel feature selection methods assume that the relation between samples and relevant labels is crisp, and the difference information hiding in the label space is lost. Given all these, in this paper, soft labels are investigated by label enhancement in multilabel feature selection. A robust feature selection algorithm is proposed using label enhancement and multilabel β-precision fuzzy rough sets. With the perspective of data distribution from samples in the same and different class, the margin-based robust fuzzy neighborhood is first defined to construct the robust fuzzy granular space. Second, label enhancement strategy is given in the robust fuzzy granular space considering multilabel data distribution. To investigate the noise-tolerant model, the underlying structure of label space after label enhancement is employed to encode the score vector of samples, which is used to search the pseudo-different class's samples of target sample. Then, the multilabel β-precision fuzzy rough set model is built to deal with multilabel data. Moreover, the fuzzy approximation degree of knowledge and the fuzzy dependency of decision classes with respect to conditional features are fused to measure the significance of features. Finally, a robust heuristic multilabel feature selection algorithm is proposed. Extensive experiments on classification performance and anti-noise ability are conducted, which verify that the proposed algorithm is effective and robust.

Multi-Label Feature Selection with Conditional Mutual Information.

Feature selection is an important way to optimize the efficiency and accuracy of classifiers. However, traditional feature selection methods cannot work with many kinds of data in the real world, such as multi-label data. To overcome this challenge, multi-label feature selection is developed. Multi-label feature selection plays an irreplaceable role in pattern recognition and data mining. This process can improve the efficiency and accuracy of multi-label classification. However, traditional multi-label feature selection based on mutual information does not fully consider the effect of redundancy among labels. The deficiency may lead to repeated computing of mutual information and leave room to enhance the accuracy of multi-label feature selection. To deal with this challenge, this paper proposed a multi-label feature selection based on conditional mutual information among labels (CRMIL). Firstly, we analyze how to reduce the redundancy among features based on existing papers. Secondly, we propose a new approach to diminish the redundancy among labels. This method takes label sets as conditions to calculate the relevance between features and labels. This approach can weaken the impact of the redundancy among labels on feature selection results. Finally, we analyze this algorithm and balance the effects of relevance and redundancy on the evaluation function. For testing CRMIL, we compare it with the other eight multi-label feature selection algorithms on ten datasets and use four evaluation criteria to examine the results. Experimental results illustrate that CRMIL performs better than other existing algorithms.

Incomplete label distribution feature selection based on neighborhood-tolerance discrimination index

Label distribution learning (LDL), focusing on the relative importance of different labels to the instance, is proposed for solving label ambiguity problem in recent years. However, for label distribution data, the annotation information may be incomplete in the real-world and complete methods cannot be directly used to process these data. In addition, with the exponential growth of data volume, data in all walks of life tend to be high-dimensional, feature selection as an efficient preprocessing technique to reduce the dimension of data. Taking the problems of the incomplete label and high-dimensional data into consideration, an incomplete label distribution feature selection method based on neighborhood-tolerance discrimination index is proposed. The neighborhood-tolerance discrimination index is utilized to explore the distinguishing ability of the feature subset, and then a novel significance metric is constructed to evaluate the importance of features, which considers the correlations between features and labels. Compared with multi-label feature selection algorithms, the proposed algorithm is designed to directly process label distribution data without discretization, which reduces information loss in the process of discretization. Compared to existing label distribution feature selection algorithms, the proposed algorithm can directly process distribution data with missing label, which avoids the interference of noisy information. Furthermore, the superiority of the proposed method over other seven state-of-the-art methods is demonstrated by conducting comprehensive experiments with eight publicly available label distribution datasets on six widely-used metrics. The experimental results show that the proposed algorithm obtains superior performance in 91.67% of cases against compared algorithms.

Real-Time Fault Diagnosis for EVs With Multilabel Feature Selection and Sliding Window Control

Real-time fault diagnosis on vehicles can effectively avoid potential accidents, which, however, is difficult and challenging to be widely deployed due to the low computational capability and limited data storage of electric vehicles (EVs). To address this issue, we propose a vehicle-mounted fault diagnosis system with low computational complexity and small data storage, for achieving real-time monitoring of vehicle status. To facilitate the accurate and optimized feature selection, we had been collecting 6.52-GB real data from three EVs in 12 months. Motivated by those data, we first propose a multilabel feature selection algorithm to obtain the feature weights, based on which the optimal number of features is then calculated through the backpropagation neural network (BPNN), thus minimizing the computational cost of real-time fault diagnosis regarding sample dimensions. To further simplify the fault diagnosis system, i.e., reducing the minimum required capacity of data storage, we design a real-time diagnosis sliding window (RDSW) where the window moves forward as new samples arrive and the stale data outside the window are discarded. In particular, we calculate the optimal size of RDSW, which controls the minimum required number of samples to guarantee the accuracy of real-time fault diagnosis. Owing to the mechanism of RDSW, vehicles no longer need to store massive data to guarantee the accuracy of real-time fault diagnosis. In addition, the results of real-time fault diagnosis at each vehicle can be shared with other vehicles in cooperative intelligent transportation systems (C-ITS). Finally, comprehensive simulation is conducted to validate the effectiveness of the proposed diagnosis system in terms of accuracy, complexity and storage capacity.