Multi-view Unsupervised Feature Selection Research Articles

Dual-level feature assessment for unsupervised multi-view feature selection with latent space learning

Recently, numerous unsupervised multi-view feature selection methods have been presented. However, these methods assess the significance of data features in each view individually or jointly evaluate the significance of data features across multiple views by concatenating data features from all data views. So, they disregard either inter-view or intra-view feature connections of data. Moreover, they ignore the complementary information from multiple insufficient views, each of which only captures partial information of multi-view data and is insufficient to characterize data distribution. To address these problems, this paper proposes a model named Dual-level Feature Assessment for Unsupervised Multi-view Feature Selection with Latent Space Learning (DFA-LSL). It first investigates the underlying complementary information from multiple views in a latent space while simultaneously learning latent representations for data. Compared with the original data representations, the latent representations contain the entire information of multi-view data. Then, it explores connections and distinguishing ability of features at two levels, namely the inter-view and intra-view levels, by jointly assessing the significance of features in the latent space and individually assessing the significance of features in each view concurrently. Following that, an effective optimization algorithm is presented. Extensive experiments demonstrate that the proposed work surpasses several state-of-the-art models.

Self-paced regularized adaptive multi-view unsupervised feature selection

Multi-view unsupervised feature selection (MUFS) is an efficient approach for dimensional reduction of heterogeneous data. However, existing MUFS approaches mostly assign the samples the same weight, thus the diversity of samples is not utilized efficiently. Additionally, due to the presence of various regularizations, the resulting MUFS problems are often non-convex, making it difficult to find the optimal solutions. To address this issue, a novel MUFS method named Self-paced Regularized Adaptive Multi-view Unsupervised Feature Selection (SPAMUFS) is proposed. Specifically, the proposed approach firstly trains the MUFS model with simple samples, and gradually learns complex samples by using self-paced regularizer. l2,p-norm (0<p≤1) is employed to measure the learning error and as the sparse regularization to accommodate various sparsity requirements across different datasets. Moreover, hypergraph Laplacian matrices are constructed for each view to better preserve the local manifold structure and encode high-order relationships within the data space. They are adaptively assigned weights to learn the underlying correlated and diverse information among different views. An iterative optimization algorithm is proposed to solve SPAMUFS and the convergence and computational complexity are also analyzed. The effectiveness of SPAMUFS is substantiated by comparing with eight state-of-the-art algorithms on nine public multi-view datasets.

Joint Multi-View Unsupervised Feature Selection and Graph Learning

Joint Multi-View Unsupervised Feature Selection and Graph Learning

Multi-view unsupervised feature selection with consensus partition and diverse graph

Multi-view unsupervised feature selection has gained significant attention in effectively reducing the dimensionality of unlabeled data collected from multiple sources. Many existing methods integrate the tasks of graph learning and feature selection to select informative features. While these methods have demonstrated promising results, they usually construct a consensus graph by merging multiple graphs or consider the consistent graph of all views. However, due to the view heterogeneity, it becomes difficult to identify a shared similarity structure. On the other hand, the clustering outcome remains consistent across all views. In light of this, we propose generating multiple graphs that are as mutually exclusive as possible to enhance the complementarity between views. Additionally, our method bridges graph learning and consensus clustering to leverage the indicator consistency. We also present an effective algorithm to optimize the objective function. Finally, extensive experiments on six benchmark datasets demonstrate that our method outperforms state-of-the-art methods. The code is available at https://github.com/HdTgon/2023-INS-CDMvFS.

Consistency–exclusivity guided unsupervised multi-view feature selection

Unsupervised multi-view feature selection (UMFS) is an effective dimension reduction for multi-view data. It aims to obtain the important feature subset from multi-view data, which can significantly minimize the impacts of noises, outliers and redundancy. Although previous UMFS methods achieve remarkable achievements, they fail to fully take into account the consistency or the exclusivity hidden in multi-view data. To address this, this article presents a consistency–exclusivity guided unsupervised multi-view feature selection (CE-UMFS) method. Specifically, we design a multi-view matrix factorization model to simultaneously explore the consistency and exclusivity in multi-view data. Meanwhile, we employ the Hilbert–Schmidt independence criterion (HSIC) to preserve the exclusivity of different views. Furthermore, we impose a nuclear norm on the consistent representation matrix to explore the consistency across views. At last, promising experimental results demonstrate the superiority of the proposed method compared with some state-of-the-art methods.

Multi-view unsupervised complementary feature selection with multi-order similarity learning

Graph construction, an open and challenging problem, is of great significance for multi-view unsupervised feature selection. So far, many graph construction methods, such as distance-based (the local structure) and self-reconstruction-based (the global structure), have been devised to serve the feature selection task. Although these methods have achieved some improvements, they fail to utilize high-order neighbor information, let alone exploit the neighbor information of different orders, to improve the feature selection task. In this paper, we propose a new insight to construct graphs that can accommodate multi-order neighbor information for selecting the relevant features. Besides, we observe that existing methods adopts the general information fusion strategy in multi-view learning, e.g. fusing graphs, without taking into account the unique characteristics of the feature selection task. Therefore, the proposed method seeks to project multi-view data onto a shared latent representation, which explores the complementarity tailored to the feature selection task at the feature level. A simple yet effective algorithm is designed to solve the optimization problem of the objective function. Extensive clustering experiments demonstrate the superiority of our method over state-of-the-art ones.

Multi-level correlation learning for multi-view unsupervised feature selection

Recently, unsupervised multi-view feature selection has received a lot of interest. However, current methods facilitate feature selection by preserving the local consistency of multi-view data, which is defined based on similarities of data within each view, but ignore the global topological consistency in data, which is defined based on the cross-view topological similarities of data between views and is essential for revealing the distribution of multi-view data. In light of this, this paper proposes a novel multi-view unsupervised feature selection method with multi-level correlation learning, termed Multi-Level Correlation Learning for Multi-View Unsupervised Feature Selection (MLCL). It simultaneously derives the global topological correlation structure from the cross-view topological similarities of data and the local geometric correlation structure from the local similarities of data within each view, to take advantage of both global and local consistencies of multi-view data. An effective optimization algorithm is then developed to resolve the optimization problem for the proposed model. Extensive experiments on eight publicly available datasets show that the proposed MLCL outperforms several state-of-the-art unsupervised multi-view feature selection models.

Structure learning with consensus label information for multi-view unsupervised feature selection

Structure learning based feature selection has attracted increasing attention for selecting these features which can preserve the learned structures. However, existing methods fail to effectively explore the heterogeneous and homogeneous information from multiple views, which leads to the suboptimal results. To solve this problem, we propose Structure Learning with Consensus Label Information for Multi-View Feature Selection (SCMvFS). Noting the heterogeneity of views, the graph of each view should be a perturbation of the intrinsic graph yet the clustering structure are shared across views. In light of this, we generate a unique clustering indicator through the spectral analysis of multiple Laplacian graphs for the structure learning based feature selection. Therefore, SCMvFS considers both the graph heterogeneity and indicator consistency to effectively explore the heterogeneous and homogeneous information for facilitating the feature selection task. Further, we carefully design an efficient algorithm to solve the resulting optimization problem. Extensive experimental results demonstrate that the proposed method outperforms the state-of-the-art methods on seven benchmark datasets with respect to two indicators. In particular, SCMvFS achieves an ACC of 61.87 (55.94) on the Outdoor Scene (Yale) dataset, which is an up to 43% (15%) performance improvement compared with the latest structure learning based method TLR. The code and datasets are available at https://github.com/HdTgon/2023-ESWA-SCMvFS.

C2IMUFS: Complementary and Consensus Learning-Based Incomplete Multi-View Unsupervised Feature Selection

Multi-view unsupervised feature selection (MUFS) has been demonstrated as an effective technique to reduce the dimensionality of multi-view unlabeled data. The existing methods assume that all of views are complete. However, multi-view data are usually incomplete, i.e., a part of instances are presented on some views but not all views. Besides, learning the complete similarity graph, as an important promising technology in existing MUFS methods, cannot achieve due to the missing views. In this paper, we propose a complementary and consensus learning-based incomplete multi-view unsupervised feature selection method (C <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^{2}$</tex-math></inline-formula> IMUFS) to address the aforementioned issues. Concretely, C <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^{2}$</tex-math></inline-formula> IMUFS integrates feature selection into an extended weighted non-negative matrix factorization model equipped with adaptive learning of view-weights and a sparse <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\ell _{2,p}$</tex-math></inline-formula> -norm, which can offer better adaptability and flexibility. By the sparse linear combinations of multiple similarity matrices derived from different views, a complementary learning-guided similarity matrix reconstruction model is presented to obtain the complete similarity graph in each view. Furthermore, C <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^{2}$</tex-math></inline-formula> IMUFS learns a consensus clustering indicator matrix across different views and embeds it into a spectral graph term to preserve the local geometric structure. Comprehensive experimental results on real-world datasets demonstrate the effectiveness of C <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^{2}$</tex-math></inline-formula> IMUFS compared with state-of-the-art methods.

Adaptive Collaborative Soft Label Learning for Unsupervised Multi-View Feature Selection

Unsupervised multi-view feature selection aims to select informative features with multi-view features and unsupervised learning. It is a challenging problem due to the absence of explicit semantic supervision. Recently, graph theory and hard pseudo-label learning have been adopted to solve multi-view feature selection problems under the unsupervised learning paradigm. However, graph-based methods are difficult to support large-scale real scenarios due to the high computational complexity of graph construction. Moreover, existing methods based on hard pseudo-label learning generally result in significant information loss. In this article, we propose an Adaptive Collaborative Soft Label Learning (ACSLL) model for unsupervised multi-view feature selection. In this model, collaborative soft label learning and multi-view feature selection are integrated into a unified framework. Specifically, we learn the pseudo soft labels from each view feature by a simple and efficient method and fuse them with an adaptive weighting strategy into a joint soft label matrix. This matrix is further used for guiding the feature selection process to identify valuable features. An effective optimization strategy guaranteed with proven convergence is derived to iteratively solve this problem. Experiments demonstrate the superiority of the proposed method in both feature selection accuracy and efficiency.

Structural regularization based discriminative multi-view unsupervised feature selection

Multi-view unsupervised feature selection (MUFS) has recently aroused considerable attention, which can select the compact representative feature subset from original multi-view data. Despite the promising preliminary performance, most previous MUFS methods fail to explore the discriminative ability of multi-view data. In addition, they usually utilize spectral analysis to maintain the geometrical structure, which will inevitably increase the difficulty of parameter selection. To address these issues, we present a novel MUFS method, named structural regularization based discriminative multi-view unsupervised feature selection (SDFS). Specifically, we calculate the similarity matrix of sample space from different views and automatically weight each view-specific graph to learn a consensus similarity graph, in which these two types of graphs can promote each other. Further, we treat the learned latent representation as the cluster indicator, and employ a graph regularization without introducing additional parameters to maintain the geometrical structure of data. Besides, a simple yet efficient iterative updating algorithm with theoretical convergence property is developed. Extensive experiments on several benchmark datasets verify that the designed model is superior to several state-of-the-art MUFS models.

Multi-view unsupervised feature selection with tensor robust principal component analysis and consensus graph learning

Recently, multi-view unsupervised feature selection has attracted much attention due to its efficiency and better interpretability in processing high-dimensional multi-view datasets. Most existing methods rely on the constructed similarity matrices to obtain reliable pseudo labels to guide the feature selection. However, the considerable adverse noise in the raw data inevitably impedes the exploration of true underlying similarity structures. Besides, the inter-view correlations are often ignored during the common representation learning, which limits the effective fusion of the essential information from multiple views. To solve these issues, we design a novel robust multi-view unsupervised feature selection framework. Specifically, our method seeks a set of noise-free view-specific similarity matrices by leveraging tensor robust principal component analysis, where the high-order connections among different views are well exploited through the constructed low-rank tensor. Meanwhile, a high-quality consensus similarity matrix is adaptively learned from the view-specific representations within the same unified framework to capture the shared local structures. To enhance the discriminative ability of the feature selection matrix, we further impose a rank constraint on the consensus similarity matrix to obtain reliable pseudo cluster indicators. We present an efficient optimization algorithm ground on the alternating direction method of multipliers to solve the proposed model. Experimental results on six multi-view datasets confirm the superiority of our method.

Consensus cluster structure guided multi-view unsupervised feature selection

As the volume of high-dimensional multi-view data continues to grow, there has been a significant development in multi-view unsupervised feature selection methods, particularly those that perform graph learning and feature selection simultaneously. These methods typically begin by constructing a consensus graph, which is then utilized to ensure that the projected samples maintain the local structure of data. However, these methods require data from multiple views to preserve the same manifold structure, which goes against the reality that similarities may vary across different views. On the other hand, despite inconsistencies between heterogeneous features, multiple views share a unique cluster structure. Inspired by this, we propose consensus cluster structure guided multi-view unsupervised feature selection (CCSFS). Specifically, we generate multiple cluster structures and fuse them into a consensus structure to guide feature selection. The proposed method unifies subspace learning, cluster analysis, consensus learning and sparse feature selection into one optimization framework. By leveraging the inherent interactions between these four subtasks, CCSFS can finally select informative and discriminative features. An efficient algorithm is carefully designed to solve the optimization problem of the objective function. We conduct extensive clustering experiments on seven multi-view datasets to demonstrate that the proposed method outperforms some of the latest competitors.

Incremental unsupervised feature selection for dynamic incomplete multi-view data

Multi-view unsupervised feature selection has been proven to be efficient in reducing the dimensionality of multi-view unlabeled data with high dimensions. The previous methods assume that all views are complete. However, in real applications, the multi-view data are often incomplete, i.e., some views of instances are missing, which will result in the failure of these methods. Besides, while the data arrive in form of streams, these existing methods will suffer the issues of high storage cost and expensive computation time. To address these issues, we propose an Incremental Incomplete Multi-view Unsupervised Feature Selection method (I2MUFS) on incomplete multi-view streaming data. By jointly considering the consistent and complementary information across different views, I2MUFS embeds the unsupervised feature selection into an extended weighted non-negative matrix factorization model, which can learn a consensus clustering indicator matrix and fuse different latent feature matrices with adaptive view weights. Furthermore, we introduce the incremental learning mechanisms to develop an alternative iterative algorithm, where the feature selection matrix is incrementally updated, rather than recomputing on the entire updated data from scratch. A series of experiments are conducted to verify the effectiveness of the proposed method by comparing with several state-of-the-art methods. The experimental results demonstrate the effectiveness and efficiency of the proposed method in terms of the clustering metrics and the computational cost.

UFODMV: Unsupervised Feature Selection for Online Dynamic Multi-Views

In most machine learning (ML) applications, data that arrive from heterogeneous views (i.e., multiple heterogeneous sources of data) are more likely to provide complementary information than does a single view. Hence, these are known as multi-view data. In real-world applications, such as web clustering, data arrive from diverse groups (i.e., sets of features) and therefore have heterogeneous properties. Each feature group is referred to as a particular view. Although multi-view learning provides complementary information for machine learning algorithms, it results in high dimensionality. However, to reduce the dimensionality, feature selection is an efficient method that can be used to select only the representative features of the views so to reduce the dimensionality. In this paper, an unsupervised feature selection for online dynamic multi-views (UFODMV) is developed, which is a novel and efficient mechanism for the dynamic selection of features from multi-views in an unsupervised stream. UFODMV consists of a clustering-based feature selection mechanism enabling the dynamic selection of representative features and a merging process whereby both features and views are received incrementally in a streamed fashion over time. The experimental evaluation demonstrates that the UFODMV model has the best classification accuracy with values of 20% and 50% compared with well-known single-view and multi-view unsupervised feature selection methods, namely OMVFS, USSSF, and SPEC.

Autoweighted Multiview Feature Selection With Graph Optimization.

In this article, we focus on the unsupervised multiview feature selection, which tries to handle high-dimensional data in the field of multiview learning. Although some graph-based methods have achieved satisfactory performance, they ignore the underlying data structure across different views. Besides, their predefined Laplacian graphs are sensitive to the noises in the original data space and fail to obtain the optimal neighbor assignment. To address the above problems, we propose a novel unsupervised multiview feature selection model based on graph learning, and the contributions are three-fold: 1) during the feature selection procedure, the consensus similarity graph shared by different views is learned. Therefore, the proposed model can reveal the data relationship from the feature subset; 2) a reasonable rank constraint is added to optimize the similarity matrix to obtain more accurate information; and 3) an autoweighted framework is presented to assign view weights adaptively, and an effective alternative iterative algorithm is proposed to optimize the problem. Experiments on various datasets demonstrate the superiority of the proposed method compared to the state-of-the-art methods.

Multi-level regularization-based unsupervised multi-view feature selection with adaptive graph learning

Multi-level regularization-based unsupervised multi-view feature selection with adaptive graph learning

Hierarchical unsupervised multi-view feature selection

Multi-view feature selection is an important research direction in the field of multi-view learning. To remove the negative effect of irrelevant and redundant information and select the important features from the multi-view data, in this paper, we propose a novel hierarchical unsupervised multi-view feature selection method, which integrates matrix decomposition and hierarchical regularization as a joint model. Specifically, to exploit the consistency information of multiple views for projection matrix, we project the multiple views into a latent basis space based on a matrix decomposition model. The hierarchical regularization containing the [Formula: see text]-norm, dependence and Frobenius-norm is imposed on projection matrix to, respectively, exploit the row-level, dependency-level and view-level feature selection. We also develop an efficient optimization algorithm to optimize our method. Extensive experimental results on six popular multi-view datasets show the effectiveness and superiority of our method by comparing with the state-of-the-art methods.

Multi-view unsupervised feature selection with tensor low-rank minimization

To describe objects more comprehensively and accurately, multi-view learning has attracted considerable attention. Recently, graph embedding based multi-view feature selection methods have been proposed and shown efficient in many real applications. The existing methods generally construct one common graph matrix to exploit the local structure of multi-view data via the linear weight fusion or learning one common graph matrix across all views. However, since all views share the identical graph structure, this emphasizes the consistency too much, resulting in restricting the diversity among different views. In this paper, a tensor low-rank constrained graph embedding method is proposed for multi-view unsupervised feature selection. To embody the view-specific information of each view, our model constructs the graph structure for each corresponding view, respectively. To capture the consistency across views, a tensor low-rank regularization constraint is imposed on the tensor data formed by these graph matrices. An efficient optimization algorithm with theoretical convergence guarantee is designed to solve the proposed method. Extensive experimental results validate that the proposed method outperforms some state-of-the-art methods. The code of our model can be found athttps://www.researchgate.net/publication/353902948_demoTLR.

Unsupervised Cross-View Feature Selection on incomplete data

Unsupervised multi-view feature selection (UMV-FS) deals with the dimension reduction problem wherein instances are unlabeled and represented by heterogeneous features. Existing mainstream UMV-FS methods incorporate instance-wise view interactions based on graphs to guide feature selection, in which within-view selection decisions are independently learned to piece up a global feature subset. However, this strategy induces a globally sub-optimal feature selection decision in the sense that unexpected redundant features across views proliferate. Furthermore, existing studies are performed in view-complete frameworks, which hardly satisfies real-world applications. To address these issues, we propose a novel cross-view feature selection (CVFS) framework in an unsupervised manner, which can process large-scale/streaming data. This is the first attempt to approach incomplete multi-view feature selection by devising and fusing two-wise view interactions. Specifically, we incorporate the traditional instance-wise view interactions based on graphs to find discriminative features in each view and model a novel kind of feature-wise view interactions to enforce selection diversity and reduce feature redundancy. These techniques can yield a globally optimal feature subset across all views. Comprehensive experiments validate the effectiveness and efficiency of the proposed CVFS.