Subspace Clustering Research Articles

VSFCM: A Novel Viewpoint-Driven Subspace Fuzzy C-Means Algorithm

Nowadays, most fuzzy clustering algorithms are sensitive to the initialization results of clustering algorithms and have a weak ability to handle high-dimensional data. To solve these problems, we developed the viewpoint-driven subspace fuzzy c-means (VSFCM) algorithm. Firstly, we propose a new cut-off distance. Based on this, we establish the cut-off distance-induced clustering initialization (CDCI) method and use it as a new strategy for cluster center initialization and viewpoint selection. Secondly, by taking the viewpoint obtained by CDCI as the entry point of knowledge, a new fuzzy clustering strategy driven by knowledge and data is formed. Based upon these, we put forward the VSFCM algorithm combined with viewpoints, separation terms, and subspace fuzzy feature weights. Moreover, compared with the symmetric weights obtained by other subspace clustering algorithms, the weights of the VSFCM algorithm exhibit significant asymmetry. That is, they assign greater weights to features that contribute more, which is validated on the artificial dataset DATA2 in the experimental section. The experimental results compared with multiple advanced clustering algorithms on the three types of datasets validate that the proposed VSFCM algorithm has the best performance in five indicators. It is demonstrated that the initialization method CDCI is more effective, the feature weight allocation of VSFCM is more consistent with the asymmetry of experimental data, and it can achieve better convergence speed while displaying better clustering efficiency.

Anchor-based multi-view subspace clustering with graph learning

Multi-view subspace clustering (MVSC) has been drawn wide attentions in the area of pattern recognition and data mining. However, for a multi-view dataset with n samples and V views from k clusters, MVSC commonly requires O(Vn2) memory for storing the view-specific graph matrices and O(n3) time for the eigenvalue decomposition of a shared graph matrix. Hence, most of MVSC methods are difficult to handle the large-scale multi-view data problem. To address this issue, this paper proposes an Anchor-based Multi-View Subspace Clustering with Graph Learning (AMVSCGL) method. Instead of constructing a n×n graph matrix, our method generates a shared coefficient matrix with the size of n×k based on few learned view-specific anchors. Moreover, through further merging a graph learning term, this shared coefficient matrix can simultaneously capture the global and local information among multiple views and few learned view-specific anchors for clustering. Experimental results on seven large-scale multi-view data verify our AMVSCGL’s effectiveness and superiority in comparison with some state-of-the-art methods.

Adaptive multi-granularity sparse subspace clustering

Sparse subspace clustering (SSC) focuses on revealing data distribution from algebraic perspectives and has been widely applied to high-dimensional data. The key to SSC is to learn the sparsest representation and derive an adjacency graph. Theoretically, the adjacency matrix with proper block diagonal structure leads to a desired clustering result. Various generalizations have been made through imposing Laplacian regularization or locally linear embedding to describe the manifold structure based on the nearest neighborhoods of samples. However, a single set of nearest neighborhoods cannot effectively characterize local information. From the perspective of granular computing, the notion of scored nearest neighborhoods is introduced to develop multi-granularity neighborhoods of samples. The multi-granularity representation of samples is integrated with SSC to collaboratively learn the sparse representation, and an adaptive multi-granularity sparse subspace clustering model (AMGSSC) is proposed. The learned adjacency matrix has a consistent block diagonal structure at all granularity levels. Furthermore, the locally linear relationship between samples is embedded in AMGSSC, and an enhanced AMGLSSC is developed to eliminate the over-sparsity of the learned adjacency graph. Experimental results show the superior performance of both models on several clustering criteria compared with state-of-the-art subspace clustering methods.

An Improved Soft Subspace Clustering Algorithm Based on Particle Swarm Optimization for MR Image Segmentation.

Soft subspace clustering (SSC), which analyzes high-dimensional data and applies various weights to each cluster class to assess the membership degree of each cluster to the space, has shown promising results in recent years. This method of clustering assigns distinct weights to each cluster class. By introducing spatial information, enhanced SSC algorithms improve the degree to which intraclass compactness and interclass separation are achieved. However, these algorithms are sensitive to noisy data and have a tendency to fall into local optima. In addition, the segmentation accuracy is poor because of the influence of noisy data. In this study, an SSC approach that is based on particle swarm optimization is suggested with the intention of reducing the interference caused by noisy data. The particle swarm optimization method is used to locate the best possible clustering center. Second, increasing the amount of geographical membership makes it possible to utilize the spatial information to quantify the link between different clusters in a more precise manner. In conclusion, the extended noise clustering method is implemented in order to maximize the weight. Additionally, the constraint condition of the weight is changed from the equality constraint to the boundary constraint in order to reduce the impact of noise. The methodology presented in this research works to reduce the amount of sensitivity the SSC algorithm has to noisy data. It is possible to demonstrate the efficacy of this algorithm by using photos with noise already present or by introducing noise to existing photographs. The revised SSC approach based on particle swarm optimization (PSO) is demonstrated to have superior segmentation accuracy through a number of trials; as a result, this work gives a novel method for the segmentation of noisy images.

Joint feature selection and optimal bipartite graph learning for subspace clustering

Recently, there has been tremendous interest in developing graph-based subspace clustering in high-dimensional data, which does not require a priori knowledge of the number of dimensions and subspaces. The general steps of such algorithms are dictionary representation and spectral clustering. Traditional methods use the dataset itself as a dictionary when performing dictionary representation. There are some limitations that the redundant information present in the dictionary and features may make the constructed graph structure unclear and require post-processing to obtain labels. To address these problems, we propose a novel subspace clustering model that first introduces feature selection to process the input data, randomly selects some samples to construct a dictionary to remove redundant information and learns the optimal bipartite graph with K-connected components under the constraint of the (normalized) Laplacian rank. Finally, the labels are obtained directly from the graphs. The experimental results on motion segmentation and face recognition datasets demonstrate the superior effectiveness and stability of our algorithm.

Hippocampus Parcellation via Discriminative Embedded Clustering of fMRI Functional Connectivity.

Dividing a pre-defined brain region into several heterogenous subregions is crucial for understanding its functional segregation and integration. Due to the high dimensionality of brain functional features, clustering is often postponed until dimensionality reduction in traditional parcellation frameworks occurs. However, under such stepwise parcellation, it is very easy to fall into the dilemma of local optimum since dimensionality reduction could not take into account the requirement of clustering. In this study, we developed a new parcellation framework based on the discriminative embedded clustering (DEC), combining subspace learning and clustering in a common procedure with alternative minimization adopted to approach global optimum. We tested the proposed framework in functional connectivity-based parcellation of the hippocampus. The hippocampus was parcellated into three spatial coherent subregions along the anteroventral-posterodorsal axis; the three subregions exhibited distinct functional connectivity changes in taxi drivers relative to non-driver controls. Moreover, compared with traditional stepwise methods, the proposed DEC-based framework demonstrated higher parcellation consistency across different scans within individuals. The study proposed a new brain parcellation framework with joint dimensionality reduction and clustering; the findings might shed new light on the functional plasticity of hippocampal subregions related to long-term navigation experience.

From Ensemble Clustering to Subspace Clustering: Cluster Structure Encoding.

In this study, we propose a novel algorithm to encode the cluster structure by incorporating ensemble clustering (EC) into subspace clustering (SC). First, the low-rank representation (LRR) is learned from a higher order data relationship induced by ensemble K-means coding, which exploits the cluster structure in a co-association matrix of basic partitions (i.e., clustering results). Second, to provide a fast predictive coding mechanism, an encoding function parameterized by neural networks is introduced to predict the LRR derived from partitions. These two steps are jointly proceeded to seamlessly integrate partition information and original features and thus deliver better representations than the ones obtained from each single source. Moreover, an alternating optimization framework is developed to learn the LRR, train the encoding function, and fine-tune the higher order relationship. Extensive experiments on eight benchmark datasets validate the effectiveness of the proposed algorithm on several clustering tasks compared with state-of-the-art EC and SC methods.

Robust Discriminant Subspace Clustering With Adaptive Local Structure Embedding.

Unsupervised dimension reduction and clustering are frequently used as two separate steps to conduct clustering tasks in subspace. However, the two-step clustering methods may not necessarily reflect the cluster structure in the subspace. In addition, the existing subspace clustering methods do not consider the relationship between the low-dimensional representation and local structure in the input space. To address the above issues, we propose a robust discriminant subspace (RDS) clustering model with adaptive local structure embedding. Specifically, unlike the existing methods which incorporate dimension reduction and clustering via regularizer, thereby introducing extra parameters, RDS first integrates them into a unified matrix factorization (MF) model through theoretical proof. Furthermore, a similarity graph is constructed to learn the local structure. A constraint is imposed on the graph to guarantee that it has the same connected components with low-dimensional representation. In this spirit, the similarity graph serves as a tradeoff that adaptively balances the learning process between the low-dimensional space and the original space. Finally, RDS adopts the l2,1 -norm to measure the residual error, which enhances the robustness to noise. Using the property of the l2,1 -norm, RDS can be optimized efficiently without introducing more penalty terms. Experimental results on real-world benchmark datasets show that RDS can provide more interpretable clustering results and also outperform other state-of-the-art alternatives.

A Personalized Low-Rank Subspace Clustering Method Based on Locality and Similarity Constraints for scRNA-seq Data Analysis.

Single-cell RNA sequencing (scRNA-seq) technology can provide expression profile of single cells, which propels biological research into a new chapter. Clustering individual cells based on their transcriptome is a critical objective of scRNA-seq data analysis. However, the high-dimensional, sparse and noisy nature of scRNA-seq data pose a challenge to single-cell clustering. Therefore, it is urgent to develop a clustering method targeting scRNA-seq data characteristics. Due to its powerful subspace learning capability and robustness to noise, the subspace segmentation method based on low-rank representation (LRR) is broadly used in clustering researches and achieves satisfactory results. In view of this, we propose a personalized low-rank subspace clustering method, namely PLRLS, to learn more accurate subspace structures from both global and local perspectives. Specifically, we first introduce the local structure constraint to capture the local structure information of the data, while helping our method to obtain better inter-cluster separability and intra-cluster compactness. Then, in order to retain the important similarity information that is ignored by the LRR model, we utilize the fractional function to extract similarity information between cells, and introduce this information as the similarity constraint into the LRR framework. The fractional function is an efficient similarity measure designed for scRNA-seq data, which has theoretical and practical implications. In the end, based on the LRR matrix learned from PLRLS, we perform downstream analyses on real scRNA-seq datasets, including spectral clustering, visualization and marker gene identification. Comparative experiments show that the proposed method achieves superior clustering accuracy and robustness.

Toward Noise-Resistant Graph Embedding With Subspace Clustering Information.

Most existing approaches of attributed network embedding often combine topology and attribute information based on the homophily assumption. In many real-world networks, such an assumption does not hold since the nodes are usually associated with many noisy or irrelevant attributes. To tackle this issue, we propose a noise-resistant graph embedding method, called NGE, by leveraging the subspace clustering information (i.e., the formation of communities is driven by different latent features in distinct subspaces). Specifically, we first construct a tensor to represent a given attributed network and then map it into different feature subspaces to capture community structure via tensor decomposition. For structure embedding, the link-level and community-level constraints are imposed. For attribute embedding, the feature-selection constraint is used to reinforce the relationship between topology and noise-removal attributes. By learning structure and attribute embedding with subspace clustering information, NGE can benefit both community detection, link prediction, and node classification. Extensive experimental results have demonstrated the superiority of NGE over many state-of-the-art approaches.

Large-scale non-negative subspace clustering based on Nyström approximation

Large-scale subspace clustering usually drops the requirements of the full similarity matrix and Laplacian matrix but constructs the anchor affinity matrix and uses matrix approximation methods to reduce the clustering complexity. However, the existing anchor affinity matrix calculation methods only consider the global structure of data. Moreover, directly using the anchor affinity matrix to approximate the full Laplacian matrix cannot guarantee the best low-rank approximation, which affects the clustering results. To address these problems, this paper proposes a large-scale non-negative subspace clustering method based on Nyström approximation. Firstly, we modify the objective function of the anchor affinity matrix by adding the local structure term, taking into account both the local and global data characteristics for better affinity learning. Secondly, a matrix iterative update rule is derived to optimize the objective function according to the non-negative Lagrangian relaxation, and its rationality and convergence are proved. Finally, two effective Laplacian matrix decomposition methods based on Nyström approximation are designed to obtain more accurate eigenvectors to improve the clustering quality. The proposed algorithms are tested on various benchmark datasets. The experimental results show that our methods have competitive performance compared with state-of-the-art large-scale clustering algorithms.

Consensus cluster structure guided multi-view unsupervised feature selection

As the volume of high-dimensional multi-view data continues to grow, there has been a significant development in multi-view unsupervised feature selection methods, particularly those that perform graph learning and feature selection simultaneously. These methods typically begin by constructing a consensus graph, which is then utilized to ensure that the projected samples maintain the local structure of data. However, these methods require data from multiple views to preserve the same manifold structure, which goes against the reality that similarities may vary across different views. On the other hand, despite inconsistencies between heterogeneous features, multiple views share a unique cluster structure. Inspired by this, we propose consensus cluster structure guided multi-view unsupervised feature selection (CCSFS). Specifically, we generate multiple cluster structures and fuse them into a consensus structure to guide feature selection. The proposed method unifies subspace learning, cluster analysis, consensus learning and sparse feature selection into one optimization framework. By leveraging the inherent interactions between these four subtasks, CCSFS can finally select informative and discriminative features. An efficient algorithm is carefully designed to solve the optimization problem of the objective function. We conduct extensive clustering experiments on seven multi-view datasets to demonstrate that the proposed method outperforms some of the latest competitors.

An Accurate Subspace Clustering Model for Unsupervised Noise-laden Hyperspectral Image Segmentation

The hyperspectral image acquisition in realistic situations is accompanied by a lot of noise, which dramatically interferes with the discriminative power of the clustering model. In this paper, a superpixel segmentation 3D regularized subspace clustering model (SS3D-SSC) is provided, which uses a superpixel segmentation method to obtain the most representative pixel in each tiny region and use it as a representative to replace other pixels in the same area, and then use sparse subspace clustering to perform the subsequent image segmentation, the proposed method in this paper is used in Indian Pines.The reliability of the algorithm is confirmed on the Indian Pines, Salinas and Pavia University datasets. Compared with the most advanced SSC-based algorithms, even using graph convolution subspace clustering, SS3D-SSC model has better classification performance and anti-noise ability.

Fully Linear Graph Convolutional Networks for Semi-Supervised and Unsupervised Classification

This article presents FLGC, a simple yet effective fully linear graph convolutional network for semi-supervised and unsupervised learning. Instead of using gradient descent, we train FLGC based on computing a global optimal closed-form solution with a decoupled procedure, resulting in a generalized linear framework and making it easier to implement, train, and apply. We show that (1) FLGC is powerful to deal with both graph-structured data and regular data, (2) training graph convolutional models with closed-form solutions improve computational efficiency without degrading performance, and (3) FLGC acts as a natural generalization of classic linear models in the non-Euclidean domain (e.g., ridge regression and subspace clustering). Furthermore, we implement a semi-supervised FLGC and an unsupervised FLGC by introducing an initial residual strategy, enabling FLGC to aggregate long-range neighborhoods and alleviate over-smoothing. We compare our semi-supervised and unsupervised FLGCs against many state-of-the-art methods on a variety of classification and clustering benchmarks, demonstrating that the proposed FLGC models consistently outperform previous methods in terms of accuracy, robustness, and learning efficiency. The core code of our FLGC is released at https://github.com/AngryCai/FLGC .

Learning Transferable Sparse Representations for Cross-Corpus Facial Expression Recognition

An assumption widely used in traditional facial expression recognition algorithms is that the training and testing are conducted on the same dataset. However, this assumption does not hold in practice, in which the training data and testing data are often from different datasets. In this scenario, directly deploying these algorithms would lead to severe information loss and performance degradation due to the domain shift. To address this challenging problem, in this paper, we propose a novel transferable sparse subspace representation method (TSSR) for cross-corpus facial expression recognition. Specifically, in order to reduce the cross-corpus mismatch, inspired by sparse subspace clustering, we advocate reconstructing the source and target samples using the source data points based on L1-norm sparse representation. Each data point in source and target corpora can be ideally represented as a combination of a few other source points from its own subspace. Moreover, we take into account the local geometrical information within the cross-corpus data by adopting a graph Laplacian regularizer, which can efficiently preserve the local manifold structure and better transfer knowledge between two corpora. Finally, experimental results on several facial expression datasets demonstrate the superiority of the proposed method over some state-of-the-art methods.

PMSSC: Parallelizable multi-subset based self-expressive model for subspace clustering

Subspace clustering methods which embrace a self-expressive model that represents each data point as a linear combination of other data points in the dataset provide powerful unsupervised learning techniques. However, when dealing with large datasets, representation of each data point by referring to all data points via a dictionary suffers from high computational complexity. To alleviate this issue, we introduce a parallelizable multi-subset based self-expressive model (PMS) which represents each data point by combining multiple subsets, with each consisting of only a small proportion of the samples. The adoption of PMS in subspace clustering (PMSSC) leads to computational advantages because the optimization problems decomposed over each subset are small, and can be solved efficiently in parallel. Furthermore, PMSSC is able to combine multiple self-expressive coefficient vectors obtained from subsets, which contributes to an improvement in self-expressiveness. Extensive experiments on synthetic and real-world datasets show the efficiency and effectiveness of our approach in comparison to other methods.

Invertible linear transforms based adaptive multi-view subspace clustering

Constructing tensor with low-rank prior is the crucial issue of tensor based multi-view subspace clustering methods, but there are still some shortcomings. First, they cannot adaptively allocate the contribution of different views, resulting in the learned tensor being suboptimal. Second, they pursue low-rank tensor for different data through Discrete Fourier Transform based tensor nuclear norm, which lacks generality. To overcome these problems, we propose an invertible linear transforms based adaptive multi-view subspace clustering method, named ILTMSC. Firstly, we mine the potential low-order representation of each view through self-representation subspace learning. Then, we capture high-order representation by integrating low-order representations with adaptive weights into a tensor and then rotated. This strategy can integrate tensor adaptively and handle the noise effectively. Finally, we approximate the low-rank tensor with a recently proposed invertible linear transforms based tensor nuclear norm. Such a new tensor nuclear norm makes our model more general because it can use different invertible linear transforms for different tensor data. Moreover, an adaptive weighted tensor singular value thresholding operator is proposed for capturing the new tensor nuclear norm. Our model could be solved by convex optimization efficiently. Extensive experiments on multi-view datasets validate the effectiveness and robustness of our method.

Summarizing User-item Matrix By Group Utility Maximization

A user-item utility matrix represents the utility (or preference) associated with each (user, item) pair, such as citation counts, rating/vote on items or locations, and clicks on items. A high utility value indicates a strong association of the pair. In this work, we consider the problem of summarizing strong association for a large user-item matrix using a small summary size. Traditional techniques fail to distinguish user groups associated with different items (such as top- l item selection) or fail to focus on high utility (such as similarity- based subspace clustering and biclustering). We formulate a new problem, called Group Utility Maximization (GUM), to summarize the entire user population through k user groups and l items for each group; the goal is to maximize the total utility of selected items over all groups collectively. We show this problem is NP-hard even for l =1. We present two algorithms. One greedily finds the next group, called Greedy algorithm, and the other iteratively refines existing k groups, called k -max algorithm. Greedy algorithm provides the \((1-\frac{1}{e})\) approximation guarantee for a nonnegative utility matrix, whereas k -max algorithm is more efficient for large datasets. We evaluate these algorithms on real-life datasets.

A self-training subspace clustering algorithm based on adaptive confidence for gene expression data.

Gene clustering is one of the important techniques to identify co-expressed gene groups from gene expression data, which provides a powerful tool for investigating functional relationships of genes in biological process. Self-training is a kind of important semi-supervised learning method and has exhibited good performance on gene clustering problem. However, the self-training process inevitably suffers from mislabeling, the accumulation of which will lead to the degradation of semi-supervised learning performance of gene expression data. To solve the problem, this paper proposes a self-training subspace clustering algorithm based on adaptive confidence for gene expression data (SSCAC), which combines the low-rank representation of gene expression data and adaptive adjustment of label confidence to better guide the partition of unlabeled data. The superiority of the proposed SSCAC algorithm is mainly reflected in the following aspects. 1) In order to improve the discriminative property of gene expression data, the low-rank representation with distance penalty is used to mine the potential subspace structure of data. 2) Considering the problem of mislabeling in self-training, a semi-supervised clustering objective function with label confidence is proposed, and a self-training subspace clustering framework is constructed on this basis. 3) In order to mitigate the negative impact of mislabeled data, an adaptive adjustment strategy based on gravitational search algorithm is proposed for label confidence. Compared with a variety of state-of-the-art unsupervised and semi-supervised learning algorithms, the SSCAC algorithm has demonstrated its superiority through extensive experiments on two benchmark gene expression datasets.

Fusion and Enhancement of Consensus Matrix for Multi-View Subspace Clustering

Multi-view subspace clustering is an effective method that has been successfully applied to many applications and has attracted the attention of scholars. Existing multi-view subspace clustering seeks to learn multiple representations from different views, then gets a consistent matrix. Until now, most of the existing efforts only consider the multi-view information and ignore the feature concatenation. It may fail to explore their high correlation. Consequently, this paper proposes a multi-view subspace clustering algorithm with a novel consensus matrix construction strategy. It learns a consensus matrix by fusing the different information from multiple views and is enhanced by the information contained in the original feature direct linkage of the data. The error matrix of the feature concatenation data is reconstructed by regularization constraints and the sparse structure of the multi-view subspace. The feature concatenation data are simultaneously used to fuse the individual views and learn the consensus matrix. Finally, the data is clustered by using spectral clustering according to the consensus matrix. We compare the proposed algorithm with its counterparts on six datasets. Experimental results verify the effectiveness of the proposed algorithm.