Sparse Subspace Clustering Research Articles

Robust and stochastic sparse subspace clustering

Sparse subspace clustering (SSC) has been widely employed in machine learning and pattern recognition, but it still faces scalability challenges when dealing with large-scale datasets. Recently, stochastic SSC (SSSC) has emerged as an effective solution by leveraging the dropout technique. However, SSSC cannot robustly handle noise, especially non-Gaussian noise, leading to unsatisfactory clustering performance. To address the above issues, we propose a novel robust and stochastic method called stochastic sparse subspace clustering with the Huber function (S3CH). The key idea is to introduce the Huber surrogate to measure the loss of the stochastic self-expression framework, thus S3CH inherits the advantage of the stochastic framework for large-scale problems while mitigating sensitivity to non-Gaussian noise. In algorithms, an efficient proximal alternating minimization (PAM)-based optimization scheme is developed. In theory, the convergence of the generated sequence is rigorously proved. Extensive numerical experiments on synthetic and six real datasets validate the advantages of the proposed method in clustering accuracy, noise robustness, parameter sensitivity, post-hoc analysis, and model stability.

Sparse subspace clustering in diverse multiplex network model

The paper considers the DIverse MultiPLEx (DIMPLE) network model, where all layers of the network have the same collection of nodes and are equipped with the Stochastic Block Models. In addition, all layers can be partitioned into groups with the same community structures, although the layers in the same group may have different matrices of block connection probabilities. To the best of our knowledge, the DIMPLE model, introduced in Pensky and Wang (2021), presents the most broad SBM-equipped binary multilayer network model on the same set of nodes and, thus, generalizes a multitude of papers that study more restrictive settings. Under the DIMPLE model, the main task is to identify the groups of layers with the same community structures since the matrices of block connection probabilities act as nuisance parameters under the DIMPLE paradigm. The main contribution of the paper is achieving the strongly consistent between-layer clustering by using Sparse Subspace Clustering (SSC), the well-developed technique in computer vision. In addition, SSC allows to handle much larger networks than spectral clustering, and is perfectly suitable for application of parallel computing. Moreover, our paper is the first one to obtain precision guarantees for SSC when it is applied to binary data.

Semi-supervised sparse subspace clustering with manifold regularization

Semi-supervised sparse subspace clustering with manifold regularization

Joint learning framework of superpixel generation and fuzzy sparse subspace clustering for color image segmentation

Sparse subspace clustering (SSC) is an important image segmentation method that constructs a self-representation coefficient matrix to represent the relationships between pixels, and then uses spectral clustering to achieve clustering. Compared with other unsupervised segmentation algorithms, SSC has good segmentation performance. However, SSC has a high computational complexity when processing large-sized image. To improve computational efficiency, many researchers have proposed superpixel-based SSC algorithms, which process superpixels instead of pixels and improve efficiency through preprocessing. Due to the sensitivity of superpixel generation to noise, superpixel-based SSC algorithms still have poor robustness. Additionally, preprocessing increases the complexity of the algorithm. To address these issues, this paper proposes a robust superpixel-based fuzzy sparse subspace clustering algorithm. This algorithm combines fuzzy sparse subspace clustering with superpixel generation, and it constructs a unified optimization learning framework through fuzzy C-multiple-means clustering to improve segmentation performance and reduce complexity. Additionally, this paper introduces additional features of superpixel in sparse subspace clustering to further enhance the segmentation performance of the algorithm. Experiment results indicate that the proposed algorithm not only outperforms existing state-of-the-art robust segmentation algorithms independent of superpixels, but also is superior to the latest superpixel-based segmentation algorithms.

Sparse subspace clustering incorporated deep convolutional transform learning for hyperspectral band selection

Sparse subspace clustering incorporated deep convolutional transform learning for hyperspectral band selection

SSCNet: learning-based subspace clustering

Sparse subspace clustering (SSC), a seminal clustering method, has demonstrated remarkable performance by effectively solving the data sparsity problem. However, it is not without its limitations. Key among these is the difficulty of incremental learning with the original SSC, accompanied by a computationally demanding recalculation process that constrains its scalability to large datasets. Moreover, the conventional SSC framework considers dictionary construction, affinity matrix learning and clustering as separate stages, potentially leading to suboptimal dictionaries and affinity matrices for clustering. To address these challenges, we present a novel clustering approach, called SSCNet, which leverages differentiable programming. Specifically, we redefine and generalize the optimization procedure of the linearized alternating direction method of multipliers (ADMM), framing it as a multi-block deep neural network, where each block corresponds to a linearized ADMM iteration step. This reformulation is used to address the SSC problem. We then use a shallow spectral embedding network as an unambiguous and differentiable module to approximate the eigenvalue decomposition. Finally, we incorporate a self-supervised structure to mitigate the non-differentiability inherent in k-means to achieve the final clustering results. In essence, we assign unique objectives to different modules and jointly optimize all module parameters using stochastic gradient descent. Due to the high efficiency of the optimization process, SSCNet can be easily applied to large-scale datasets. Experimental evaluations on several benchmarks confirm that our method outperforms traditional state-of-the-art approaches.

Adaptive fuzzy weighted C-mean image segmentation algorithm combining a new distance metric and prior entropy

The fuzzy C-mean clustering algorithm (FCM) is effective in image segmentation. However, its sensitivity to initialization settings and the limitation of the Euclidean distance metric in measuring similarity lead to unsatisfactory image segmentation. To overcome these shortcomings, we propose an adaptive fuzzy weighted C-mean image segmentation algorithm that combines a new distance metric and prior entropy. This algorithm significantly contributes in three aspects. First, we employ an improved sparse subspace clustering (SSC) algorithm based on the superpixel image for initialization settings. This step aims to obtain the initial number of clusters and the membership matrix, ensuring the algorithm’s convergence to the optimal solution. Second, to better capture differences between image regions, we introduce a novel distance metric that combines Euclidean and non-Euclidean distances. Simultaneously, the fuzzy weight is introduced to mitigate redundant feature interference, making the clustering result more reasonable. Finally, leveraging prior entropy – a maximum entropy under conditional constraints – we refine the algorithm’s adaptability to unknown features, thereby improving clustering accuracy. Comparative experiments with several state-of-the-art algorithms validate the effectiveness and robustness of our proposed algorithm.

Grouping Subspace Segmentation

This work studies the subspace segmentation problem. Given a set of data points which are drawn from a union of multiple subspaces. The goal of subspace segmentation is to cluster the data into the underlying subspaces from which the data are drawn from. The most recent works use spectral clustering based on the affinity matrix obtained by Sparse Subspace Clustering (SSC), Low-Rank Representation (LRR) and Least Squares Regression (LSR). They write each data point as a linear combination of all the data points, and use the representation coefficients to form the affinity matrix. LSR shows that it is more effective due to the grouping effect for data with high correlation. In this work, we propose the Grouping Subspace Segmentation (GSS) method by enhancing the grouping effect of correlated data points. The affinity graph is constructed to encode the local structure of data. Experiments on the real data sets show that the proposed method GSS is more effective than the state-of-the-art methods.

Fault diagnosis method for bogie bearings of high-speed trains based on weighted-sparse subspace clustering

The bogie bearings are crucial components of the high-speed train transmission system, and any failure of them can severely impact the normal functioning of the vehicle. Nowadays, sparse subspace clustering (SSC) is an effective technology for diagnosing mechanical system faults. However, SSC is easily affected by noise. To address this issue, we introduce the weighted-SSC algorithm, which incorporates weighting coefficients to enhance the connections between similar sample points. Our approach involves extracting the fault characteristic parameters of the vibration signal through wavelet packet transform and singular value decomposition. Subsequently, these parameters are employed to construct corresponding weighted-sparse subspaces. We also examined the selection of a hyperparameter in the weighted coefficients. The resultant sparse representation vectors are then harnessed for the purpose of transductive semi-supervised learning clustering and diagnosing the specific type of fault. To validate the effectiveness of our proposed method, we designed and built a bearing experimental platform. We compare our method with existing clustering algorithms, including K-means, SSC, dimension reduction graph-based SSC, and ant colony optimization-K-means clustering algorithms. The results demonstrate that our proposed weighted-SSC method achieves higher accuracy in fault diagnosis than the other clustering algorithms.

Robust subspace clustering via multi-affinity matrices fusion

Subspace clustering (SC) methods have attracted widespread attention from interested researchers, and a variety of SC methods have been proposed recently. In general, SC methods obtain an affinity matrix, on which spectral clustering is used. Obviously, affinity matrices obtained by different algorithms are not the same, and their clustering effects are also different. A question then arises of whether the affinity matrices obtained by different algorithms can be merged to improve the clustering performance. The answer may be yes I n this paper, we design a new SC method, i.e., robust subspace clustering via multi-affinity matrices fusion (RSC/MAMF). Specifically, several classical SC algorithms, i.e., sparse subspace clustering (SSC), low-rank representation (LRR) and least squares regression (LSR), are first chosen to derive their own affinity matrices. To make full use of the information of different affinity matrices, and to mine the subspace structure thoroughly, we further fuse the derived affinity matrices, i.e., splice them into a 3-order tensor, and impose a weighted tensor nuclear norm (WTNN) to it, which not only mines and fuses the information of the different affinity matrices but also removes their noise. Additionally, to further explore the consistency of the different affinity matrices, spectral embedding is also unified into the final objective function. We propose an optimization algorithm to address the optimization problem of RSC/MAMF, which utilizes the Augmented Lagrange Multiplier (ALM) method. Experiments show that the multi-affinity matrices fusion idea is feasible, and RSC/MAMF outperforms the state-of-the-art SC methods.

ARGLRR: A Sparse Low-Rank Representation Single-Cell RNA-Sequencing Data Clustering Method Combined with a New Graph Regularization.

The development of single-cell transcriptome sequencing technologies has opened new ways to study biological phenomena at the cellular level. A key application of such technologies involves the employment of single-cell RNA sequencing (scRNA-seq) data to identify distinct cell types through clustering, which in turn provides evidence for revealing heterogeneity. Despite the promise of this approach, the inherent characteristics of scRNA-seq data, such as higher noise levels and lower coverage, pose major challenges to existing clustering methods and compromise their accuracy. In this study, we propose a method called Adjusted Random walk Graph regularization Sparse Low-Rank Representation (ARGLRR), a practical sparse subspace clustering method, to identify cell types. The fundamental low-rank representation (LRR) model is concerned with the global structure of data. To address the limited ability of the LRR method to capture local structure, we introduced adjusted random walk graph regularization in its framework. ARGLRR allows for the capture of both local and global structures in scRNA-seq data. Additionally, the imposition of similarity constraints into the LRR framework further improves the ability of the proposed model to estimate cell-to-cell similarity and capture global structural relationships between cells. ARGLRR surpasses other advanced comparison approaches on nine known scRNA-seq data sets judging by the results. In the normalized mutual information and Adjusted Rand Index metrics on the scRNA-seq data sets clustering experiments, ARGLRR outperforms the best-performing comparative method by 6.99% and 5.85%, respectively. In addition, we visualize the result using Uniform Manifold Approximation and Projection. Visualization results show that the usage of ARGLRR enhances the separation of different cell types within the similarity matrix.

Design of feature selection algorithm for high-dimensional network data based on supervised discriminant projection.

High dimension and complexity of network high-dimensional data lead to poor feature selection effect network high-dimensional data. To effectively solve this problem, feature selection algorithms for high-dimensional network data based on supervised discriminant projection (SDP) have been designed. The sparse representation problem of high-dimensional network data is transformed into an Lp norm optimization problem, and the sparse subspace clustering method is used to cluster high-dimensional network data. Dimensionless processing is carried out for the clustering processing results. Based on the linear projection matrix and the best transformation matrix, the dimensionless processing results are reduced by combining the SDP. The sparse constraint method is used to achieve feature selection of high-dimensional data in the network, and the relevant feature selection results are obtained. The experimental findings demonstrate that the suggested algorithm can effectively cluster seven different types of data and converges when the number of iterations approaches 24. The F1 value, recall, and precision are all kept at high levels. High-dimensional network data feature selection accuracy on average is 96.9%, and feature selection time on average is 65.1 milliseconds. The selection effect for network high-dimensional data features is good.

Adaptive multi-granularity sparse subspace clustering

Sparse subspace clustering (SSC) focuses on revealing data distribution from algebraic perspectives and has been widely applied to high-dimensional data. The key to SSC is to learn the sparsest representation and derive an adjacency graph. Theoretically, the adjacency matrix with proper block diagonal structure leads to a desired clustering result. Various generalizations have been made through imposing Laplacian regularization or locally linear embedding to describe the manifold structure based on the nearest neighborhoods of samples. However, a single set of nearest neighborhoods cannot effectively characterize local information. From the perspective of granular computing, the notion of scored nearest neighborhoods is introduced to develop multi-granularity neighborhoods of samples. The multi-granularity representation of samples is integrated with SSC to collaboratively learn the sparse representation, and an adaptive multi-granularity sparse subspace clustering model (AMGSSC) is proposed. The learned adjacency matrix has a consistent block diagonal structure at all granularity levels. Furthermore, the locally linear relationship between samples is embedded in AMGSSC, and an enhanced AMGLSSC is developed to eliminate the over-sparsity of the learned adjacency graph. Experimental results show the superior performance of both models on several clustering criteria compared with state-of-the-art subspace clustering methods.

Learning Transferable Sparse Representations for Cross-Corpus Facial Expression Recognition

An assumption widely used in traditional facial expression recognition algorithms is that the training and testing are conducted on the same dataset. However, this assumption does not hold in practice, in which the training data and testing data are often from different datasets. In this scenario, directly deploying these algorithms would lead to severe information loss and performance degradation due to the domain shift. To address this challenging problem, in this paper, we propose a novel transferable sparse subspace representation method (TSSR) for cross-corpus facial expression recognition. Specifically, in order to reduce the cross-corpus mismatch, inspired by sparse subspace clustering, we advocate reconstructing the source and target samples using the source data points based on L1-norm sparse representation. Each data point in source and target corpora can be ideally represented as a combination of a few other source points from its own subspace. Moreover, we take into account the local geometrical information within the cross-corpus data by adopting a graph Laplacian regularizer, which can efficiently preserve the local manifold structure and better transfer knowledge between two corpora. Finally, experimental results on several facial expression datasets demonstrate the superiority of the proposed method over some state-of-the-art methods.

An Accurate Subspace Clustering Model for Unsupervised Noise-laden Hyperspectral Image Segmentation

The hyperspectral image acquisition in realistic situations is accompanied by a lot of noise, which dramatically interferes with the discriminative power of the clustering model. In this paper, a superpixel segmentation 3D regularized subspace clustering model (SS3D-SSC) is provided, which uses a superpixel segmentation method to obtain the most representative pixel in each tiny region and use it as a representative to replace other pixels in the same area, and then use sparse subspace clustering to perform the subsequent image segmentation, the proposed method in this paper is used in Indian Pines.The reliability of the algorithm is confirmed on the Indian Pines, Salinas and Pavia University datasets. Compared with the most advanced SSC-based algorithms, even using graph convolution subspace clustering, SS3D-SSC model has better classification performance and anti-noise ability.

Attention reweighted sparse subspace clustering

Subspace clustering has attracted much attention in many applications of computer vision and pattern recognition. Spectral clustering based methods, such as sparse subspace clustering (SSC) and low-rank representation (LRR), have become popular due to their theoretical guarantees and impressive performance. However, many state-of-the-art subspace clustering methods specify the mean square error (MSE) criterion as the loss function, which is sensitive to outliers and complex noises in reality. These methods have poor performance when the data are corrupted by complex noise. In this paper, we propose a robust sparse subspace clustering method, termed Attention Reweighted SSC (ARSSC), by paying less attention to the corrupted entries (adaptively assigning small weights to the corrupted entries in each data point). To reduce the extra bias in estimation introduced by ℓ1 regularization, we also utilize non-convex penalties to overcome the overpenalized problem. In addition, we provide theoretical guarantees for ARSSC and theoretically show that our method gives a subspace-preserving affinity matrix under appropriate conditions. To solve the ARSSC optimization problem, we devise an optimization algorithm using an Alternating Direction Method of Multipliers (ADMM) method. Experiments on real-world databases validate the effectiveness of the proposed method.

Maximum Block Energy Guided Robust Subspace Clustering.

Subspace clustering is useful for clustering data points according to the underlying subspaces. Many methods have been presented in recent years, among which Sparse Subspace Clustering (SSC), Low-Rank Representation (LRR) and Least Squares Regression clustering (LSR) are three representative methods. These approaches achieve good results by assuming the structure of errors as a prior and removing errors in the original input space by modeling them in their objective functions. In this paper, we propose a novel method from an energy perspective to eliminate errors in the projected space rather than the input space. Since the block diagonal property can lead to correct clustering, we measure the correctness in terms of a block in the projected space with an energy function. A correct block corresponds to the subset of columns with the maximal energy. The energy of a block is defined based on the unary column, pairwise and high-order similarity of columns for each block. We relax the energy function of a block and approximate it by a constrained homogenous function. Moreover, we propose an efficient iterative algorithm to remove errors in the projected space. Both theoretical analysis and experiments show the superiority of our method over existing solutions to the clustering problem, especially when noise exists.

Fuzzy Sparse Subspace Clustering for Infrared Image Segmentation.

Infrared image segmentation is a challenging task, due to interference of complex background and appearance inhomogeneity of foreground objects. A critical defect of fuzzy clustering for infrared image segmentation is that the method treats image pixels or fragments in isolation. In this paper, we propose to adopt self-representation from sparse subspace clustering in fuzzy clustering, aiming to introduce global correlation information into fuzzy clustering. Meanwhile, to apply sparse subspace clustering for non-linear samples from an infrared image, we leverage membership from fuzzy clustering to improve conventional sparse subspace clustering. The contributions of this paper are fourfold. First, by introducing self-representation coefficients modeled in sparse subspace clustering based on high-dimensional features, fuzzy clustering is capable of utilizing global information to resist complex background as well as intensity inhomogeneity of objects, so as to improve clustering accuracy. Second, fuzzy membership is tactfully exploited in the sparse subspace clustering framework. Thereby, the bottleneck of conventional sparse subspace clustering methods, that they could be barely applied to nonlinear samples, can be surmounted. Third, as we integrate fuzzy clustering and subspace clustering in a unified framework, features from two different aspects are employed, contributing to precise clustering results. Finally, we further incorporate neighbor information into clustering, thus effectively solving the uneven intensity problem in infrared image segmentation. Experiments examine the feasibility of proposed methods on various infrared images. Segmentation results demonstrate the effectiveness and efficiency of the proposed methods, which proves the superiority compared to other fuzzy clustering methods and sparse space clustering methods.

ScDSSC: Deep Sparse Subspace Clustering for scRNA-seq Data.

Single cell RNA sequencing (scRNA-seq) enables researchers to characterize transcriptomic profiles at the single-cell resolution with increasingly high throughput. Clustering is a crucial step in single cell analysis. Clustering analysis of transcriptome profiled by scRNA-seq can reveal the heterogeneity and diversity of cells. However, single cell study still remains great challenges due to its high noise and dimension. Subspace clustering aims at discovering the intrinsic structure of data in unsupervised fashion. In this paper, we propose a deep sparse subspace clustering method scDSSC combining noise reduction and dimensionality reduction for scRNA-seq data, which simultaneously learns feature representation and clustering via explicit modelling of scRNA-seq data generation. Experiments on a variety of scRNA-seq datasets from thousands to tens of thousands of cells have shown that scDSSC can significantly improve clustering performance and facilitate the interpretability of clustering and downstream analysis. Compared to some popular scRNA-deq analysis methods, scDSSC outperformed state-of-the-art methods under various clustering performance metrics.

A novel conformal deformation based sparse subspace clustering

A novel conformal deformation based sparse subspace clustering