Subspace Clustering Methods Research Articles

Diffusion process with structural changes for subspace clustering

Spectral clustering-based methods have gained significant popularity in subspace clustering due to their ability to capture the underlying data structure effectively. Standard spectral clustering focuses on only pairwise relationships between data points, neglecting interactions among high-order neighboring points. Integrating the diffusion process can address this limitation by leveraging a Markov random walk. However, ensuring that diffusion methods capture sufficient information while maintaining stability against noise remains challenging. In this paper, we propose the Diffusion Process with Structural Changes (DPSC) method, a novel affinity learning framework that enhances the robustness of the diffusion process. Our approach broadens the scope of nearest neighbors and leverages the dropout idea to generate random transition matrices. Furthermore, inspired by the structural changes model, we use two transition matrices to optimize the iteration rule. The resulting affinity matrix undergoes self-supervised learning and is subsequently integrated back into the diffusion process for refinement. Notably, the convergence of the proposed DPSC is theoretically proven. Extensive experiments on benchmark datasets demonstrate that the proposed method outperforms existing subspace clustering methods. The code of our proposed DPSC is available at https://github.com/zhudafa/DPSC.

Multi-dimensional weighted deep subspace clustering with feature classification

Deep subspace clustering (DSC) methods based on deep autoencoder (DAE) and self-expression layers have achieved impressive performance. However, traditional DSC method often loses useful information in the feature extraction of DAE, leading to incomplete learning of the self-expression coefficient matrix. Besides, existing deep subspace clustering methods always extract feature information from a single dimension, ignoring the potential information that may exist in the self-expression coefficient matrix across multiple dimensions. In this paper, we propose a novel method called multi-dimensional weighted deep subspace clustering based on feature classification (MWDSC). Specifically, we use the multivariate feature classification module (MFC) to learn the potential information of multiple local feature matrix for local pre-clustering. To further improve the clustering performance, we propose a self-flip weighted fusion strategy (SWF). The SWF strategy obtains a self-expression coefficient matrix with rich features through self-flipping weighting, thereby aiding in achieving global optimal solutions. Extensive experiments conducted on five benchmark datasets validate the superiority and effectiveness of our proposed method over other state-of-the-art approaches.

Multi-view clustering with adaptive anchor and bipartite graph learning

Multi-view subspace clustering optimizes and integrates the graph structure information of each view. At present, the subspace clustering methods based on the abstract graph have better performance and improve the clustering results. Although the existing clustering algorithms have achieved excellent results, there are still four deficiencies: 1) Expensive time overhead, with most algorithms having a high time complexity; 2) Using fixed anchor points and the separation of anchor graph learning from subsequent graph construction, resulting in inadequate use of underlying graph structure between views; 3) Multi-view data have inevitable noise and in the original high-dimensional space data fusion may cause the loss of important information; 4) Most algorithms require additional post-processing to obtain clustering labels. To solve the above problems, we innovate a novel anchor-adaptive multi-view clustering algorithm based on a bipartite graph . Specifically, anchor graph learning and subspace graph construction are adaptively combined into a unified optimization framework. The projection matrix, consensus anchor matrix, and similarity matrix optimize each other to promote the clustering effect and structure a constrained bipartite graph to describe the relationship between representative points and sample points. At the same time, connectivity constraints are used to ensure that the connected components represent clusters directly. Our method has linear time complexity about sample size. Comparison experiments with the state-of-the-art algorithms demonstrate the effectiveness of the proposed method on various benchmark datasets. Moreover, our method is robust to noisy data and suitable for large-scale datasets.

A coarse and fine-grained deep multi view subspace clustering method for unsupervised fault diagnosis of rolling bearings

Abstract To address the issue of insufficient characterization of fault features in inherent vibration data that affects the performance of unsupervised learning-based fault diagnosis, a coarse and fine-grained deep multi view subspace clustering method (CFG-DMVSC) for unsupervised fault diagnosis of rolling bearings is proposed. The proposed method designs a convolutional autoencoder network based on the Gramian angular field transformation for multi-signal analysis domains. A multi-view coarse-grained self-expressive method based on information entropy is designed to handle differences in information across different views. Furthermore, a fine-grained common and independent information separation loss function based on mutual information is proposed to ensure compactness among multiple views. Both the Case Western Reserve University rolling bearing dataset and privately built bearing fault test bench data demonstrate that, compared to existing methods, the proposed method can perform coarse and fine-grained division in multi-view subspaces, achieving better clustering diagnosis performance on the extracted common information among views.

Multiview ensemble clustering of hypergraph p-Laplacian regularization with weighting and denoising

Multiview clustering has gained attention for its ability to incorporate complementary information from multiple sources of data, leading to better clustering results. However, these methods don't sufficiently mine high-dimensional information of non-linear subspace or ignore important high-level information between basic partitions (BPs), which are obtained via single-view clustering that could help to overcome differences between heterogeneous feature spaces and defined in Eq. (4). Additionally, existing multiview ensemble clustering methods neglect the noise arising from the generation phase. In light of this, we first propose a novel multiview subspace clustering method with hypergraph p-Laplacian regularization and denoising. Specifically, to utilize the high-dimensional information, a hypergraph p-Laplacian regularized term is added to the model along with low-rank subspace learning to capture the different hierarchical structures in the data. A denoising algorithm based on KMeans and K-nearest neighbor is used to minimize the noise of similarity matrix. Then utilizing this approach as a backbone, a multiview ensemble clustering of hypergraph p-Laplacian regularization with weighting and denoising method is proposed. A hybrid strategy and a novel global weighting ensemble strategy is further proposed to extract high-level information across all the BPs. By integrating hypergraph p-Laplacian operator, low-rank subspace learning, denoising, and weighting ensemble strategy in a unified framework, this ensemble approach adequately learns latent structures and complementary information. Experimental results on multiple datasets demonstrate the efficacy of this approach.

Coupled double self-expressive subspace clustering with low-rank tensor learning

In recent years, subspace clustering (SC) methods have been widely used in machine learning and computer vision. However, the self-expressive matrix obtained by the existing SC methods is not clean block diagram and still contains some of typical noise in data, which leads to a decrease in clustering performance. In addition, most SC methods do not make full use of the structural information of self-expressive matrices. To solve the above problems, we propose coupled double self-expressive subspace clustering with low-rank tensor learning (CDSESLRTL) method. Specifically, first, we use the self-expressive characteristics of the data to obtain the self-expressive matrix of the data. However, the structural information of the self-expressive matrix is not fully utilized, and the block diagonal structure is often not sufficiently clear. Then, we obtain a new self-expressive matrix by viewing the obtained self-expressive matrix as a feature representation of the data and use self-expressive characteristics again. Third, we stack the two self-expressive matrices into a tensor, which is constrained by the tensor nuclear norm (TNN). Then, the two self-expressive matrices can be learned in a mutually reinforcing manner, which not only gives these matrices a clearer block diagonal structure but also sufficiently utilizes their structural information. Finally, the combination of these three steps forms a framework, which is addressed using the augmented Lagrange multiplier (ALM) method. The experimental results on various datasets demonstrate that the CDSESLRTL algorithm outperforms the state-of-the-art algorithms.

Dual contrastive learning for multi-view clustering

Self-representation subspace clustering, based on the self-representation property of linear subspaces, has demonstrated superior performance in subspace clustering. However, many existing methods learn self-representation in the embedding space, which fails to accurately capture the clustering structure of the data. Additionally, in the data embedding process, only the shared features among clusters are removed, and the discriminative features that distinguish different clusters cannot be effectively selected. To address these limitations, we propose a contrastive learning-based self-representation subspace clustering method named Multi-view Contrastive Subspace Clustering (MCSC). This method integrates feature selection and self-representation learning into a unified framework. It learns a shared self-representation coefficient in both the embedding space and the original space using contrastive learning, which allows for a more accurate description of the linear relationship between samples and a better depiction of the inter-sample structure simultaneously. Moreover, we employ the ℓ1,2-norm for feature selection, which eliminates shared redundancies while preserving important features specific to each cluster, making the samples more distinguishable. Extensive experiments on six datasets verify the validity of our proposed model.

An adaptive kernel dictionary-based low-rank representation method for subspace clustering

Low-rank representation (LRR) is a classic subspace clustering (SC) algorithm, and many LRR-based methods have been proposed. Generally, LRR-based methods use denoized data as dictionaries for data reconstruction purpose. However, the dictionaries used in LRR-based algorithms are fixed, leading to poor clustering performance. In addition, most of these methods assume that the input data are linearly correlated. However, in practice, data are mostly nonlinearly correlated. To address these problems, we propose a novel adaptive kernel dictionary-based LRR (AKDLRR) method for SC. Specifically, to explore nonlinear information, the given data are mapped to the Hilbert space via the kernel technique. The dictionary in AKDLRR is not fixed; it adaptively learns from the data in the kernel space, making AKDLRR robust to noise and yielding good clustering performance. To solve the AKDLRR model, an efficient procedure including an alternative optimization strategy is proposed. In addition, a theoretical analysis of the convergence performance of AKDLRR is presented, which reveals that AKDLRR can converge in at most three iterations under certain conditions. The experimental results show that AKDLRR can achieve the best clustering performance and has excellent speed in comparison with other algorithms.

Adaptive weighted multi-view subspace clustering method for recognizing urban functions from multi-source social sensing data

ABSTRACT Multi-source social sensing data provide new opportunities to identify urban functions from the perspective of human activity. The information embedded in multi-source data typically needs to be fused to obtain a comprehensive view of urban functions. Although multi-view clustering has been successfully used to fuse multi-source social sensing data, the adaptive determination of fusion weights for high-dimensional and noisy multi-source social sensing data remains challenging. Therefore, this study proposes an adaptive weighted multi-view subspace clustering (AWMSC) method. First, we use two neural networks to map multi-source data into a common latent representation and multiple specific latent representations, which serve as the query vector and input vectors of the attention mechanism, respectively. Then, the weight of each type of data is calculated based on the attention mechanism. Finally, the specific latent representations of the multi-source data are weighted and fused into a shared subspace representation, which is used as the input of the spectral clustering algorithm to obtain clustering results. AWMSC is applied to identify urban functional zones in Beijing using bus transactions, taxi trajectories, and points of interest datasets. The results show that AWMSC outperforms the typical single-view, weighted-average, and representative multi-view methods. AWMSC can obtain a comprehensive understanding of urban functional zones which may help government departments make more accurate strategic decisions.

Markov-Embedded Affinity Learning with Connectivity Constraints for Subspace Clustering

Subspace clustering algorithms have demonstrated remarkable success across diverse fields, including object segmentation, gene clustering, and recommendation systems. However, they often face challenges, such as omitting cluster information and the neglect of higher-order neighbor relationships within the data. To address these issues, a novel subspace clustering method named Markov-Embedded Affinity Learning with Connectivity Constraints for Subspace Clustering is proposed. This method seamlessly embeds Markov transition probability information into the self-expression, leveraging a fine-grained neighbor matrix to uncover latent data structures. This matrix preserves crucial high-order local information and complementary details, ensuring a comprehensive understanding of the data. To effectively handle complex nonlinear relationships, the method learns the underlying manifold structure from a cross-order local neighbor graph. Additionally, connectivity constraints are applied to the affinity matrix, enhancing the group structure and further improving the clustering performance. Extensive experiments demonstrate the superiority of this novel method over baseline approaches, validating its effectiveness and practical utility.

Nonconvex submodule clustering via joint sliced sparse gradient and cluster-aware approach

Most existing subspace clustering methods preprocess image data by converting them into vectors, which lacks exploration of the spatial structure of high-dimensional data. Therefore, we proposes a nonconvex submodule clustering model (NSSGCA) via joint sliced sparse gradient and cluster-aware approach. NSSGCA arranges each 2D image as lateral slices of a 3rd-order tensor, and utilizes the t-product under the model of the union of free submodules to represent 3rd-order tensor samples, thereby exploring the latent spatial structure of samples. To more accurately approximate tensor rank, a nonconvex Schatten p-norm constraint is imposed on the rotated representation tensor. Under the submodule framework, a consistent gradient matrix is derived based on the δ-nearest neighbor adjacency graph to construct sliced sparse gradient (SSG) regularization, which is more conducive to clustering tasks. NSSGCA learns representation tensor with clearer block-like structure based on ℓq norm and cluster-aware attention mechanism. The convergence of the constructed sequence to the stationary Karush–Kuhn–Tucker (KKT) point is proven. Experimental results on real-world image datasets confirm the effectiveness of NSSGCA.

Clean and robust multi-level subspace representations learning for deep multi-view subspace clustering

Deep multi-view subspace clustering has achieved increasing attention owing to its encouraging ability to address the nonlinear multi-view data. Despite significant progress, most existing deep multi-view subspace clustering methods still suffer from the following drawbacks. First, they usually focus on utilizing the last-level feature from the last encoder layer of the auto-encoder related to each view, yet often overlook exploiting the earlier-level features from the earlier encoder layers as well as integrating the multi-level features from multiple encoder layers. Second, when learning the subspace representation from the self-expressive layer, they rarely consider the cleanliness and robustness of the learned subspace representation. To overcome these drawbacks, this paper proposes a novel clean and robust multi-level subspace representations learning for deep multi-view subspace clustering (CRMSRL). Specifically, the auto-encoder is adopted to nonlinearly map each view into the multi-level features, and then multiple self-expressive layers are added between the encoder layers and their corresponding decoder layers to provide multiple information flow paths through the auto-encoder and learn the multi-level subspace representations corresponding to the multi-level features. Consequently, the intricate hierarchical information embedded in the multi-level features is passed to the corresponding multi-level subspace representations. Moreover, with the idea of robust principal component analysis (RPCA), the learned multi-level subspace representations can be further purified to achieve the cleaner and more robust ones. In addition, to excavate the structural consistency among different views, the common layer is designed to interact with different view-specific self-expressive layers and achieve the high-quality common subspace representation, which can be applied to the spectral clustering algorithm to obtain the final clustering results. Experimental results show that CRMSRL outperforms twelve state-of-the-art baseline methods on six benchmark datasets. In particular, CRMSRL achieves 4.5% improvements on the MSRCV1 dataset in terms of accuracy, compared with the best baseline method.

Efficient subspace clustering and feature extraction via [formula omitted]-norm and [formula omitted]-norm minimization

The nuclear norm-based Latent Low-Rank Representation (LatLRR) has gained much attention due to its success in subspace clustering and feature extraction. However, it suffers from high computational costs due to the calculation of singular value decomposition for large matrices. To this end, we develop an efficient subspace clustering and feature extraction method (ESCFE) which substitutes the nuclear norm with the ℓ2,1-norm and ℓ1,2-norm respectively. Theoretically proof shows both the ℓ2,1-norm and ℓ1,2-norm can serve as the convex surrogates of the nuclear norm while can derive closed-form solutions. Furthermore, the ℓ2,1-norm (or ℓ1,2-norm) regularization promotes column (or row) structure sparsity due to the discriminative nature inherited from the ℓ1-norm. Thus the proposed ESCFE is robust to outliers in data and can extract features with joint sparsity. Extensive experiments on multiple benchmark datasets demonstrate the superiority of our method in both efficiency and effectiveness.

Multi-stage exponential model based on subspace clustering distribution for bearing remaining useful life prediction

As one of the key components in rotating machinery, the rolling element bearing has been widely used in actual production, such as wind turbines, vehicles and machine tools. A bearing’s remaining useful life (RUL) is an important indicator for its performance assessment, which is related to maintenance and production safety. To overcome the insensitivity of the conventional health indicator (HI) on bearing degradation assessment, this study proposes a subspace clustering method based on manifold learning to evaluate the evolution of health status, which describes the degenerate distribution via a two-class model and realizes the identification of the degradation of each stage. Motivated by the inconsistent degradation process in the application of actual bearing, this study proposes a multi-stage degradation identification criterion in an adaptive way, which can effectively identify different degradation rates of bearing. Based on the different degradation states, a multi-stage degradation exponential model is established to accurately predict the RUL. The effectiveness of the proposed method is validated through open datasets. The experimental results prove that the proposed method can effectively identify different degradation rates and accurately give the boundary time of the multi-stage degradation. The RUL prediction accuracy is significantly improved compared with traditional HI.

Comprehensive multi-view self-representations for clustering

Subspace learning-based methods have shown excellent performance for multi-view clustering, yet have the following problems: (1) most existing methods obtain the subspace representation from the original space, which might contain noises and cannot guarantee a clean enough subspace representation; (2) existing methods mainly focus on the consistency of the subspace representation, while the unique information of each view is not sufficiently exploited. To solve these two problems, we propose a novel multi-view subspace clustering method called comprehensive multi-view self-representations (CMSR). Specifically, we learn the original coefficient matrix of each view through the self-representation, which can reduce the noise of the original space to some extent. Then, we learn the subspace representation of the original coefficient matrix and decompose it into a consistent coefficient matrix and multiple diverse coefficient matrices, which can exploit the consistent and complementary information of multi-view data. Further, we impose the Schatten p-norm constraint on the consistent coefficient matrix to capture robust consistent information. Finally, the comprehensive results on eight real datasets demonstrate the versatility and effectiveness of the proposed method.

Multi-view Subspace Clustering Based on Weighted Tensor Schatten-P Norm

In the era of advancing technology and continuous societal evolution, an abundance of voluminous data from various domains has become prevalent in the public domain. Managing this wealth of data necessitates a series of preprocessing procedures, encompassing tasks such as data storage, computation, and analysis, with the aim of unveiling intrinsic features and patterns contained within. Among these procedures, clustering stands out as a crucial step, with the primary goal of unveiling the hidden structures within the data and organizing data samples into meaningful clusters based on specific criteria. Early clustering efforts were based on a singleview. Real world data, however, often contains interrelated information, leading to the emergence of multi-view clustering. Multiview subspace clustering (T-SVD) has been widely used in recent years. However, the equal treatment of each singular value in this approach lacks relevance when the data is noisy or contaminated. In order to improve the robustness and clustering performance, this paper presents a T-SVD weighted tensor norm. In order to improve the robustness, a Schatten-P norm was proposed. This paper explores multi-view subspace clustering is based on the Schatten-P norm weighted tensor. Experiments are carried out on 6 benchmark data sets, and the results are compared with 4 existing ones. Experiments show that the proposed approach is better than the most advanced multi-view subspace clustering methods.

Unified Framework for Faster Clustering via Joint Schatten p-Norm Factorization With Optimal Mean.

To enhance the effectiveness and efficiency of subspace clustering in visual tasks, this work introduces a novel approach that automatically eliminates the optimal mean, which is embedded in the subspace clustering framework of low-rank representation (LRR) methods, along with the computationally factored formulation of Schatten p -norm. By addressing the issues related to meaningful computations involved in some LRR methods and overcoming biased estimation of the low-rank solver, we propose faster nonconvex subspace clustering methods through joint Schatten p -norm factorization with optimal mean (JS p NFOM), forming a unified framework for enhancing performance while reducing time consumption. The proposed approach employs tractable and scalable factor techniques, which effectively address the disadvantages of higher computational complexity, particularly when dealing with large-scale coefficient matrices. The resulting nonconvex minimization problems are reformulated and further iteratively optimized by multivariate weighting algorithms, eliminating the need for singular value decomposition (SVD) computations in the developed iteration procedures. Moreover, each subproblem can be guaranteed to obtain the closed-form solver, respectively. The theoretical analyses of convergence properties and computational complexity further support the applicability of the proposed methods in real-world scenarios. Finally, comprehensive experimental results demonstrate the effectiveness and efficiency of the proposed nonconvex clustering approaches compared to existing state-of-the-art methods on several publicly available databases. The demonstrated improvements highlight the practical significance of our work in subspace clustering tasks for visual data analysis. The source code for the proposed algorithms is publicly accessible at https://github.com/ZhangHengMin/TRANSUFFC.

A game model for semi-supervised subspace clustering with dynamic affinity and label learning

With the aid of partial supervised information, semi-supervised subspace clustering methods aim to obtain affinity matrices directly derived from raw data, and then those affinity matrices are utilized to get assignment matrices. These affinity matrices are susceptible to disturbances such as noise and outliers, which could significantly impact their quality. To mitigate this, it becomes essential to dynamically update the affinity matrix using the clustering result, reducing the high dependency on raw data. This paper presents a Game Model for Semi-supervised Subspace Clustering (GMSSC). The first submodule of GMSSC utilizes a Graph Convolutional Network to learn a mapping from raw data to the assignment matrix with the assistance of the affinity matrix. The second submodule constructs a semi-supervised self-expressive model to learn a discriminative affinity matrix. By integrating these submodules into a game model, GMSSC achieves a Nash Equilibrium during adversarial training, resulting in stable and robust affinity and assignment matrices. Additionally, we propose an iterative algorithm based on data-driven and model-driven approaches to solve the model. Experimental results on four open real-world datasets demonstrate that the proposed method not only achieves elegant results but also outperforms state-of-the-art semi-supervised subspace clustering algorithms.

A symmetric low-rank subspace clustering method for cooperative spectrum sensing in complex environments

A new cooperative spectrum sensing (CSS) method based on low-rank symmetric subspace clustering is proposed to improve spectrum sensing performance in complex environments. To align with the evolution of cellular networks and leverage spatial observations, a CSS model with multiple primary users and antennas is considered. To address the asymmetric weighting issue of data pairs, a low-rank representation with a symmetric constraint is proposed that reduces the signal dimensions. Finally, features are extracted from the symmetric coefficient matrix and clustered using the K-means algorithm. Through simulation, the effectiveness of the novel algorithm is numerically validated through comparative experiments.

Multi-task subspace clustering

In recent years, subspace clustering and multi-task clustering have received extensive attention due to their wide practical applications. Traditional subspace clustering is limited to exploring the underlying subspaces within a single task, while traditional multi-task clustering usually ignores the data distributed in some low-dimensional subspaces. However, multiple subspace clustering tasks, in reality, may be related. To resolve this issue, we introduce a multi-task subspace clustering method, which unifies transfer learning, local structure learning, and self-representation learning into a single paradigm. In particular, we suggest learning a projection matrix to perform dimensionality reduction, and exploring correlations among multiple tasks simultaneously. In addition, we design an efficient algorithm to solve the objective. The experimental findings demonstrate that the approach suggested in this study exhibits superior performance compared to the existing state-of-the-art methods in the domains of subspace clustering and multi-task clustering. Our MATLAB codes are available at https://github.com/HeKind/MTSC.