Subspace Clustering Research Articles

Enhanced honey badger algorithm for multi-view subspace clustering based on consensus representation

Enhanced honey badger algorithm for multi-view subspace clustering based on consensus representation

Nonlinear subspace clustering by functional link neural networks

Nonlinear subspace clustering based on a feed-forward neural network has been demonstrated to provide better clustering accuracy than some advanced subspace clustering algorithms. While this approach demonstrates impressive outcomes, it involves a balance between effectiveness and computational cost. In this study, we employ a functional link neural network to transform data samples into a nonlinear domain. Subsequently, we acquire a self-representation matrix through a learning mechanism that builds upon the mapped samples. As the functional link neural network is a single-layer neural network, our proposed method achieves high computational efficiency while ensuring desirable clustering performance. By incorporating the local similarity regularization to enhance the grouping effect, our proposed method further improves the quality of the clustering results. We name our method as Functional Link Neural Network Subspace Clustering (FLNNSC). Furthermore, we propose a convex combination subspace clustering scheme that combines a linear subspace clustering method with the functional link neural network subspace clustering approach. This combination method is named as Convex Combination Subspace Clustering (CCSC), which allows for a dynamic balance between linear and nonlinear representations. Extensive experiments conducted on four widely used datasets, including Extended Yale B, USPS, COIL20, and ORL, demonstrate that both FLNNSC and CCSC outperform several state-of-art subspace clustering methods in terms of clustering accuracy. Our affinity graph experiments reveal that FLNNSC exhibits clear block diagonal structures. We provide recommendations for hyperparameters in FLNNSC by performing a parameter sensitivity analysis, and empirically verify the convergence of FLNNSC. Additionally, we show that FLNNSC has a lower computational cost compared to two high-performing methods.

Graph regularized least squares regression for automated breast ultrasound imaging

Breast cancer, recognized as one of the most pervasive malignancies affecting females, manifests a perpetual escalation in its worldwide morbidity. Timely screening offers patients interventions that are not only more expeditious but also more efficacious. ABUS, as an automated breast ultrasound imaging technology, has the advantages of high resolution, reproducible results, and strong objectivity. However, manual interpretation of ABUS images can be highly tedious and laborious when handling huge amounts of data from images, may leading to the risk of missed diagnosis. In practical clinical diagnosis, the incorporation of Computer-Aided Diagnosis (CAD) systems emerges as a more favorable option. In consideration of the holistic factors encompassing model construction efficiency, training cost and overall performance, we provide a unique subspace clustering approach known as Graph regularized Least Squares Regression (GLSR) in this paper. Firstly, we encode the data's intrinsic geometry into the framework of least squares regression to respect local and global correlation structures at the same time. Secondly, we construct a nearest neighbor graph to model the smooth structure of the representation, which may lead to a more accurate representation. Finally, a nonnegative constraint is introduced into our framework so that the representation obtained by GLSR has clearer physical meaning. We provide an optimization approach for solving such a framework based on iterative updates of four blocks. Extensive experimental results indicate the effectiveness of the proposed method compared with state-of-the-art approaches on real-world breast cancer images and other biological data.

Tensorized and Compressed Multi-View Subspace Clustering via Structured Constraint.

Multi-view learning has raised more and more attention in recent years. However, traditional approaches only focus on the difference while ignoring the consistency among views. It may make some views, with the situation of data abnormality or noise, ineffective in the progress of view learning. Besides, the current datasets have become high-dimensional and large-scale gradually. Therefore, this paper proposes a novel multi-view compressed subspace learning method via low-rank tensor constraint, which incorporates the clustering progress and multi-view learning into a unified framework. First, for each view, we take the partial samples to build a small-size dictionary, which can reduce the effect of both redundancy information and computation cost greatly. Then, to find the consistency and difference among views, we impose a low-rank tensor constraint on these representations and further design an auto-weighted mechanism to learn the optimal representation. Last, due to the non-square of the learned representation, the bipartite graph has been introduced, and under the structured constraint, the clustering results can be obtained directly from this graph without any post-processing. Extensive experiments on synthetic and real-world benchmark datasets demonstrate the efficacy and efficiency of our method, especially for the views with noise or outliers.

Hierarchical bipartite graph based multi-view subspace clustering

Multi-view subspace clustering has attracted much attention because of its effectiveness in unsupervised learning. The high time consumption and hyper-parameters are the main obstacles to its development. In this paper, we present a novel method to effectively solve these two defects. First, we employ the bisecting k-means method to generate anchors and construct the hierarchical bipartite graph, which greatly reduce the time consumption. Moreover, we adopt an auto-weighted allocation strategy to learn appropriate weight factors for each view, which can avoid the influence of hyper-parameters. Furthermore, by imposing low rank constraints on the fusion graph, our proposed method can directly obtained the cluster indicators without any post-processing operations. Finally, numerous experiments verify the superiority of proposed method.

Specific and coupled double consistency multi-view subspace clustering with low-rank tensor learning

Multi-view clustering (MVC) has gained widespread attention due to its ability to utilize different features from different views. However, the existing MVC methods fail to fully exploit the consistency across multiple views, leading to information loss. Additionally, the performance of the algorithms is not satisfactory due to the inherent noise in the data. To address the above-mentioned issues, this paper proposes the specific and coupled double consistency multi-view subspace clustering with low-rank tensor learning (SCDCMV) method. Specifically, firstly, we simultaneously incorporate the consistency and specificity of multiple views into self-expressive learning. However, the information within the consistency matrix has not been fully utilized, and there still exists some noise. Then, the obtained consistency matrix is once again integrated into self-expressive learning to obtain a new consistency matrix. Thirdly, we combine the two consistency matrices into a tensor and constrain it using tensor nuclear norm (TNN). Then, under the constraint of TNN, the two consistency matrices mutually reinforce each other, which helps fully utilize the consistency information and reduce the impact of noise, ultimately leading to better clustering results. Ultimately, these three steps constitute a framework that is tackled utilizing the augmented Lagrange multiplier method. The performance of SCDCMV has improved by 55.94 %. Experimental results on different datasets indicate that the SCDCMV algorithm outperforms state-of-the-art algorithms. In other words, these experimental results validate the importance of effectively utilizing consistent information from multiple views while reducing the impact of noise. The code is publicly available at https://github.com/TongWuahpu/SCDCMV.

A new nonconvex multi-view subspace clustering via learning a clean low-rank representation tensor

Abstract Recently, low-rank tensor representation has achieved impressive results for multi-view subspace clustering (MSC). The typical MSC methods utilize the tensor nuclear norm as a convex surrogate of the tensor multi-rank to obtain a low-rank representation, which exhibits limited robustness when dealing with noisy and complex data scenarios. In this paper, we introduce an innovative clean low-rank tensor representation approach that combines the idea of tensor robust principal component analysis with a new nonconvex tensor multi-rank approximation regularization. This integration enhances the robustness of the low-rank representation, resulting in improved performance. Furthermore, to better capture the local geometric features, we employ a high-order manifold regularization term. To effectively address our new model, we develop an iterative algorithm that can be proved to converge to the desired Karush-Kuhn-Tucker critical point. The numerical experiments on widely used datasets serve to demonstrate the efficacy and effectiveness of our new method.

Partition-level fusion induced multi-view Subspace Clustering with Tensorial Geman Rank

Partition-level fusion induced multi-view Subspace Clustering with Tensorial Geman Rank

Locality-constrained double-layer structure scaled simplex multi-view subspace clustering

Locality-constrained double-layer structure scaled simplex multi-view subspace clustering

Diffusion process with structural changes for subspace clustering

Spectral clustering-based methods have gained significant popularity in subspace clustering due to their ability to capture the underlying data structure effectively. Standard spectral clustering focuses on only pairwise relationships between data points, neglecting interactions among high-order neighboring points. Integrating the diffusion process can address this limitation by leveraging a Markov random walk. However, ensuring that diffusion methods capture sufficient information while maintaining stability against noise remains challenging. In this paper, we propose the Diffusion Process with Structural Changes (DPSC) method, a novel affinity learning framework that enhances the robustness of the diffusion process. Our approach broadens the scope of nearest neighbors and leverages the dropout idea to generate random transition matrices. Furthermore, inspired by the structural changes model, we use two transition matrices to optimize the iteration rule. The resulting affinity matrix undergoes self-supervised learning and is subsequently integrated back into the diffusion process for refinement. Notably, the convergence of the proposed DPSC is theoretically proven. Extensive experiments on benchmark datasets demonstrate that the proposed method outperforms existing subspace clustering methods. The code of our proposed DPSC is available at https://github.com/zhudafa/DPSC.

An Integer Programming Approach to Subspace Clustering with Missing Data

In the subspace clustering with missing data (SCMD) problem, we are given a collection of n partially observed d-dimensional vectors. The data points are assumed to be concentrated near a union of low-dimensional subspaces. The goal of SCMD is to cluster the vectors according to their subspace membership and recover the underlying basis, which can then be used to infer their missing entries. State-of-the-art algorithms for SCMD can fail on instances with a high proportion of missing data, with full-rank data, or if the underlying subspaces are similar to each other. We propose a novel integer programming approach for SCMD. The approach is based on dynamically determining a set of candidate subspaces and optimally assigning points to selected subspaces. The problem structure is identical to the classical facility-location problem, with subspaces playing the role of facilities and data points playing that of customers. We propose a column-generation approach for identifying candidate subspaces combined with a Benders decomposition approach for solving the linear programming relaxation of the formulation. An empirical study demonstrates that the proposed approach can achieve better clustering accuracy than state-of-the-art methods when the data are high rank, the percentage of missing data is high, or the subspaces are similar. Funding: Support for this research was provided by American Family Insurance through a research partnership with the University of Wisconsin–Madison’s Data Science Institute. Supplemental Material: The online appendix is available at https://doi.org/10.1287/ijoo.2023.0020 .

Robust and stochastic sparse subspace clustering

Sparse subspace clustering (SSC) has been widely employed in machine learning and pattern recognition, but it still faces scalability challenges when dealing with large-scale datasets. Recently, stochastic SSC (SSSC) has emerged as an effective solution by leveraging the dropout technique. However, SSSC cannot robustly handle noise, especially non-Gaussian noise, leading to unsatisfactory clustering performance. To address the above issues, we propose a novel robust and stochastic method called stochastic sparse subspace clustering with the Huber function (S3CH). The key idea is to introduce the Huber surrogate to measure the loss of the stochastic self-expression framework, thus S3CH inherits the advantage of the stochastic framework for large-scale problems while mitigating sensitivity to non-Gaussian noise. In algorithms, an efficient proximal alternating minimization (PAM)-based optimization scheme is developed. In theory, the convergence of the generated sequence is rigorously proved. Extensive numerical experiments on synthetic and six real datasets validate the advantages of the proposed method in clustering accuracy, noise robustness, parameter sensitivity, post-hoc analysis, and model stability.

Unsupervised Cross-View Subspace Clustering via Adaptive Contrastive Learning

Unsupervised Cross-View Subspace Clustering via Adaptive Contrastive Learning

Multi-geometric block diagonal representation subspace clustering with low-rank kernel

Multi-geometric block diagonal representation subspace clustering with low-rank kernel

Towards attributed graph clustering using enhanced graph and reconstructed graph structure

Attributed graph clustering, leveraging both structural and attribute information, is crucial in various real-world applications. However, current approaches face challenges stemming from the sparsity of graphs and sensitivity to noise in Graph Convolutional Networks (GCNs). Moreover, GCN-based methods are often designed based on the assumption of homophilic graph and ignore heterophilic graph. To address these, we propose a graph clustering method that consists of four phases: graph enhance, graph reconstruction, graph refine, and dual-guidance supervisor module. An enhanced graph module is defined by an auxiliary graph to consider distant relationships in the topology structure to alleviate the limitations of sparse graphs. The graph reconstruction phase includes the creation and integration of homophily and heterophily graphs to achieve graph-agnostic. In graph refine, the auxiliary graph is iteratively improved to enhance the generalization of the representation. In this phase, a subspace clustering module is applied to convert attribute-based embeddings into relationship-based representations. Finally, the extracted graphs are fed to a dual-guidance supervisor module to obtain the final clustering result. Experimental validation on several benchmark datasets demonstrates the efficiency of our model. Meanwhile, the findings offer significant advancements in attributed graph clustering, promising improved applicability in various domains.

Multi-dimensional weighted deep subspace clustering with feature classification

Deep subspace clustering (DSC) methods based on deep autoencoder (DAE) and self-expression layers have achieved impressive performance. However, traditional DSC method often loses useful information in the feature extraction of DAE, leading to incomplete learning of the self-expression coefficient matrix. Besides, existing deep subspace clustering methods always extract feature information from a single dimension, ignoring the potential information that may exist in the self-expression coefficient matrix across multiple dimensions. In this paper, we propose a novel method called multi-dimensional weighted deep subspace clustering based on feature classification (MWDSC). Specifically, we use the multivariate feature classification module (MFC) to learn the potential information of multiple local feature matrix for local pre-clustering. To further improve the clustering performance, we propose a self-flip weighted fusion strategy (SWF). The SWF strategy obtains a self-expression coefficient matrix with rich features through self-flipping weighting, thereby aiding in achieving global optimal solutions. Extensive experiments conducted on five benchmark datasets validate the superiority and effectiveness of our proposed method over other state-of-the-art approaches.

Multi-view clustering with adaptive anchor and bipartite graph learning

Multi-view subspace clustering optimizes and integrates the graph structure information of each view. At present, the subspace clustering methods based on the abstract graph have better performance and improve the clustering results. Although the existing clustering algorithms have achieved excellent results, there are still four deficiencies: 1) Expensive time overhead, with most algorithms having a high time complexity; 2) Using fixed anchor points and the separation of anchor graph learning from subsequent graph construction, resulting in inadequate use of underlying graph structure between views; 3) Multi-view data have inevitable noise and in the original high-dimensional space data fusion may cause the loss of important information; 4) Most algorithms require additional post-processing to obtain clustering labels. To solve the above problems, we innovate a novel anchor-adaptive multi-view clustering algorithm based on a bipartite graph . Specifically, anchor graph learning and subspace graph construction are adaptively combined into a unified optimization framework. The projection matrix, consensus anchor matrix, and similarity matrix optimize each other to promote the clustering effect and structure a constrained bipartite graph to describe the relationship between representative points and sample points. At the same time, connectivity constraints are used to ensure that the connected components represent clusters directly. Our method has linear time complexity about sample size. Comparison experiments with the state-of-the-art algorithms demonstrate the effectiveness of the proposed method on various benchmark datasets. Moreover, our method is robust to noisy data and suitable for large-scale datasets.

Low-Rank Robust Subspace Tensor Clustering for Metro Passenger Flow Modeling

Tensor clustering has become an important topic, specifically in spatiotemporal modeling, because of its ability to cluster spatial modes (e.g., stations or road segments) and temporal modes (e.g., time of day or day of the week). Our motivating example is from subway passenger flow modeling, where similarities between stations are commonly found. However, the challenges lie in the innate high-dimensionality of tensors and also the potential existence of anomalies. This is because the three tasks, that is, dimension reduction, clustering, and anomaly decomposition, are intercorrelated with each other, and treating them in a separate manner will render a suboptimal performance. Thus, in this work, we design a tensor-based subspace clustering and anomaly decomposition technique for simultaneous outlier-robust dimension reduction and clustering for high-dimensional tensors. To achieve this, a novel low-rank robust subspace clustering decomposition model is proposed by combining Tucker decomposition, sparse anomaly decomposition, and subspace clustering. An effective algorithm based on Block Coordinate Descent is proposed to update the parameters. Prudent experiments prove the effectiveness of the proposed framework via the simulation study, with a gain of +25% clustering accuracy over benchmark methods in a hard case. The interrelations of the three tasks are also analyzed via ablation studies, validating the interrelation assumption. Moreover, a case study in station clustering based on real passenger flow data is conducted, with quite valuable insights discovered. History: Bianca Maria Colosimo served as the senior editor for this article. Funding: Dr. Yan is partially funded by DOE [DE-EE0009354] and NSF [CMMI 2316654]. Dr. Fugee is partially funded with the RGC [GRF 16201718 and 16216119]. The authors appreciate the help from the Hong Kong MTR Co. research, marketing, and customer service teams. Data Ethics & Reproducibility Note: The code capsule is available on Code Ocean at https://codeocean.com/capsule/6536164/tree and in the e-Companion to this article (available at https://doi.org/10.1287/ijds.2022.0028 ).

Collaborative Structure-Preserved Missing Data Imputation for Single-Cell RNA-Seq Clustering.

Clustering of the single-cell RNA-seq (scRNA-seq) transcriptome profiles is able to identify cell types, which is beneficial to improve the understanding of disease progression. However, in practice, the single-cell expression data often contains a significant number of missing values as a result of technical variability. Missing data is a critical challenge in scRNA-seq clustering analysis since the unknown value does not reflect the underlying true expression level and makes it difficult to discovering cell types by applying clustering algorithms directly. Various approaches have been developed to overcome missing data issue in scRNA-seq clustering. Most of them recover missing expression values by borrowing observed data from similar cells or synthesizing data via generative adversarial networks. Such that the biologically meaningful cluster structure has not been sufficiently exploited. In this work, we introduce ColImpute, a collaborative structure-preserved missing data imputation approach for the scRNA-seq clustering. Specifically, a cluster structure-preserved imputation module and a subspace clustering module, which respectively perform missing data imputation and cell subtypes identification, are integrated into a unified optimization framework to train the two networks in a collaborative manner. Consequently, the clustering module effectively contributes cluster-structure information to guide the trainning process of the missing data imputation module. Simultaneously, the cluster structure-preserved imputation module reciprocally enhances the performance of the clustering module by generating more precise recovered samples. Promising experimental results show that the proposed method is effective for both the data imputation and the cell types identification.

Dual subspace clustering for spectral-spatial hyperspectral image clustering

Subspace clustering supposes that hyperspectral image (HSI) pixels lie in a union vector spaces of multiple sample subspaces without considering their dual space, i.e., spectral space. In this article, we propose a promising dual subspace clustering (DualSC) for improving spectral-spatial HSIs clustering by relaxing subspace clustering. To this end, DualSC simultaneously optimizes row and column subspace-representations of HSI superpixels to capture the intrinsic connection between spectral and spatial information. From the new perspective, the original subspace clustering can be treated as a special case of DualSC that has larger solution space, so tends to finding better sample representation matrix for applying spectral clustering. Besides, we provide theoretical proofs that show the proposed method relaxes the subspace space clustering with dual subspace, and can recover subspace-sparse representation of HSI samples. To the best of our knowledge, this work could be one of the first dual clustering method leveraging sample and spectral subspaces simultaneously. As a result, we conduct several clustering experiments on four canonical data sets, implying that our proposed method with strong interpretability reaches comparable performance and computing efficiency with other state-of-the-art methods.