Semi-supervised Subspace Clustering Research Articles

A game model for semi-supervised subspace clustering with dynamic affinity and label learning

With the aid of partial supervised information, semi-supervised subspace clustering methods aim to obtain affinity matrices directly derived from raw data, and then those affinity matrices are utilized to get assignment matrices. These affinity matrices are susceptible to disturbances such as noise and outliers, which could significantly impact their quality. To mitigate this, it becomes essential to dynamically update the affinity matrix using the clustering result, reducing the high dependency on raw data. This paper presents a Game Model for Semi-supervised Subspace Clustering (GMSSC). The first submodule of GMSSC utilizes a Graph Convolutional Network to learn a mapping from raw data to the assignment matrix with the assistance of the affinity matrix. The second submodule constructs a semi-supervised self-expressive model to learn a discriminative affinity matrix. By integrating these submodules into a game model, GMSSC achieves a Nash Equilibrium during adversarial training, resulting in stable and robust affinity and assignment matrices. Additionally, we propose an iterative algorithm based on data-driven and model-driven approaches to solve the model. Experimental results on four open real-world datasets demonstrate that the proposed method not only achieves elegant results but also outperforms state-of-the-art semi-supervised subspace clustering algorithms.

An end-to-end Graph Convolutional Network for Semi-supervised Subspace Clustering via label self-expressiveness

Semi-supervised clustering provides accurate assignments by leveraging a limited amount of supervisory information. There are two main groups of conventional semi-supervised subspace clustering algorithms: Pairwise Constraints (PC)-based methods and Gaussian Fields and Harmonic Function (GFHF)-based methods. Nevertheless, PC-based methods are limited because of their reliance on weak supervisory information. GFHF-based methods, on the other hand, focus on the local relationship between the affinity matrix and the assignment matrix, which restricts their clustering efficacy. Recently, graph convolutional network (GCN) has been employed to effectively transmit the local graph topology of the raw data into the feature domain, which motivates the proposed GCN model for Semi-supervised Subspace Clustering via label self-expressiveness (GSSC). Conventional semi-supervised clustering has been enhanced by incorporating a novel concept known as label self-expressiveness, which serves as a guiding factor for the clustering process. Furthermore, GSSC considers the global geometric structure of the assignment matrix and generates a discriminable assignment matrix using an end-to-end mechanism. Additionally, a well-designed loss function is introduced to train the novel GSSC. The experimental results conducted on four benchmark datasets demonstrate that the proposed GSSC algorithm exhibits superior performance compared to existing semi-supervised clustering algorithms.

Semi-Supervised Subspace Clustering via Tensor Low-Rank Representation

In this letter, we propose a novel semi-supervised subspace clustering method, which is able to simultaneously augment the initial supervisory information and construct a discriminative affinity matrix. By representing the limited amount of supervisory information as a pairwise constraint matrix, we observe that the ideal affinity matrix for clustering shares the same low-rank structure as the ideal pairwise constraint matrix. Thus, we stack the two matrices into a 3-D tensor, where a global low-rank constraint is imposed to promote the affinity matrix construction and augment the initial pairwise constraints synchronously. Besides, we use the local geometry structure of input samples to complement the global low-rank prior to achieve better affinity matrix learning. The proposed model is formulated as a Laplacian graph regularized convex low-rank tensor representation problem, which is further solved with an alternative iterative algorithm. In addition, we propose to refine the affinity matrix with the augmented pairwise constraints. Comprehensive experimental results on eight commonly-used benchmark datasets demonstrate the superiority of our method over state-of-the-art methods. The code is publicly available at https://github.com/GuanxingLu/Subspace-Clustering.

Constrained Tensor Representation Learning for Multi-View Semi-Supervised Subspace Clustering

Multi-view subspace clustering is an effective method to partition data into their corresponding categories. Nevertheless, existing multi-view subspace clustering approaches generally operate in a purely unsupervised manner, while ignoring the valuable weakly supervised information that can be readily obtained in many practical applications. In this paper, we consider the weakly supervised form of sample pair constraints, and devote to promoting the performance of multi-view subspace clustering with the aid of such prior knowledge. To achieve this goal, inspired by the intrinsic block diagonal structure of ideal low-rank representation (LRR), we propose a novel regularization to integrate must-link, cannot-link and normalization constraints into a unified formulation. The proposed regularization can be regarded as a general description for sample pairwise constraints, and thus provides a flexible framework for multi-view semi-supervised subspace clustering task. Furthermore, we devise a contrained tensor representation learning (CTRL) model that takes advantage of our proposed regularization to facilitate the learning of the desired representation tensor. An efficient optimization algorithm based on alternating direction minimization strategy is carefully designed to solve the proposed CTRL model. Extensive experiments on eight challenging real-world datasets are conducted, and the results validate the effectiveness of our designed pairwise constraints regularization, as well as the superiority of the proposed CTRL model.

A Comparative Study of Incremental Semi-Supervised Subspace Clustering Ensemble Approaches and Performance Outcomes for High Dimensional Data

A Comparative Study of Incremental Semi-Supervised Subspace Clustering Ensemble Approaches and Performance Outcomes for High Dimensional Data

A splitting method for the locality regularized semi-supervised subspace clustering

Graph-based semi-supervised learning (G-SSL) methods play an increasingly important role in machine learning systems. Recently, latent low-rank representation (LatLRR) graph has gained great success in subspace clustering. However, LatLRR only considers the global structure, while the local geometric information, which is often important to many real applications, is ignored. In this paper, we propose a locality regularized LatLRR model (LR-LatLRR) for semi-supervised subspace clustering problems. This model incorporates two regularization terms into LatLRR by taking the local structure of data into account. Then, we develop an efficient splitting algorithm for solving LR-LatLRR. In addition, we also prove the global convergence of the proposed algorithm. Furthermore, we extend the LR-LatLRR model to a case of including the non-negative constraint. Finally, we conduct experiments on a synthetic data and several real data sets for the semi-supervised clustering problems. Experimental results show that our method can obtain high classification accuracy and outperforms several state-of-the-art G-SSL methods.

ISVS3CE: Incremental Support Vector Semi-Supervised Subspace Clustering Ensemble and ENhanced Bat Algorithm (ENBA) for High Dimensional Data Clustering

In the recent work, Incremental Soft Subspace Based Semi-Supervised Ensemble Clustering (IS4EC) framework was proposed which helps in detecting clusters in the dataset. IS4EC framework also increases the results of clustering by reducing the intra-cluster distance and increasing the inter-cluster distance with increased cluster quality. It cannot attain acceptable results while handling high dimensional data. However, decreasing the dimensional subspace becomes extremely difficult issue. In IS4EC framework, to choose the optimal ensemble members also extremely becomes challenging issue. In order to solve these issues of traditional cluster ensemble methods, first propose an Incremental Support vector Semi-Supervised Subspace Clustering Ensemble (ISVS3CE) framework which makes utilized of benefits of the random subspace algorithm and the Constraint Propagation (CP) algorithm. Here the centroid values were selected by using the Support Vector Machine (SVM) classifier. In the ISVS3CE framework, Incremental Ensemble Member Chosen (IEMC) process is performed by using the ENhanced Bat Algorithm (ENBA), and the normalized cut algorithm is introduced to perform high dimensional data clustering. The ISVS3CE framework is successful for solving high dimensional data issue, at the same time as the CP algorithm is valuable for incorporating the prior information. Results demonstrate that the proposed ISVS3CE framework performs well on datasets by means of very high dimensionality, and better than the traditional clustering ensemble methods.

A framework for expert-driven subpopulation discovery and evaluation using subspace clustering for epidemiological data

Objective: We propose an intelligent system that assists epidemiology experts in analysing the data of a population-based epidemiological study, in identifying relevant variables for an outcome and subpopulations with increased disease prevalence, and in validating the findings concerning variables and subpopulations in a further, expert-specified cohort. At present, the study of an outcome on a population-based cohort is hypothesis-driven, i.e. the expert must specify the variables to be studied. Our approach rather operates in a data-driven, semi-automated way, enabling the expert to identify variables of relevance and generate hypotheses on them.Methods: Our system DIVA supports the Discovery, Inspection and VAlidation of subpopulations with increased prevalence of an outcome, without requiring parameter tuning. DIVA takes as input the cohort of an epidemiological population-based study with all variables specified in the study’s protocol, as well as inputs from the expert on the similarity of a small number of cohort participants. DIVA uses semi-supervised subspace clustering and subspace construction to identify sets of variables – subspaces – that promote participant similarity with respect to the outcome and with respect to the expert inputs, and then discovers subpopulations with increased outcome prevalence in those subspaces (DIVA module “DRESS”). DIVA uses visual analytics techniques to assist the expert in juxtaposing, filtering and inspecting the characteristics of these subpopulations (web-based DIVA module “D-INSPECTOR”). If the expert has access to a second cohort on a comparable population, DIVA aligns the cohort used for discovery to this second cohort, and then checks whether the subpopulations found in the original cohort are also present in the second one (DIVA module “VALIDATOR”).Results:We applied DIVA to the third wave (SHIP-2) of the SHIP-CORE cohort of the Study of Health in Pomerania (Völzke et al., 2011) for the liver disorder “hepatic steatosis”, and on the first wave (TREND-0) of the SHIP-TREND cohort of the same study for the thyroid gland disorder “goitre”. We found that most of the subpopulations extracted automatically, and subsequently ranked and filtered by the modules of DIVA, had significantly higher disease prevalence than the general population. We varied the amount of inputs needed from the expert to drive the subpopulation extraction process and found that a very small amount of information, namely the outcome of as few as 4 cohort participants, is sufficient for the identification of several relevant variables and subpopulations. We used a subset of TREND-0 for the validation on goitre and the complete TREND-0 for the validation on hepatic steatosis and found that the significant difference in prevalence for the identified subpopulation also holds in the validation data.Conclusions: We have shown that DIVA discovers subpopulations and variables of importance with respect to an outcome, while requiring a very small amount of expert inputs. Each combination of variables and each subpopulation corresponds to a hypothesis, the validation of which would have required substantial human effort. Thus, DIVA allows for a more effective exploitation of population-based data, not fully automated but driven by the expert and without the need for technical parameter tuning.A shortcoming of DIVA design is the demand of a specific type of expert inputs, namely “constraints” on the similarity of pairs of participants. Currently, we generate the constraints with a naive utility that is based on random sampling, but we work on the development of an interactive algorithm that would allow the epidemiology expert to inspect a small choice of study participant and give statements on their similarity.The present version of DIVA considers a single wave of the cohort data, ignoring the evolution of the population during the horizon of the study. Hence, subspace and subpopulation discovery do not take account of changes in the importance of variables. We currently work on the incorporation of algorithms that derive additional variables from the longitudinal data and use them in the Discovery module.

Unified Discriminative and Coherent Semi-supervised Subspace Clustering.

The ubiquitous large, complex and high dimensional datasets in computer vision and machine learning generates the problem of subspace clustering, which aims to partition the data into several low dimensional subspaces. By utilizing relatively limited labeled data and sufficient unlabeled data, the semi-supervised subspace clustering is more effective, practical and become more popular. In this work, we present a new regularity combing the labels and the affinity to ensure the coherence of the affinity between data points from the same subspace as well as the discrimination of cluster labels for data points from different subspaces. We combine it with the manifold smoothing term of the existing methods and the Gaussian fields and harmonic functions method to give a new unified optimization framework for semi-supervised subspace clustering. Analysis shows the proposed model fully combines the affinity and the labels to guide each other so that both are discriminative between clusters and coherent within clusters. Extensive experiments show that our method outperforms the existing state-of-the-art methods, thus suggests that the property of discriminative between clusters and coherent within clusters of our method is advantageous to semi-supervised subspace clustering.

Constrained Low-Rank Representation for Robust Subspace Clustering.

Subspace clustering aims to partition the data points drawn from a union of subspaces according to their underlying subspaces. For accurate semisupervised subspace clustering, all data that have a must-link constraint or the same label should be grouped into the same underlying subspace. However, this is not guaranteed in existing approaches. Moreover, these approaches require additional parameters for incorporating supervision information. In this paper, we propose a constrained low-rank representation (CLRR) for robust semisupervised subspace clustering, based on a novel constraint matrix constructed in this paper. While seeking the low-rank representation of data, CLRR explicitly incorporates supervision information as hard constraints for enhancing the discriminating power of optimal representation. This strategy can be further extended to other state-of-the-art methods, such as sparse subspace clustering. We theoretically prove that the optimal representation matrix has both a block-diagonal structure with clean data and a semisupervised grouping effect with noisy data. We have also developed an efficient optimization algorithm based on alternating the direction method of multipliers for CLRR. Our experimental results have demonstrated that CLRR outperforms existing methods.

Robust Semi-Supervised Subspace Clustering via Non-Negative Low-Rank Representation.

Low-rank representation (LRR) has been successfully applied in exploring the subspace structures of data. However, in previous LRR-based semi-supervised subspace clustering methods, the label information is not used to guide the affinity matrix construction so that the affinity matrix cannot deliver strong discriminant information. Moreover, these methods cannot guarantee an overall optimum since the affinity matrix construction and subspace clustering are often independent steps. In this paper, we propose a robust semi-supervised subspace clustering method based on non-negative LRR (NNLRR) to address these problems. By combining the LRR framework and the Gaussian fields and harmonic functions method in a single optimization problem, the supervision information is explicitly incorporated to guide the affinity matrix construction and the affinity matrix construction and subspace clustering are accomplished in one step to guarantee the overall optimum. The affinity matrix is obtained by seeking a non-negative low-rank matrix that represents each sample as a linear combination of others. We also explicitly impose the sparse constraint on the affinity matrix such that the affinity matrix obtained by NNLRR is non-negative low-rank and sparse. We introduce an efficient linearized alternating direction method with adaptive penalty to solve the corresponding optimization problem. Extensive experimental results demonstrate that NNLRR is effective in semi-supervised subspace clustering and robust to different types of noise than other state-of-the-art methods.

Exploiting constraint inconsistence for dimension selection in subspace clustering: A semi-supervised approach

Selecting correct dimensions is very important to subspace clustering and is a challenging issue. This paper studies semi-supervised approach to the problem. In this setting, limited domain knowledge in the form of space level pair-wise constraints, i.e., must-links and cannot-links, are available. We propose a semi-supervised subspace clustering (S 3C) algorithm that exploits constraint inconsistence for dimension selection. Our algorithm firstly correlates globally inconsistent constraints to dimensions in which they are consistent, then unites constraints with common correlating dimensions, and finally forms the subspaces according to the constraint unions. Experimental results show that S 3C is superior to the typical unsupervised subspace clustering algorithm FINDIT, and the other constraint based semi-supervised subspace clustering algorithm SC-MINER.