Semi-supervised Clustering Research Articles

GSSCL: A framework for Graph Self-Supervised Curriculum Learning based on clustering label smoothing

Graph self-supervised learning is an effective technique for learning common knowledge from unlabeled graph data through pretext tasks. To capture the interrelationships between nodes and their essential roles globally, existing methods use clustering labels as self-supervised signals. However, in some cases, these methods may introduce noise, leading to over-fitting of the model and a reduction in performance. To address these issues, a novel framework for Graph Self-Supervised Curriculum Learning based on clustering label smoothing called GSSCL has been proposed. GSSCL clusters knowledge in an easy-to-difficult manner, reducing the heavy dependence on the reliability of clustering and improving the generalizability of the model. Moreover, the Silhouette Coefficient is employed to evaluate the clustering confident scores for all nodes. Some nodes are selected based on high confident scores to perform self-supervised learning. To account for the possibility of complex heterophilous information in graphs (e.g., noisy links), clustering pseudo-label smoothing is performed on K-nearest neighbor graphs built upon the similarities between node features instead of the original graph structures. The obtained multi-scale knowledge is then applied to curriculum learning. Finally, comprehensive experiments conducted across diverse public graph benchmarks demonstrate the superior performance of the proposed framework. It exhibits comparable results to state-of-the-art methods across semi-supervised node classification and clustering tasks.

Dual semi-supervised hypergraph regular multi-view NMF with anchor graph embedding

Graph regularized nonnegative matrix factorization (GNMF) has been widely used in multi-view clustering tasks due to its good clustering properties. However, it uses a simple graph to describe the complex data relationship of multiple views and tries to obtain a consistent low-dimensional representation, which undoubtedly brings challenges to its clustering performance. In addition, clustering a large amount of high-dimensional data from multiple views undoubtedly faces a huge computational burden. In order to effectively improve the performance and efficiency of GNMF based multi-view clustering algorithm, this paper proposes a dual semi-supervised hypergraph regular multi-view clustering method with anchor graph embedding (DSSHMNMFAE). Specifically, DSSHMNMFAE develops a new anchor selection method to generate anchors and the anchor bipartite graph is constructed to embed the matrix factorization process. DSSHMNMFAE constructs a hypergraph to effectively learn the high-order relationship between data from multiple views. In order to perform semi-supervised learning more efficiently, DSSHMNMFAE integrates pairwise constraint information and label constraint information into the clustering process as dual label information. In addition, DSSHMNMFAE considers the learning of both consistency information and complementarity information, and adopts adaptive measures to distinguish the contributions of different views. We use the alternating iterative algorithm to optimize the objective function of DSSHMNMFAE. The experimental results on eight real datasets show that the performance of DSSHMNMFAE is comparable to other algorithms.

ImmCellTyper facilitates systematic mass cytometry data analysis for deep immune profiling.

Mass cytometry is a cutting-edge high-dimensional technology for profiling marker expression at the single-cell level, advancing clinical research in immune monitoring. Nevertheless, the vast data generated by cytometry by time-of-flight (CyTOF) poses a significant analytical challenge. To address this, we describe ImmCellTyper (https://github.com/JingAnyaSun/ImmCellTyper), a novel toolkit for CyTOF data analysis. This framework incorporates BinaryClust, an in-house developed semi-supervised clustering tool that automatically identifies main cell types. BinaryClust outperforms existing clustering tools in accuracy and speed, as shown in benchmarks with two datasets of approximately 4 million cells, matching the precision of manual gating by human experts. Furthermore, ImmCellTyper offers various visualisation and analytical tools, spanning from quality control to differential analysis, tailored to users' specific needs for a comprehensive CyTOF data analysis solution. The workflow includes five key steps: (1) batch effect evaluation and correction, (2) data quality control and pre-processing, (3) main cell lineage characterisation and quantification, (4) in-depth investigation of specific cell types; and (5) differential analysis of cell abundance and functional marker expression across study groups. Overall, ImmCellTyper combines expert biological knowledge in a semi-supervised approach to accurately deconvolute well-defined main cell lineages, while maintaining the potential of unsupervised methods to discover novel cell subsets, thus facilitating high-dimensional immune profiling.

ScVSC: Deep Variational Subspace Clustering for Single-Cell Transcriptome Data.

Single-cell RNA sequencing (scRNA-seq) is a potent advancement for analyzing gene expression at the individual cell level, allowing for the identification of cellular heterogeneity and subpopulations. However, it suffers from technical limitations that result in sparse and heterogeneous data. Here, we propose scVSC, an unsupervised clustering algorithm built on deep representation neural networks. The method incorporates the variational inference into the subspace model, which imposes regularization constraints on the latent space and further prevents overfitting. In a series of experiments across multiple datasets, scVSC outperforms existing state-of-the-art unsupervised and semi-supervised clustering tools regarding clustering accuracy and running efficiency. Moreover, the study indicates that scVSC could visually reveal the state of trajectory differentiation, accurately identify differentially expressed genes, and further discover biologically critical pathways.

Interpretable semi-supervised clustering enables universal detection and intensity assessment of diverse aviation hazardous winds

The identification of aviation hazardous winds is crucial and challenging in air traffic management for assuring flight safety, particularly during the take-off and landing phases. Existing criteria are typically tailored for special wind types, and whether there exists a universal feature that can effectively detect diverse types of hazardous winds from radar/lidar observations remains as an open question. Here we propose an interpretable semi-supervised clustering paradigm to solve this problem, where the prior knowledge and probabilistic models of winds are integrated to overcome the bottleneck of scarce labels (pilot reports). Based on this paradigm, a set of high-dimensional hazard features is constructed to effectively identify the occurrence of diverse hazardous winds and assess the intensity metrics. Verification of the paradigm across various scenarios has highlighted its high adaptability to diverse input data and good generalizability to diverse geographical and climate zones.

Machine learning identifies clusters of the normal adolescent spine based on sagittal balance.

This study applied a machine learning semi-supervised clustering approach to radiographs of adolescent sagittal spines from a single pediatric institution to identify patterns of sagittal alignment in the normal adolescent spine. We sought to explore the inherent variability found in adolescent sagittal alignment using machine learning to remove bias and determine whether clusters of sagittal alignment exist. Multiple semi-supervised machine learning clustering algorithms were applied to 111 normal adolescent sagittal spines. Sagittal parameters for resultant clusters were determined. Machine learning analysis found that the spines did cluster into distinct groups with an optimal number of clusters ranging from 3 to 5. We performed an analysis on both 3 and 5-cluster groups. The 3-cluster groups analysis found good consistency between methods with 96 of 111, while the analysis of 5-cluster groups found consistency with 105 of 111 spines. When assessing for differences in sagittal parameters between the groups for both analyses, there were differences in T4-12 TK, L1-S1 LL, SS, SVA, PI-LL mismatch, and TPA. However, the only parameter that was statistically different for all groups was SVA. Based on machine learning, the adolescent sagittal spine alignments do cluster into distinct groups. While there were distinguishing features with TK and LL, the most important parameter distinguishing these groups was SVA. Further studies may help to understand these findings in relation to spinal deformities.

Adaptive channel-weight dual-constrained semi-supervised EEG clustering

When processing Electroencephalography(EEG) for applications such as brain disease diagnosis, rehabilitation, and brain–computer interfaces, we encounter challenges related to missing labels or incorrect annotations. These challenges may arise from subjective factors, technical limitations, or data quality issues. In traditional EEG processing, unsupervised clustering algorithms are commonly used; however, due to the lack of labels, these methods may struggle to fully leverage latent information, resulting in uncertainty and instability in clustering outcomes. To address this issue, we propose a semi-supervised clustering method tailored for partially labeled EEG signals, named Adaptive Channel-weight Dual-constrained Semi-supervised EEG Clustering (ACDSEEGc). This method incorporates dual constraints: a connectivity constraint for the interconnections between EEG points and a neighborhood constraint for the spatial neighborhood relationships among EEG data points. Additionally, it integrates channel weights to emphasize information from important channels. The objective function of ACDSEEGc integrates cross-entropy evaluation, channel weight learning, pseudo-label learning, connectivity constraint, and least squares error minimization. Our aim is to maximize the use of limited prior knowledge to achieve better EEG clustering results. Experimental validation demonstrates that ACDSEEGc effectively produces superior clustering outcomes compared to state-of-the-art standard unsupervised and semi-supervised EEG/time-series clustering algorithms. Furthermore, we conducted sensitivity experiments and ablation studies to verify the performance of ACDSEEGc under different settings.

Combining Semi-supervised Clustering and Classification Under a Generalized Framework

Combining Semi-supervised Clustering and Classification Under a Generalized Framework

S-wave log construction through semi-supervised regression clustering using machine learning: A case study of North Sea fields

Accurate prediction of S-wave velocity from well logs is essential for understanding subsurface geological formations and hydrocarbon reservoirs. Machine learning techniques, including clustering and regression, have emerged as effective methods for indirectly estimating S-wave logs and other rock properties. In this study, we employed clustering algorithms to identify similarities among well log datasets, encompassing depth, sonic, porosity, neutron, and apparent density, facilitating the discovery of correlations among various wells. These identified correlations served as a foundation for predicting S-wave values using a novel semi-supervised approach. Our approach combined clustering, specifically k-means clustering, with different types of regressors, including Least Squares Regression (LSR), Support Vector Regression (SVR), and Multi-Layer Perceptron (MLP). Our results demonstrate the superior performance of this integrated approach compared to traditional regression methods. We validated our methodology using various parametric and non-parametric regression techniques, showcasing its effectiveness not only on wells within the training region but also on wells outside the study area. We achieved a significant improvement in the R2 score metric, ranging from 2.22% to 6.51%, and a reduction in Mean Square Error (MSE) of at least 31% when compared to predictions made without clustering. This study underscores the potential of machine learning techniques for accurate prediction of S-wave velocity and other rock properties, thereby enhancing our comprehension of subsurface geology and hydrocarbon reservoirs.

Contrastive subspace distribution learning for novel category discovery in high-dimensional visual data

Discovering novel visual categories, particularly in recognition tasks, is a prominent area of research in artificial intelligence. Visual data, such as images and videos, is inherently high-dimensional and they often exhibit low-dimensional subspace structures, making the task of uncovering novel low-dimensional subspace structures a substantial challenge. To address this challenge, we develop a novel subspace discovery method, consisting of four steps. First, we introduce a base-expressive network (BENet) designed to learn coefficients that express the relationships between samples and bases. Additionally, these coefficients are normalized as a probability distribution using the softmax function. Second, to enhance this process, we propose a Contrastive Subspace Distribution Learning (CSDL) technique, utilizing Rényi Divergence. CSDL combines self-supervised and supervised contrastive distances between the normalized coefficients, considering all samples and labeled samples, respectively. Third, the base-expressive coefficients are subsequently computed by the learned SENet, which effectively transfers subspace information from labeled samples to unlabeled ones. Finally, we employ spectral clustering on the predicted coefficients to automatically assign categories to unlabeled samples. Extensive experimental results demonstrate that our approach surpasses related methods in the realms of category discovery, unsupervised clustering, and semi-supervised (subspace) clustering.

Effective semi-supervised graph clustering with pairwise constraints

Semi-supervised graph clustering with constraints has received considerable attention in the last decade. They use pre-given constraints to guide the clustering process and improve the performance. Nonetheless, most of related research works still face two main issues: (1) cannot-link problem: how to ensure that instances under the cannot-link constraint are located in different clusters as much as possible. (2) The high computational cost caused by eigendecomposition. To tackle these problems, this paper proposes a novel method for directly solving the clustering indicator matrix without high complexity computational steps. Then, we propose an effective and simple algorithm guided by the pre-given constraints, which is able to simultaneously optimize the associated multiple pairs of constraints, allowing the associated constraints to achieve the related co-optimal solution. Extensive experiments are conducted to verify that our approach significantly reduces the violation rate of the prior constraints and the clustering performance of the proposed method outperforms seven other state-of-the-art semi-supervised clustering methods on 16 real benchmark datasets.

Transcriptomic analysis of the human habenula in schizophrenia.

Transcriptomic analysis of the human habenula and identification of molecular changes associated with schizophrenia risk and illness state.

Study on the Design Parameters of Landscape Elements of Urban Tunnel Body Segments in China Based on Biomass Perception

To provide a comprehensive analysis of how an intricate landscape environment comprising various landscape elements influences driving safety and comfort. Beginning with the individual design of the tunnel body section, which includes the roof, sidewalls, road surface, and sidewalk facade markings, this paper references the driver’s biomass index to identify design alterations that satisfy basic safety requirements through simulated driving experiments. Subsequently, employing orthogonal experimental design and semi-supervised clustering algorithms, we ascertain the optimal combination of the four main landscape elements of the tunnel body section that align with both safety and comfort objectives. The experimental findings demonstrate that a safe and comfortable landscape design for urban tunnel sections does not entail a single optimal design; rather, any landscape design conforming to the criterion of “patterned landscapes are set on the side walls and roofs of the tunnel instead of the surface of the road” is the optimal design.

Localization of protoporphyrin IX during glioma-resection surgery via paired stimulated Raman histology and fluorescence microscopy.

The most widely used fluorophore in glioma-resection surgery, 5-aminolevulinic acid (5-ALA), is thought to cause the selective accumulation of fluorescent protoporphyrin IX (PpIX) in tumour cells. Here we show that the clinical detection of PpIX can be improved via a microscope that performs paired stimulated Raman histology and two-photon excitation fluorescence microscopy (TPEF). We validated the technique in fresh tumour specimens from 115 patients with high-grade gliomas across four medical institutions. We found a weak negative correlation between tissue cellularity and the fluorescence intensity of PpIX across all imaged specimens. Semi-supervised clustering of the TPEF images revealed five distinct patterns of PpIX fluorescence, and spatial transcriptomic analyses of the imaged tissue showed that myeloid cells predominate in areas where PpIX accumulates in the intracellular space. Further analysis of external spatially resolved metabolomics, transcriptomics and RNA-sequencing datasets from glioblastoma specimens confirmed that myeloid cells preferentially accumulate and metabolize PpIX. Our findings question 5-ALA-induced fluorescence in glioma cells and show how 5-ALA and TPEF imaging can provide a window into the immune microenvironment of gliomas.

Semi-supervised clustering guided by pairwise constraints and local density structures

Clustering based on local density peaks and graph cut (LDP-SC) is one of the state-of-the-art algorithms in unsupervised clustering, which first divides the data set to be multiple local trees, and then aggregates these local trees to obtain the final clustering result. However, for complex data sets, there might exist data points from different classes in the same local tree. In this article, we use pairwise constraint information to resolve this issue and propose a semi-supervised local density peaks and graph cut based clustering algorithm (SLDPC). In particular, SLDPC proposes intra-cluster conflict resolution and inter-cluster conflict resolution steps to split the local trees which are inconsistent with the provided pairwise constraint information. Theoretically, we show that the two steps will finish in a finite number of operations and the split local trees will be consistent with the pairwise constraint information. Subsequently, root node redirection and noise filtering steps are designed to avoid the local trees becoming too fragmented. Finally, we exploit the E2CP algorithm to further improve the similarity matrix between local trees using the pairwise constraint information, and the spectral clustering algorithm is adopted to obtain the clustering result. Experiments on multiple widely used synthetic and real-world data sets show that SLDPC is superior to LDP-SC and several other semi-supervised prominent clustering algorithms for most of the cases.

Hypergraph-based convex semi-supervised unconstraint symmetric matrix factorization for image clustering

Semi-supervised symmetric nonnegative matrix factorization (SNMF) has been extensively utilized in both linear and nonlinear data clustering tasks. However, the current SNMF model's non-convex objective function faces challenges in global optimization and time efficiency. In this study, we leverage label information to propose a convex and unconstrained symmetric matrix factorization (SMF) model that is thoroughly analyzed for its convexity properties. In order to capture high-order relationships among data, a hypergraph is utilized in the model, which is computationally simple, translation invariant, and naturally normalized. Moreover, based on the analysis and the corresponding experiments in the paper, the model exhibits robustness towards outliers to some extent. Due to the convexity of our proposed model without constraint, it can be efficiently optimized using the Conjugate Gradient (CG) method, one of the most efficient methods available. Therefore, we propose a novel Convex Combination-based Sufficient Descent CG (CSDCG) method, which outperforms other methods across 284 optimization problems within the CUTEst library. In order to evaluate the effectiveness of the proposed method, the semi-supervised clustering experiments are conducted on the eight datasets by comparison with ten state-of-the-art matrix factorization (MF) methods. The experiment results demonstrate its superiority over the other compared methods to handle the clustering problem with better performance and less computational time. The code is available at https://github.com/Pokemer/HCSSMF.

Efficient semi-supervised clustering with pairwise constraint propagation for multivariate time series

Semi-supervised clustering is an effective method, which improves the clustering performance based on pairwise constraints. However, state-of-the-art methods suffer from two issues: 1) due to the high dimensionality and multiple variables of multivariate time series (MTS) data, the competitive similarity approach DTW is time consuming on large-scale MTS data. 2) Traditional semi-supervised clustering methods could not make full use of pairwise constraints information, which affects clustering performance. To deal with these issues, we propose an efficient semi-supervised clustering with pairwise constraint propagation for MTS data. First, two approximate distance measure methods are designed based on dynamic time warping (DTW) from the perspectives of boundary and dimension reduction, which greatly improve the efficiency of clustering without sacrificing its accuracy. Then, a graph-based clustering method with pairwise constraints propagation (GCPCP) is advanced on multivariate time series data. In GCPCP, the pairwise constraint propagation matrix and the affinity matrix are jointly optimized to exploit the dependence between them, which finally improves the clustering performance. Experimental results on 12 multivariate time series datasets show the effectiveness and efficiency of our proposed method.

Semi-supervised topic representation through sentiment analysis and semantic networks

This paper proposes a novel approach to topic detection aimed at improving the semi-supervised clustering of customer reviews in the context of customers' services. The proposed methodology, named SeMi-supervised clustering for Assessment of Reviews using Topic and Sentiment (SMARTS) for Topic-Community Representation with Semantic Networks, combines semantic and sentiment analysis of words to derive topics related to positive and negative reviews of specific services. To achieve this, a semantic network of words is constructed based on word embedding semantic similarity to identify relationships between words used in the reviews. The resulting network is then used to derive the topics present in users' reviews, which are grouped by positive and negative sentiment based on words related to specific services. Clusters of words, obtained from the network's communities, are used to extract topics related to particular services and to improve the interpretation of users' assessments of those services. The proposed methodology is applied to tourism review data from Booking.com, and the results demonstrate the efficacy of the approach in enhancing the interpretability of the topics obtained by semi-supervised clustering. The methodology has the potential to provide valuable insights into the sentiment of customers toward tourism services, which could be utilized by service providers and decision-makers to enhance the quality of their services.

A lie group semi-supervised FCM clustering method for image segmentation

As an unsupervised clustering method with low overhead, Fuzzy C-means (FCM) clustering has been widely used in a variety of image segmentation tasks. However, existing FCM clustering methods are sensitive to image noises and are either suffer from losing of image detail or falling into local optima in identifying cluster centers. Aiming at these problems, this paper proposes a Lie group semi-supervised FCM (LieSSFCM) clustering method for image segmentation. The method maps the input image from Euclidean space to Lie group manifold by representing each image pixel as a matrix Lie group and calculates geodesic distances between group elements and cluster centers on Lie group manifold. Prior information of the image and neighborhood relationships of pixels are used to guide the initialization and constrain the update of cluster centers and the corresponding fuzzy membership matrix. The proposed LieSSFCM has been validated against two medical image datasets and was compared with seven FCM clustering methods. Experimental results along with a systematic evaluation demonstrated that the method was superior in segmentation accuracy both visually and statistically, robustness to noises, adaptability to different tasks, and stability while maintaining a moderate computational complexity.

Semi-supervised spectral clustering using shared nearest neighbour for data with different shape and density

<p>In the absence of supervisory information in spectral clustering algorithms, it is difficult to construct suitable similarity graphs for data with complex shapes and varying densities. To address this issue, this paper proposes a Semi-supervised Spectral Clustering algorithm based on shared nearest neighbor. The proposed algorithm combines the idea of semi-supervised clustering, adding Shared Nearest Neighbor information to the calculation of the distance matrix, and using pairwise constraint information to find the relationship between two data points, while providing a portion of supervised information. Comparative experiments were conducted on artificial data sets and University of California Irvine machine learning repository datasets. The experimental results show that the proposed algorithm achieves better clustering results compared to traditional K-means and spectral clustering algorithms.</p>