Semi-supervised Clustering Algorithm Research Articles

Semi-supervised spectral clustering using shared nearest neighbour for data with different shape and density

<p>In the absence of supervisory information in spectral clustering algorithms, it is difficult to construct suitable similarity graphs for data with complex shapes and varying densities. To address this issue, this paper proposes a Semi-supervised Spectral Clustering algorithm based on shared nearest neighbor. The proposed algorithm combines the idea of semi-supervised clustering, adding Shared Nearest Neighbor information to the calculation of the distance matrix, and using pairwise constraint information to find the relationship between two data points, while providing a portion of supervised information. Comparative experiments were conducted on artificial data sets and University of California Irvine machine learning repository datasets. The experimental results show that the proposed algorithm achieves better clustering results compared to traditional K-means and spectral clustering algorithms.</p>

Electrical Sensor Calibration by Fuzzy Clustering with Mandatory Constraint.

Electrical tomography sensors have been widely used for pipeline parameter detection and estimation. Before they can be used in formal applications, the sensors must be calibrated using enough labeled data. However, due to the high complexity of actual measuring environments, the calibrated sensors are inaccurate since the labeling data may be uncertain, inconsistent, incomplete, or even invalid. Alternatively, it is always possible to obtain partial data with accurate labels, which can form mandatory constraints to correct errors in other labeling data. In this paper, a semi-supervised fuzzy clustering algorithm is proposed, and the fuzzy membership degree in the algorithm leads to a set of mandatory constraints to correct these inaccurate labels. Experiments in a dredger validate the proposed algorithm in terms of its accuracy and stability. This new fuzzy clustering algorithm can generally decrease the error of labeling data in any sensor calibration process.

Online semi-supervised active learning ensemble classification for evolving imbalanced data streams

Concept drift is a core challenge in classification tasks of data streams. Although many drift adaptation methods have been presented, most of them assume that labels of all data are available, which is impractical in many real-world applications. Additionally, the absence of label makes the imbalance ratio of an imbalanced data stream difficultly being obtained in time, providing the inaccurate guidance for resampling and causing poor generalization. To tackle the joint challenges, an online semi-supervised active learning method is proposed to classifier imbalanced data streams with concept drift. A newly-arrived data is first added to the sliding window, and then assigned a pseudo label in terms of its nearest cluster. Meanwhile, semi-supervised clustering algorithm offers its predicted label. Based on the above two predictive labels, cluster-based query strategy provides the criteria for the evaluation and selection of representative instances. More especially, the uncertainty and importance of instances are defined to synthetically evaluate its representativeness. After obtaining true labels of typical ones, ensemble classifier is updated by all instances in current sliding window. Experimental results on 13 synthetic and real data streams indicate that the proposed method outperforms six comparative methods on both G-mean and Recall under various labeling budgets.

A game model for semi-supervised subspace clustering with dynamic affinity and label learning

With the aid of partial supervised information, semi-supervised subspace clustering methods aim to obtain affinity matrices directly derived from raw data, and then those affinity matrices are utilized to get assignment matrices. These affinity matrices are susceptible to disturbances such as noise and outliers, which could significantly impact their quality. To mitigate this, it becomes essential to dynamically update the affinity matrix using the clustering result, reducing the high dependency on raw data. This paper presents a Game Model for Semi-supervised Subspace Clustering (GMSSC). The first submodule of GMSSC utilizes a Graph Convolutional Network to learn a mapping from raw data to the assignment matrix with the assistance of the affinity matrix. The second submodule constructs a semi-supervised self-expressive model to learn a discriminative affinity matrix. By integrating these submodules into a game model, GMSSC achieves a Nash Equilibrium during adversarial training, resulting in stable and robust affinity and assignment matrices. Additionally, we propose an iterative algorithm based on data-driven and model-driven approaches to solve the model. Experimental results on four open real-world datasets demonstrate that the proposed method not only achieves elegant results but also outperforms state-of-the-art semi-supervised subspace clustering algorithms.

A new semi-supervised clustering algorithm for probability density functions and applications

A new semi-supervised clustering algorithm for probability density functions and applications

An end-to-end Graph Convolutional Network for Semi-supervised Subspace Clustering via label self-expressiveness

Semi-supervised clustering provides accurate assignments by leveraging a limited amount of supervisory information. There are two main groups of conventional semi-supervised subspace clustering algorithms: Pairwise Constraints (PC)-based methods and Gaussian Fields and Harmonic Function (GFHF)-based methods. Nevertheless, PC-based methods are limited because of their reliance on weak supervisory information. GFHF-based methods, on the other hand, focus on the local relationship between the affinity matrix and the assignment matrix, which restricts their clustering efficacy. Recently, graph convolutional network (GCN) has been employed to effectively transmit the local graph topology of the raw data into the feature domain, which motivates the proposed GCN model for Semi-supervised Subspace Clustering via label self-expressiveness (GSSC). Conventional semi-supervised clustering has been enhanced by incorporating a novel concept known as label self-expressiveness, which serves as a guiding factor for the clustering process. Furthermore, GSSC considers the global geometric structure of the assignment matrix and generates a discriminable assignment matrix using an end-to-end mechanism. Additionally, a well-designed loss function is introduced to train the novel GSSC. The experimental results conducted on four benchmark datasets demonstrate that the proposed GSSC algorithm exhibits superior performance compared to existing semi-supervised clustering algorithms.

Semi-supervised fuzzy C means based on membership integration mechanism and its application in brain infarction lesion segmentation in DWI images

In this paper, a novel semi-supervised fuzzy clustering algorithm, MFM-SFCM, based on a membership fusion mechanism is proposed for Diffusion-weighted imaging (DWI) brain infarction lesion segmentation. The proposed MFM-SFCM algorithm addresses the issue of weakened constraints and insufficient influence of labeled samples on the clustering process that arises in the semi-supervised fuzzy C-means clustering (SFCM) when emphasizing supervised information. By using a new membership fusion mechanism, MFM-SFCM eliminates this issue, greatly improving the accuracy of clustering results and accelerating convergence speed. This allows fuzzy clustering to achieve good results in the segmentation of DWI brain infarction lesions using a small amount of labeled information. The effectiveness of the MFM-SFCM algorithm is demonstrated through experiments conducted on a real-world dataset of DWI brain images.

Semi-supervised possibilistic c-means clustering algorithm based on feature weights for imbalanced data

The possibilistic c-means clustering (PCM) algorithm improves the robustness of fuzzy c-means clustering (FCM) to noise and outliers by releasing the probabilistic constraint of memberships. The semi-supervised possibilistic c-means clustering (SSPCM) algorithm improves the clustering effect on datasets with imbalanced sizes by introducing a small amount of label information. However, the traditional semi-supervised algorithm still faces the problem of low utilization of supervision information for datasets with large differences in sample sizes. Moreover, the Euclidean distance, which treats features equally, cannot handle feature-imbalanced data. Therefore, this paper proposes a semi-supervised possibilistic c-means clustering algorithm based on feature weights (FW-SSPCM) by introducing the ideas of supervised centers. First, the algorithm introduces the supervised center into the objective function of the SSPCM to improve the utilization rate of supervision information and thus guide the center iteration of small clusters. Second, the feature weighting strategy is introduced in the objective function to adaptively assign feature weights according to the importance of different features in different clusters, thus improving the adaptability of the algorithm to feature-imbalanced datasets. In addition, to improve the robustness of the antinoise effect and retain additional image details, a new image segmentation algorithm based on FW-SSPCM and local information (LFW-SSPCM) is proposed by introducing local spatial information obtained by bilateral filtering. Finally, through clustering experiments on synthetic data, UCI datasets and on color images characteristic of multiple features, including imbalanced sizes, imbalanced features and strong noise injection, the clustering performances of the proposed FW-SSPCM and LFW-SSPCM proposed in this paper are significantly better than those of several related clustering algorithms.

Toward Addressing Ambiguous Interactions and Inferring User Intent with Dimension Reduction and Clustering Combinations in Visual Analytics

Direct manipulation interactions on projections are often incorporated in visual analytics applications. These interactions enable analysts to provide incremental feedback to the system in a semi-supervised manner, demonstrating relationships that the analyst wishes to find within the data. However, determining the precise intent of the analyst is a challenge. When an analyst interacts with a projection, the inherent ambiguity of interactions can lead to a variety of possible interpretations that the system can infer. Previous work has demonstrated the utility of clusters as an interaction target to address this “With Respect to What” problem in dimension-reduced projections. However, the introduction of clusters introduces interaction inference challenges as well. In this work, we discuss the interaction space for the simultaneous use of semi-supervised dimension reduction and clustering algorithms. We introduce a novel pipeline representation to disambiguate between interactions on observations and clusters, as well as which underlying model is responding to those analyst interactions. We use a prototype visual analytics tool to demonstrate the effects of these ambiguous interactions, their properties, and the insights that an analyst can glean from each.

A semi-supervised ensemble clustering algorithm for discovering relationships between different diseases by extracting cell-to-cell biological communications.

In recent decades, many theories have been proposed about the cause of hereditary diseases such as cancer. However, most studies state genetic and environmental factors as the most important parameters. It has been shown that gene expression data are valuable information about hereditary diseases and their analysis can identify the relationships between these diseases. Identification of damaged genes from various diseases can be done through the discovery of cell-to-cell biological communications. Also, extraction of intercellular communications can identify relationships between different diseases. For example, gene disorders that cause damage to the same cells in both breast and blood cancers. Hence, the purpose is to discover cell-to-cell biological communications in gene expression data. The identification of cell-to-cell biological communications for various cancer diseases has been widely performed by clustering algorithms. However, this field remains open due to the abundance of unprocessed gene expression data. Accordingly, this paper focuses on the development of a semi-supervised ensemble clustering algorithm that can discover relationships between different diseases through the extraction of cell-to-cell biological communications. The proposed clustering framework includes a stratified feature sampling mechanism and a novel similarity metric to deal with high-dimensional data and improve the diversity of primary partitions. The performance of the proposed clustering algorithm is verified with several datasets from the UCI machine learning repository and then applied to the FANTOM5 dataset to extract cell-to-cell biological communications. The used version of this dataset contains 108 cells and 86,427 promoters from 702 samples. The strength of communication between two similar cells from different diseases indicates the relationship of those diseases. Here, the strength of communication is determined by promoter, so we found the highest cell-to-cell biological communication between "basophils" and "ciliary.epithelial.cells" with 62,809 promoters. The maximum cell-to-cell biological similarity in each cluster can be used to detect the relationship between different diseases such as cancer.

A cooperative spectrum sensing method based on semi-supervised clustering with variational mode decomposition and information geometry

In practice, obtaining a large number of sample labels for the supervised learning-based cooperative spectrum sensing (CSS) method is challenging. Additionally, the CSS method based on information geometry (IG) and clustering algorithm exhibited low detection performance in low signal-to-noise ratio (SNR) environment. This paper proposes a cooperative spectrum sensing method based on variational mode decomposition and IG with semi-supervised clustering. First, the secondary users (SUs) reconstruct the received signals using variational mode decomposition. Next, the fusion center combines all the reconstructed signals into a single matrix and calculates its corresponding covariance matrix. Finally, a binary classifier is obtained via training offline on the matrix manifold space using a semi-supervised clustering algorithm based on IG. The simulation results show that increasing the number of SUs and sampling points can improve the CSS performance of the proposed algorithm. Furthermore, the proposed algorithm is compared with other CSS algorithms, demonstrating its superior spectrum sensing performance regardless of whether the SUs are in the same very low SNR environment.

Semi-supervised fuzzy clustering algorithm based on prior membership degree matrix with expert preference

Existing pre-processing methods for the prior membership degree matrix suffer from the following issues: (1) The labeling constraints for prior membership degree matrix have an effect on the expert’s judgment on the prior membership degree, which easily causes the distortion problem of the prior membership degree labeling information; (2) There exists the problem of inconsistency between the filling information and the labeling information in the prior membership degree matrix to be filled in the missing values with zeros. To address these problems, we propose an unconstrained labeling idea for the prior membership degree matrix and the corresponding pre-processing method for the missing values by introducing the statistical characteristics of extreme value distribution and simultaneously apply it to the semi-supervised fuzzy clustering algorithm. More specifically, we focus on learning an expert preference value from the prior membership degree matrix and filling in the missing values with the expert preference value. Thus, we propose an unconstrained pre-processing method for the prior membership degree matrix by filling in missing values with an expert preference to keep the filling information consistent with the labeling information in the prior membership degree matrix as much as possible. In addition, we design a semi-supervised fuzzy clustering algorithm based on an unconstrained prior membership degree matrix with expert preference (SFCM-EP) by introducing the K-L divergence to improve the applicability, utility and running performance of semi-supervised fuzzy clustering algorithm. Our experimental results on the simulation dataset and the UCI datasets show the feasibility and effectiveness of the proposed pre-processing method of the prior membership degree matrix with encouraging results.

Nonsmooth Optimization-Based Model and Algorithm for Semisupervised Clustering.

Using a nonconvex nonsmooth optimization approach, we introduce a model for semisupervised clustering (SSC) with pairwise constraints. In this model, the objective function is represented as a sum of three terms: the first term reflects the clustering error for unlabeled data points, the second term expresses the error for data points with must-link (ML) constraints, and the third term represents the error for data points with cannot-link (CL) constraints. This function is nonconvex and nonsmooth. To find its optimal solutions, we introduce an adaptive SSC (A-SSC) algorithm. This algorithm is based on the combination of the nonsmooth optimization method and an incremental approach, which involves the auxiliary SSC problem. The algorithm constructs clusters incrementally starting from one cluster and gradually adding one cluster center at each iteration. The solutions to the auxiliary SSC problem are utilized as starting points for solving the nonconvex SSC problem. The discrete gradient method (DGM) of nonsmooth optimization is applied to solve the underlying nonsmooth optimization problems. This method does not require subgradient evaluations and uses only function values. The performance of the A-SSC algorithm is evaluated and compared with four benchmarking SSC algorithms on one synthetic and 12 real-world datasets. Results demonstrate that the proposed algorithm outperforms the other four algorithms in identifying compact and well-separated clusters while satisfying most constraints.

Application of Semi-supervised Fuzzy Clustering Based on Knowledge Weighting and Cluster Center Learning to Mammary Molybdenum Target Image Segmentation.

Breast cancer is commonly diagnosed with mammography. Using image segmentation algorithms to separate lesion areas in mammography can facilitate diagnosis by doctors and reduce their workload, which has important clinical significance. Because large, accurately labeled medical image datasets are difficult to obtain, traditional clustering algorithms are widely used in medical image segmentation as an unsupervised model. Traditional unsupervised clustering algorithms have limited learning knowledge. Moreover, some semi-supervised fuzzy clustering algorithms cannot fully mine the information of labeled samples, which results in insufficient supervision. When faced with complex mammography images, the above algorithms cannot accurately segment lesion areas. To address this, a semi-supervised fuzzy clustering based on knowledge weighting and cluster center learning (WSFCM_V) is presented. According to prior knowledge, three learning modes are proposed: a knowledge weighting method for cluster centers, Euclidean distance weights for unlabeled samples, and learning from the cluster centers of labeled sample sets. These strategies improve the clustering performance. On real breast molybdenum target images, the WSFCM_V algorithm is compared with currently popular semi-supervised and unsupervised clustering algorithms. WSFCM_V has the best evaluation index values. Experimental results demonstrate that compared with the existing clustering algorithms, WSFCM_V has a higher segmentation accuracy than other clustering algorithms, both for larger lesion regions like tumor areas and for smaller lesion areas like calcification point areas.

Medoid based semi-supervised fuzzy clustering algorithms for multi-view relational data

In this paper we present two novel families of semi-supervised multi-view clustering algorithms for relational data, providing relevance weights for views and cluster representatives. The families differ by their cluster representative approach: set-medoids and vector set-medoids. Each family possesses four models, referred to as variants, differing by their weighting approach and the constraints imposed on those weights. Semi-supervision is attained through the use of must-link and cannot-link pairwise constraints. The algorithms perform the clustering task through the minimisation of an objective function; such minimisation is achieved through the iteration of three steps until convergence, namely: search for the best representatives, computation of the best relevance weights, and computation of the best fuzzy partition. Experiments performed on several real datasets show that those algorithms have greater robustness than other semi-supervised multi-view fuzzy clustering algorithms when handling the pairwise constraints and provide better interpretability of the results. Moreover, those experiments reveal that one family may perform better than the other depending on the data being considered; this can also be observed for the variants within each family. Furthermore, the proposed algorithms were tested in a real multi-view dataset where a synthetic noise view was added; it was observed that those algorithms managed to maintain a good level of quality in the resulting partition as well as discriminating the noise view by assigning it a lower relevance weight.

An efficient semi-supervised balanced cut with hard pairwise constraints and partial labels

With a small scale of prior information, including pairwise constraints and partial labels, semi-supervised clustering algorithms can improve clustering performance by imposing certain restrictions on the clustering results, and have accordingly attracted a lot of attention in recent years. Moreover, a large number of graph-based semi-supervised clustering algorithms have been proposed, since graphs are naturally suited to pairwise constraints. However, most graph-based approaches involve a two-step process and are not scalable to large-scale datasets due to their high time complexity. Additionally, they also encounter a serious problem of constraint violation. To deal with this issue, we propose a new non-parametric model to directly obtain the indicator matrix by minimizing the cut of the graph with a regularization term that corresponds to the balance of the clusters. To simultaneously support the two forms of prior information, our approach restricts the indicator matrix during optimization, while also ensuring that the clusters obtained by our approach do not violate any constraints. The computational complexity of the proposed approach is nearly linear to the data size n. Experiments show that the algorithm proposed in this paper achieves significant improvements in both clustering performance and efficiency.

Semi-supervised hierarchical ensemble clustering based on an innovative distance metric and constraint information

Agglomerative Hierarchical Clustering (AHC) is a bottom-up clustering strategy in which each object is originally a cluster, and more pairs of clusters are formed by traversing the hierarchy. It has been proven that there is no individual AHC clustering algorithm that can be efficient in all situations. In order to address this problem, ensemble clustering techniques have been introduced. These techniques combine the results of several output partitions to achieve a consensus with higher accuracy compared to an individual clustering algorithm. This paper proposes an AHC-based ensemble semi-supervised clustering algorithm to improve performance. In semi-supervised clustering, class membership information is used in some objects. Here, we introduce the Semi-Supervised Ensemble Hierarchical Clustering based on Constraints Information (SSEHCCI) algorithm. SSEHCCI is developed using several individual clustering algorithms based on AHC. SSEHCCI includes a flexible weighting policy to generate base partitions and uses the constraints information to configure the semi-supervised clustering. In addition, SSEHCCI uses an innovative distance measure to calculate the distance between each pair of objects. Experimental results show that SSEHCCI performs better than existing semi-supervised algorithms on some University of California Irvine (UCI) datasets. Specifically, we observed an average accuracy of SSEHCCI compared to SSDC and RSSC of 2.6% and 1.8%, respectively.

Segmentation of breast molybdenum target image lesions based on semi-supervised fuzzy clustering

Currently, breast cancer is one of the most common cancers among women. To aid clinicians in diagnosis, lesion regions in mammography pictures can be segmented using an artificial intelligence system. This has significant clinical implications. Clustering algorithms, as unsupervised models, are widely used in medical image segmentation. However, due to the different sizes and shapes of lesions in mammography images and the low contrast between lesion areas and the surrounding pixels, it is difficult to use traditional unsupervised clustering methods for image segmentation. In this study, we try to apply the semisupervised fuzzy clustering algorithm to lesion segmentation in mammography molybdenum target images and propose semisupervised fuzzy clustering based on the cluster centres of labelled samples (called SFCM_V, where V stands for cluster centre). The algorithm refers to the cluster centre of the labelled sample dataset during the clustering process and uses the information of the labelled samples to guide the unlabelled samples during clustering to improve the clustering performance. We compare the SFCM_V algorithm with the current popular semisupervised clustering algorithm and an unsupervised clustering algorithm and perform experiments on real patient mammogram images using DICE and IoU as evaluation metrics; SFCM_V has the highest evaluation metric coefficient. Experiments demonstrate that SFCM_V has higher segmentation accuracy not only for larger lesion regions, such as tumours, but also for smaller lesion regions, such as calcified spots, compared with existing clustering algorithms.

Consumer-feeder relationship identification method for low-voltage distribution station area considering feature fusion of voltage variation and spatial distribution

Consumer-feeder relationship is an important basis for the fault location, outage management, load adjustment and power quality management in low-voltage distribution station area (LVDSA). The current identification method of the consumer-feeder relationship based on electrical characteristics analysis is single and lack of reliability. Besides, the consumers of different feeders which locate near the bus at the low voltage side of distribution transformer have high voltage variation similarity, which makes it difficult to distinguish reliably. The spatial distribution of consumers can directly reflects the feeder trend. Therefore, a consumer-feeder relationship identification method for LVDSA considering feature fusion of voltage variation and spatial distribution is proposed in this paper. Firstly, the electrical characteristics based on voltage variation and spatial distribution characteristics based on geographical location are analyzed, and the correlation between the characteristics above and consumer-feeder relationship is explained. Secondly, a semi-supervised spectral clustering (SSC) algorithm based on the feature of voltage variation and spatial distribution is proposed to identify the consumer-feeder relationship. Finally, the proposed method is tested with the real data of LVDSAs in China. The results indicate that the feature fusion of voltage variation and spatial distribution can effectively improve the identification accuracy of consumer-feeder relationship.

Stratification-based semi-supervised clustering algorithm for arbitrary shaped datasets

Semi-supervised clustering is not only an important branch of semi-supervised learning but also an improvement direction for clustering. Semi-supervised clustering algorithms designed based on Kmeans, such as the classical Seeded-Kmeans and Constrained-Kmeans, where supervision information is used to guide clustering iterations, have the same disadvantages as the original Kmeans algorithm: they are confined to the assumption of isotropic spherical clusters, leading to the narrow adaptability in handling data of various characteristics. To solve the problem, we propose the scattered centroids initialization clustering algorithm based on Stratification (SCICS). First, based on the concept of influence space, a method for modeling the cluster-level location of any object is presented, according to which we can obtain well-defined cluster decision boundaries through stratification. On this basis, by extending the seed thought, we propose a semi-supervised subclustering algorithm that can break through the limitations of partitional clustering methods that rely on strict assumptions on particular cluster distributions. Experiments on artificial and real-world datasets show that the proposed algorithm gains the ability of clustering arbitrary shaped data and surpasses the competitors in terms of performance and adaptability.