Semi-supervised Clustering Algorithm Research Articles

Nonsmooth Optimization-Based Model and Algorithm for Semisupervised Clustering.

Using a nonconvex nonsmooth optimization approach, we introduce a model for semisupervised clustering (SSC) with pairwise constraints. In this model, the objective function is represented as a sum of three terms: the first term reflects the clustering error for unlabeled data points, the second term expresses the error for data points with must-link (ML) constraints, and the third term represents the error for data points with cannot-link (CL) constraints. This function is nonconvex and nonsmooth. To find its optimal solutions, we introduce an adaptive SSC (A-SSC) algorithm. This algorithm is based on the combination of the nonsmooth optimization method and an incremental approach, which involves the auxiliary SSC problem. The algorithm constructs clusters incrementally starting from one cluster and gradually adding one cluster center at each iteration. The solutions to the auxiliary SSC problem are utilized as starting points for solving the nonconvex SSC problem. The discrete gradient method (DGM) of nonsmooth optimization is applied to solve the underlying nonsmooth optimization problems. This method does not require subgradient evaluations and uses only function values. The performance of the A-SSC algorithm is evaluated and compared with four benchmarking SSC algorithms on one synthetic and 12 real-world datasets. Results demonstrate that the proposed algorithm outperforms the other four algorithms in identifying compact and well-separated clusters while satisfying most constraints.

Application of Semi-supervised Fuzzy Clustering Based on Knowledge Weighting and Cluster Center Learning to Mammary Molybdenum Target Image Segmentation.

Breast cancer is commonly diagnosed with mammography. Using image segmentation algorithms to separate lesion areas in mammography can facilitate diagnosis by doctors and reduce their workload, which has important clinical significance. Because large, accurately labeled medical image datasets are difficult to obtain, traditional clustering algorithms are widely used in medical image segmentation as an unsupervised model. Traditional unsupervised clustering algorithms have limited learning knowledge. Moreover, some semi-supervised fuzzy clustering algorithms cannot fully mine the information of labeled samples, which results in insufficient supervision. When faced with complex mammography images, the above algorithms cannot accurately segment lesion areas. To address this, a semi-supervised fuzzy clustering based on knowledge weighting and cluster center learning (WSFCM_V) is presented. According to prior knowledge, three learning modes are proposed: a knowledge weighting method for cluster centers, Euclidean distance weights for unlabeled samples, and learning from the cluster centers of labeled sample sets. These strategies improve the clustering performance. On real breast molybdenum target images, the WSFCM_V algorithm is compared with currently popular semi-supervised and unsupervised clustering algorithms. WSFCM_V has the best evaluation index values. Experimental results demonstrate that compared with the existing clustering algorithms, WSFCM_V has a higher segmentation accuracy than other clustering algorithms, both for larger lesion regions like tumor areas and for smaller lesion areas like calcification point areas.

Medoid based semi-supervised fuzzy clustering algorithms for multi-view relational data

In this paper we present two novel families of semi-supervised multi-view clustering algorithms for relational data, providing relevance weights for views and cluster representatives. The families differ by their cluster representative approach: set-medoids and vector set-medoids. Each family possesses four models, referred to as variants, differing by their weighting approach and the constraints imposed on those weights. Semi-supervision is attained through the use of must-link and cannot-link pairwise constraints. The algorithms perform the clustering task through the minimisation of an objective function; such minimisation is achieved through the iteration of three steps until convergence, namely: search for the best representatives, computation of the best relevance weights, and computation of the best fuzzy partition. Experiments performed on several real datasets show that those algorithms have greater robustness than other semi-supervised multi-view fuzzy clustering algorithms when handling the pairwise constraints and provide better interpretability of the results. Moreover, those experiments reveal that one family may perform better than the other depending on the data being considered; this can also be observed for the variants within each family. Furthermore, the proposed algorithms were tested in a real multi-view dataset where a synthetic noise view was added; it was observed that those algorithms managed to maintain a good level of quality in the resulting partition as well as discriminating the noise view by assigning it a lower relevance weight.

An efficient semi-supervised balanced cut with hard pairwise constraints and partial labels

With a small scale of prior information, including pairwise constraints and partial labels, semi-supervised clustering algorithms can improve clustering performance by imposing certain restrictions on the clustering results, and have accordingly attracted a lot of attention in recent years. Moreover, a large number of graph-based semi-supervised clustering algorithms have been proposed, since graphs are naturally suited to pairwise constraints. However, most graph-based approaches involve a two-step process and are not scalable to large-scale datasets due to their high time complexity. Additionally, they also encounter a serious problem of constraint violation. To deal with this issue, we propose a new non-parametric model to directly obtain the indicator matrix by minimizing the cut of the graph with a regularization term that corresponds to the balance of the clusters. To simultaneously support the two forms of prior information, our approach restricts the indicator matrix during optimization, while also ensuring that the clusters obtained by our approach do not violate any constraints. The computational complexity of the proposed approach is nearly linear to the data size n. Experiments show that the algorithm proposed in this paper achieves significant improvements in both clustering performance and efficiency.

Semi-supervised hierarchical ensemble clustering based on an innovative distance metric and constraint information

Agglomerative Hierarchical Clustering (AHC) is a bottom-up clustering strategy in which each object is originally a cluster, and more pairs of clusters are formed by traversing the hierarchy. It has been proven that there is no individual AHC clustering algorithm that can be efficient in all situations. In order to address this problem, ensemble clustering techniques have been introduced. These techniques combine the results of several output partitions to achieve a consensus with higher accuracy compared to an individual clustering algorithm. This paper proposes an AHC-based ensemble semi-supervised clustering algorithm to improve performance. In semi-supervised clustering, class membership information is used in some objects. Here, we introduce the Semi-Supervised Ensemble Hierarchical Clustering based on Constraints Information (SSEHCCI) algorithm. SSEHCCI is developed using several individual clustering algorithms based on AHC. SSEHCCI includes a flexible weighting policy to generate base partitions and uses the constraints information to configure the semi-supervised clustering. In addition, SSEHCCI uses an innovative distance measure to calculate the distance between each pair of objects. Experimental results show that SSEHCCI performs better than existing semi-supervised algorithms on some University of California Irvine (UCI) datasets. Specifically, we observed an average accuracy of SSEHCCI compared to SSDC and RSSC of 2.6% and 1.8%, respectively.

Segmentation of breast molybdenum target image lesions based on semi-supervised fuzzy clustering

Currently, breast cancer is one of the most common cancers among women. To aid clinicians in diagnosis, lesion regions in mammography pictures can be segmented using an artificial intelligence system. This has significant clinical implications. Clustering algorithms, as unsupervised models, are widely used in medical image segmentation. However, due to the different sizes and shapes of lesions in mammography images and the low contrast between lesion areas and the surrounding pixels, it is difficult to use traditional unsupervised clustering methods for image segmentation. In this study, we try to apply the semisupervised fuzzy clustering algorithm to lesion segmentation in mammography molybdenum target images and propose semisupervised fuzzy clustering based on the cluster centres of labelled samples (called SFCM_V, where V stands for cluster centre). The algorithm refers to the cluster centre of the labelled sample dataset during the clustering process and uses the information of the labelled samples to guide the unlabelled samples during clustering to improve the clustering performance. We compare the SFCM_V algorithm with the current popular semisupervised clustering algorithm and an unsupervised clustering algorithm and perform experiments on real patient mammogram images using DICE and IoU as evaluation metrics; SFCM_V has the highest evaluation metric coefficient. Experiments demonstrate that SFCM_V has higher segmentation accuracy not only for larger lesion regions, such as tumours, but also for smaller lesion regions, such as calcified spots, compared with existing clustering algorithms.

Consumer-feeder relationship identification method for low-voltage distribution station area considering feature fusion of voltage variation and spatial distribution

Consumer-feeder relationship is an important basis for the fault location, outage management, load adjustment and power quality management in low-voltage distribution station area (LVDSA). The current identification method of the consumer-feeder relationship based on electrical characteristics analysis is single and lack of reliability. Besides, the consumers of different feeders which locate near the bus at the low voltage side of distribution transformer have high voltage variation similarity, which makes it difficult to distinguish reliably. The spatial distribution of consumers can directly reflects the feeder trend. Therefore, a consumer-feeder relationship identification method for LVDSA considering feature fusion of voltage variation and spatial distribution is proposed in this paper. Firstly, the electrical characteristics based on voltage variation and spatial distribution characteristics based on geographical location are analyzed, and the correlation between the characteristics above and consumer-feeder relationship is explained. Secondly, a semi-supervised spectral clustering (SSC) algorithm based on the feature of voltage variation and spatial distribution is proposed to identify the consumer-feeder relationship. Finally, the proposed method is tested with the real data of LVDSAs in China. The results indicate that the feature fusion of voltage variation and spatial distribution can effectively improve the identification accuracy of consumer-feeder relationship.

Stratification-based semi-supervised clustering algorithm for arbitrary shaped datasets

Semi-supervised clustering is not only an important branch of semi-supervised learning but also an improvement direction for clustering. Semi-supervised clustering algorithms designed based on Kmeans, such as the classical Seeded-Kmeans and Constrained-Kmeans, where supervision information is used to guide clustering iterations, have the same disadvantages as the original Kmeans algorithm: they are confined to the assumption of isotropic spherical clusters, leading to the narrow adaptability in handling data of various characteristics. To solve the problem, we propose the scattered centroids initialization clustering algorithm based on Stratification (SCICS). First, based on the concept of influence space, a method for modeling the cluster-level location of any object is presented, according to which we can obtain well-defined cluster decision boundaries through stratification. On this basis, by extending the seed thought, we propose a semi-supervised subclustering algorithm that can break through the limitations of partitional clustering methods that rely on strict assumptions on particular cluster distributions. Experiments on artificial and real-world datasets show that the proposed algorithm gains the ability of clustering arbitrary shaped data and surpasses the competitors in terms of performance and adaptability.

Safe semi-supervised clustering based on Dempster–Shafer evidence theory

In this paper, we propose a safe semi-supervised clustering algorithm based on Dempster–Shafer (D–S) evidence theory. The motivation is that D–S evidence theory can be used to fuse multiple base clustering results and obtain robust confidence estimations of mislabeled samples. Firstly, the proposed algorithm constructs multiple base clusters using fuzzy c-means and kernel fuzzy c-means. Base clusters with good performance are selected according to a clustering validity function. Then, D–S evidence theory is used to fuse the results of the selected base clusters, and the confidence of labeled samples is calculated based on the fused results. Finally, we construct a p-nearest neighbor graph to limit the outputs of labeled samples with low confidence to be those of the p nearest unlabeled samples. It is desired to reduce the negative influence of labeled samples with low confidence and achieve safe exploitation. To verify the effectiveness of the proposed algorithm, we compare it to several unsupervised and semi-supervised clustering algorithms. The results demonstrate that our algorithm yields higher accuracy and is more stable.

MSC-CSMC: A multi-objective semi-supervised clustering algorithm based on constraints selection and multi-source constraints for gene expression data.

Many clustering techniques have been proposed to group genes based on gene expression data. Among these methods, semi-supervised clustering techniques aim to improve clustering performance by incorporating supervisory information in the form of pairwise constraints. However, noisy constraints inevitably exist in the constraint set obtained on the practical unlabeled dataset, which degenerates the performance of semi-supervised clustering. Moreover, multiple information sources are not integrated into multi-source constraints to improve clustering quality. To this end, the research proposes a new multi-objective semi-supervised clustering algorithm based on constraints selection and multi-source constraints (MSC-CSMC) for unlabeled gene expression data. The proposed method first uses the gene expression data and the gene ontology (GO) that describes gene annotation information to form multi-source constraints. Then, the multi-source constraints are applied to the clustering by improving the constraint violation penalty weight in the semi-supervised clustering objective function. Furthermore, the constraints selection and cluster prototypes are put into the multi-objective evolutionary framework by adopting a mixed chromosome encoding strategy, which can select pairwise constraints suitable for clustering tasks through synergistic optimization to reduce the negative influence of noisy constraints. The proposed MSC-CSMC algorithm is testified using five benchmark gene expression datasets, and the results show that the proposed algorithm achieves superior performance.

A semi-supervised resampling method for class-imbalanced learning

Clustering analysis is widely used as a pre-process to discover the data distribution for resampling. Existing clustering-based resampling methods mostly run unsupervised clustering on labeled data without taking advantage of the class information to guide the clustering. When there are not enough labeled data, the clustering can hardly capture the underlying data distribution. In this paper, we propose a semi-supervised hybrid resampling (SSHR) method which runs semi-supervised clustering to capture the data distribution for both over-sampling and under-sampling. Firstly, we design a semi-supervised hierarchical clustering algorithm (SSHC) which uses labeled data to guide the clustering procedure on the whole dataset. Specifically, labeled data are used to initialize a clustering model and then guide its updating via an iterative cluster-splitting process. In this way, original classes are divided into multiple disjunct clusters, which contributes o disclosing not only the inter-class imbalance but also the intra-class imbalance. Subsequently, a hybrid resampling is performed according to the result of SSHC Labeled data of the majority class are under-sampled according to the distances to their cluster centroids and the adjacency to minority cluster centroids. Furthermore, we propose a novel over-sampling approach which selects some confident unlabeled data in minority clusters as pseudo-labeled data to enlarge the training set Compared with traditional over-sampling methods, our approach contributes to discovering more about the distribution of the minority class. In order to validate the effectiveness of SSHR, we conduct extensive experiments on 44 benchmark datasets. Our method achieves the best performances in terms of both F-measure and AUC. The Friedman test demonstrates that SSHR significantly outperforms the compared state-of-the-art resampling algorithms.

A Compressed Sensing Based Least Squares Approach to Semi-supervised Local Cluster Extraction

A least squares semi-supervised local clustering algorithm based on the idea of compressed sensing is proposed to extract clusters from a graph with known adjacency matrix. The algorithm is based on a two-stage approach similar to the one proposed by Lai and Mckenzie (SIAM J Math Data Sci 2:368–395, 2020). However, under a weaker assumption and with less computational complexity, our algorithms are shown to be able to find a desired cluster with high probability. The “one cluster at a time" feature of our method distinguishes it from other global clustering methods. Numerical experiments are conducted on the synthetic data such as stochastic block model and real data such as MNIST, political blogs network, AT &T and YaleB human faces data sets to demonstrate the effectiveness and efficiency of our algorithms.

Semi-Supervised Clustering Algorithm Based on Deep Feature Mapping

Clustering analysis is one of the main concerns in data mining. A common approach to the clustering process is to bring together points that are close to each other and separate points that are away from each other. Therefore, measuring the distance between sample points is crucial to the effectiveness of clustering. Filtering features by label information and measuring the distance between samples by these features is a common supervised learning method to reconstruct distance metric. However, in many application scenarios, it is very expensive to obtain a large number of labeled samples. In this paper, to solve the clustering problem in the few supervised sample and high data dimensionality scenarios, a novel semi-supervised clustering algorithm is proposed by designing an improved prototype network that attempts to reconstruct the distance metric in the sample space with a small amount of pairwise supervised information, such as Must-Link and Cannot-Link, and then cluster the data in the new metric space. The core idea is to make the similar ones closer and the dissimilar ones further away through embedding mapping. Extensive experiments on both real-world and synthetic datasets show the effectiveness of this algorithm. Average clustering metrics on various datasets improved by 8% compared to the comparison algorithm.

Semi-Supervised Clustering Ensemble Based on Cluster Consensus Selection

Ensemble clustering emerged as an important extension of classical clustering problems and is one of the most recent advances in unsupervised learning. Its purpose is to combine the results obtained using different algorithms by a consensus function so that the final solution is more favorable than the individual clustering algorithms. In this study, we propose a semi-supervised clustering ensemble framework using cluster consensus selection, which tries to improve the accuracy of clustering results. In general, there are two types of semi-supervised clustering algorithms, including constraint-based and metric-based. Here, the proposed ensemble clustering algorithm is equipped with a semi-supervised clustering mechanism based on pairwise constraints. Since the complexity of consensus functions scales with the number of clustering methods, processing big data for ensemble clustering is sometimes slow or impossible. Usually, all primary clusters from all clustering methods are used in the consensus function. However, the merit of clusters from different methods can be considered to improve the consensus quality. Accordingly, we propose a cluster consensus selection approach that selects a subset of meriting primary clusters to participate in the final consensus. Here, Normalized Mutual Information (NMI) is developed to measure the merit of clusters. Meanwhile, reducing the number of primary clusters in the consensus function can enable big data clustering. The proposed algorithm is very computationally efficient and provides linear complexity in clustering. Experimental results show the effectiveness of the proposed algorithm in terms of different performance metrics such as NMI, ARI and CPCC.

LFSC: A linear fast semi-supervised clustering algorithm that integrates reference-bulk and single-cell transcriptomes.

The identification of cell types in complex tissues is an important step in research into cellular heterogeneity in disease. We present a linear fast semi-supervised clustering (LFSC) algorithm that utilizes reference samples generated from bulk RNA sequencing data to identify cell types from single-cell transcriptomes. An anchor graph is constructed to depict the relationship between reference samples and cells. By applying a connectivity constraint to the learned graph, LFSC enables the preservation of the underlying cluster structure. Moreover, the overall complexity of LFSC is linear to the size of the data, which greatly improves effectiveness and efficiency. By applying LFSC to real single-cell RNA sequencing datasets, we discovered that it has superior performance over existing baseline methods in clustering accuracy and robustness. An application using infiltrating T cells in liver cancer demonstrates that LFSC can successfully find new cell types, discover differently expressed genes, and explore new cancer-associated biomarkers.

Semi-Supervised Clustering via Cannot Link Relationship for Multiview Data

Due to the diversity of data modalities, the research interest of multi-view clustering is gradually increasing, in the field of large-data analytics, particularly in clustering. However, the greater part of current multi-view clustering methods is mainly in view of unsupervised learning, which leads to unpredictable results and algorithmic instability. Besides, they ignore the diversity of graphs, which is not desirable in practical applications, because the characteristic properties of each view are different. To solve these problems, inspired by the outstanding performance of semi-supervised learning in machine learning, we propose a valid semi-supervised multi-view spectral clustering algorithm. We use the pre-set labels as prior knowledge to obtain the overall distribution of the remaining unlabeled data. Tensor minimization Schatten <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$p$ </tex-math></inline-formula> -norm is utilized to mine the mutual information hidden in multiple views. Meanwhile, we also use the cannot-link as another semi-supervised constraint to update the graph. Our proposed algorithm is generally 5%-10% better than the comparison algorithms in view of the experimental results on five datasets, and our algorithm is relatively fast with the computational complexity of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mathcal {O}({T({n^{2}}\log (n) + {n^{2}} + {u^{2}}l + ulc + uc\log (c))})$ </tex-math></inline-formula> , where <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$T$ </tex-math></inline-formula> denotes the number of iterations and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$n$ </tex-math></inline-formula> , <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$l$ </tex-math></inline-formula> , <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$u$ </tex-math></inline-formula> represent the number of samples, the number of labeled and unlabeled samples, respectively, which shows that our proposed method has broad application prospects.

Intelligent fault diagnosis of rolling bearings based on clustering algorithm of fast search and find of density peaks

Rolling bearings based rotating machinery are widely used in various industrial applications. The failure of rolling bearings, as one of the most critical components, would lead to disastrous consequences to the machinery. Therefore, it’s paramount to deliver an effective intelligent fault diagnosis method for rolling bearings to ensure the machinery’s stability and reliability. With this aim, this article proposes a novel approach that features are extracted via an improved complete ensemble empirical mode decomposition with adaptive noise (CEEMDAN) and the faults are identified based on a semi-supervised clustering algorithm, that is, clustering approach of fast search and discovery of density peaks (CFSFDP). The proposed method provides two main contributions: (1) highly representative important features may be derived from common high-dimensional features, and (2) the intelligent semi-supervised classifier can identify faults type adaptively without large amount of type-labelled data unlike other supervised classifiers. Benchmarking studies were carried out to indicate that the proposed methodology for the fault diagnostic is superior to other common-used approaches.

A label-guided weighted semi-supervised neutrosophic clustering algorithm

The traditional neutrosophic clustering method only performs cluster analysis on the data itself, and often ignores the supervision information of data. In order to solve the above problems, a label-guided weighted semi-supervised neutrosophic clustering algorithm is proposed in the paper. On the one hand, the paired constraint information is used to construct the supervision weight coefficient and the distance measurement learning is combined to re-measure the degree of membership of the data and the cluster center; On the other hand, by minimizing the sum of squares of error between membership matrix and label matrix, the purpose of clustering results guided by label information is realized. Experiments on various data sets and comparisons with other clustering algorithms show that the new clustering algorithm can make full use of supervisory information and improve the accuracy of clustering.

Transdiagnostic inflammatory subgroups among psychiatric disorders and their relevance to role functioning: a nested case-control study of the ALSPAC cohort

Individuals with psychotic disorders and depressive disorder exhibit altered concentrations of peripheral inflammatory markers. It has been suggested that clinical trials of anti-inflammatory therapies for psychiatric disorders should stratify patients by their inflammatory profile. Hence, we investigated whether different subgroups of individuals exist across psychiatric disorders, based on their inflammatory biomarker signatures. We measured the plasma concentrations of 17 inflammatory markers and receptors in 380 participants with psychotic disorder, depressive disorder or generalised anxiety disorder and 399 controls without psychiatric symptoms from the ALSPAC cohort at age 24. We employed a semi-supervised clustering algorithm, which discriminates multiple clusters of psychiatric disorder cases from controls. The best fit was for a two-cluster model of participants with psychiatric disorders (Adjusted Rand Index (ARI) = 0.52 ± 0.01) based on the inflammatory markers. Permutation analysis indicated the stability of the clustering solution performed better than chance (ARI = 0.43 ± 0.11; p < 0.001), and the clusters explained the inflammatory marker data better than a Gaussian distribution (p = 0.021). Cluster 2 exhibited marked increases in sTNFR1/2, suPAR, sCD93 and sIL-2RA, compared to cluster 1. Participants in the cluster exhibiting higher inflammation were less likely to be in employment, education or training, indicating poorer role functioning. This study found evidence for a novel pattern of inflammatory markers specific to psychiatric disorders and strongly associated with a transdiagnostic measure of illness severity. sTNFR1/2, suPAR, sCD93 and sIL-2RA could be used to stratify clinical trials of anti-inflammatory therapies for psychiatric disorders.

Application of big data adaptive semi-supervised clustering method based on deep learning

In order to solve the problems of high computational complexity, poor dimensionality reduction, and reduced clustering effect when the clustering task faces a large amount of big data, the application of a large data adaptive semi-supervised clustering method based on deep learning is proposed. Through the self-encoder of the deep clustering network, the analysis of the confrontation network is generated, and the semi-supervised deep clustering algorithm and algorithm of the adaptive strategy are optimized. Through the encoder layer structure of the deep coding network, different parameters are set for all data sets for algorithm experimental analysis. The results show that the data obtained by this method is faster, more accurate and more optimized than the traditional clustering method, which proves the effectiveness of the method.