Semi-supervised Clustering Technique Research Articles

Constrained Density Peak Clustering

Clustering is a commonly used tool for discovering knowledge in data mining. Density peak clustering (DPC) has recently gained attention for its ability to detect clusters with various shapes and noise, using just one parameter. DPC has shown advantages over other methods, such as DBSCAN and K-means, but it struggles with datasets that have both high and low-density clusters. To overcome this limitation, the paper introduces a new semi-supervised DPC method that improves clustering results with a small set of constraints expressed as must-link and cannot-link. The proposed method combines constraints and a k-nearest neighbor graph to filter out peaks and find the center for each cluster. Constraints are also used to support label assignment during the clustering procedure. The efficacy of this method is demonstrated through experiments on well-known data sets from UCI and benchmarked against contemporary semi-supervised clustering techniques.

Knowledge augmentation-based soft constraints for semi-supervised clustering

Cluster analysis discovers natural structures from different perspectives of data objects and has become an effective method in data mining. The emergence of semi-supervised clustering techniques has improved the performance of unsupervised clustering algorithms. Clustering with guidance information is a clustering method variant that uses pairwise constraints based on background knowledge. This method increases the interpretability of the results through a knowledge-guided perspective but simultaneously suffers from the problem of constraint conflict. This paper designs knowledge augmentation-based soft constraints as a new pairwise constraint representation and proposes a Soft Constraints Kmeans (SCop-Kmeans) method to resolve constraint conflicts. By describing constraint knowledge from multiple perspectives, the association strength of pairwise constraints is calculated to obtain the assignment basis of objects. SCop-Kmeans can solve the sample allocation conflict problem caused by the contradiction between different constraints and improve the clustering stability. Finally, experiments are performed using UCI public standard datasets. The proposed method further improves the accuracy of clustering and performs well in experiments with different numbers of constraints, which shows that the proposed method has advantages in using constraint information to guide clustering.

MSC-CSMC: A multi-objective semi-supervised clustering algorithm based on constraints selection and multi-source constraints for gene expression data.

Many clustering techniques have been proposed to group genes based on gene expression data. Among these methods, semi-supervised clustering techniques aim to improve clustering performance by incorporating supervisory information in the form of pairwise constraints. However, noisy constraints inevitably exist in the constraint set obtained on the practical unlabeled dataset, which degenerates the performance of semi-supervised clustering. Moreover, multiple information sources are not integrated into multi-source constraints to improve clustering quality. To this end, the research proposes a new multi-objective semi-supervised clustering algorithm based on constraints selection and multi-source constraints (MSC-CSMC) for unlabeled gene expression data. The proposed method first uses the gene expression data and the gene ontology (GO) that describes gene annotation information to form multi-source constraints. Then, the multi-source constraints are applied to the clustering by improving the constraint violation penalty weight in the semi-supervised clustering objective function. Furthermore, the constraints selection and cluster prototypes are put into the multi-objective evolutionary framework by adopting a mixed chromosome encoding strategy, which can select pairwise constraints suitable for clustering tasks through synergistic optimization to reduce the negative influence of noisy constraints. The proposed MSC-CSMC algorithm is testified using five benchmark gene expression datasets, and the results show that the proposed algorithm achieves superior performance.

Trust Cop-Kmeans Clustering Analysis and Minimum-Cost Consensus Model Considering Voluntary Trust Loss in Social Network Large-Scale Decision-Making

Social network large-scale decision-making (SN-LSDM) has attracted much attention in the field of decision science. Clustering and consensus building are two important processes for solving SN-LSDM problems. Traditional clustering methods are typically based on the opinion similarity. When extending decision-making to social network contexts, we argue that trust relationships among decision-makers (DMs) serve as a reliable resource for clustering. This study proposes a trust Cop-Kmeans clustering algorithm, which is a semisupervised clustering technique using prior knowledge of trust constraints generated by trust relationships. The consensus-reaching process (CRP) is an effective tool for reducing differences of opinion. In general, the costs and resources associated with the CRP are limited. Therefore, minimum-cost consensus (MCC) models have been developed and used widely in various group decision-making contexts. As an important resource influencing decision-making, trust provides a common-sense perception that the opinion of a DM with a high trust degree is considered to be widely recognized by others. We hold that the DM with high trust and low consensus can reduce the consensus cost by voluntarily losing some trust. Consequently, an improved MCC model considering voluntary trust loss is developed. Finally, a numerical example is presented to illustrate the feasibility of the proposed clustering algorithm and consensus model. A comparative analysis is implemented to explore the influence of the trust constraint on clustering and the influence of trust loss on the CRP.

A New semi-supervised clustering for incomplete data

Semi-supervised clustering technique partitions the unlabeled data based on prior knowledge of labeled data. Most of the semi-supervised clustering algorithms exist only for the clustering of complete data, i.e., the data sets with no missing features. In this paper, an effort has been made to check the effectiveness of semi-supervised clustering when applied to incomplete data sets. The novelty of this approach is that it considers the missing features along with available knowledge (labels) of the data set. The linear interpolation imputation technique initially imputes the missing features of the data set, thus completing the data set. A semi-supervised clustering is now employed on this complete data set, and missing features are regularly updated within the clustering process. In the proposed work, the labeled percentage range used is 30, 40, 50, and 60% of the total data. Data is further altered by arbitrarily eliminating certain features of its components, which makes the data incomplete with partial labeling. The proposed algorithm utilizes both labeled and unlabeled data, along with certain missing values in the data. The proposed algorithm is evaluated using three performance indices, namely the misclassification rate, random index metric, and error rate. Despite the additional missing features, the proposed algorithm has been successfully implemented on real data sets and showed better/competing results than well-known standard semi-supervised clustering methods.

Multi-scale semi-supervised clustering of brain images: Deriving disease subtypes.

Disease heterogeneity is a significant obstacle to understanding pathological processes and delivering precision diagnostics and treatment. Clustering methods have gained popularity for stratifying patients into subpopulations (i.e., subtypes) of brain diseases using imaging data. However, unsupervised clustering approaches are often confounded by anatomical and functional variations not related to a disease or pathology of interest. Semi-supervised clustering techniques have been proposed to overcome this and, therefore, capture disease-specific patterns more effectively. An additional limitation of both unsupervised and semi-supervised conventional machine learning methods is that they typically model, learn and infer from data using a basis of feature sets pre-defined at a fixed anatomical or functional scale (e.g., atlas-based regions of interest). Herein we propose a novel method, "Multi-scAle heteroGeneity analysIs and Clustering" (MAGIC), to depict the multi-scale presentation of disease heterogeneity, which builds on a previously proposed semi-supervised clustering method, HYDRA. It derives multi-scale and clinically interpretable feature representations and exploits a double-cyclic optimization procedure to effectively drive identification of inter-scale-consistent disease subtypes. More importantly, to understand the conditions under which the clustering model can estimate true heterogeneity related to diseases, we conducted extensive and systematic semi-simulated experiments to evaluate the proposed method on a sizeable healthy control sample from the UK Biobank (N=4403). We then applied MAGIC to imaging data from Alzheimer's disease (ADNI, N=1728) and schizophrenia (PHENOM, N=1166) patients to demonstrate its potential and challenges in dissecting the neuroanatomical heterogeneity of common brain diseases. Taken together, we aim to provide guidance regarding when such analyses can succeed or should be taken with caution. The code of the proposed method is publicly available at https://github.com/anbai106/MAGIC.

A semi-supervised clustering-based classification model for classifying imbalanced data streams in the presence of scarcely labelled data

Data streams are potentially infinite in length, fast changing and scarcely labelled. It is practically impossible to label all the observed instances. Online frameworks for classifying data streams are generally supervised in nature assuming the availability of labelled data and hence cannot be used for data streams. Semi-supervised learning (SSL) addresses this problem of scarcely labelled data by using large amount of unlabelled data together with labelled data to build classifiers. Data streams may also suffer from the problem of imbalanced data. Previous works in learning from data streams have analysed problems of imbalanced data. But to the best of our knowledge no work has applied semi-supervised learning approaches for classifying imbalanced data streams so far. This paper proposes a model using a semi-supervised clustering technique to classify an imbalanced data stream in the presence of scarcely labelled data. The results prove that the model outperforms many state-of-the-art techniques.

An Intensified System for Identifying Distinct Clusters over the Video Using Improved Semi-Supervised Clustering Technique

The improved video clustering technique called Automatic Video Frame Separation System. The proposed (AVFS) system aims to automatically separate the video frames into finite number of distinct cluster with higher similarity using improved semi-supervised clustering technique. The proposed (AVFS) system contains five stages, in the first stage, the automatic video separation system gets the different input videos with different size from different data store. Next stage, the (AVFS) system converts each and every individual video into sequence of frames using standard video to frame converter tool. The third stage has a proposed (AVFS) system pre-processing the video frames to improve the frame quality and optimize the size of the frames using improved arithmetical operation. Similarly, in the fourth stage the proposed system extract the predetermined distinct number of different feature from each and every individual frame over the individual video using standard statistical operation. Following the previous stage; the fifth stage has the proposed system following the previous stage result and split the feature set of input video frames into distinct number of highly relative clusters using optimum N means technique (ONM). The proposed Automatic Video Frame Separation System is well suitable to robotically separate the video frames into distinct clusters with high intra resemblance and higher inter diversity among the clusters.

A Two-Level Semi-supervised Clustering Technique for News Articles

The web and social media are overcrowded with news pieces in terms of amount and diversity. Document clustering is a useful technique that is widely used in organizing and managing data into smaller groups. One of the factors influencing the quality of clustering is the way documents are represented. Some traditional methods of document representation depend on word frequencies and create sparse and large-sized document vectors. These methods cannot preserve proximity information between documents. In addition, neural network-based methods that preserve proximity information suffer from poor interpretability. Conceptual text representation methods have overcome the shortcomings of previous methods, but semi-supervised text clustering does not currently use concept-based document representation. This paper presents a two-level semi-supervised text clustering method that uses labeled and unlabeled data simultaneously to achieve higher clustering quality. In the first level, documents are represented based on the concepts extracted from the raw corpus. Second, the semi-supervised clustering process applies unlabeled data to capture the overall structure of the clusters and a small amount of labeled data to adjust the center of the clusters. Experiments on the Reuters-21578 data collection show that the proposed model is superior to other semi-supervised approaches in both text classification and text clustering.

An Improved Non-Schedulable Load Forecasting Strategy for Enhancing the Performance of the Energy Management in a Nearly Zero Energy Building

This paper proposes an improved non-schedulable load forecasting (NLF) technique that can be utilized to enhance the performance of the energy management system (EMS) in a nearly zero energy building (nZEB). The suggested NLF is based on a long short-term memory (LSTM) framework in conjunction with a semi-supervised clustering (SSC) technique, considering the most important features which may affect the energy consumption of the non-schedulable appliances (NAs), i.e. number and identity of residents, energy consumption, weather, temperature, humidity, outdoor irradiance, correlation with other loads, day of the week and holidays. The SSC algorithm is utilized to fill the uncompleted information for the residents’ presence in the house and its output constitutes one of the inputs of the LSTM based technique which provides as output a set of forecasting sequences of the NAs’ energy consumption. Unlike the published techniques, the proposed NLF method is not only based on the modeling of the residents’ preferences and habits, but it considers them as variables which affect the nZEB’s microgrid and EMS performance. Therefore, it predicts the residents’ behavior considering its interdependence with the nZEB’s microgrid, which can considerably contribute to the enhancement of the EMS effectiveness and performance. For the implementation of the proposed NLF, no additional hardware is required, but only amendments in the EMS to consider the NLF’s outcomes. To validate the effectiveness of the proposed NLF, selective Hardware-in-the-Loop results from a real nZEB are presented.

Pattern recognition of UAV flight data based on semi-supervised clustering

UAV is an unmanned aerial vehicle controlled by a remote radio signal or a trajectory planning software carried itself. It is widely used in military, civil and scientific research fields. However, due to the lack of real-time decision-making ability, the UAV has high fault rate. The flight quality assessment of UAV and the construction of fault prediction model can be used for debugging and fault-removing to customer’s aircraft, and also to increase the added value of the civilian UAV products. Before building a fault prediction model, a very important step is to identify the pattern of sampled data. For each group of flight data, the efficiency and accuracy rate of manual quality evaluation and fault identification are not acceptable. Based on the UAV flight data accumulated in the big data platform of an UAV production company in Shenyang, Liaoning Province of China, this paper proposes a semi-supervised clustering technique to do automatic pattern recognition for the sampling points. According to the characteristics of UAV flight data, two different methods are designed to choose initial centroids. Meanwhile, we use the existing normal flight data to train distance thresholds to combine some clusters to eliminate the resulting error clustering. Real flight data or flight test data with manually added labels are used to run the proposed algorithms to verify the recognition results. The experimental results show that the proposed methods greatly improve the efficiency and accuracy of adding precise labels to the historical flight data and play a role in assisting the manual recognition of sampling points while strengthening the management and statistics.

Semi-Supervised Deep Fuzzy C-Mean Clustering for Software Fault Prediction

Software fault prediction is a consequential research area in software quality promise. In this paper, we propose a semi-supervised deep fuzzy C-mean (DFCM) clustering for software fault prediction, which is the cumulation of semi-supervised DFCM clustering and feature compression techniques. Deep is utilized for the feature-based multi clusters of unlabeled and labeled data sets along with their labeled classes. In our approach, for the training model, we simultaneously deal with the unsupervised data and supervised data to exploit the obnubilated information from unlabeled data to labeled data to support the construction of the precise model. We utilize DFCM clustering to handle the class imbalance problem and withal fuzzy theory logic is very akin to human logic and it is facile to comprehend. We further ameliorate the prediction performance with the coalescence of feature learning techniques-feature extraction and feature selection (using random-under sampling) to generate good features and remove irrelevant and redundant features to reduce the noisy data for classification. However, by the performance of the model results, the amalgamation of deep multi clusters and feature techniques work better due to their ability to identify and amalgamation essential information in data feature. The classification model is predicted on the maximum homogeneous between the features of labeled and unlabeled data, the model is trained on the un-noisy data set obtained by the deep coalescence of multi clusters and feature techniques. To check the efficacy of our approach, we chose data sets from real-world software project (NASA & Eclipse), and then we compared our approach with a number of latest classical base-line methods, and investigate the performance by using performance measures such as probability of detection, F-measure, and area under the curve.

Building energy modeling (BEM) using clustering algorithms and semi-supervised machine learning approaches

Abstract Energy efficiency is a critical element of building energy conservation. Energy Information Administration (EIA) and International Electrotechnical Commission (IEC) estimated that over 6% of electrical energy was lost during transmission and distribution. Sensing and tracking technologies, and data-mining offer new windows to better understanding these losses in real-time. Recent developments in energy optimization computational methods also allow engineers to better characterize energy consumption load profiles. The paper focuses on developing new and robust data-mining techniques to explore large and complex data generated by sensing and tracking technologies. These techniques would potentially offer new avenues to understand and prevent energy losses during transmission. The paper presents two new concepts: First, a set of clustering algorithms that model the supply-demand characterization of four different substations clusters, and second, a semi-supervised machine learning and clustering technique are developed to optimize the losses and automate the process of identifying loss factors contributing to the total loss. This three-step process uses real-time data from buildings and the substations that supply electricity to the buildings to develop the proposed technique. The preliminary findings of this paper help the utility service providers to understand the energy supply-demand requirements.

Brain image segmentation using semi-supervised clustering

The objective of brain image segmentation is to partition the brain images into different non-overlapping homogeneous regions representing the different anatomical structures. Magnetic resonance brain image segmentation has large number of applications in diagnosis of neurological disorders like Alzheimer diseases, Parkinson related syndrome etc. But automatically segmenting the MR brain image is not an easy task. To solve this problem, several unsupervised and supervised based classification techniques have been developed in the literature. But supervised classification techniques are more time consuming and cost-sensitive due to the requirement of sufficient labeled data. In contrast, unsupervised classification techniques work without using any prior information but it suffers from the local trap problems. So, to overcome the problems associated with unsupervised and supervised classification techniques, we have proposed a new semi-supervised clustering technique using the concepts of multiobjective optimization and applied this technique for automatic segmentation of MR brain images in the intensity space. Multiple centers are used to encode a cluster in the form of a string. The proposed clustering technique utilizes intensity values of the brain pixels as the features. Additionally it also assumes that the actual class label information of 10% points of a particular image data set is also known. Three cluster validity indices are utilized as the objective functions, which are simultaneously optimized using AMOSA, a modern multiobjective optimization technique based on the concepts of simulated annealing. First two cluster validity indices are symmetry distance based Sym-index and Euclidean distance based I-index, which are based on unsupervised properties. Last one is a supervised information based cluster validity index, Minkowski Index. The effectiveness of this proposed semi-supervised clustering technique is demonstrated on several simulated MR normal brain images and MR brain images having some multiple sclerosis lesions. The performance of the proposed semi-supervised clustering technique is compared with some other popular image segmentation techniques like Fuzzy C-means, Expectation Maximization and some recent image clustering techniques like multi-objective based MCMOClust technique, and Fuzzy-VGAPS clustering techniques.

ClusPath: a temporal-driven clustering to infer typical evolution paths

We propose ClusPath, a novel algorithm for detecting general evolution tendencies in a population of entities. We show how abstract notions, such as the Swedish socio-economical model (in a political dataset) or the companies fiscal optimization (in an economical dataset) can be inferred from low-level descriptive features. Such high-level regularities in the evolution of entities are detected by combining spatial and temporal features into a spatio-temporal dissimilarity measure and using semi-supervised clustering techniques. The relations between the evolution phases are modeled using a graph structure, inferred simultaneously with the partition, by using a "slow changing world" assumption. The idea is to ensure a smooth passage for entities along their evolution paths, which catches the long-term trends in the dataset. Additionally, we also provide a method, based on an evolutionary algorithm, to tune the parameters of ClusPath to new, unseen datasets. This method assesses the fitness of a solution using four opposed quality measures and proposes a balanced compromise.

Multi-objective semi-supervised clustering of tissue samples for cancer diagnosis

In the domain of bioinformatics, the clustering of gene expression profiles of different tissue samples over different experimental conditions has gained importance with the invention of micro-array based technology. This study also has some impact on cancer diagnosis. The proper classification of cancer tissue samples generated using the micro-array technology helps in detecting cancers in an automated way. In the current paper we have developed a semi-supervised clustering technique for proper partitioning of these gene expression data sets. Semi-supervised clustering is a combination of unsupervised and supervised classification techniques. It uses some amount of supervised information and a large collection of unsupervised data. Here a multi-objective based semi-supervised clustering technique is developed for solving the cancer tissue classification problem. Different combinations of objective functions are used. As the supervised information we assume that class labels of 10 % data are available. The proposed technique is evaluated for three open source benchmark cancer data sets (brain tumor data set, adult malignancy and small round blood cell tumors). Two classification quality measures, viz., Adjusted Rand Index and Classification Accuracy are used to measure the goodness of the obtained partitionings. Obtained results are compared with several state-of-the-art clustering techniques. Moreover, significant gene markers have been identified and demonstrated visually from the clustering solutions obtained.

Use of Semisupervised Clustering and Feature-Selection Techniques for Identification of Co-expressed Genes.

Studying the patterns hidden in gene-expression data helps to understand the functionality of genes. In general, clustering techniques are widely used for the identification of natural partitionings from the gene expression data. In order to put constraints on dimensionality, feature selection is the key issue because not all features are important from clustering point of view. Moreover some limited amount of supervised information can help to fine tune the obtained clustering solution. In this paper, the problem of simultaneous feature selection and semisupervised clustering is formulated as a multiobjective optimization (MOO) task. A modern simulated annealing-based MOO technique namely AMOSA is utilized as the background optimization methodology. Here, features and cluster centers are represented in the form of a string and the assignment of genes to different clusters is done using a point symmetry-based distance. Six optimization criteria based on several internal and external cluster validity indices are utilized. In order to generate the supervised information, a popular clustering technique, Fuzzy C-mean, is utilized. Appropriate subset of features, proper number of clusters and the proper partitioning are determined using the search capability of AMOSA. The effectiveness of this proposed semisupervised clustering technique, Semi-FeaClustMOO, is demonstrated on five publicly available benchmark gene-expression datasets. Comparison results with the existing techniques for gene-expression data clustering again reveal the superiority of the proposed technique. Statistical and biological significance tests have also been carried out.

Multi-objective semi-supervised clustering for automatic pixel classification from remote sensing imagery

Classifying the pixels of satellite images into homogeneous regions is a very challenging task as different regions have different types of land covers. Some land covers contain more regions, while some contain relatively smaller regions (e.g., bridges, roads). In satellite image segmentation, no prior information is available about the number of clusters. Here, in this paper, we have solved this problem using the concepts of semi-supervised clustering which utilizes the property of unsupervised and supervised classification. Three cluster validity indices are utilized, which are simultaneously optimized using AMOSA, a modern multiobjective optimization technique based on the concepts of simulated annealing. The first two cluster validity indices, symmetry distance based Sym-index, and Euclidean distance based I-index, are based on unsupervised properties. The last one is a supervised information based cluster validity index, Minkowski index. For supervised information, initially fuzzy C-mean clustering technique is used. Thereafter, based on the highest membership values of the data points to their respective clusters, randomly 10 % data points with their class labels are chosen. The effectiveness of this proposed semi-supervised clustering technique is demonstrated on three satellite image data sets of different cities of India. Results are also compared with existing clustering techniques.

A new semi-supervised clustering technique using multi-objective optimization

Semi-supervised clustering techniques have been proposed in the literature to overcome the problems associated with unsupervised and supervised classification. It considers a small amount of labeled data and the whole data distribution during the process of clustering a data. In this paper, a new approach towards semi-supervised clustering is implemented using multiobjective optimization (MOO) framework. Four objective functions are optimized using the search capability of a multiobjective simulated annealing based technique, AMOSA. These objective functions are based on some unsupervised and supervised information. First three objective functions represent, respectively, the goodness of the partitioning in terms of Euclidean distance, total symmetry present in the clusters and the cluster connectedness. For the last objective function, we have considered different external cluster validity indices, including adjusted rand index, rand index, a newly developed min-max distance based MMI index, NMMI index and Minkowski Score. Results show that the proposed semi-supervised clustering technique can effectively detect the appropriate number of clusters as well as the appropriate partitioning from the data sets having either well-separated clusters of any shape or symmetrical clusters with or without overlaps. Twenty four artificial and five real-life data sets have been used in the evaluation. We develop five different versions of Semi-GenClustMOO clustering technique by varying the external cluster validity indices. Obtained partitioning results are compared with another recently developed multiobjective semi-supervised clustering technique, Mock-Semi. At the end of the paper the effectiveness of the proposed Semi-GenClustMOO clustering technique is shown in segmenting one remote sensing satellite image on the part from the city of Kolkata.

An Efficient DNA Molecule Clustering using GCC Algorithm

AbstractResearchers in the biotechnology field have accomplished many achievements in the past century. They can now measure expression levels for thousands of genes, testing different conditions over varying periods of time. The analysis of the measurement results is essential to understand gene patterns and extract information about their functions and their biological roles. This paper describes a novel approach for clustering large-scale next-generation sequences (NGS). It also facilitates the process of predicting patterns and the likelihood of mutations based on a semi-supervised clustering technique. The process is based on the previously developed construction of FuzzyFind Dictionary utilizing the Golay Code for error correction. The introduced method is exceptional; it has linear time complexity with one passage through the file.