Standard Clustering Algorithms Research Articles

Image clustering using generated text centroids

In recent years, deep neural networks pretrained on large-scale datasets have been used to address data deficiency and achieve better performance through prior knowledge. Contrastive language–image pretraining (CLIP), a vision-language model pretrained on an extensive dataset, achieves better performance in image recognition. In this study, we harness the power of multimodality in image clustering tasks, shifting from a single modality to a multimodal framework using the describability property of image encoder of the CLIP model. The importance of this shift lies in the ability of multimodality to provide richer feature representations. By generating text centroids corresponding to image features, we effectively create a common descriptive language for each cluster. It generates text centroids assigned by the image features and improves the clustering performance. The text centroids use the results generated by using the standard clustering algorithm as a pseudo-label and learn a common description of each cluster. Finally, only text centroids were added when the image features on the same space were assigned to the text centroids, but the clustering performance improved significantly compared to the standard clustering algorithm, especially on complex datasets. When the proposed method is applied, the normalized mutual information score rises by 32% on the Stanford40 dataset and 64% on ImageNet-Dog compared to the k-means clustering algorithm.

Unsupervised Single-Cell Clustering with Asymmetric Within-Sample Transformation and Per-Cluster Supervised Features Selection.

This chapter shows applying the Asymmetric Within-Sample Transformation to single-cell RNA-Seq data matched with a previous dropout imputation. The asymmetric transformation is a special winsorization that flattens low-expressed intensities and preserves highly expressed gene levels. Before a standard hierarchical clustering algorithm, an intermediate step removes noninformative genes according to a threshold applied to a per-gene entropy estimate. Following the clustering, a time-intensive algorithm is shown to uncover the molecular features associated with each cluster. This step implements a resampling algorithm to generate a random baseline to measure up/downregulated significant genes. To this aim, we adopt a GLM model as implemented in DESeq2 package. We render the results in graphical mode. While the tools are standard heat maps, we introduce some data scaling to clarify the results' reliability.

Improving Clustering Accuracy of K-Means and Random Swap by an Evolutionary Technique Based on Careful Seeding

K-Means is a “de facto” standard clustering algorithm due to its simplicity and efficiency. K-Means, though, strongly depends on the initialization of the centroids (seeding method) and often gets stuck in a local sub-optimal solution. K-Means, in fact, mainly acts as a local refiner of the centroids, and it is unable to move centroids all over the data space. Random Swap was defined to go beyond K-Means, and its modus operandi integrates K-Means in a global strategy of centroids management, which can often generate a clustering solution close to the global optimum. This paper proposes an approach which extends both K-Means and Random Swap and improves the clustering accuracy through an evolutionary technique and careful seeding. Two new algorithms are proposed: the Population-Based K-Means (PB-KM) and the Population-Based Random Swap (PB-RS). Both algorithms consist of two steps: first, a population of J candidate solutions is built, and then the candidate centroids are repeatedly recombined toward a final accurate solution. The paper motivates the design of PB-KM and PB-RS, outlines their current implementation in Java based on parallel streams, and demonstrates the achievable clustering accuracy using both synthetic and real-world datasets.

A model-based clustering algorithm with covariates adjustment and its application to lung cancer stratification.

Usually, the clustering process is the first step in several data analyses. Clustering allows identify patterns we did not note before and helps raise new hypotheses. However, one challenge when analyzing empirical data is the presence of covariates, which may mask the obtained clustering structure. For example, suppose we are interested in clustering a set of individuals into controls and cancer patients. A clustering algorithm could group subjects into young and elderly in this case. It may happen because the age at diagnosis is associated with cancer. Thus, we developed CEM-Co, a model-based clustering algorithm that removes/minimizes undesirable covariates' effects during the clustering process. We applied CEM-Co on a gene expression dataset composed of 129 stage I non-small cell lung cancer patients. As a result, we identified a subgroup with a poorer prognosis, while standard clustering algorithms failed.

Fuzzy-Rough induced spectral ensemble clustering

Ensemble clustering helps achieve fast clustering under abundant computing resources by constructing multiple base clusterings. Compared with the standard single clustering algorithm, ensemble clustering integrates the advantages of multiple clustering algorithms and has stronger robustness and applicability. Nevertheless, most ensemble clustering algorithms treat each base clustering result equally and ignore the difference of clusters. If a cluster in a base clustering is reliable/unreliable, it should play a critical/uncritical role in the ensemble process. Fuzzy-rough sets offer a high degree of flexibility in enabling the vagueness and imprecision present in real-valued data. In this paper, a novel fuzzy-rough induced spectral ensemble approach is proposed to improve the performance of clustering. Specifically, the significance of clusters is differentiated, and the unacceptable degree and reliability of clusters formed in base clustering are induced based on fuzzy-rough lower approximation. Based on defined cluster reliability, a new co-association matrix is generated to enhance the effect of diverse base clusterings. Finally, a novel consensus spectral function is defined by the constructed adjacency matrix, which can lead to significantly better results. Experimental results confirm that the proposed approach works effectively and outperforms many state-of-the-art ensemble clustering algorithms and base clustering, which illustrates the superiority of the novel algorithm.

Synonyms, Antonyms and Factual Knowledge in BERT Heads

In recent years, many studies have been devoted to discovering the inner workings of Transformer-based models, such as BERT, for instance, attempting to identify what information is contained within them. However, little is known about how these models store this information in their millions of parameters and which parts of the architecture are the most important. In this work, we propose an approach to identify self-attention mechanisms, called heads, that contain semantic and real-world factual knowledge in BERT. Our approach includes a metric computed from attention weights and exploits a standard clustering algorithm for extracting the most relevant connections between tokens in a head. In our experimental analysis, we focus on how heads can connect synonyms, antonyms and several types of factual knowledge regarding subjects such as geography and medicine.

Cluster-based ensemble learning model for improving sentiment classification of Arabic documents

AbstractThis article reports on designing and implementing a multiclass sentiment classification approach to handle the imbalanced class distribution of Arabic documents. The proposed approach, sentiment classification of Arabic documents (SCArD), combines the advantages of a clustering-based undersampling (CBUS) method and an ensemble learning model to aid machine learning (ML) classifiers in building accurate models against highly imbalanced datasets. The CBUS method applies two standard clustering algorithms:K-means and expectation–maximization, to balance the ratio between the major and the minor classes by decreasing the number of the major class instances and maintaining the number of the minor class instances at the cluster level. The merits of the proposed approach are that it does not remove the majority class instances from the dataset nor injects the dataset with artificial minority class instances. The resulting balanced datasets are used to train two ML classifiers, random forest and updateable Naïve Bayes, to develop prediction data models. The best prediction data models are selected based on F1-score rates. We applied two techniques to test SCArD and generate new predictions from the imbalanced test dataset. The first technique uses the best prediction data models. The second technique uses the majority voting ensemble learning model, which combines the best prediction data models to generate the final predictions. The experimental results showed that SCArD is promising and outperformed the other comparative classification models based on the F1-score rates.

An improved differential bond energy algorithm with fuzzy merging method to improve the document clustering for information mining

AbstractThe vast and diversified text materials on the internet in recent years have drastically increased the importance of information mining. By organizing documents into cohesive groupings, a document clustering method is a suitable tool for dealing with massive amounts of documents. Text documents, on the other hand, include sparse and uninformative features like noise, unrelated, and unneeded features, which reduce the efficiency of the document clustering methods. For noise removal, this work employs the hyperclique pattern‐based data cleaner pre‐processing approach. Then, differential bond energy algorithm (DBEA) is combined with a fuzzy merging approach and termed improved differential bond energy algorithm with fuzzy merging (IDBEFM) to handle the issues present in text document clustering. It seeks to discover and display natural variable clusters among large amounts of data. IDBEFM clusters are documented in three steps: in the first step, a cluster similarity matrix is instantiated by applying the DBEA. The second step develops an innovative technique for automatically dividing the cluster matrix into compact cohesive clusters, and in the third step, a fuzzy merging approach is used for combining identical clusters by the correlations and interrelationships between the resultant clusters. The test findings demonstrated that the accomplishment of the proposed technique is much superior to standard clustering algorithms.

A new method based on ensemble time series for fast and accurate clustering

PurposeThe common methods for clustering time series are the use of specific distance criteria or the use of standard clustering algorithms. Ensemble clustering is one of the common techniques used in data mining to increase the accuracy of clustering. In this study, based on segmentation, selecting the best segments, and using ensemble clustering for selected segments, a multistep approach has been developed for the whole clustering of time series data.Design/methodology/approachFirst, this approach divides the time series dataset into equal segments. In the next step, using one or more internal clustering criteria, the best segments are selected, and then the selected segments are combined for final clustering. By using a loop and how to select the best segments for the final clustering (using one criterion or several criteria simultaneously), two algorithms have been developed in different settings. A logarithmic relationship limits the number of segments created in the loop.FindingAccording to Rand's external criteria and statistical tests, at first, the best setting of the two developed algorithms has been selected. Then this setting has been compared to different algorithms in the literature on clustering accuracy and execution time. The obtained results indicate more accuracy and less execution time for the proposed approach.Originality/valueThis paper proposed a fast and accurate approach for time series clustering in three main steps. This is the first work that uses a combination of segmentation and ensemble clustering. More accuracy and less execution time are the remarkable achievements of this study.

CLUE: a clustering algorithm for current and future experiments

CLUE (CLUstering of Energy) is a fast parallel clustering algorithm for High Granularity Calorimeters in High Energy Physics. In these types of detectors, the standard clustering algorithms using combinatorics are expected to fail due to large number of digitized energy deposits (hits) in the reconstruction stage bringing to a consequent memory/timing explosion. This innovative algorithm uses a grid spatial index for fast querying of neighbors and its timing scales linearly with the number of hits within the range considered. Initially CLUE was developed in a standalone repository that allows performance benchmarking with respect to its CPU and GPU implementations, demonstrating the power of algorithmic parallelization in the coming era of heterogeneous computing. CLUE has been successfully used in simulation and beam tests of the High Granularity Calorimeter to be installed for the upgrade of the CMS detector in Phase-2 of the HL-LHC. Recently CLUE was also imported in the key4hep framework and first results will be shown for detectors proposed in future collider projects.

DIRICHLET-TREE MULTINOMIAL MIXTURES FOR CLUSTERING MICROBIOME COMPOSITIONS.

Studying the human microbiome has gained substantial interest in recent years, and a common task in the analysis of these data is to cluster microbiome compositions into subtypes. This subdivision of samples into subgroups serves as an intermediary step in achieving personalized diagnosis and treatment. In applying existing clustering methods to modern microbiome studies including the American Gut Project (AGP) data, we found that this seemingly standard task, however, is very challenging in the microbiome composition context due to several key features of such data. Standard distance-based clustering algorithms generally do not produce reliable results as they do not take into account the heterogeneity of the cross-sample variability among the bacterial taxa, while existing model-based approaches do not allow sufficient flexibility for the identification of complex within-cluster variation from cross-cluster variation. Direct applications of such methods generally lead to overly dispersed clusters in the AGP data and such phenomenon is common for other microbiome data. To overcome these challenges, we introduce Dirichlet-tree multinomial mixtures (DTMM) as a Bayesian generative model for clustering amplicon sequencing data in microbiome studies. DTMM models the microbiome population with a mixture of Dirichlet-tree kernels that utilizes the phylogenetic tree to offer a more flexible covariance structure in characterizing within-cluster variation, and it provides a means for identifying a subset of signature taxa that distinguish the clusters. We perform extensive simulation studies to evaluate the performance of DTMM and compare it to state-of-the-art model-based and distance-based clustering methods in the microbiome context, and carry out a validation study on a publicly available longitudinal data set to confirm the biological relevance of the clusters. Finally, we report a case study on the fecal data from the AGP to identify compositional clusters among individuals with inflammatory bowel disease and diabetes. Among our most interesting findings is that enterotypes (i.e., gut microbiome clusters) are not always defined by the most dominant species as previous analyses had assumed, but can involve a number of less abundant OTUs, which cannot be identified with existing distance-based and method-based approaches.

Deep Temporal Iterative Clustering for Satellite Image Time Series Land Cover Analysis

The extensive amount of Satellite Image Time Series (SITS) data brings new opportunities and challenges for land cover analysis. Many supervised machine learning methods have been applied in SITS, but the labeled SITS samples are time- and effort-consuming to acquire. It is necessary to analyze SITS data with an unsupervised learning method. In this paper, we propose a new unsupervised learning method named Deep Temporal Iterative Clustering (DTIC) to deal with SITS data. The proposed method jointly learns a neural network’s parameters and the resulting features’ cluster assignments, which uses a standard clustering algorithm, K-means, to iteratively cluster the features produced by the feature extraction network and then uses the subsequent assignments as supervision to update the network’s weights. We apply DTIC to the unsupervised training of neural networks on both SITS datasets. Experimental results demonstrate that DTIC outperforms the state-of-the-art K-means clustering algorithm, which proves that the proposed approach successfully provides a novel idea for unsupervised training of SITS data.

Efficient Crime Analysis Based on Hybrid Approach by Combining Dynamic Time Wrapping Algorithm with K-Means Clustering Approach.

Abstract: Crime is a preeminent issue where the main concern has been worried by individual, the local area and government. Wrongdoing forecast utilizes past information and in the wake of investigating information, anticipate the future wrongdoing with area and time. In present days sequential criminal cases quickly happen so it is a provoking assignment to anticipate future wrongdoing precisely with better execution. Clustering different time series into similar groups is a challenging clustering task because each data point is an ordered sequence. The most common approach to time series clustering is to flatten the time series into a table, with a column for each time index (or aggregation of the series) and directly apply standard clustering algorithms like k-means. But this doesn’t always work well on Time Series Data. The paper focuses on combining the features of K-Means Clustering algorithm with Dynamic Time Wrapping Algorithm for efficient Crime prediction and analysis

Finding an optimised infrastructure for electricity distribution networks in rural areas - A comparison of different approaches

We introduce two novel evolutionary algorithms for the problem of optimising electricity distribution networks in remote areas. Traditionally, electricity networks have been built using a centralised network infrastructure. In remote areas these typically include long distances between sparsely distributed consumers that lead to high costs and several risks that accompany these long connection lines. Therefore, alternative distributed infrastructures including potential microgrids or stand-alone power systems can be beneficial. Given a set of electricity consumers, their loads and locations, the aim is to find an optimal electricity network topology that might not be interconnected. Previously, this problem has been addressed using different standard clustering approaches comparing the total network costs of each solution. We propose a problem-specific evolutionary algorithm that alters the network structure directly as well as a novel quadtree-based evolutionary algorithm. The algorithms minimise a non-linear objective function representing the cost of the network while approximating real-world constraints. They are tested on a large set of various problem instances and compared to the standard clustering algorithms that have been used previously and an intuitive baseline method. The problem-specific evolutionary algorithm outperforms most standard clustering methods on a range of different test cases while the quadtree-based algorithm is shown to be effective in certain settings.

Spectral Clustering on Spherical Coordinates Under the Degree-Corrected Stochastic Blockmodel

ABSTRACT Spectral clustering is a popular method for community detection in network graphs: starting from a matrix representation of the graph, the nodes are clustered on a low-dimensional projection obtained from a truncated spectral decomposition of the matrix. Estimating correctly the number of communities and the dimension of the reduced latent space is critical for good performance of spectral clustering algorithms. Furthermore, many real-world graphs, such as enterprise computer networks studied in cyber-security applications, often display heterogeneous within-community degree distributions. Such heterogeneous degree distributions are usually not well captured by standard spectral clustering algorithms. In this article, a novel spectral clustering algorithm is proposed for community detection under the degree-corrected stochastic blockmodel. The proposed method is based on a transformation of the spectral embedding to spherical coordinates, and a novel modeling assumption in the transformed space. The method allows for simultaneous and automated selection of the number of communities and the latent dimension for spectral embeddings of graphs with uneven node degrees. Results show improved performance over competing methods in representing computer networks.

Load Clustering Method Based on Improved Density Peak and Gaussian Mixture Model

A load clustering method based on improved density peak and Gaussian mixture was proposed to solve the problems that standard Gaussian mixture clustering algorithm (GMM) with random initial clustering center was easy to fall into local optimum in the clustering analysis of residential electricity behavior, and the density peak algorithm (DPC) needed to manually select the clustering center.The method combines the Cosine distance and Euclidean distance, which reflect the similarity of curve shape, and selects the clustering center automatically through the sine value relation of relative Angle between two points in γ descending graph. The clustering center selected by the improved DPC algorithm was used to initialize the parameters of the GMM model. Finally, load clustering is carried out by GMM model. The results of calculation examples show that the proposed method can effectively improve the efficiency of daily load curve clustering.

Deep Semi-Supervised Algorithm for Learning Cluster-Oriented Representations of Medical Images Using Partially Observable DICOM Tags and Images.

The task of automatically extracting large homogeneous datasets of medical images based on detailed criteria and/or semantic similarity can be challenging because the acquisition and storage of medical images in clinical practice is not fully standardised and can be prone to errors, which are often made unintentionally by medical professionals during manual input. In this paper, we propose an algorithm for learning cluster-oriented representations of medical images by fusing images with partially observable DICOM tags. Pairwise relations are modelled by thresholding the Gower distance measure which is calculated using eight DICOM tags. We trained the models using 30,000 images, and we tested them using a disjoint test set consisting of 8000 images, gathered retrospectively from the PACS repository of the Clinical Hospital Centre Rijeka in 2017. We compare our method against the standard and deep unsupervised clustering algorithms, as well as the popular semi-supervised algorithms combined with the most commonly used feature descriptors. Our model achieves an NMI score of with respect to the anatomic region, and an NMI score of with respect to the modality. The results suggest that DICOM data can be used to generate pairwise constraints that can help improve medical images clustering, even when using only a small number of constraints.

Weighted Clustering

We investigate a natural generalization of the classical clustering problem, considering clustering tasks in which different instances may have different weights. We conduct the first extensive theoretical analysis on the influence of weighted data on standard clustering algorithms in both the partitional and hierarchical settings, characterizing the conditions under which algorithms react to weights. Extending a recent framework for clustering algorithm selection, we propose intuitive properties that would allow users to choose between clustering algorithms in the weighted setting and classify algorithms accordingly.

A fast and efficient deep learning procedure for tracking droplet motion in dense microfluidic emulsions.

We present a deep learning-based object detection and object tracking algorithm to study droplet motion in dense microfluidic emulsions. The deep learning procedure is shown to correctly predict the droplets' shape and track their motion at competitive rates as compared to standard clustering algorithms, even in the presence of significant deformations. The deep learning technique and tool developed in this work could be used for the general study of the dynamics of biological agents in fluid systems, such as moving cells and self-propelled microorganisms in complex biological flows. This article is part of the theme issue 'Progress in mesoscale methods for fluid dynamics simulation'.

Multi-view subspace clustering networks with local and global graph information

This study investigates the problem of multi-view subspace clustering, the goal of which is to explore the underlying grouping structure of data collected from different fields or measurements. Since data do not always comply with the linear subspace models in many real-world applications, most existing multi-view subspace clustering methods based on the shallow linear subspace models may fail in practice. Furthermore, the underlying graph information of multi-view data is usually ignored in most existing multi-view subspace clustering methods. To address the aforementioned limitations, we proposed the novel multi-view subspace clustering networks with local and global graph information, termed MSCNLG, in this paper. Specifically, autoencoder networks are employed on multiple views to achieve latent smooth representations that are suitable for the linear assumption. Simultaneously, by integrating fused multi-view graph information into self-expressive layers, the proposed MSCNLG obtains the common shared multi-view subspace representation, which can be used to get clustering results by employing the standard spectral clustering algorithm. As an end-to-end trainable framework, the proposed method fully investigates the valuable information of multiple views. Comprehensive experiments on six benchmark datasets validate the effectiveness and superiority of the proposed MSCNLG.