Standard Clustering Algorithms Research Articles

A cluster Analysis for Binary Data Using Genetic Algorithms

This research was initially driven by the lack of clustering algorithms that focus on binary data. A promising technique to analyze this type of data, namely Genetic Clustering for Unknown K (GCUK) became the main subject in this research. GCUK was applied to cluster four binary data and there is a presence of an imbalanced data in one of the data sets. The results show that GCUK is an efficient and effective clustering algorithm compared to K-means. The other contribution is the capability of GCUK for clustering the unbalanced data. Standard clustering algorithms cannot simply be applied to this type of data sets as it can cause a misclassification results.

CATS: A Tool for Clustering the Ensemble of Intrinsically Disordered Peptides on a Flat Energy Landscape.

We introduce the combinatorial averaged transient structure (CATS) clustering method as a means to cluster protein structure ensembles based on the distributions of protein backbone descriptor coordinates. In our study, we use phi and psi dihedral angle coordinates of the protein backbone as descriptors due to their translational and rotational invariance. The CATS method was developed to produce unique structure ensembles that are typically difficult to obtain from flat energy landscapes using a one-dimensional separation value (e.g., RMSD cutoff). Through the use of higher-dimensional descriptor coordinates, we remedy structure resolution shortcomings of standard clustering algorithms due to large RMSD fluctuations between structures. We compare the performance of CATS to an RMSD-based clustering method GROMOS, which may not be the best choice for IDP clustering since separation quality heavily relies on cutoff values instead of energy landscape minima. We demonstrate the performance of CATS and GROMOS by analyzing the all-atom molecular dynamics trajectories of the Tau/R2(273-284) fragment in solution with TMAO and urea osmolytes from prior studies. Our study reveals that the CATS method produces more unique clusters than the GROMOS method as a result of higher-dimensional distributions of the descriptor coordinates. The cluster centers produced by CATS correspond to local minima in the multidimensional potential mean force, which generates a structure ensemble that adequately samples the energy landscape.

Clustering with missing features: a penalized dissimilarity measure based approach

Many real-world clustering problems are plagued by incomplete data characterized by missing or absent features for some or all of the data instances. Traditional clustering methods cannot be directly applied to such data without preprocessing by imputation or marginalization techniques. In this article, we overcome this drawback by utilizing a penalized dissimilarity measure which we refer to as the feature weighted penalty based dissimilarity (FWPD). Using the FWPD measure, we modify the traditional k-means clustering algorithm and the standard hierarchical agglomerative clustering algorithms so as to make them directly applicable to datasets with missing features. We present time complexity analyses for these new techniques and also undertake a detailed theoretical analysis showing that the new FWPD based k-means algorithm converges to a local optimum within a finite number of iterations. We also present a detailed method for simulating random as well as feature dependent missingness. We report extensive experiments on various benchmark datasets for different types of missingness showing that the proposed clustering techniques have generally better results compared to some of the most well-known imputation methods which are commonly used to handle such incomplete data. We append a possible extension of the proposed dissimilarity measure to the case of absent features (where the unobserved features are known to be undefined).

A Comparison between Standard and Functional Clustering Methodologies: Application to Agricultural Fields for Yield Pattern Assessment

The recognition of spatial patterns within agricultural fields, presenting similar yield potential areas, stable through time, is very important for optimizing agricultural practices. This study proposes the evaluation of different clustering methodologies applied to multispectral satellite time series for retrieving temporally stable (constant) patterns in agricultural fields, related to within-field yield spatial distribution. The ability of different clustering procedures for the recognition and mapping of constant patterns in fields of cereal crops was assessed. Crop vigor patterns, considered to be related to soils characteristics, and possibly indicative of yield potential, were derived by applying the different clustering algorithms to time series of Landsat images acquired on 94 agricultural fields near Rome (Italy). Two different approaches were applied and validated using Landsat 7 and 8 archived imagery. The first approach automatically extracts and calculates for each field of interest (FOI) the Normalized Difference Vegetation Index (NDVI), then exploits the standard K-means clustering algorithm to derive constant patterns at the field level. The second approach applies novel clustering procedures directly to spectral reflectance time series, in particular: (1) standard K-means; (2) functional K-means; (3) multivariate functional principal components clustering analysis; (4) hierarchical clustering. The different approaches were validated through cluster accuracy estimates on a reference set of FOIs for which yield maps were available for some years. Results show that multivariate functional principal components clustering, with an a priori determination of the optimal number of classes for each FOI, provides a better accuracy than those of standard clustering algorithms. The proposed novel functional clustering methodologies are effective and efficient for constant pattern retrieval and can be used for a sustainable management of agricultural fields, depending on farming systems and environmental conditions in different regions.

Unsupervised Robotic Sorting : Towards Autonomous Decision Making Robots

Autonomous sorting is a crucial task in industrial robotics which can be very challenging depending on the expected amount of automation. Usually, to decide where to sort an object, the system needs to solve either an instance retrieval (known object) or a supervised classification (predefined set of classes) problem. In this paper, we introduce a new decision making module, where the robotic system chooses how to sort the objects in an unsupervised way. We call this problem Unsupervised Robotic Sorting (URS) and propose an implementation on an industrial robotic system, using deep CNN feature extraction and standard clustering algorithms. We carry out extensive experiments on various standard datasets to demonstrate the efficiency of the proposed image clustering pipeline. To evaluate the robustness of our URS implementation, we also introduce a complex real world dataset containing images of objects under various background and lighting conditions. This dataset is used to fine tune the design choices (CNN and clustering algorithm) for URS. Finally, we propose a method combining our pipeline with ensemble clustering to use multiple images of each object. This redundancy of information about the objects is shown to increase the clustering results.

TEXT DOCUMENT CLUSTERING USING ARTIFICIAL BEE COLONY WITH BISECTING K-MEANS ALGORITHM

Recently, document clustering with optimization techniques has gained the attention of many researchers, especially those who are dealing with a huge volume of documents. The main goal of document clustering is to place the documents with similar content in one group, and the documents with dissimilar contents in another group. Document clustering with optimization algorithm achieves the global optimal solution. The main aim of this research work is to cluster the documents based on their content. In order to perform this task, this research work proposes a new hybrid algorithm called Artificial Bee Colony with Bisecting K-Means (ABC-BK). The proposed algorithm was verified with the benchmark dataset in contrast to the widely used document clustering algorithms. Experimental results show that the proposed algorithm gives a better performance compared to the standard ABC clustering algorithm, K-means, and the Bisecting K-means algorithm.

Generalised kernel weighted fuzzy C-means clustering algorithm with local information

To improve the performance of segmentation for the images corrupted by noise, many variants of standard fuzzy C-means (FCM) clustering algorithm have been proposed that incorporate the local spatial neighbourhood information to perform image segmentation. Among them, the kernel weighted fuzzy local information C-means (KWFLICM) algorithm gives robust to noise image segmentation results by using local spatial image neighbourhood information, it is limited to one-dimensional input data i.e. image intensity. In this paper, we propose a generalisation of KWFLICM (GKWFLICM) that is applicable to M-dimensional input data sets. The proposed algorithm incorporates neighbourhood information among the M-dimensional data, which mitigates the disadvantages of the standard FCM clustering algorithm (sensitive to noise and outliers, poor performance for differently sized clusters and for different density clusters) and greatly improves the clustering performance. Experiments have been performed on several noisy and non-noisy data sets, as well as natural and real-world images, to demonstrate the effectiveness, efficiency, and robustness to noise of the GKWFLICM algorithm by comparing it to kernel fuzzy C-means (KFCM), kernel possibilistic fuzzy C-means (KPFCM), fuzzy local information C-means (FLICM), and KWFLICM.

181. The Phenomenological Construct of Self, Voices, and Other Extreme States

Abstract Traditionally, hallucinations have been understood as abnormalities of perception and delusions as abnormalities of cognition. However, descriptive psychopathologists have long noted the presence of “silent” or thought-like AVHs as well as AVHs with ambiguous perceptual and cognitive qualities. Some researchers have proposed that delusions of reference, control, and communication may at times overlap with or even be indistinguishable from AVHs. This study reports the analyses into the intercept of AVHs and forms of delusions. Ninety-two participants experiencing psychosis participated in the study. Inclusion criteria for the study included participants between the ages of 21–60 who met criteria for schizophrenia or bipolar disorder/psychosis and were currently experiencing AVHs. To advance the area of phenomenology of voices and their interrelatedness to forms of delusions, this study investigated the prevalence and interrelatedness of co-occurring auditory verbal hallucinations (AVHs) and delusions. Additionally, we explored the characterization of distinct subcategories/clusters of AVHs and delusions. Group comparisons of prevalence of co-occurring AVHs and delusions show a significant increased prevalence within the schizophrenia group (χ(1, N = 92) = 11.47, < .001). To identify specific subtypes, we conducted 2-step cluster analysis of preclustering and standard hierarchical clustering algorithm of SCID items pertaining to voices and delusions. The analysis yielded 2 distinct cluster groups. Cluster One: Voices and Thought Delusions and Cluster Two: Voices and Thematic Delusions. The primary cluster predictor algorithm show that delusions of control, thought insertion, thought withdrawal, and thought broadcasting are more discrete variables in identifying cluster subtypes. An independent sample t-test was conducted to compare PANSS 5-factor subscale items between cluster groups. These results show that, participants in the Voices and Thought Delusions cluster experienced increased positive, cognitive, and depressive factor scores. Our analyses provisionally support a reframing of AVHs and thought delusions as endpoints on a shared continuum of attenuated sense of basic self (ipseity). Within this disturbance of ipseity, aspects of agency, authorship, and mineness are experienced as foreign, not self-generated, and with absence of intentionality. These cluster groups begin to disentangle the intrasubjectivity of voices and delusion by characterizing Cluster One: AVH and Thought Delusions as a structure that centers on alterations of basic ipseity and internal/external boundaries, while Cluster Two: AVH and Thematic Delusions centers on exaggerated processes of social internalization (guilt) or externalization (jealousy/delusions of infidelity).

Automatic, exploratory mineralogical mapping of CRISM imagery using summary product signatures

Martian spectroscopic and mineralogical analysis is usually performed using Compact Reconnaissance Imaging Spectrometer for Mars (CRISM) browse products – false color images which show the spatial distribution of absorption features at key wavelengths. This manual, time-consuming method is ill-suited for exploratory surveys of a large number of images – for such surveys an automatic methodology is needed. In this paper we propose a method for exploratory but fully automatic mineralogical mapping of CRISM images. In our approach pixels are characterized by vectors of CRISM summary product values instead of spectral functions, and mineralogical units are discovered using a clustering principle. Moreover, the rare class discovery algorithm DEMUD is used in place of a standard clustering algorithm to identify mineralogical units - enabling the identification of only scientifically interesting, possibly rare, mineralogical deposits. The method outputs a map for each site showing the spatial distribution of mineralogical units – areas characterized by similar mineralogy. It also provides, without using a spectral library, semantic labels for each unit. We envision our method as a focus-of-attention tool to facilitate fast exploratory surveys of a large number of images. An analyst needs only to examine manually regions within an image where our pipeline indicates the existence of mineral units of interest. In this paper the method for our computational pipeline is described in detail and its performance is evaluated using a sample of 20 CRISM images – the mineralogical content of which is known from manual analysis. We find that our pipeline identifies most deposits found through manual analysis as well as some additional deposits which were not targeted by those analyses. Overall, we conclude that our fully automatic mineralogical mapper works well for exploratory purposes. Thus, it adds a new, valuable functionality to existing tools for CRISM imagery analysis.

Functional grouping of similar genes using eigenanalysis on minimum spanning tree based neighborhood graph

Gene expression data clustering is an important biological process in DNA microarray analysis. Although there have been many clustering algorithms for gene expression analysis, finding a suitable and effective clustering algorithm is always a challenging problem due to the heterogeneous nature of gene profiles. Minimum Spanning Tree (MST) based clustering algorithms have been successfully employed to detect clusters of varying shapes and sizes. This paper proposes a novel clustering algorithm using Eigenanalysis on Minimum Spanning Tree based neighborhood graph (E-MST). As MST of a set of points reflects the similarity of the points with their neighborhood, the proposed algorithm employs a similarity graph obtained from k′ rounds of MST (k′-MST neighborhood graph). By studying the spectral properties of the similarity matrix obtained from k′-MST graph, the proposed algorithm achieves improved clustering results. We demonstrate the efficacy of the proposed algorithm on 12 gene expression datasets. Experimental results show that the proposed algorithm performs better than the standard clustering algorithms.

Topic modeling of behavioral modes using sensor data

The field of movement ecology is experiencing a period of rapid growth in availability of data. As the volume rises, traditional methods are giving way to machine learning and data science, which are playing an increasingly large part in turning these data into science-driving insights. One rich and interesting source is the biologger. These small electronic wearable devices are attached to animals free to roam in their natural habitats and report back readings from multiple sensors, including GPS and accelerometer bursts. A common use of accelerometer data is for supervised learning of behavioral modes. However, we need unsupervised analysis tools as well, in order to overcome the inherent difficulties of obtaining a labeled dataset, which in some cases either is infeasible or does not successfully encompass the full repertoire of behavioral modes of interest. Here, we present a matrix factorization-based topic model method for accelerometer bursts, derived using a linear mixture property of patch features. Our method is validated via comparison with a labeled dataset and is further compared to standard clustering algorithms.

Modelling-based experiment retrieval: a case study with gene expression clustering.

Public and private repositories of experimental data are growing to sizes that require dedicated methods for finding relevant data. To improve on the state of the art of keyword searches from annotations, methods for content-based retrieval have been proposed. In the context of gene expression experiments, most methods retrieve gene expression profiles, requiring each experiment to be expressed as a single profile, typically of case versus control. A more general, recently suggested alternative is to retrieve experiments whose models are good for modelling the query dataset. However, for very noisy and high-dimensional query data, this retrieval criterion turns out to be very noisy as well. We propose doing retrieval using a denoised model of the query dataset, instead of the original noisy dataset itself. To this end, we introduce a general probabilistic framework, where each experiment is modelled separately and the retrieval is done by finding related models. For retrieval of gene expression experiments, we use a probabilistic model called product partition model, which induces a clustering of genes that show similar expression patterns across a number of samples. The suggested metric for retrieval using clusterings is the normalized information distance. Empirical results finally suggest that inference for the full probabilistic model can be approximated with good performance using computationally faster heuristic clustering approaches (e.g. k-means). The method is highly scalable and straightforward to apply to construct a general-purpose gene expression experiment retrieval method. The method can be implemented using standard clustering algorithms and normalized information distance, available in many statistical software packages. paul.blomstedt@aalto.fi or samuel.kaski@aalto.fi Supplementary data are available at Bioinformatics online.

Term Frequency Based Cosine Similarity Measure for Clustering Categorical Data using Hierarchical Algorithm

Object in real world are categorical in nature. Categorical data are not analyzed as numerical data because of the absence of inherit ordering. In this study performance of cosine based hierarchical clustering algorithm for categorical data is evaluated. It make use of two functions such as Frequency Computation, Term Frequency based Cosine Similarity Matrix (TFCSM) computation. Clusters are formed using TFCSM based hierarchical clustering algorithm. Results are evaluated for vote real life data set using TFCSM based hierarchical clustering and standard hierarchical clustering algorithm using single link, complete link and average link method.

Distance Measure to Cluster Spatiotemporal Scenarios for Strategic Air Traffic Management

Convective weather, recognized as the leading contributor to delays in the National Airspace System, causes demand–capacity imbalance in the airspace. Strategic air traffic management aims to resolve the imbalance for offnominal conditions (for example, convective weather events) through redistributing flows and resources at the strategic timeframe (2–15 h in advance). As a component of scenario-driven strategic air traffic management, this paper develops a multiresolution spatiotemporal distance measure that, combined with standard distance-based clustering algorithms, can be used to group a wide range of possible weather-impact scenarios into a few representative clusters to facilitate corresponding management strategy design. Motivations of this new distance measure, its generation algorithm, and parameter impact analysis are described in detail to facilitate practical implementation and subsequent use in decision support. This multiresolution spatiotemporal distance measure not only addresses the needs of scenario-driven strategic air traffic management but also generally applies to broad data-driven decision support applications that involve large-scale physical processes of spatiotemporal spread dynamics, as the measure captures the similarity between scenarios of spatiotemporal spread patterns of varying shape, size, location, and intensity; corrects the “boundary effects”; and is flexible to a variety of data features in temporal and spatial dimensions. Performance evaluation is conducted through comparative studies and sensitivity analysis, using real weather-impact datasets.

Tensor Spectral Clustering for Partitioning Higher-order Network Structures.

Spectral graph theory-based methods represent an important class of tools for studying the structure of networks. Spectral methods are based on a first-order Markov chain derived from a random walk on the graph and thus they cannot take advantage of important higher-order network substructures such as triangles, cycles, and feed-forward loops. Here we propose a Tensor Spectral Clustering (TSC) algorithm that allows for modeling higher-order network structures in a graph partitioning framework. Our TSC algorithm allows the user to specify which higher-order network structures (cycles, feed-forward loops, etc.) should be preserved by the network clustering. Higher-order network structures of interest are represented using a tensor, which we then partition by developing a multilinear spectral method. Our framework can be applied to discovering layered flows in networks as well as graph anomaly detection, which we illustrate on synthetic networks. In directed networks, a higher-order structure of particular interest is the directed 3-cycle, which captures feedback loops in networks. We demonstrate that our TSC algorithm produces large partitions that cut fewer directed 3-cycles than standard spectral clustering algorithms.

ClusTrack: feature extraction and similarity measures for clustering of genome-wide data sets.

Clustering is a popular technique for explorative analysis of data, as it can reveal subgroupings and similarities between data in an unsupervised manner. While clustering is routinely applied to gene expression data, there is a lack of appropriate general methodology for clustering of sequence-level genomic and epigenomic data, e.g. ChIP-based data. We here introduce a general methodology for clustering data sets of coordinates relative to a genome assembly, i.e. genomic tracks. By defining appropriate feature extraction approaches and similarity measures, we allow biologically meaningful clustering to be performed for genomic tracks using standard clustering algorithms. An implementation of the methodology is provided through a tool, ClusTrack, which allows fine-tuned clustering analyses to be specified through a web-based interface. We apply our methods to the clustering of occupancy of the H3K4me1 histone modification in samples from a range of different cell types. The majority of samples form meaningful subclusters, confirming that the definitions of features and similarity capture biological, rather than technical, variation between the genomic tracks. Input data and results are available, and can be reproduced, through a Galaxy Pages document at http://hyperbrowser.uio.no/hb/u/hb-superuser/p/clustrack. The clustering functionality is available as a Galaxy tool, under the menu option "Specialized analyzis of tracks", and the submenu option "Cluster tracks based on genome level similarity", at the Genomic HyperBrowser server: http://hyperbrowser.uio.no/hb/.

H-CLAP: hierarchical clustering within a linear array with an application in genetics.

In most cases where clustering of data is desirable, the underlying data distribution to be clustered is unconstrained. However clustering of site types in a discretely structured linear array, as is often desired in studies of linear sequences such as DNA, RNA or proteins, represents a problem where data points are not necessarily exchangeable and are directionally constrained within the array. Each position in the linear array is fixed, and could be either "marked" (i.e., of interest such as polymorphic or substitute sites) or "non-marked." Here we describe a method for clustering of those marked sites. Since the cluster-generating process is constrained by discrete locality inside such an array, traditional clustering methods need adjustment to be appropriate. We develop a hierarchical Bayesian approach. We adopt a Markov clustering algorithm, revealing any natural partitioning in the pattern of marked sites. The resulting recursive partitioning and clustering algorithm is named hierarchical clustering in a linear array (H-CLAP). It employs domain-specific directional constraints directly in the likelihood construction. Our method, being fully Bayesian, is more flexible in cluster discovery compared to a standard agglomerative hierarchical clustering algorithm. It not only provides hierarchical clustering, but also cluster boundaries, which may have their own biological significance. We have tested the efficacy of our method on data sets, including two biological and several simulated ones.

Clustering-based fragmentation and data replication for flexible query answering in distributed databases

One feature of cloud storage systems is data fragmentation (or sharding) so that data can be distributed over multiple servers and subqueries can be run in parallel on the fragments. On the other hand, flexible query answering can enable a database system to find related information for a user whose original query cannot be answered exactly. Query generalization is a way to implement flexible query answering on the syntax level. In this paper we study a clustering-based fragmentation for the generalization operator Anti-Instantiation with which related information can be found in distributed data. We use a standard clustering algorithm to derive a semantic fragmentation of data in the database. The database system uses the derived fragments to support an intelligent flexible query answering mechanism that avoids overgeneralization but supports data replication in a distributed database system. We show that the data replication problem can be expressed as a special Bin Packing Problem and can hence be solved by an off-the shelf solver for integer linear programs. We present a prototype system that makes use of a medical taxonomy to determine similarities between medical expressions.

A “Density-Based” Algorithm for Cluster Analysis Using Species Sampling Gaussian Mixture Models

We propose a new model for cluster analysis in a Bayesian nonparametric framework. Our model combines two ingredients, species sampling mixture models of Gaussian distributions on one hand, and a deterministic clustering procedure (DBSCAN) on the other. Here, two observations from the underlying species sampling mixture model share the same cluster if the distance between the densities corresponding to their latent parameters is smaller than a threshold; this yields a random partition which is coarser than the one induced by the species sampling mixture. Since this procedure depends on the value of the threshold, we suggest a strategy to fix it. In addition, we discuss implementation and applications of the model; comparison with more standard clustering algorithms will be given as well. Supplementary materials for the article are available online.

Similarity-Driven Semantic Role Induction via Graph Partitioning

As in many natural language processing tasks, data-driven models based on supervised learning have become the method of choice for semantic role labeling. These models are guaranteed to perform well when given sufficient amount of labeled training data. Producing this data is costly and time-consuming, however, thus raising the question of whether unsupervised methods offer a viable alternative. The working hypothesis of this article is that semantic roles can be induced without human supervision from a corpus of syntactically parsed sentences based on three linguistic principles: (1) arguments in the same syntactic position (within a specific linking) bear the same semantic role, (2) arguments within a clause bear a unique role, and (3) clusters representing the same semantic role should be more or less lexically and distributionally equivalent. We present a method that implements these principles and formalizes the task as a graph partitioning problem, whereby argument instances of a verb are represented as vertices in a graph whose edges express similarities between these instances. The graph consists of multiple edge layers, each one capturing a different aspect of argument-instance similarity, and we develop extensions of standard clustering algorithms for partitioning such multi-layer graphs. Experiments for English and German demonstrate that our approach is able to induce semantic role clusters that are consistently better than a strong baseline and are competitive with the state of the art.