Probabilistic Clustering Algorithm Research Articles

SoF: Soft-Cluster Matrix Factorization for Probabilistic Clustering

We propose SoF (Soft-cluster matrix Factorization), a probabilistic clustering algorithm which softly assigns each data point into clusters. Unlike model-based clustering algorithms, SoF does not make assumptions about the data density distribution. Instead, we take an axiomatic approach to define 4 properties that the probability of co-clustered pairs of points should satisfy. Based on the properties, SoF utilizes a distance measure between pairs of points to induce the conditional co-cluster probabilities. The objective function in our framework establishes an important connection between probabilistic clustering and constrained symmetric Nonnegative Matrix Factorization (NMF), hence providing a theoretical interpretation for NMF-based clustering algorithms. To optimize the objective, we derive a sequential minimization algorithm using a penalty method. Experimental results on both synthetic and real-world datasets show that SoF significantly outperforms previous NMF-based algorithms and that it is able to detect non-convex patterns as well as cluster boundaries.

Ensuring the validity of private forest owner typologies by controlling for response style bias and the robustness of statistical methods

In survey-based segmentation of forest owners, two threats to the validity of results have largely been ignored: (1) response style bias and (2) the robustness of the statistical methods. This study demonstrates response style bias detection, presents an approach for correcting for acquiescence – the systematic tendency to agree with survey items, and explores the sensitivity of a probabilistic clustering algorithm to requirements for the validity of the typology. Structural equation modeling and Monte Carlo data generation techniques were employed to detect acquiescence and estimate its effect on construct validity. A survey of the relevance of management information for private forest owners (N = 364) was used as an example. Although acquiescence was confirmed, it had minor effect on the results and no effect on the substantive construct. Uncertainty about the number of forest owner types and membership can be reduced by using probabilistic clustering and observing the number of clusters while changing the requirements for the validity of clusters. The expectation maximization algorithm proved to be robust even to stringent requirements for the validity of clusters. By controlling for response style and the robustness of statistical methods, the validity of private forest owner typologies can be better ensured.

The Bioinformatic Basis of Pan-Asianism

The socially calculated Asian body is an abstract discursive space bridging early twentieth- and twenty-first-century Pan-Asianism across multiple scientific understandings of race and ethnicity. In the early twentieth century, the pan-Asian body was a static, statistical taxonomy of precisely measured blood and body parts. As an administrative tool of empire and nation building, the quantitatively defined Asian was plotted along Cartesian coordinates of racial purity. By the twenty-first century, new computational technologies flexibly supported both national and transcendent pan-Asian ethnic identities by constructing regional populations as dynamic probabilistic clusters over time. This paper focuses on how the Pan-Asian SNP Consortium (PASNP) of the Human Genome Organisation (HUGO), the first inter-Asian genomics collaboration, embodied a revival of Pan-Asianism in both the members' collaborative network and scientific research. As a network of scientists, the PASNP members heralded the spirit of regional cooperation to bring about the rise of a pan-Asian research area in science. Through their research, the members reflexively calculated a new narrative of the shared ethnic origin and genetic unity of the region. Biochip data, probabilistic clustering algorithms, and computer simulations in the hands of Asian scientists calculated that the region was most likely populated as a single wave of historical migration. This overturned the dominant theory supported by Western-led international projects that divided Asian populations. The PASNP thus mapped the metageography of Pan-Asianism through big data computation.

A Global Approach to the Comparison of Clustering Results

Summary The discovery of knowledge in the case of Hierarchical Cluster Analysis (HCA) depends on many factors, such as the clustering algorithms applied and the strategies developed in the initial stage of Cluster Analysis. We present a global approach for evaluating the quality of clustering results and making a comparison among different clustering algorithms using the relevant information available (e.g. the stability, isolation and homogeneity of the clusters). In addition, we present a visual method to facilitate evaluation of the quality of the partitions, allowing identification of the similarities and differences between partitions, as well as the behaviour of the elements in the partitions. We illustrate our approach using a complex and heterogeneous dataset (real horse data) taken from the literature. We apply HCA based on the generalized affinity coefficient (similarity coefficient) to the case of complex data (symbolic data), combined with 26 (classic and probabilistic) clustering algorithms. Finally, we discuss the obtained results and the contribution of this approach to gaining better knowledge of the structure of data.

A EM Probabilistic Clustering Algorithm for Large Scale Data Sets based on Partial Constraints Information

A EM Probabilistic Clustering Algorithm for Large Scale Data Sets based on Partial Constraints Information

C^4: Exploring Multiple Solutions in Graphical Models by Cluster Sampling

This paper presents a novel Markov Chain Monte Carlo (MCMC) inference algorithm called C(4)--Clustering with Cooperative and Competitive Constraints--for computing multiple solutions from posterior probabilities defined on graphical models, including Markov random fields (MRF), conditional random fields (CRF), and hierarchical models. The graphs may have both positive and negative edges for cooperative and competitive constraints. C(4) is a probabilistic clustering algorithm in the spirit of Swendsen-Wang. By turning the positive edges on/off probabilistically, C(4) partitions the graph into a number of connected components (ccps) and each ccp is a coupled subsolution with nodes connected by positive edges. Then, by turning the negative edges on/off probabilistically, C(4) obtains composite ccps (called cccps) with competing ccps connected by negative edges. At each step, C(4) flips the labels of all nodes in a cccp so that nodes in each ccp keep the same label while different ccps are assigned different labels to observe both positive and negative constraints. Thus, the algorithm can jump between multiple competing solutions (or modes of the posterior probability) in a single or a few steps. It computes multiple distinct solutions to preserve the intrinsic ambiguities and avoids premature commitments to a single solution that may not be valid given later context. C(4) achieves a mixing rate faster than existing MCMC methods, such as various Gibbs samplers and Swendsen-Wang cuts. It is also more "dynamic" than common optimization methods such as ICM, LBP, and graph cuts. We demonstrate the C(4) algorithm in line drawing interpretation, scene labeling, and object recognition.

Naturally Occurring Human Urinary Peptides for Use in Diagnosis of Chronic Kidney Disease

Because of its availability, ease of collection, and correlation with physiology and pathology, urine is an attractive source for clinical proteomics/peptidomics. However, the lack of comparable data sets from large cohorts has greatly hindered the development of clinical proteomics. Here, we report the establishment of a reproducible, high resolution method for peptidome analysis of naturally occurring human urinary peptides and proteins, ranging from 800 to 17,000 Da, using samples from 3,600 individuals analyzed by capillary electrophoresis coupled to MS. All processed data were deposited in an Structured Query Language (SQL) database. This database currently contains 5,010 relevant unique urinary peptides that serve as a pool of potential classifiers for diagnosis and monitoring of various diseases. As an example, by using this source of information, we were able to define urinary peptide biomarkers for chronic kidney diseases, allowing diagnosis of these diseases with high accuracy. Application of the chronic kidney disease-specific biomarker set to an independent test cohort in the subsequent replication phase resulted in 85.5% sensitivity and 100% specificity. These results indicate the potential usefulness of capillary electrophoresis coupled to MS for clinical applications in the analysis of naturally occurring urinary peptides.

Identifying functional connectivity in large-scale neural ensemble recordings: a multiscale data mining approach.

Identifying functional connectivity between neuronal elements is an essential first step toward understanding how the brain orchestrates information processing at the single-cell and population levels to carry out biological computations. This letter suggests a new approach to identify functional connectivity between neuronal elements from their simultaneously recorded spike trains. In particular, we identify clusters of neurons that exhibit functional interdependency over variable spatial and temporal patterns of interaction. We represent neurons as objects in a graph and connect them using arbitrarily defined similarity measures calculated across multiple timescales. We then use a probabilistic spectral clustering algorithm to cluster the neurons in the graph by solving a minimum graph cut optimization problem. Using point process theory to model population activity, we demonstrate the robustness of the approach in tracking a broad spectrum of neuronal interaction, from synchrony to rate co-modulation, by systematically varying the length of the firing history interval and the strength of the connecting synapses that govern the discharge pattern of each neuron. We also demonstrate how activity-dependent plasticity can be tracked and quantified in multiple network topologies built to mimic distinct behavioral contexts. We compare the performance to classical approaches to illustrate the substantial gain in performance.

Fuzzy c-means clustering with prior biological knowledge

We propose a novel semi-supervised clustering method called GO Fuzzy c-means, which enables the simultaneous use of biological knowledge and gene expression data in a probabilistic clustering algorithm. Our method is based on the fuzzy c-means clustering algorithm and utilizes the Gene Ontology annotations as prior knowledge to guide the process of grouping functionally related genes. Unlike traditional clustering methods, our method is capable of assigning genes to multiple clusters, which is a more appropriate representation of the behavior of genes. Two datasets of yeast ( Saccharomyces cerevisiae) expression profiles were applied to compare our method with other state-of-the-art clustering methods. Our experiments show that our method can produce far better biologically meaningful clusters even with the use of a small percentage of Gene Ontology annotations. In addition, our experiments further indicate that the utilization of prior knowledge in our method can predict gene functions effectively. The source code is freely available at http://sysbio.fulton.asu.edu/gofuzzy/.

EDISA: extracting biclusters from multiple time-series of gene expression profiles.

BackgroundCells dynamically adapt their gene expression patterns in response to various stimuli. This response is orchestrated into a number of gene expression modules consisting of co-regulated genes. A growing pool of publicly available microarray datasets allows the identification of modules by monitoring expression changes over time. These time-series datasets can be searched for gene expression modules by one of the many clustering methods published to date. For an integrative analysis, several time-series datasets can be joined into a three-dimensional gene-condition-time dataset, to which standard clustering or biclustering methods are, however, not applicable. We thus devise a probabilistic clustering algorithm for gene-condition-time datasets.ResultsIn this work, we present the EDISA (Extended Dimension Iterative Signature Algorithm), a novel probabilistic clustering approach for 3D gene-condition-time datasets. Based on mathematical definitions of gene expression modules, the EDISA samples initial modules from the dataset which are then refined by removing genes and conditions until they comply with the module definition. A subsequent extension step ensures gene and condition maximality. We applied the algorithm to a synthetic dataset and were able to successfully recover the implanted modules over a range of background noise intensities. Analysis of microarray datasets has lead us to define three biologically relevant module types: 1) We found modules with independent response profiles to be the most prevalent ones. These modules comprise genes which are co-regulated under several conditions, yet with a different response pattern under each condition. 2) Coherent modules with similar responses under all conditions occurred frequently, too, and were often contained within these modules. 3) A third module type, which covers a response specific to a single condition was also detected, but rarely. All of these modules are essentially different types of biclusters.ConclusionWe successfully applied the EDISA to different 3D datasets. While previous studies were mostly aimed at detecting coherent modules only, our results show that coherent responses are often part of a more general module type with independent response profiles under different conditions. Our approach thus allows for a more comprehensive view of the gene expression response. After subsequent analysis of the resulting modules, the EDISA helped to shed light on the global organization of transcriptional control. An implementation of the algorithm is available at http://www-ra.informatik.uni-tuebingen.de/software/IAGEN/.

Identifying neuronal assemblies with local and global connectivity with scale space spectral clustering

A nonparametric approach is proposed to identify clusters of functionally interdependent neurons, independent of the time scale at which they are maximally correlated. The neural point processes are represented in a N-dimensional scale space using the Haar wavelet transform. A similarity measure between any given pair of neurons is defined in the scale space. Clusters of “similar” neurons are identified by first reducing the N-dimensional scale space representation using principal components to obtain a Q-dimensional space. The weighted principal components are subsequently used to connect each neuron to the others in a graph representation. A probabilistic spectral clustering algorithm is used to perform graph partitioning by maximizing cluster compactness. Performance is compared to that of the k-means and the expectation–maximization algorithms for 120 neurons with time-varying intensity functions consisting of spontaneous background activity and phased response elicited at distinct time scales.

Extending fuzzy and probabilistic clustering to very large data sets

Approximating clusters in very large (VL=unloadable) data sets has been considered from many angles. The proposed approach has three basic steps: (i) progressive sampling of the VL data, terminated when a sample passes a statistical goodness of fit test; (ii) clustering the sample with a literal (or exact) algorithm; and (iii) non-iterative extension of the literal clusters to the remainder of the data set. Extension accelerates clustering on all (loadable) data sets. More importantly, extension provides feasibility—a way to find (approximate) clusters—for data sets that are too large to be loaded into the primary memory of a single computer. A good generalized sampling and extension scheme should be effective for acceleration and feasibility using any extensible clustering algorithm. A general method for progressive sampling in VL sets of feature vectors is developed, and examples are given that show how to extend the literal fuzzy ( c -means) and probabilistic (expectation-maximization) clustering algorithms onto VL data. The fuzzy extension is called the generalized extensible fast fuzzy c -means (geFFCM) algorithm and is illustrated using several experiments with mixtures of five-dimensional normal distributions.

Digital Topography Models for Martian Surfaces

We propose to use an unsupervised automated classification of topographic features on Mars in order to speed up geomorphic and geologic mapping of the planet. We construct a digital topography model (DTM), a multilayer grid that stores various kinds of topographical information for every pixel in a site. The method uses a probabilistic clustering algorithm to assign topographically meaningful labels to all pixels in the DTM. The results are displayed as a thematic map of topography. Resultant topographical features are characterized and compared using statistics of their constituent pixels. We demonstrate the usage of our method by classifying and characterizing the topography of a landscape in the Tisia Valles region on Mars. We discuss extensions and further applications of our method.

Identification of hair cycle-associated genes from time-course gene expression profile data by using replicate variance

The hair-growth cycle is an example of a cyclic process that is well characterized morphologically but understood incompletely at the molecular level. As an initial step in discovering regulators in hair-follicle morphogenesis and cycling, we used DNA microarrays to profile mRNA expression in mouse back skin from eight representative time points. We developed a statistical algorithm to identify the set of genes expressed within skin that are associated specifically with the hair-growth cycle. The methodology takes advantage of higher replicate variance during asynchronous hair cycles in comparison with synchronous cycles. More than one-third of genes with detectable skin expression showed hair-cycle-related changes in expression, suggesting that many more genes may be associated with the hair-growth cycle than have been identified in the literature. By using a probabilistic clustering algorithm for replicated measurements, these genes were grouped into 30 time-course profile clusters, which fall into four major classes. Distinct genetic pathways were characteristic for the different time-course profile clusters, providing insights into the regulation of hair-follicle cycling and suggesting that this approach is useful for identifying hair follicle regulators. In addition to revealing known hair-related genes, we identified genes that were not previously known to be hair cycle-associated and confirmed their temporal and spatial expression patterns during the hair-growth cycle by quantitative real-time PCR and in situ hybridization. The same computational approach should be generally useful for identifying genes associated with cyclic processes from complex tissues.

Finding structure in home videos by probabilistic hierarchical clustering

Accessing, organizing, and manipulating home videos present technical challenges due to their unrestricted content and lack of storyline. We present a methodology to discover cluster structure in home videos, which uses video shots as the unit of organization, and is based on two concepts: (1) the development of statistical models of visual similarity, duration, and temporal adjacency of consumer video segments and (2) the reformulation of hierarchical clustering as a sequential binary Bayesian classification process. A Bayesian formulation allows for the incorporation of prior knowledge of the structure of home video and offers the advantages of a principled methodology. Gaussian mixture models are used to represent the class-conditional distributions of intra- and inter-segment visual and temporal features. The models are then used in the probabilistic clustering algorithm, where the merging order is a variation of highest confidence first, and the merging criterion is maximum a posteriori. The algorithm does not need any ad-hoc parameter determination. We present extensive results on a 10-h home-video database with ground truth which thoroughly validate the performance of our methodology with respect to cluster detection, individual shot-cluster labeling, and the effect of prior selection.