Nonparametric Bayesian Clustering Research Articles

Nonparametric Bayesian functional clustering with applications to racial disparities in breast cancer

AbstractAs we have easier access to massive data sets, functional analyses have gained more interest. However, such data sets often contain large heterogeneities, noises, and dimensionalities. When generalizing the analyses from vectors to functions, classical methods might not work directly. This paper considers noisy information reduction in functional analyses from two perspectives: functional clustering to group similar observations and thus reduce the sample size and functional variable selection to reduce the dimensionality. The complicated data structures and relations can be easily modeled by a Bayesian hierarchical model due to its flexibility. Hence, this paper proposes a nonparametric Bayesian functional clustering and peak point selection method via weighted Dirichlet process mixture (WDPM) modeling that automatically clusters and provides accurate estimations, together with conditional Laplace prior, which is a conjugate variable selection prior. The proposed method is named WDPM‐VS for short, and is able to simultaneously perform the following tasks: (1) Automatic cluster without specifying the number of clusters or cluster centers beforehand; (2) Cluster for heterogeneously behaved functions; (3) Select vibrational peak points; and (4) Reduce noisy information from the two perspectives: sample size and dimensionality. The method will greatly outperform its comparison methods in root mean squared errors. Based on this proposed method, we are able to identify biological factors that can explain the breast cancer racial disparities.

Proteome trait regulation of marine Synechococcus elemental stoichiometry under global change.

Recent studies have demonstrated regional differences in marine ecosystem C:N:P with implications for carbon and nutrient cycles. Due to strong co-variance, temperature and nutrient stress explain variability in C:N:P equally well. A reductionistic approach can link changes in individual environmental drivers with changes in biochemical traits and cell C:N:P. Thus, we quantified effects of temperature and nutrient stress on Synechococcus chemistry using laboratory chemostats, chemical analyses, and data-independent acquisition mass spectrometry proteomics. Nutrient supply accounted for most C:N:Pcell variability and induced tradeoffs between nutrient acquisition and ribosomal proteins. High temperature prompted heat-shock, whereas thermal effects via the "translation-compensation hypothesis" were only seen under P-stress. A Nonparametric Bayesian Local Clustering algorithm suggested that changes in lipopolysaccharides, peptidoglycans, and C-rich compatible solutes may also contribute to C:N:P regulation. Physiological responses match field-based trends in ecosystem stoichiometry and suggest a hierarchical environmental regulation of current and future ocean C:N:P.

Clustering spatial functional data using a geographically weighted Dirichlet process

AbstractWe propose a Bayesian nonparametric clustering approach to study the spatial heterogeneity effect for functional data observed at spatially correlated locations. We consider a geographically weighted Chinese restaurant process equipped with a conditional autoregressive prior to capture fully the spatial correlation of function curves. To sample efficiently from our model, we customize a prior called Quadratic Gamma, which ensures conjugacy. We design a Markov chain Monte Carlo algorithm to infer simultaneously the posterior distributions of the number of groups and the grouping configurations. The superior numerical performance of the proposed method over competing methods is demonstrated using simulated examples and a U.S. annual precipitation study.

Bayesian Nonparametric Model-based Clustering with Intractable Distributions: An ABC Approach

Bayesian Nonparametric Model-based Clustering with Intractable Distributions: An ABC Approach

A Bayesian Nonparametric Modeling Approach to Settlement Patterns of Pastoralists Population in Kenya

Pastoralists' settlement patterns in Kenya have been studied for decades using various statistical and mathematical models. However, traditional models have often relied on restrictive assumptions, such as the normality of the data or the linearity of relationships. In this paper, we apply a Bayesian nonparametric approach to model the settlement patterns of pastoralists in Kenya, allowing for more flexible and realistic representations of the data. We first collected settlement data for pastoralists in Kenya and compiled a database of environmental covariates, such as distance to water sources, vegetation cover, and road networks. We then applied a Bayesian nonparametric clustering method to identify distinct settlement patterns and tested the performance of the model against other commonly used clustering techniques. Our results indicate that the Bayesian nonparametric approach outperforms other clustering techniques in terms of model fit and accuracy in identifying distinct settlement patterns. Additionally, we conducted a spatial regression analysis to investigate the relationship between settlement patterns and environmental covariates, revealing that distance to water sources and road networks are significant predictors of settlement patterns. Overall, our study highlights the usefulness of Bayesian nonparametric methods in modelling settlement patterns of pastoralists in Kenya and provides valuable insights into the relationship between environmental factors and settlement patterns.

DNB: A Joint Learning Framework for Deep Bayesian Nonparametric Clustering.

Clustering algorithms based on deep neural networks have been widely studied for image analysis. Most existing methods require partial knowledge of the true labels, namely, the number of clusters, which is usually not available in practice. In this article, we propose a Bayesian nonparametric framework, deep nonparametric Bayes (DNB), for jointly learning image clusters and deep representations in a doubly unsupervised manner. In doubly unsupervised learning, we are dealing with the problem of "unknown unknowns," where we estimate not only the unknown image labels but also the unknown number of labels as well. The proposed algorithm alternates between generating a potentially unbounded number of clusters in the forward pass and learning the deep networks in the backward pass. With the help of the Dirichlet process mixtures, the proposed method is able to partition the latent representations space without specifying the number of clusters a priori. An important feature of this work is that all the estimation is realized with an end-to-end solution, which is very different from the methods that rely on post hoc analysis to select the number of clusters. Another key idea in this article is to provide a principled solution to the problem of "trivial solution" for deep clustering, which has not been much studied in the current literature. With extensive experiments on benchmark datasets, we show that our doubly unsupervised method achieves good clustering performance and outperforms many other unsupervised image clustering methods.

Human–Robot Shared Control Based on Locally Weighted Intent Prediction for a Teleoperated Hydraulic Manipulator System

Human–robot shared control provides a promising opportunity to improve operational safety and efficiency for a teleoperated hydraulic manipulator system that works in unstructured environments, such as rescue response and underwater exploration. However, the poor dynamic of the heavy-duty manipulator (e.g., oscillation tendency and response delay) brings a challenging problem on intent prediction and effective assistance. To overcome this problem, a blended shared control method is proposed via locally weighted intent prediction. First, a task learning method from demonstration is presented by task-based action primitives with the Bayesian nonparametric clustering to extract subgoals of the remote arm trajectory, especially a locally weighted chattering cancellation method is proposed to reduce unintended action primitives due to velocity fluctuations. A dynamic intent prediction method is presented based on the empirical and real-time motion information by introducing an iterative transition matrix and a locally weighted dynamic angle, respectively. A prediction-based arbitration rule is finally established to blend the controllers of the human and robot seamlessly. Comparative tests in two typical scenes were carried out on a teleoperated hydraulic manipulator system. The results validate the proposed shared control method with fewer collision accidents and reduced task completion time, and subjective feelings on operation are also improved.

Clustering Gene Expressions Using the Table Invitation Prior.

A prior for Bayesian nonparametric clustering called the Table Invitation Prior (TIP) is used to cluster gene expression data. TIP uses information concerning the pairwise distances between subjects (e.g., gene expression samples) and automatically estimates the number of clusters. TIP's hyperparameters are estimated using a univariate multiple change point detection algorithm with respect to the subject distances, and thus TIP does not require an analyst's intervention for estimating hyperparameters. A Gibbs sampling algorithm is provided, and TIP is used in conjunction with a Normal-Inverse-Wishart likelihood to cluster 801 gene expression samples, each of which belongs to one of five different types of cancer.

Analysis of Professional Basketball Field Goal Attempts via a Bayesian Matrix Clustering Approach

We propose a Bayesian nonparametric matrix clustering approach to analyze the latent heterogeneity structure in the shot selection data collected from professional basketball players in the National Basketball Association (NBA). The proposed method adopts a mixture of finite mixtures framework and fully uses the spatial information via a mixture of matrix normal distribution representation. We propose an efficient Markov chain Monte Carlo algorithm for posterior sampling that allows simultaneous inference on both the number of clusters and the cluster configurations. We also establish large-sample convergence properties for the posterior distribution. The compelling empirical performance of the proposed method is demonstrated via simulation studies and an application to shot chart data from selected players in the NBAs 2017–2018 regular season. Supplementary materials for this article are available online.

Similarity Search on Wafer Bin Map Through Nonparametric and Hierarchical Clustering

Searching for and comparing similar wafer maps can provide crucial information for root cause analysis in the manufacturing process of integrated circuits. Owing to the high dimensionality and complexity of defect patterns, comparison of similar maps in their entirety is inefficient. This paper proposes an automated similarity ranking system with a novel feature set as a reduced representation of wafer maps. To detect systematic failure patterns across wafer maps, we use nonparametric Bayesian clustering based on the Dirichlet process Gaussian mixture model, and hierarchical clustering based on the symmetric Kullback–Leibler divergence. The proposed features are efficient because they require minimal computation and storage; furthermore, they allow for highly discriminative rankings of similar failure patterns. Thus they are suitable for large-scale analysis of wafer maps. The proposed method is experimentally verified using a real wafer map dataset from a semiconductor manufacturing company, and a subset of WM-811K.

BNPmix: An R Package for Bayesian Nonparametric Modeling via Pitman-Yor Mixtures

BNPmix is an R package for Bayesian nonparametric multivariate density estimation, clustering, and regression, using Pitman-Yor mixture models, a flexible and robust generalization of the popular class of Dirichlet process mixture models. A variety of model specifications and state-of-the-art posterior samplers are implemented. In order to achieve computational efficiency, all sampling methods are written in C++ and seamless integrated into R by means of the Rcpp and RcppArmadillo packages. BNPmix exploits the ggplot2 capabilities and implements a series of generic functions to plot and print summaries of posterior densities and induced clustering of the data.

Prior selection method using likelihood confidence region and Dirichlet process Gaussian mixture model for Bayesian inference of building energy models

It is widely acknowledged that Bayesian inference is only beneficial when prior information is properly defined. However, there is no clear rule for prior selection, and it is apparently a matter of subjective selection by the domain expert(s). In other words, because the posterior inference results can vary depending on how the prior is set, a proper definition of the prior is important in terms of objectivity and accuracy for Bayesian inference of building energy models. Hence, the authors suggest a new prior selection method using Dirichlet process Gaussian mixture model (DPGMM) and the likelihood confidence region (hereafter referred to as likelihood CR). The DPGMM is a Bayesian nonparametric clustering technique that optimizes both the cluster shape and the number of clusters. Using the DPGMM, the finite probability distributions that make up the likelihood CR can be estimated, where the distribution with the highest maximum likelihood is applied as the informative prior. In this study, a reference office building of the United States Department of Energy was selected, and a surrogate model was generated using an artificial neural network. Based on a comparison between the authors’ suggestion and traditional informative (and/or non-informative) priors by domain experts, the proposed method requires only minimum information about the parameters (min and max) and performs better than the traditional approach

Bayesian nonparametric clustering as a community detection problem

A wide class of Bayesian nonparametric priors leads to the representation of the distribution of the observable variables as a mixture density with an infinite number of components. Such a representation induces a clustering structure in the data. However, due to label switching, cluster identification is not straightforward a posteriori and some post-processing of the MCMC output is usually required. Alternatively, observations can be mapped on a weighted undirected graph, where each node represents a sample item and edge weights are given by the posterior pairwise similarities. It is shown how, after building a particular random walk on such a graph, it is possible to apply a community detection algorithm, known as map equation, leading to the minimisation of the expected description length of the partition. A relevant feature of this method is that it allows for the quantification of the posterior uncertainty of the classification.

BnpC: Bayesian non-parametric clustering of single-cell mutation profiles.

MotivationThe high resolution of single-cell DNA sequencing (scDNA-seq) offers great potential to resolve intratumor heterogeneity (ITH) by distinguishing clonal populations based on their mutation profiles. However, the increasing size of scDNA-seq datasets and technical limitations, such as high error rates and a large proportion of missing values, complicate this task and limit the applicability of existing methods.ResultsHere, we introduce BnpC, a novel non-parametric method to cluster individual cells into clones and infer their genotypes based on their noisy mutation profiles. We benchmarked our method comprehensively against state-of-the-art methods on simulated data using various data sizes, and applied it to three cancer scDNA-seq datasets. On simulated data, BnpC compared favorably against current methods in terms of accuracy, runtime and scalability. Its inferred genotypes were the most accurate, especially on highly heterogeneous data, and it was the only method able to run and produce results on datasets with 5000 cells. On tumor scDNA-seq data, BnpC was able to identify clonal populations missed by the original cluster analysis but supported by Supplementary Experimental Data. With ever growing scDNA-seq datasets, scalable and accurate methods such as BnpC will become increasingly relevant, not only to resolve ITH but also as a preprocessing step to reduce data size.Availability and implementationBnpC is freely available under MIT license at https://github.com/cbg-ethz/BnpC.Supplementary information Supplementary data are available at Bioinformatics online.

A Differentially Private Big Data Nonparametric Bayesian Clustering Algorithm in Smart Grid

Smart systems, including smart grid (SG) and Internet of Things (IoT), have been playing a critical role in addressing contemporary issues. Taking full advantage of the big data generated by the smart grid can enhance the system stability and reliability, increase asset utilization, and offer better customer experience. To better support the data-driven smart grid, the machine learning technologies such as cluster analysis can be applied to process the massive data generated in smart grid. However, the process of cluster analysis may cause the disclosure of personal private information. In this paper, to achieve privacy-preserving cluster analysis in smart grid, we propose IDPC, a Differentially Private Clustering algorithm based on the Infinite Gaussian mixture model (IGMM). IDPC uses a combination of nonparametric Bayesian method and differential privacy. The nonparametric Bayesian method allows certain parameters to change along with the data and it is usually adopted in a clustering algorithm without a fixed number of clusters. The Laplace mechanism is used in data releasing process to make IDPC differentially private. We present how to make the nonparametric Bayesian clustering algorithm differentially private by adding Laplace noise. By security analysis and performance evaluation, IDPC is proved to be privacy-preserving as well as efficient.

Bayesian cluster hierarchical model for subgroup borrowing in the design and analysis of basket trials with binary endpoints.

Master protocol designs are often proposed to improve the efficiency of drug development with multiple subgroups. In the basket trial design, different subgroups can have similar biological pathogenesis pathways. Hence, a target therapy can result in similar responses. A good information sharing strategy between different subgroups can potentially improve the efficiency of evaluating treatment efficacy. In traditional hierarchical models, based on the exchangeability assumption, all subgroups are placed into the same sharing pool for cross subgroup information sharing. However, due to the heterogeneity between subgroups, there can be large differences in drug efficacy. Under such cases, strong borrowing across all subgroups is not suitable and no borrowing can be inefficient, because the treatment effect is analyzed in each subgroup separately. We propose a Bayesian cluster hierarchical model (BCHM) to improve the operating characteristics of estimating the treatment effect in multiple subgroups in basket trials. Bayesian nonparametric method is applied to dynamically calculate the number of clusters by conducting a multiple cluster classification based on subgroup outcomes. A hierarchical model is used to compute the posterior probability of the treatment effect, with the borrowing strength determined by the Bayesian nonparametric clustering and the similarities between subgroups. We apply the BCHM to clinical trials with binary endpoints. For treatment effect estimation, the BCHM yields lower mean squared error values, when compared to the independent analyses. In scenarios with a heterogeneous treatment effect, the BCHM provides lower mean squared error values compared to traditional hierarchical models. In addition, we can construct a loss function to optimize the design parameters. BCHM provides a balanced approach and smart borrowing, which yields better results in assessing the treatment effect in different scenarios compared to other conventional methods.

Task Learning, Intent Prediction, and Adaptive Blended Shared Control With Application to Excavators

Human operators of construction equipment usually exhibit strong situational awareness, which enables them to execute tasks while handling unexpected uncertainties and adapting to environmental changes. Automatic control, on the other hand, is generally capable of executing repetitive and well-defined tasks more efficiently and precisely. In dynamic and uncertain environments, significant safety and performance benefits can be derived by collaboratively achieving the tasks via the blending of control inputs from the human operator and automatic control. The focus of this article is on developing efficient methods for such blending and its application to hydraulic excavators, which involves the developments in task learning, intent prediction, and human-machine shared control. We propose a new task learning method by segmenting the tasks with the operator primitive-based segmentation (OPbS) and clustering of subgoals via Bayesian nonparametric clustering with temporal ordering (BNPC/TO). We introduce a method for dynamically predicting the operator's intent in seeking a particular subgoal by proposing an empirical stochastic transition matrix (ESTM) and a dynamic angle difference exponential (DADE). We then propose a method for blended shared control with conflict awareness extended from dynamic angle difference. Finally, we apply our algorithms and evaluate the results on a scaled hydraulic excavator test platform for a typical earth-moving task with novice learning operators and a skilled demonstration operator.

GPseudoClust: deconvolution of shared pseudo-profiles at single-cell resolution.

MotivationMany methods have been developed to cluster genes on the basis of their changes in mRNA expression over time, using bulk RNA-seq or microarray data. However, single-cell data may present a particular challenge for these algorithms, since the temporal ordering of cells is not directly observed. One way to address this is to first use pseudotime methods to order the cells, and then apply clustering techniques for time course data. However, pseudotime estimates are subject to high levels of uncertainty, and failing to account for this uncertainty is liable to lead to erroneous and/or over-confident gene clusters.ResultsThe proposed method, GPseudoClust, is a novel approach that jointly infers pseudotemporal ordering and gene clusters, and quantifies the uncertainty in both. GPseudoClust combines a recent method for pseudotime inference with non-parametric Bayesian clustering methods, efficient Markov Chain Monte Carlo sampling and novel subsampling strategies which aid computation. We consider a broad array of simulated and experimental datasets to demonstrate the effectiveness of GPseudoClust in a range of settings.Availability and implementationAn implementation is available on GitHub: https://github.com/magStra/nonparametricSummaryPSM and https://github.com/magStra/GPseudoClust.Supplementary information Supplementary data are available at Bioinformatics online.

NOn-parametric Bayesian channEls cLustering (NOBEL) Scheme for Wireless Multimedia Cognitive Radio Networks

In wireless multimedia cognitive radio networks (WMCRNs), to optimize multimedia transmissions and scarce wireless spectrum utilization, a multimedia secondary user (MSU) needs to estimate and/or identify the achievable quality of service (QoS)-levels over the available licensed channels. However, due to the lack of signaling information among MSUs and the primary users (PUs) in uncoordinated environments, identification of the achievable QoS-levels on the available licensed channels is a challenging problem and has not yet been fully explored. To address this challenge, we propose a novel NOn-parametric Bayesian channEls cLustering (NOBEL) scheme. In NOBEL, an infinite Gaussian mixture model-based collapsed Gibbs sampler is adopted to identify the achievable QoS-levels over the feature space, i.e., bitrate, packet delay variation, and packet delivery ratio on the PUs’ licensed channels. Real trace-driven evaluation results demonstrate that NOBEL outperforms other baseline clustering techniques and guarantee high accuracy from 98% to 99.5%.

Identifying Transcriptional Regulatory Modules Among Different Chromatin States in Mouse Neural Stem Cells.

Gene expression regulation is a complex process involving the interplay between transcription factors and chromatin states. Significant progress has been made toward understanding the impact of chromatin states on gene expression. Nevertheless, the mechanism of transcription factors binding combinatorially in different chromatin states to enable selective regulation of gene expression remains an interesting research area. We introduce a nonparametric Bayesian clustering method for inhomogeneous Poisson processes to detect heterogeneous binding patterns of multiple proteins including transcription factors to form regulatory modules in different chromatin states. We applied this approach on ChIP-seq data for mouse neural stem cells containing 21 proteins and observed different groups or modules of proteins clustered within different chromatin states. These chromatin-state-specific regulatory modules were found to have significant influence on gene expression. We also observed different motif preferences for certain TFs between different chromatin states. Our results reveal a degree of interdependency between chromatin states and combinatorial binding of proteins in the complex transcriptional regulatory process. The software package is available on Github at - https://github.com/BSharmi/DPM-LGCP.