Dirichlet Process Mixture Model Research Articles

Evaluating the measurement uncertainty at hydrogen refueling stations using a Bayesian non-parametric approach

The ability to evaluate measurement error at hydrogen refueling stations plays a vital role in the sustainability of the hydrogen vehicle industry. Most previous work in this application investigates the measurement accuracy of mass flow meters in controlled experiments, using testing equipment. The focus of our work is to estimate the measurement accuracy of fueling using data from hydrogen refueling stations collected under real operation. Accuracy is estimated by comparing the observed mass count readings with reference mass counts calculated using the pressure-volume-temperature method. To quantify the measurement uncertainty, we propose using Dirichlet process mixture models, a class of Bayesian non-parametric methods. The Dirichlet process mixture model approach is tested on five hydrogen refueling stations in real operation. Our results show that the model is able to capture the complex structure of the data and successfully estimate the probability distribution of measurement uncertainty. Our work demonstrates the effectiveness of the Bayesian non-parametric approach for evaluating the measurement uncertainty of hydrogen refueling stations.

Data-Driven-Based Stochastic Robust Optimization for a Virtual Power Plant With Multiple Uncertainties

Virtual power plant (VPP) has gradually become a key technology to the increasing penetration of renewable energy. The uncertainty and variability of renewable energy and load demand pose significant challenges to the efficiency and stability of VPP's operation. In this paper, a data-driven two-stage stochastic robust optimization (SRO) scheduling model is proposed for a VPP considering wind power, solar power, load demand, and market price uncertainties. The objective is to maximize the profit of the VPP in the energy market and minimize the total system cost of the VPP in the reserve market under the worst-case realization of the uncertainties. The Dirichlet process mixture model (DPMM) and variational inference algorithm are employed for constructing the data-driven uncertainty ambiguity set considering the correlations among multiple uncertainties. The tailored column-and-constraint generation algorithm is developed to solve the SRO model iteratively by reformulating the second stage with the application of the Karush-Kuhn-Tucker conditions. Results from a case study illustrate the effectiveness and superiority of the proposed model.

Bayesian clustering for continuous‐time hidden Markov models

We develop clustering procedures for longitudinal trajectories based on a continuous‐time hidden Markov model (CTHMM) and a generalized linear observation model. Specifically, in this article we carry out finite and infinite mixture model‐based clustering for a CTHMM and achieve inference using Markov chain Monte Carlo (MCMC). For a finite mixture model with a prior on the number of components, we implement reversible‐jump MCMC to facilitate the trans‐dimensional move between models with different numbers of clusters. For a Dirichlet process mixture model, we utilize restricted Gibbs sampling split–merge proposals to improve the performance of the MCMC algorithm. We apply our proposed algorithms to simulated data as well as a real‐data example, and the results demonstrate the desired performance of the new sampler.

Updating Variational Bayes: fast sequential posterior inference

Variational Bayesian (VB) methods produce posterior inference in a time frame considerably smaller than traditional Markov Chain Monte Carlo approaches. Although the VB posterior is an approximation, it has been shown to produce good parameter estimates and predicted values when a rich classes of approximating distributions are considered. In this paper, we propose the use of recursive algorithms to update a sequence of VB posterior approximations in an online, time series setting, with the computation of each posterior update requiring only the data observed since the previous update. We show how importance sampling can be incorporated into online variational inference allowing the user to trade accuracy for a substantial increase in computational speed. The proposed methods and their properties are detailed in two separate simulation studies. Additionally, two empirical illustrations are provided, including one where a Dirichlet Process Mixture model with a novel posterior dependence structure is repeatedly updated in the context of predicting the future behaviour of vehicles on a stretch of the US Highway 101.

Unit Commitment under Uncertainty using Data-Driven Optimization with Clustering Techniques

This paper proposes a novel robust unit commitment (UC) framework with data-driven disjunctive uncertainty sets for volatile wind power outputs, assisted by machine learning techniques. To flexibly identify the uncertainty space based on wind power forecast error data with disjunctive structures, the uncertainty data are grouped using K-means and density-based spatial clustering of applications with noise following the optimal cluster number determined by the Calinski-Harabasz index. The disjunctive uncertainty sets are constructed accordingly as the union of multiple basic uncertainty sets, including conventional box and budget uncertainty sets, and data-driven uncertainty sets using Dirichlet process mixture model, principal component analysis coupled with kernel density estimation, and support vector clustering. The problem is formulated into a two-stage adaptive robust UC model with data-driven disjunctive uncertainty sets and with a multi-level optimization structure. To facilitate the solution process, a tailored decomposition-based optimization algorithm is developed. The effectiveness of the proposed framework is illustrated using an application on the IEEE 39-bus system. The proposed approach can reduce the price of robustness by 8-38 % compared to the conventional “one-set-fits-all” approaches. Benchmarking with stochastic programming indicates that the proposed framework can achieve the same or better economic performance with over 30 % less computational time.

Clustering with label constrained Dirichlet process mixture model

In this paper, we present a constrained Dirichlet process mixture model with labels as side information. Specifically, the labeled information is incorporated through the use of a product partition prior to give clusters of instances with similar labels a higher prior preference. The proposed formulation is further extended to handle multiple side information. The empirical results on several benchmark datasets show that our method can consistently improve its clustering performance as more labeled data become available. Even in the presence of noisy labels, the proposed method rarely performs worse than its unsupervised counterpart. The effectiveness of the proposed method is also demonstrated through an application of magnetic resonance imaging for identifying major brain tissues.

Dynamic Clustering for Wafer Map Patterns Using Self-Supervised Learning on Convolutional Autoencoders

Defect pattern analysis in wafer bin maps (WBM) plays a significant role in the semiconductor manufacturing process because it helps identify problematic steps or equipment so that process engineers can take appropriate actions to improve the overall yield. Clustering algorithms have been widely used to detect different defect patterns. However, most clustering algorithms, such as K-means clustering and self-organizing map, are required to determine the number of clusters in advance. To resolve this issue, we propose a self-supervised learning-based dynamic WBM clustering method. The proposed model first uses pseudo-labeled data, of which, the labels are dynamically determined by the Dirichlet process mixture model (DPMM). Thereafter, it is fine-tuned using pseudo-labels in a self-supervised manner. Experimental results based on the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">WM-811K</i> dataset indicate that the proposed model not only outperforms the benchmark models but also demonstrates robustness to hyperparameters. In addition, the defect patterns identified by our model are more accurately and distinctively localized than those identified by the benchmark models.

An evolutionary event detection model using the Matrix Decomposition Oriented Dirichlet Process

With the huge expansion of user generated content on social networks, event detection has emerged as a major challenge and source of knowledge discovery. This knowledge is employed in different applications such as recommender systems, crisis management systems, and decision support systems. Dynamicity, overlapping, and evolutionary behavior are the most important issues in event detection. This paper proposes a novel evolutionary model for event detection to capture the dynamism and evolving behavior of events. The proposed method uses a matrix decomposition technique and a Dirichlet Process to detect events and handle their dynamicity. This model consists of two components, namely preliminary event detection and event evolvement tracking. The former component extracts preliminary events from the available data using the matrix decomposition method. Then, subsequent data is employed into a Non-Parametric Bayesian Network, namely Dirichlet Process Mixture Model to evolve the preliminary events. During the evolvement process, data may migrate between extracted events or new events may be discovered. The experimental results and comparisons with several recently developed approaches show the superiority of the proposed approach, and its ability to capture the evolutionary behavior of events over time.

Tracking clusters and anomalies in evolving data streams

AbstractData‐driven anomaly detection methods typically build a model for the normal behavior of the target system, and score each data instance with respect to this model. A threshold is invariably needed to identify data instances with high (or low) scores as anomalies. This presents a practical limitation on the applicability of such methods, since most methods are sensitive to the choice of the threshold, and it is challenging to set optimal thresholds. The issue is exacerbated in a streaming scenario, where the optimal thresholds vary with time. We present a probabilistic framework to explicitly model the normal and anomalous behaviors and probabilistically reason about the data. An extreme value theory based formulation is proposed to model the anomalous behavior as the extremes of the normal behavior. As a specific instantiation, a joint nonparametric clustering and anomaly detection algorithm (INCAD) is proposed that models the normal behavior as a Dirichlet process mixture model. Results on a variety of datasets, including streaming data, show that the proposed method provides effective and simultaneous clustering and anomaly detection without requiring strong initialization and threshold parameters.

Nonlinear random vibration analysis: A Bayesian nonparametric approach

Random vibration analysis aims to estimate the response statistics of dynamical systems subject to stochastic excitations. Stochastic differential equations (SDEs) that govern the response of general nonlinear systems are often complicated, and their analytical solutions are scarce. Thus, a range of approximate methods and simulation techniques have been developed. This paper develops a hybrid approach that approximates the governing SDE of nonlinear systems using a small number of response simulations and information available a priori. The main idea is to identify a set of surrogate linear systems such that their response probability distributions collectively estimate the response probability distribution of the original nonlinear system. To identify the surrogate linear systems, the proposed method integrates the simulated responses of the original nonlinear system with information available a priori about the number and parameters of the surrogate linear systems. There will be epistemic uncertainty in the number and parameters of the surrogate linear systems because of the limited data. This paper proposes a Bayesian nonparametric approach, called a Dirichlet Process Mixture Model, to capture these uncertainties. The Dirichlet process models the uncertainty over an infinite-dimensional parameter space, representing an infinite number of potential surrogate linear systems. Specifically, the proposed method allows the number of surrogate linear systems to grow indefinitely as the nonlinear system observed dynamic unveil new patterns. The quantified uncertainty in the estimates of the unknown model parameters propagates into the response probability distribution. The paper then shows that, under some mild conditions, the estimated probability distribution approaches, as close as desired, to the original nonlinear system’s response probability distribution. As a measure of model accuracy, the paper provides the convergence rate of the response probability distribution. Because the posterior distribution of the unknown model parameters is often not analytically tractable, a Gibbs sampling algorithm is presented to draw samples from the posterior distribution. Variational Bayesian inference is also introduced to derive an approximate closed-form expression for the posterior distribution. The paper illustrates the proposed method through the random vibration analysis of a nonlinear elastic and a nonlinear hysteretic system.

Oil Spill Detection in SAR Images Using Online Extended Variational Learning of Dirichlet Process Mixtures of Gamma Distributions

In this paper, we propose a Dirichlet process (DP) mixture model of Gamma distributions, which is an extension of the finite Gamma mixture model to the infinite case. In particular, we propose a novel online nonparametric Bayesian analysis method based on the infinite Gamma mixture model where the determination of the number of clusters is bypassed via an infinite number of mixture components. The proposed model is learned via an online extended variational Bayesian inference approach in a flexible way where the priors of model’s parameters are selected appropriately and the posteriors are approximated effectively in a closed form. The online setting has the advantage to allow data instances to be treated in a sequential manner, which is more attractive than batch learning especially when dealing with massive and streaming data. We demonstrated the performance and merits of the proposed statistical framework with a challenging real-world application namely oil spill detection in synthetic aperture radar (SAR) images.

Bayesian Nonparametric Object Association for Semantic SLAM

Semantic simultaneous localization and mapping (SLAM) can provide the foundation for robots to perform more advanced tasks than traditional geometric SLAM. Object association is a crucial factor in semantic SLAM. Previous work utilizes observations at the current moment and a single geometric feature to associate objects. The number and measurement of objects change over time. The ambiguity of object association reduces the performance of semantic SLAM. In this letter, we present a Bayesian Nonparametric model approach for object association. The temporal Dirichlet process mixture (TDPM) model is introduced to provide prior information. We couple the evolution of time with the object association, which considers the influence of historical association information on the current association results. Besides, we propose a Multi-Rules Likelihood (MRL) model to eliminate the ambiguity of object association by considering multiple object properties. We validate our approach both in simulation and real data from the KITTI dataset. The experimental results show that our approach improves the performance of object association and semantic SLAM.

Abstract 3033: Molecular classification of pediatric high-risk leukemias using expression profiles of multimodally expressed genes

Abstract Introduction: Leukemia is the most common cancer in children, accounting for approximately one third of all malignancies that occur in the pediatric age group. Acute Lymphoblastic Leukemia (ALL) and Acute Myeloid Leukemia (AML) account for most leukemia diagnosed in this age group. While known markers for poor prognosis include higher age, higher white blood cell count at diagnosis and certain translocations, innovative approaches in tumor RNA sequencing (RNA-Seq) data analysis can discover novel prognostic factors that could be exploited for future therapeutic development in fusion-negative ALL and AML. Methods: To reveal gene expression signatures among fusion-negative leukemias, we used a novel unsupervised analysis model called Hydra. Hydra uses a Dirichlet process mixture model to detect multimodally expressed genes to use in characterizing clusters within cancer cohorts. This approach can detect subtle yet robust differences in gene expression without the reliance on reference normal RNA-Seq datasets. The Hydra model reveals clusters of the cancer cohort, and differences among these clusters can be investigated by finding enriched pathways via Gene Set Enrichment Analysis (GSEA). The cluster-specific enriched pathways can be used in conjunction with survival data to determine how certain pathways are associated with outcome. This analysis used publicly available data from the National Cancer Institute (NCI) Therapeutically Applicable Research to Generate Effective Treatments (TARGET) database that was uniformly processed by the Treehouse Childhood Cancer Initiative. Results: First, 202 fusion-negative AML and fusion-negative B-cell precursor ALL samples were run through Hydra and five clusters were identified. These clusters had different enriched pathways, such as high mitochondrial activity, high cell proliferation, and high cell signaling. Though these are characteristics of all cancer cells, each cluster demonstrated that one pathway was most distinctive of those samples. Most clusters were differentiated by disease, however, one cluster with enriched heme metabolism and immunoglobulin pathways contained almost equal amounts of AML and ALL samples, suggesting that specific cohorts of AML and ALL patients had increased inflammatory response. Another cluster contained 72 AML samples and 4 ALL samples. The four ALL samples in this cluster showed lowered expression of CD19, a B-cell lineage immune marker, and elevated expression of CD14, a myeloid lineage immune marker. These ALL patients exhibited genomic characteristics of AML, which may suggest a more specialized treatment regimen. Discussion: Despite extensive characterization of pediatric high-risk leukemias using genomic approaches, there is ample opportunity to study RNA-Seq-derived gene expression profiles to help accurately diagnose and treat pediatric patients. Citation Format: Sneha S. Jariwala, Alfred Geoffrey Lyle, Jacob Pfeil, Lauren Sanders, Holly C. Beale, Ellen T. Kephart, Katrina Learned, Allison Cheney, Olena M. Vaske. Molecular classification of pediatric high-risk leukemias using expression profiles of multimodally expressed genes [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2021; 2021 Apr 10-15 and May 17-21. Philadelphia (PA): AACR; Cancer Res 2021;81(13_Suppl):Abstract nr 3033.

A BAYESIAN NONPARAMETRIC MODEL FOR INFERRING SUBCLONAL POPULATIONS FROM STRUCTURED DNA SEQUENCING DATA.

There are distinguishing features or "hallmarks" of cancer that are found across tumors, individuals, and types of cancer, and these hallmarks can be driven by specific genetic mutations. Yet, within a single tumor there is often extensive genetic heterogeneity as evidenced by single-cell and bulk DNA sequencing data. The goal of this work is to jointly infer the underlying genotypes of tumor subpopulations and the distribution of those subpopulations in individual tumors by integrating single-cell and bulk sequencing data. Understanding the genetic composition of the tumor at the time of treatment is important in the personalized design of targeted therapeutic combinations and monitoring for possible recurrence after treatment. We propose a hierarchical Dirichlet process mixture model that incorporates the correlation structure induced by a structured sampling arrangement and we show that this model improves the quality of inference. We develop a representation of the hierarchical Dirichlet process prior as a Gamma-Poisson hierarchy and we use this representation to derive a fast Gibbs sampling inference algorithm using the augment-and-marginalize method. Experiments with simulation data show that our model outperforms standard numerical and statistical methods for decomposing admixed count data. Analyses of real acute lymphoblastic leukemia cancer sequencing dataset shows that our model improves upon state-of-the-art bioinformatic methods. An interpretation of the results of our model on this real dataset reveals co-mutated loci across samples.

Multi-task learning by hierarchical Dirichlet mixture model for sparse failure prediction

Sparsity and noisy labels occur inherently in real-world data. Previously, strong assumptions were made by domain experts to use their experience and expertise to select parameters for their models. Similar approach has been adopted in machine learning for hyper-parameter setting. However, these assumptions are often subjective and are not necessarily the optimal choice. To address this problem, we propose a data-driven approach to automate model parameter learning via a Bayesian nonparametric formulation. We propose hierarchical Dirichlet process mixture model (HDPMM) as a multi-task learning framework. It is used to learn the common parameters across different datasets in the same industry. In our experiments, we verified the capability of HDPMM for multi-task learning in infrastructure failure predictions. It was done by combining HDPMM with hierarchical beta process, which is our failure prediction model. In particular, multi-task learning was used to gain additional knowledge from failure records of water supply networks managed by other utility companies to improve prediction accuracy of our model. Notably, we have achieved superior accuracy for sparse predictions than previous state-of-the-art models. Moreover, we have demonstrated the capability of our proposed model in supporting preventive maintenance of critical infrastructure.

Bayesian mixture modelling for glass refractive index measurement

Glass is a common type of physical evidence in forensic science. Broken glass recovered from a suspect may have similar physical characteristics to glass collected at a crime scene and therefore can be used as evidence. Statistical treatment of this evidence involves computing a measure of the weight of evidence. This may be done in a Bayesian framework that incorporates information from the circumstances of the crime. One of the most crucial quantities in this calculation is the assessment of the relative rarity of the characteristics of the glass, essentially the probability distribution used to model the physical characteristics of recovered glass. Typical characteristics used in casework are the elemental composition of glass and the refractive index measurement. There is a considerable body of scientific literature devoted to the modelling of this information. For example a kernel density estimation has been used to model the background population of glass based on the refractive index measurement and a multivariate Gaussian finite mixture model has been used to model the elemental composition of glass. In this paper, we present an alternative approach, the Dirichlet Process Mixture Model, to model the glass refractive index measurement in a Bayesian methodology. A key advantage is that using this method allows us to model the probability density distribution of refractive index measurements in a more flexible way.

Dynamic Dirichlet process mixture model for identifying voting coalitions in the United Nations General Assembly human rights roll call votes

ABSTRACT Scholars have been interested in the politicization of humans rights within the United Nations for some time. However, previous research typically looks at simple associations between voting coalitions and observable variables, such as geographic location or membership in international organizations. Our study is the first attempt at estimating the latent coalition structure based on the voting data. We propose a Bayesian Dynamic Dirichlet Process Mixture (DDPM) model to identify voting coalitions based on roll call vote data across multiple time periods. We also propose post-processing methods for analyzing the outputs of the DDPM model. We apply these methods to the United Nations General Assembly (UNGA) human rights roll call vote data from 1992 to 2017. We identify human rights voting coalitions in the UNGA after the Cold War, and polarizing resolutions that divide countries into different coalitions.

Modeling wildfires via marked spatio-temporal Poisson processes

From a statistical viewpoint, characteristics such as ignition time, location and duration are relevant components for wildfire modeling. The observed ignition sites and starting times constitute a space-time point pattern, and a natural framework to model this type of data is via point processes. In this work, we propose a marked Poisson process to model fire patterns in space-time, considering durations as marks. The collected data correspond to fires observed in the Valencian Community, Spain, between 2010 and 2015. The methodology relies on writing the intensity function of such a process, jointly for starting times, locations and durations, as a weighted Dirichlet process mixture model. A particular choice of the kernel that determines such mixture was made, compatible with data features. We conducted posterior inference on some characteristics of interest for understanding wildfire behavior, showing high flexibility to emulate data patterns.

New York State's 100% renewable electricity transition planning under uncertainty using a data-driven multistage adaptive robust optimization approach with machine-learning

Abstract Power system decarbonization is critical for combating climate change, and handling systems uncertainties is essential for designing robust renewable transition pathways. In this study, a bottom-up data-driven multistage adaptive robust optimization (MARO) framework is proposed to address the power systems’ renewable transition under uncertainty. To illustrate the applicability of the proposed framework, a case study for New York State is presented. Machine learning techniques, including a variational algorithm for Dirichlet process mixture model, principal component analysis, and kernel density estimation, are applied for constructing data-driven uncertainty sets, which are integrated into the proposed MARO framework to systematically handle uncertainty. The results show that the total renewable electricity transition costs under uncertainty are 21%-42% higher than deterministic planning, and the costs under the data-driven uncertainty sets are 2%-17% lower than the conventional uncertainty sets. By 2035, on-land wind and offshore wind would be the major power source for the deterministic planning case and robust optimization cases, respectively.

Bayesian subgroup analysis in regression using mixture models

Heterogeneity occurs in many regression problems, where members from different latent subgroups respond differently to the covariates of interest (e.g., treatments) even after adjusting for other covariates. A Bayesian model called the mixture of finite mixtures (MFM) can be used to identify these subgroups, a key feature of which is that the number of subgroups is modeled as a random variable and its distribution is learned from the data. The Bayesian MFM model was not commonly used in earlier applications largely due to computational difficulties. In comparison, an alternative infinite mixture model called the Dirichlet Process Mixture (DPM) model has been a main Bayesian tool for clustering even though it is a mis-specified model for many applications. The popularity of DPM is partly due to its convenient mathematical properties that enable efficient computing algorithms.A class of Bayesian models tailored to regression problems, the conditional MFMs (cMFM), are described and studied. Computing for the cMFM is developed by extending the efficient MCMC algorithms for general MFMs. Using simulation and real data examples, the cMFM is compared to existing frequentist methods, the conditional DPM, and the original MFM and DPM models that model response and covariates jointly. The cMFM is shown to be favorable in clustering accuracy and is robust to different covariates and noise distributions.