Dirichlet Process Gaussian Mixture Model Research Articles

A data-driven monitoring scheme for multivariate multimodal data

Multivariate multimodal data, obtained from complex processes with multiple correlated quality characteristics and multiple operating modes, are commonly used in many applications. Online monitoring of multivariate multimodal data has been receiving increasing attention because timely detection of process problems is important. However, existing statistical process monitoring methods are often inadequate for addressing the challenges of multivariate multimodal data monitoring. For example, they may require the process data to be unimodal or identically distributed or may assume the information regarding the multimodal processes to be known a priori. Therefore, a data-driven monitoring scheme is proposed in this paper. First, the Dirichlet process Gaussian mixture model is adopted, which can depend entirely on data characteristics and automatically cluster the data without any prior knowledge. On this basis, an exponentially weighted moving average scheme is then constructed by incorporating the negative log-likelihood statistic. Using thorough simulations and a real example, we analyse and compare the performance of the model estimation and online monitoring for detecting various out-of-control scenarios. Extensive results show that our proposed monitoring scheme is sensitive to not only model parameter shifts but also model order changes. Moreover, the proposed scheme generally performs better and more robustly than existing methods.

Multi-Indicator Fused Resilience Assessment of Power Grids Considering Wind-Photovoltaic Output Uncertainty during Typhoon Disasters

Extreme weather events such as typhoons pose a serious threat to the safe operation of power grids. In the field of power system resilience assessment during typhoon disasters, a parametric typhoon wind field model combined with actual historical meteorological data has not been well adopted, and the conventional renewable energy uncertainty modeling methods are not suitable for typhoon disaster periods. In this paper, a multi-indicator fused resilience assessment strategy considering wind-photovoltaic uncertainty and component failure during typhoon disasters is proposed. Firstly, based on the actual historical meteorological data of typhoons, an uncertainty model of typhoon wind speed is established by a rolling non-parametric Dirichlet process Gaussian mixture model. Then, a spatial–temporal contingency set is constructed by considering the best-fit wind field model and stress–strength interference model for failure probability of transmission lines. On this basis, a holistic resilience assessment framework is established from the perspectives of priority, robustness, rapidity, and sustainability, and the entropy weight method combined with the technology for order preference by similarity to an ideal solution is leveraged to obtain the comprehensive resilience indicator. Finally, numerical studies are performed on the IEEE-30 bus test system to identify vulnerable lines and improve system resilience during typhoon disasters.

Probabilistic Forecasting-Based Reserve Determination Considering Multi-Temporal Uncertainty of Renewable Energy Generation

Operating reserve, among the most important ancillary services, is a powerful prescription for mitigating increasing uncertainty of renewable generation. Traditional reserve determination neglects uncertainty of renewable generation variations within the dispatch interval, which cannot gurantee sufficient reserve deliverability in real-time operation. This paper proposes a probabilistic forecasting-based reserve determination method to efficiently deal with multi-temporal uncertainty of renewable energy in power systems. A novel probabilistic forecasting approach is proposed by integrating Dirichlet process Gaussian mixture model into bootstrap-based extreme learning machine. A unified uncertainty model is constructed for depicting both interval-averaged uncertainty and intra-interval uncertainty by combing the probabilistic forecasts and linearized Itô-process model. In current business practices of many independent system operators, the coordinated determination of two well-designed reserve products, including regulating reserve and ramp capability reserve, is formulated in a two-stage robust optimization framework. A Bernstein polynomial-based model reformulation approach is then employed to handle the existing heterogeneity in the sub-models of each stage. Consequently, the integrated reserve determination is embedded in a generic two-stage robust optimization problem, which can be efficiently solved by the adopted modified column-and-constraint generation method. Finally, numerical simulations are implemented to validate the effectiveness and profitability of the proposed approach.

Self-coloring-Driven Plume Source Localization Strategy for Multiple Robots Using Dirichlet Process Gaussian Mixture Model and Mutation Random Salp Swarm Algorithm

Self-coloring-Driven Plume Source Localization Strategy for Multiple Robots Using Dirichlet Process Gaussian Mixture Model and Mutation Random Salp Swarm Algorithm

Density-adaptive registration of pointclouds based on Dirichlet Process Gaussian Mixture Models

We propose an algorithm for rigid registration of pre- and intra-operative patient anatomy, represented as pointclouds, during minimally invasive surgery. This capability is essential for development of augmented reality systems for guiding such interventions. Key challenges in this context are differences in the point density in the pre- and intra-operative pointclouds, and potentially low spatial overlap between the two. Solutions, correspondingly, must be robust to both of these phenomena. We formulated a pointclouds registration approach which considers the pointclouds after rigid transformation to be observations of a global non-parametric probabilistic model named Dirichlet Process Gaussian Mixture Model. The registration problem is solved by minimizing the Kullback-Leibler divergence in a variational Bayesian inference framework. By this means, all unknown parameters are recursively inferred, including, importantly, the optimal number of mixture model components, which ensures the model complexity efficiently matches that of the observed data. By presenting the pointclouds as KDTrees, both the data and model are expanded in a coarse-to-fine style. The scanning weight of each point is estimated by its neighborhood, imparting the algorithm with robustness to point density variations. Experiments on several datasets with different levels of noise, outliers and pointcloud overlap show that our method has a comparable accuracy, but higher efficiency than existing Gaussian Mixture Model methods, whose performance is sensitive to the number of model components.

An online collision-free trajectory generation algorithm for human–robot collaboration

The premise of human–robot collaboration is that robots have adaptive trajectory planning strategies in hybrid work cell. The aim of this paper is to propose a new online collision avoidance trajectory planning algorithm for moderate dynamic environments to insure human safety when sharing collaborative tasks. The algorithm contains two parts: trajectory generation and local optimization. Firstly, based on empirical Dirichlet Process Gaussian Mixture Model (DPGMM) distribution learning, a neural network trajectory planner called Collaborative Waypoint Planning network (CWP-net) is proposed to generate all key waypoints required for dynamic obstacle avoidance in joint space according to environmental inputs. These points are used to generate quintic spline smooth motion trajectories with velocity and acceleration constraints. Secondly, we present an improved Stochastic Trajectory Optimization for Motion Planning (STOMP) algorithm which locally optimizes the generated trajectories of CWP-net by constraining the trajectory optimization range and direction through the DPGMM model. Simulations and real experiments from an industrial use case of human–robot collaboration in the field of aircraft assembly testing show that the proposed algorithm can smoothly adjust the nominal path online and effectively avoid collisions during the collaboration. • An online trajectory planning algorithm for dynamic obstacles avoidance is proposed. • A neural network CWP-net with multiple collision-free waypoints output is designed. • An improved STOMP algorithm is proposed for local trajectory optimization. • Continuous smooth trajectories are generated and satisfy joint constraints. • This work builds a bridge between advanced offline planners and online planning tasks.

Break and Splice: A Statistical Method for Non‐Rigid Point Cloud Registration

Abstract3D object matching and registration on point clouds are widely used in computer vision. However, most existing point cloud registration methods have limitations in handling non‐rigid point sets or topology changes (e.g. connections and separations). As a result, critical characteristics such as large inter‐frame motions of the point clouds may not be accurately captured. This paper proposes a statistical algorithm for non‐rigid point sets registration, addressing the challenge of handling topology changes without the need to estimate correspondence. The algorithm uses a novel Break and Splice framework to treat the non‐rigid registration challenges as a reproduction process and a Dirichlet Process Gaussian Mixture Model (DPGMM) to cluster a pair of point sets. Labels are assigned to the source point set with an iterative classification procedure, and the source is registered to the target with the same labels using the Bayesian Coherent Point Drift (BCPD) method. The results demonstrate that the proposed approach achieves lower registration errors and efficiently registers point sets undergoing topology changes and large inter‐frame motions. The proposed approach is evaluated on several data sets using various qualitative and quantitative metrics. The results demonstrate that the Break and Splice framework outperforms state‐of‐the‐art methods, achieving an average error reduction of about 60% and a registration time reduction of about 57.8%.

Wind turbine performance degradation monitoring using DPGMM and Mahalanobis distance

Real-time monitoring of wind turbine performance degradation can improve the economics and safety of wind farms. Normal operational data can accurately reflect the generation performance of a wind turbine and in the wind-speed coordinate system these normal data constitute the “main power band”. This paper invokes a Dirichlet Process Gaussian Mixture Model (DPGMM) to cluster operational data in each horizontal power bin, and the number of Gaussian components can be determined automatically. The confidence ellipses of Gaussian components can be used to identify the contour of the main power band which is then used as baseline performance model. In the monitoring phase, Mahalanobis distance is used to judge whether new monitoring data lies outside the contour of main power band and thus should be labeled as degraded operational data. When the proportion of such data exceeds a set value in a sliding window, a wind turbine performance degradation alarm is triggered. Degradation degree and rate can quantitatively measure the severity of performance degradation. For an industrial performance degradation case caused by gearbox oil over temperature, the method proposed timely gives alarm only 12 points (2 h) later than the first degraded operational data appears and is proved to be effective.

Task adaptive siamese neural networks for open-set recognition of encrypted network traffic with bidirectional dropout

• Create highly distinguishable similarity scores by a bidirectional dropout method. • Handle task adaptive open-set recognition by a non-parametric Bayesian based metric. • Improve the confidence of similarity scores by a hierarchical cross entropy loss. • Experiments demonstrate our method achieves high performance and generalizability . Existing deep learning approaches have achieved high performance in encrypted network traffic analysis tasks. However, practical requirements such as open-set recognition on dynamically changing tasks (e.g., changes in the target website list), challenge existing methods. While few-shot learning and open-set recognition methods have been proposed for domains such as computer vision , few-shot open-set recognition for encrypted network traffic remains an unexplored area. This paper proposes a task adaptive siamese neural network for open-set recognition of encrypted network traffic with bidirectional dropout data augmentation. Our contributions are three-fold: First, we introduce generated positive and negative pairs into the siamese neural network training process to shape a more precise similarity boundary through bidirectional dropout data augmentation. Second, we utilize Dirichlet Process Gaussian Mixture Model (DPGMM) distribution to fit the similarity scores of the negative pairs constructed by the support set of each query task, and create a new open-set recognition metric. Third, by leveraging the extracted features at coarse and fine granular levels , we construct a hierarchical cross entropy loss to improve the confidence of the similarity score. Extensive experiments on a network traffic dataset and the Omniglot dataset demonstrate the superiority and generalizability of our proposed approach.

MetaDecoder: a novel method for clustering metagenomic contigs

BackgroundClustering the metagenomic contigs into potential genomes is a key step to investigate the functional roles of microbial populations. Existing algorithms have achieved considerable success with simulated or real sequencing datasets. However, accurately classifying contigs from complex metagenomes is still a challenge.ResultsWe introduced a novel clustering algorithm, MetaDecoder, which can classify metagenomic contigs based on the frequencies of k-mers and coverages. MetaDecoder was built as a two-layer model with the first layer being a GPU-based modified Dirichlet process Gaussian mixture model (DPGMM), which controls the weight of each DPGMM cluster to avoid over-segmentation by dynamically dissolving contigs in small clusters and reassigning them to the remaining clusters. The second layer comprises a semi-supervised k-mer frequency probabilistic model and a modified Gaussian mixture model for modeling the coverage based on single copy marker genes. Benchmarks on simulated and real-world datasets demonstrated that MetaDecoder can be served as a promising approach for effectively clustering metagenomic contigs.ConclusionsIn conclusion, we developed the GPU-based MetaDecoder for effectively clustering metagenomic contigs and reconstructing microbial communities from microbial data. Applying MetaDecoder on both simulated and real-world datasets demonstrated that it could generate more complete clusters with lower contamination. Using MetaDecoder, we identified novel high-quality genomes and expanded the existing catalog of bacterial genomes.6-kuJzX4_XG_RYHXitZZH9Video

Contextual anomaly detection for high-dimensional data using Dirichlet process variational autoencoder

Due to recent advances in sensing technologies, response measurements of various sensors are frequently used for system monitoring purposes. However, response data are often affected by some contextual variables, such as equipment settings and time, resulting in different patterns, even when the system is in the normal state. In this case, anomaly detection methods that do not consider contextual variables may be unable to distinguish between abnormal and normal patterns of the response data affected by the contextual variables. Motivated by this problem, we propose a method for contextual anomaly detection, particularly in the case where the response and contextual variables are both high-dimensional and complex. The proposed method is based on Variational AutoEncoders (VAEs), which are neural-network-based generative models suitable for modeling high-dimensional and complex data. The proposed method combines two VAEs: one for response variables and the other for contextual variables. Specifically, in the latent space of the VAE for contextual variables, we model the latent variables using a Dirichlet process Gaussian mixture model. Consequently, the effects of the contextual variables can be modeled using several clusters, each representing a different contextual environment. The latent contextual variables are then used as additional inputs to the other VAE’s decoder for reconstructing response data from their latent representations. We then detect the anomalies based on the negative reconstruction loss of a new response observation. The effectiveness of the proposed method is demonstrated using several benchmark datasets and a case study based on a global tire company.

Modeling Unsupervised Empirical Adaptation by DPGMM and DPGMM-RNN Hybrid Model to Extract Perceptual Features for Low-Resource ASR

Speech feature extraction is critical for ASR systems. Such successful features as MFCC and PLP use filterbank techniques to model log-scaled speech perception but fail to model the adaptation of human speech perception by hearing experiences. Infant perception that is adapted by hearing speech without text may cause permanent brain state modifications (engrams) that serve as a physical fundamental basis for lifetime speech perception formation. This realization motivates us to propose to model such an unsupervised adaptation process, where adaptation denotes perception that is affected or changed by the history of experiences, with the Dirichlet Process Gaussian Mixture Model (DPGMM) and the DPGMM-RNN hybrid model to extract perceptual features to improve ASR. Our proposed features extend MFCC features with posteriorgrams extracted from the DPGMM algorithm or the DPGMM-RNN hybrid model. Our analysis shows that the DPGMM and DPGMM-RNN model perplexities agree with infant auditory perplexity to support that the proposed features are perceptual. Our ASR results verify the effectiveness of the proposed unsupervised features in such tasks as LVCSR on WSJ and ASR on noisy low-resource telephone conversations, compared with the supervised bottleneck features from Kaldi in ASR performance.

(H)DPGMM: a hierarchy of Dirichlet process Gaussian mixture models for the inference of the black hole mass function

ABSTRACT We introduce (H)DPGMM, a hierarchical Bayesian non-parametric method based on the Dirichlet process Gaussian mixture model, designed to infer data-driven population properties of astrophysical objects without being committal to any specific physical model. We investigate the efficacy of our model on simulated data sets and demonstrate its capability to reconstruct correctly a variety of population models without the need of fine-tuning of the algorithm. We apply our method to the problem of inferring the black hole mass function given a set of gravitational wave observations from LIGO and Virgo, and find that the (H)DPGMM infers a binary black hole mass function that is consistent with previous estimates without the requirement of a theoretically motivated parametric model. Although the number of systems observed is still too small for a robust inference, (H)DPGMM confirms the presence of at least two distinct modes in the observed merging black hole mass function, hence suggesting in a model-independent fashion the presence of at least two classes of binary black hole systems.

Similarity Search on Wafer Bin Map Through Nonparametric and Hierarchical Clustering

Searching for and comparing similar wafer maps can provide crucial information for root cause analysis in the manufacturing process of integrated circuits. Owing to the high dimensionality and complexity of defect patterns, comparison of similar maps in their entirety is inefficient. This paper proposes an automated similarity ranking system with a novel feature set as a reduced representation of wafer maps. To detect systematic failure patterns across wafer maps, we use nonparametric Bayesian clustering based on the Dirichlet process Gaussian mixture model, and hierarchical clustering based on the symmetric Kullback–Leibler divergence. The proposed features are efficient because they require minimal computation and storage; furthermore, they allow for highly discriminative rankings of similar failure patterns. Thus they are suitable for large-scale analysis of wafer maps. The proposed method is experimentally verified using a real wafer map dataset from a semiconductor manufacturing company, and a subset of WM-811K.

Two Class Pruned Log Message Anomaly Detection.

Log messages are widely used in cloud servers and other systems. Millions of logs are generated each day which makes them important for anomaly detection. However, they are complex unstructured text messages which makes this task difficult. In this paper, a hybrid log message anomaly detection technique is proposed which employs pruning of positive and negative logs. Reliable positive log messages are first selected using a Gaussian mixture model algorithm. Then reliable negative logs are selected using the K-means, Gaussian mixture model and Dirichlet process Gaussian mixture model methods iteratively. It is shown that the precision for positive and negative logs with pruning is high. Anomaly detection is done using a deep learning long short-term memory network. The proposed model is evaluated using the well-known BGL, Openstack, and Thunderbird data sets. The results obtained indicate that the proposed model performs better than several well-known algorithms.

Discovering key transcriptomic regulators in pancreatic ductal adenocarcinoma using Dirichlet process Gaussian mixture model

Pancreatic Ductal Adenocarcinoma (PDAC) is the most lethal type of pancreatic cancer, late detection leading to its therapeutic failure. This study aims to determine the key regulatory genes and their impacts on the disease’s progression, helping the disease’s etiology, which is still mostly unknown. We leverage the landmark advantages of time-series gene expression data of this disease and thereby identified the key regulators that capture the characteristics of gene activity patterns in the cancer progression. We have identified the key gene modules and predicted the functions of top genes from a reconstructed gene association network (GAN). A variation of the partial correlation method is utilized to analyze the GAN, followed by a gene function prediction task. Moreover, we have identified regulators for each target gene by gene regulatory network inference using the dynamical GENIE3 (dynGENIE3) algorithm. The Dirichlet process Gaussian process mixture model and cubic spline regression model (splineTimeR) are employed to identify the key gene modules and differentially expressed genes, respectively. Our analysis demonstrates a panel of key regulators and gene modules that are crucial for PDAC disease progression.

A deep learning framework for autonomous flame detection

This paper proposes a novel framework of flame region-based convolutional neural network for autonomous flame detection. The task of flame detection is especially challenging since flames have greater diversity in colour, texture, and shape than regular rigid objects. To cope with these difficulties due to the various appearances and unclear edges of flames, a proposal generation approach is developed to effectively select candidate flame regions based on two crucial properties of flames, i.e., their dynamics and colours. The candidate flame regions together with a convolutional feature map are further processed by additional layers to output detected flames. The diversity in flame colours is well represented by approximating the distribution using a Dirichlet Process Gaussian mixture model with variational inference. The proposed framework is evaluated on publicly available videos and achieves an average frame-wise accuracy higher than 88%, which outperforms the state-of-the-art methods.

Variational Low-Rank Matrix Factorization with Multi-Patch Collaborative Learning for Hyperspectral Imagery Mixed Denoising

In this study, multi-patch collaborative learning is introduced into variational low-rank matrix factorization to suppress mixed noise in hyperspectral images (HSIs). Firstly, based on the spatial consistency and nonlocal self-similarities, the HSI is partitioned into overlapping patches with a full band. The similarity metric with fusing features is exploited to select the most similar patches and construct the corresponding collaborative patches. Secondly, considering that the latent clean HSI holds the low-rank property across the spectra, whereas the noise component does not, variational low-rank matrix factorization is proposed in the Bayesian framework for each collaborative patch. Using Gaussian distribution adaptively adjusted by a gamma distribution, the noise-free data can be learned by exploring low-rank properties of collaborative patches in the spatial/spectral domain. Additionally, the Dirichlet process Gaussian mixture model is utilized to approximate the statistical characteristics of mixed noises, which is constructed by exploiting the Gaussian distribution, the inverse Wishart distribution, and the Dirichlet process. Finally, variational inference is utilized to estimate all variables and solve the proposed model using closed-form equations. Widely used datasets with different settings are adopted to conduct experiments. The quantitative and qualitative results indicate the effectiveness and superiority of the proposed method in reducing mixed noises in HSIs.

Data-Driven Chance-Constrained Optimal Gas-Power Flow Calculation: A Bayesian Nonparametric Approach

This paper proposes a data-driven chance-constrained optimal gas-power flow (OGPF) calculation method without any prior assumption on the distribution of uncertainties of wind power generation. The Gaussian mixture model is employed to fit the uncertainty distribution, where the Bayesian nonparametric Dirichlet process is adopted to tune the component number. To facilitate the online application of the proposed methods, an online-offline double-track distribution construction approach is established, where the frequency of training the relatively time-consuming Dirichlet process Gaussian mixture model can be reduced. On account of the quadratic gas consumption expression of gas-fired generators as well as the linear decision rule based uncertainty mitigation mechanism, the chance constraints would become quadratic ones with quadratic terms of uncertainties, which makes the proposed model more intractable. An equivalent linear separable counterpart is then provided for the quadratic chance constraints, after which the intractable chance constraints could be converted into traditional linear ones. The convex-concave procedure is used to crack the nonconvex Weymouth equation in the gas network and the auxiliary quadratic equalities. Simulation results on two test systems validate the effectiveness of the proposed methods.

Parameter clustering in Bayesian functional principal component analysis of neuroscientific data.

The extraordinary advancements in neuroscientific technology for brain recordings over the last decades have led to increasingly complex spatiotemporal data sets. To reduce oversimplifications, new models have been developed to be able to identify meaningful patterns and new insights within a highly demanding data environment. To this extent, we propose a new model called parameter clustering functional principal component analysis (PCl-fPCA) that merges ideas from functional data analysis and Bayesian nonparametrics to obtain a flexible and computationally feasible signal reconstruction and exploration of spatiotemporal neuroscientific data. In particular, we use a Dirichlet process Gaussian mixture model to cluster functional principal component scores within the standard Bayesian functional PCA framework. This approach captures the spatial dependence structure among smoothed time series (curves) and its interaction with the time domain without imposing a prior spatial structure on the data. Moreover, by moving the mixture from data to functional principal component scores, we obtain a more general clustering procedure, thus allowing a higher level of intricate insight and understanding of the data. We present results from a simulation study showing improvements in curve and correlation reconstruction compared with different Bayesian and frequentist fPCA models and we apply our method to functional magnetic resonance imaging and electroencephalogram data analyses providing a rich exploration of the spatiotemporal dependence in brain time series.