Unknown Number Of Components Research Articles

Model-Based Clustering of Categorical Data Based on the Hamming Distance

A model-based approach is developed for clustering categorical data with no natural ordering. The proposed method exploits the Hamming distance to define a family of probability mass functions to model the data. The elements of this family are then considered as kernels of a finite mixture model with an unknown number of components. Conjugate Bayesian inference has been derived for the parameters of the Hamming distribution model. The mixture is framed in a Bayesian nonparametric setting, and a transdimensional blocked Gibbs sampler is developed to provide full Bayesian inference on the number of clusters, their structure, and the group-specific parameters, facilitating the computation with respect to customary reversible jump algorithms. The proposed model encompasses a parsimonious latent class model as a special case when the number of components is fixed. Model performances are assessed via a simulation study and reference datasets, showing improvements in clustering recovery over existing approaches. Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work.

A note on estimating multivariate Gaussian mixtures with unknown number of components

. In this article, a new penalized likelihood method is proposed for finite multivariate Gaussian mixture models to conduct order selection and model estimation. The method is proved to achieve order selection consistency and have root-n convergence rate. Asymptotic normality is established for the proposed estimator. A modified EM algorithm is developed for computation. Extensive simulations and a real data analysis are conducted to illustrate the performance of our method.

Kn-out-of-n: F System with a Random Number of Components

k-out-of-n systems are among the most important models in practical engineering, they have been extensively studied in the past decades. The system fails if and only if there exist at least k failed components. Generally these systems are considered with a fixed number of components. However, many systems have a variable number of components which can be considered a random variable. In the present paper, we studied the [Formula: see text]-out-of-[Formula: see text]: F system, we assume that the system has an unknown number of components, and we search for this number, assuming it is a random variable, by optimizing the system’s cost. This variable is supposed following a power series distribution such as geometric, Poisson and logarithmic distributions. This work discusses the case where [Formula: see text] for optimization problems. Optimal replacement policies, the optimal number of components and the optimal replacement time are determined to make maintenance and avoid failure, this will minimize the mean cost rates. We started with the case of constant number of components, where we give real and approximate values of the MTTF, optimal number of components and optimal replacement time, then, we considered variable number of components, where the results are compared to the constant case. Numerical examples are also given to illustrate our results.

Inference on latent factor models for informative censoring.

This work discusses the problem of informative censoring in survival studies. A joint model for the time to event and the time to censoring is presented. Their hazard functions include a latent factor in order to identify this joint model without sacrificing the flexibility of the parametric specification. Furthermore, a fully Bayesian formulation with a semi-parametric proportional hazard function is provided. Similar latent variable models have been described in literature, but here the emphasis is on the performance of the inferential task of the resulting mixture model with unknown number of components. The posterior distribution of the parameters is estimated using Hamiltonian Monte Carlo methods implemented in Stan. Simulation studies are provided to study its performance and the methodology is implemented for the analysis of the ACTG175 clinical trial dataset yielding a better fit. The results are also compared to the non-informative censoring case to show that ignoring informative censoring may lead to serious biases.

Statistical inference for normal mixtures with unknown number of components

Statistical inference for normal mixture models with unknown number of components has long been challenging due to the issues of nonidentifiability, degenerated Fisher matrix, and boundary parameters. In this paper, a penalized likelihood estimation procedure is proposed for mixtures of normals with unknown number of components to achieve both the order selection consistency and the root-n convergence rate for the component parameters estimators. We show that the proposed new estimator could avoid being trapped in certain degenerated regions of the nonidentifiable subset of the parameter space for over-fitted normal mixture models so that a regular asymptotic quadratic Taylor expansion of the mixture log-likelihood could be derived. With a suitable penalty function on mixing proportions, the new estimator is proved to be consistent on the order selection, and have an asymptotic normal distribution. Our derived sparsity conditions also reveal some surprising but interesting differences among some commonly used penalty functions and explain why the performance of some popularly used penalty functions, such as Lasso and SCAD, provide unsatisfactory results in the order selection. Extensive simulations and a real data analysis are conducted to demonstrate the effectiveness of the newly proposed estimator.

Variable selection in finite mixture of regression models with an unknown number of components

A Bayesian framework for finite mixture models to deal with model selection and the selection of the number of mixture components simultaneously is presented. For that purpose, a feasible reversible jump Markov Chain Monte Carlo algorithm is proposed to model each component as a sparse regression model. This approach is made robust to outliers by using a prior that induces heavy tails and works well under multicollinearity and with high-dimensional data. Finally, the framework is applied to cross-sectional data investigating early warning indicators. The results reveal two distinct country groups for which estimated effects of vulnerability indicators vary considerably.

Bayesian MISE convergence rates of Polya urn based density estimators: asymptotic comparisons and choice of prior parameters

ABSTRACT Bhattacharya [Gibbs sampling based Bayesian analysis of mixtures with unknown number of components. Sankhya B. 2008;70:133–155] introduced a mixture model based on the Dirichlet process, where an upper bound on the unknown number of components is to be specified. Defining a Bayesian analogue of the mean integrated squared error (Bayesian MISE), here we consider a Bayesian asymptotic density estimation framework forobjectively specifying the upper bound, as well as the precision parameter of the Dirichlet process, such that the Bayesian MISE converges at a desired rate. As a byproduct of our approach, we also investigate Bayesian MISE convergence rate of the traditional Dirichlet process mixture model, which leads to asymptotic specification of the precision parameter. Various asymptotic issues related to the two aforementioned mixtures, including comparative performances, are also investigated. The theoretical studies, supplemented with simulation experiments, bring out the superiority of the approach of Bhattacharya (2008).

Empirical Probability Distributions with Unknown Number of Components

Empirical Probability Distributions with Unknown Number of Components

Improved Initialization of the EM Algorithm for Mixture Model Parameter Estimation

A commonly used tool for estimating the parameters of a mixture model is the Expectation–Maximization (EM) algorithm, which is an iterative procedure that can serve as a maximum-likelihood estimator. The EM algorithm has well-documented drawbacks, such as the need for good initial values and the possibility of being trapped in local optima. Nevertheless, because of its appealing properties, EM plays an important role in estimating the parameters of mixture models. To overcome these initialization problems with EM, in this paper, we propose the Rough-Enhanced-Bayes mixture estimation (REBMIX) algorithm as a more effective initialization algorithm. Three different strategies are derived for dealing with the unknown number of components in the mixture model. These strategies are thoroughly tested on artificial datasets, density–estimation datasets and image–segmentation problems and compared with state-of-the-art initialization methods for the EM. Our proposal shows promising results in terms of clustering and density-estimation performance as well as in terms of computational efficiency. All the improvements are implemented in the rebmix R package.

Clustering multivariate data using factor analytic Bayesian mixtures with an unknown number of components

Recent work on overfitting Bayesian mixtures of distributions offers a powerful framework for clustering multivariate data using a latent Gaussian model which resembles the factor analysis model. The flexibility provided by overfitting mixture models yields a simple and efficient way in order to estimate the unknown number of clusters and model parameters by Markov chain Monte Carlo (MCMC) sampling. The present study extends this approach by considering a set of eight parameterizations, giving rise to parsimonious representations of the covariance matrix per cluster. A Gibbs sampler combined with a prior parallel tempering scheme is implemented in order to approximately sample from the posterior distribution of the overfitting mixture. The parameterization and number of factors is selected according to the Bayesian Information Criterion. Identifiability issues related to label switching are dealt by post-processing the simulated output with the Equivalence Classes Representatives algorithm. The contributed method and software are demonstrated and compared to similar models estimated using the Expectation-Maximization algorithm on simulated and real datasets. The software is available online at https://CRAN.R-project.org/package=fabMix.

Data Integrative Bayesian Inference for Mixtures of Regression Models

Summary Modern data collection techniques, which often produce different types of relevant information, call for new statistical learning methods that are adapted to cope with data integration. In the paper Bayesian inference is considered for mixtures of regression models with an unknown number of components, that facilitates data integration and variable selection for high dimensional data. In the approach presented, named data integrative mixture of regressions, data integration is accomplished by introducing a new data allocation scheme that summarizes additional data in the form of an informative prior on latent variables. To cope with high dimensionality, a shrinkage-type prior is assumed on the regression parameters, and a posteriori variable selection is conducted based on Bayesian credible intervals. Posterior estimation is achieved via a Markov chain Monte Carlo algorithm. The method is validated through simulation studies and illustrated by its performance on real data.

Data-Driven Background Subtraction Algorithm for In-Camera Acceleration in Thermal Imagery

Detection of moving objects in videos is a crucial step towards successful surveillance and monitoring applications. A key component for such tasks is called background subtraction and tries to extract regions of interest from the image background for further processing or action. For this reason, its accuracy and real-time performance is of great significance. Although, effective background subtraction methods have been proposed, only a few of them take into consideration the special characteristics of thermal imagery. In this work, we propose a background subtraction scheme, which models the thermal responses of each pixel as a mixture of Gaussians with unknown number of components. Following a Bayesian approach, our method automatically estimates the mixture structure, while simultaneously it avoids over/under fitting. The pixel density estimate is followed by an efficient and highly accurate updating mechanism, which permits our system to be automatically adapted to dynamically changing operation conditions. We propose a reference implementation of our method in reconfigurable hardware achieving both adequate performance and low power consumption. Adopting a High Level Synthesis design, demanding floating point arithmetic operations are mapped in reconfigurable hardware; demonstrating fast-prototyping and on-field customization at the same time.

Clustering and variable selection in the presence of mixed variable types and missing data.

We consider the problem of model-based clustering in the presence of many correlated, mixed continuous, and discrete variables, some of which may have missing values. Discrete variables are treated with a latent continuous variable approach, and the Dirichlet process is used to construct a mixture model with an unknown number of components. Variable selection is also performed to identify the variables that are most influential for determining cluster membership. The work is motivated by the need to cluster patients thought to potentially have autism spectrum disorder on the basis of many cognitive and/or behavioral test scores. There are a modest number of patients (486) in the data set along with many (55) test score variables (many of which are discrete valued and/or missing). The goal of the work is to (1) cluster these patients into similar groups to help identify those with similar clinical presentation and (2) identify a sparse subset of tests that inform the clusters in order to eliminate unnecessary testing. The proposed approach compares very favorably with other methods via simulation of problems of this type. The results of the autism spectrum disorder analysis suggested 3 clusters to be most likely, while only 4 test scores had high (>0.5) posterior probability of being informative. This will result in much more efficient and informative testing. The need to cluster observations on the basis of many correlated, continuous/discrete variables with missing values is a common problem in the health sciences as well as in many other disciplines.

Overfitting Bayesian mixtures of factor analyzers with an unknown number of components

Recent advances on overfitting Bayesian mixture models provide a solid and straightforward approach for inferring the underlying number of clusters and model parameters in heterogeneous datasets. The applicability of such a framework in clustering correlated high dimensional data is demonstrated. For this purpose an overfitting mixture of factor analyzers is introduced, assuming that the number of factors is fixed. A Markov chain Monte Carlo (MCMC) sampler combined with a prior parallel tempering scheme is used to estimate the posterior distribution of model parameters. The optimal number of factors is estimated using information criteria. Identifiability issues related to the label switching problem are dealt by post-processing the simulated MCMC sample by relabeling algorithms. The method is benchmarked against state-of-the-art software for maximum likelihood estimation of mixtures of factor analyzers using an extensive simulation study. Finally, the applicability of the method is illustrated in publicly available data.

偏正态全波激光雷达数据的可变分量波形分解

偏正态全波激光雷达数据的可变分量波形分解

Mixture Models With a Prior on the Number of Components

ABSTRACTA natural Bayesian approach for mixture models with an unknown number of components is to take the usual finite mixture model with symmetric Dirichlet weights, and put a prior on the number of components—that is, to use a mixture of finite mixtures (MFM). The most commonly used method of inference for MFMs is reversible jump Markov chain Monte Carlo, but it can be nontrivial to design good reversible jump moves, especially in high-dimensional spaces. Meanwhile, there are samplers for Dirichlet process mixture (DPM) models that are relatively simple and are easily adapted to new applications. It turns out that, in fact, many of the essential properties of DPMs are also exhibited by MFMs—an exchangeable partition distribution, restaurant process, random measure representation, and stick-breaking representation—and crucially, the MFM analogues are simple enough that they can be used much like the corresponding DPM properties. Consequently, many of the powerful methods developed for inference in DPMs can be directly applied to MFMs as well; this simplifies the implementation of MFMs and can substantially improve mixing. We illustrate with real and simulated data, including high-dimensional gene expression data used to discriminate cancer subtypes. Supplementary materials for this article are available online.

Bayesian Analysis of Mixture Structural Equation Models With an Unknown Number of Components

Multivariate heterogenous data with latent variables are common in many fields such as biological, medical, behavioral, and social-psychological sciences. Mixture structural equation models are multivariate techniques used to examine heterogeneous interrelationships among latent variables. In the analysis of mixture models, determination of the number of mixture components is always an important and challenging issue. This article aims to develop a full Bayesian approach with the use of reversible jump Markov chain Monte Carlo method to analyze mixture structural equation models with an unknown number of components. The proposed procedure can simultaneously and efficiently select the number of mixture components and conduct parameter estimation. Simulation studies show the satisfactory empirical performance of the method. The proposed method is applied to study risk factors of osteoporotic fractures in older people.

BayesBinMix: an R Package for Model Based Clustering of Multivariate Binary Data

The BayesBinMix package offers a Bayesian framework for clustering binary data with or without missing values by fitting mixtures of multivariate Bernoulli distributions with an unknown number of components. It allows the joint estimation of the number of clusters and model parameters using Markov chain Monte Carlo sampling. Heated chains are run in parallel and accelerate the convergence to the target posterior distribution. Identifiability issues are addressed by implementing label switching algorithms. The package is demonstrated and benchmarked against the Expectation-Maximization algorithm using a simulation study as well as a real dataset.

Two Algorithms for Modeling and Tracking of Dynamic Time-Frequency Spectra

This paper considers track-before-detect algorithms that can be applied to modeling and tracking of dynamic time-frequency spectra where the number of spectral components is allowed to vary in time. The algorithms are the adaptive Dirichlet process mixture model-based Rao-Blackwellised particle filter (ADPM-RBPF) and the histogram probabilistic multihypothesis tracker with random matrices (H-PMHT-RM). The ADPM-RBPF uses a time-varying nonparametric Bayesian method based on the Dirichlet process prior to model spectral densities as mixtures of normal distributions with an unknown number of components. This approach estimates the time-varying mean and variance of each spectral mixture component and allows for the inclusion of outliers or clutter measurements. The H-PMHT-RM models each spectrum as a histogram observation of a mixture process. It uses a randomly evolving matrix to estimate the spread of each spectral component and a track management model that enables automated track initiation and termination. The algorithms are compared using simulated and experimental data.

Estimating number of components in Gaussian mixture model using combination of greedy and merging algorithm

The brain must deal with a massive flow of sensory information without receiving any prior information. Therefore, when creating cognitive models, it is important to acquire as much information as possible from the data itself. Moreover, the brain has to deal with an unknown number of components (concepts) contained in a dataset without any prior knowledge. Most of the algorithms in use today are not able to effectively copy this strategy. We propose a novel approach based on neural modelling fields theory (NMF) to overcome this problem. The algorithm combines NMF and greedy Gaussian mixture models. The novelty lies in the combination of information criterion with the merging algorithm. The performance of the algorithm was compared with other well-known algorithms and tested both on artificial and real-world datasets.