Infinite Mixture Research Articles

THE GENERALIZED MARSHALL-OLKIN-KUMARASWAMY-G FAMILY OF DISTRIBUTIONS

A family of distribution is proposed by using Kumaraswamy-G ( Kw − G ) distribution as the base line distribution in the generalized Marshall-Olkin (GMO) construction. By expanding the probability density function and the survival function as infinite series the proposed family is seen as infinite mixtures of the Kw − G distribution. Series expansions of the density function for order statistics are also obtained. Moments, moment generating function, Rényi entropy, quantile function, random sample generation, asymptotes, shapes and stochastic orderings are also investigated. Maximum likelihood estimation, their large sample standard error, confidence intervals and method of moment are presented. Three real life illustrations of comparative data modeling applications with some of the important sub mode

Gene expression profiling (GEP) to identify metabolic gene signature predictive of recurrence after surgery in stage III clear-cell renal cell carcinoma (ccRCC).

355 Background: Management of patients with localized or locally advanced renal cell cancer (RCC) involves surgical resection. However, 20-40% of all localized kidney cancer patients experience a recurrence. Adjuvant treatment in high-risk kidney cancer has not proven to successfully improve overall survival (OS) and the only drug approved so far in this setting is vascular endothelial growth factor (VEGF) inhibitor, sunitinib. As multiple trials are evaluating drugs including immune checkpoint inhibitors in patients with high risk localized clear cell RCC, there is an urgent need to identify biomarkers to help with therapeutic decisions as well as for risk stratification. Here we present analysis pertaining to genes involved in metabolic reprogramming in kidney cancer. Methods: A deep transcriptomic analysis of patients with stage III ccRCC in the TCGA KIRC Firehose Legacy cohort was undertaken. Caucasian males with stage III ccRCC in the TCGA cohort for whom recurrence data was available with a minimum follow-up of 2 years were identified for the analysis. Expression profiles of differentially expressed genes were clustered using the Bayesian infinite mixture model and samples clustered using average linkage hierarchical clustering based on pairwise Pearson’s correlations as the measure of similarity. Enrichment analysis of up- and down-regulated genes was performed using logistic regression. R package was used to generate Kaplan-Meier curves and assess statistical significance of differences in overall and disease-free survival using log-rank test. Results: The cohort evaluated for this study included Caucasian male patients that remained disease free for at least 24 months after surgery (n = 22) and patients whose cancer recurred within 24 months (n = 20). We identified metabolic genes, encoding subunits of mitochondrial electron transport chain as well as malate aspartate shuttle (MAS), where loss of coordinated co-expression between these genes identified patients at risk of recurrence after surgery. The gene signature stratified the 42 patients into three subtypes: significantly enriched for a) recurrence (subtype 1), for b) disease free status (subtype 2) and b) intermediate (subtype 3). Work is underway to combine the use of individual gene expression and ratios of gene expression within the signature to optimize prognostic biomarkers for localized ccRCC. Conclusions: In this analysis, we have identified a novel metabolic gene signature which can help identify patients with localized ccRCC at high risk of recurrence after surgery to guide aggressive therapy. These findings require further validation in tumors collected from patients with stage III ccRCC.

Lifelong Incremental Reinforcement Learning With Online Bayesian Inference.

A central capability of a long-lived reinforcement learning (RL) agent is to incrementally adapt its behavior as its environment changes and to incrementally build upon previous experiences to facilitate future learning in real-world scenarios. In this article, we propose lifelong incremental reinforcement learning (LLIRL), a new incremental algorithm for efficient lifelong adaptation to dynamic environments. We develop and maintain a library that contains an infinite mixture of parameterized environment models, which is equivalent to clustering environment parameters in a latent space. The prior distribution over the mixture is formulated as a Chinese restaurant process (CRP), which incrementally instantiates new environment models without any external information to signal environmental changes in advance. During lifelong learning, we employ the expectation-maximization (EM) algorithm with online Bayesian inference to update the mixture in a fully incremental manner. In EM, the E-step involves estimating the posterior expectation of environment-to-cluster assignments, whereas the M-step updates the environment parameters for future learning. This method allows for all environment models to be adapted as necessary, with new models instantiated for environmental changes and old models retrieved when previously seen environments are encountered again. Simulation experiments demonstrate that LLIRL outperforms relevant existing methods and enables effective incremental adaptation to various dynamic environments for lifelong learning.

Bayesian inference for infinite asymmetric Gaussian mixture with feature selection

Data clustering is a fundamental unsupervised learning approach that impacts several domains such as data mining, computer vision, information retrieval, and pattern recognition. In this work, we develop a statistical framework for data clustering which uses Dirichlet processes and asymmetric Gaussian distributions. The parameters of this framework are learned using Markov Chain Monte Carlo inference approaches. We also integrate a feature selection technique to choose the features that are most informative in order to construct an appropriate model in terms of clustering accuracy. This paper reports results based on experiments that concern dynamic textures clustering as well as scene categorization.

Estimation and visualization of process states using latent variable models based on Gaussian process.

The estimation and visualization of process states are important for process control in chemical and industrial plants. Since industrial processes are related to Gaussian distributions theoretically, this study focused on Gaussian process latent variable models. Process state estimation and visualization methods are proposed using two latent variables based on the Bayesian Gaussian process latent variable model (BGPLVM), infinite warped mixture model (iWMM), and Gaussian process dynamical models (GPDM). The Tennessee Eastman process dataset was analyzed and it was confirmed that the performance of estimating the process states was highest in the order of GPDM, iWMM, and BGPLVM. Moreover, time-delayed process variables were added to the process variables to consider the process dynamics, which further improved the performance of estimating the process states. Particularly in the case of GPDM, only two latent variables could estimate the process states, with approximately 100% accuracy for four process states. Additionally, even 10 process states could be estimated with approximately 90% accuracy, and it was confirmed that the process state estimation and process state visualization could be achieved simultaneously.

On the distribution of the likelihood ratio test of independence for random sample size — a computational approach

The test of independence of two groups of variables is addressed in the case where the sample size N is considered randomly distributed. This assumption may lead to a more realist testing procedure since in many situations the sample size is not known in advance. Three sample schemes are considered where N may have a Poisson, Binomial or Hypergeometric distribution. For the case of two groups with p1 and p2 variables, it is shown that when either p1 or p2 (or both) are even the exact distribution corresponds to a finite or an infinite mixture of Exponentiated Generalized Integer Gamma distributions. In these cases a computational module is made available for the cumulative distribution function of the test statistic. When both p1 and p2 are odd, the exact distribution of the test statistic may be represented as a finite or an infinite mixture of products of independent Beta random variables whose density and cumulative distribution functions do not have a manageable closed form. Therefore, a computational approach for the evaluation of the cumulative distribution function is provided based on a numerical inversion formula originally developed for Laplace transforms. When the exact distribution is represented through infinite mixtures, an upper bound for the error of truncation of the cumulative distribution function is provided. Numerical studies are developed in order to analyze the precision of the results and the accuracy of the upper bounds proposed. A simulation study is provided in order to assess the power of the test when the sample size N is considered randomly distributed. The results are compared with the ones obtained for the fixed sample size case.

Geodesic projection of the von Mises–Fisher distribution for projection pursuit of directional data

We investigate geodesic projections of von Mises–Fisher (vMF) distributed directional data. The vMF distribution for random directions on the (p−1)-dimensional unit hypersphere Sp−1⊂Rp plays the role of multivariate normal distribution in directional statistics. For one-dimensional circle S1, the vMF distribution is called von Mises (vM) distribution. Projections onto geodesics are one of main ingredients of modeling and exploring directional data. We show that the projection of vMF distributed random directions onto any geodesic is approximately vM-distributed, albeit not exactly the same. In particular, the distribution of the geodesic-projected score is an infinite scale mixture of vM distributions. Approximations by vM distributions are given along various asymptotic scenarios including large and small concentrations (κ→∞,κ→0), high-dimensions (p→∞), and two important cases of double-asymptotics (p,κ→∞, κ∕p→c or κ∕p→λ), to support our claim: geodesic projections of the vMF are approximately vM. As one of potential applications of the result, we contemplate a projection pursuit exploration of high-dimensional directional data. We show that in a high dimensional model almost all geodesic-projections of directional data are nearly vM, thus measures of non-vM-ness are a viable candidate for projection index.

Indoor Multi-Speaker Localization Based on Bayesian Nonparametrics in the Circular Harmonic Domain

Circular microphone arrays have been used for multi-speaker localization in computational auditory scene analysis, for their high flexibility in sound field analysis, including the generation of frequency-invariant eigenbeams for wideband acoustic sources. However, the localization performance of existing circular harmonic approaches, such as circular harmonics beamformer (CHB) depends strongly on the physical characteristics (such as shape) of sensor arrays, and the level of uncertainties presented in acoustic environments (such as background noise, room reverberation, and the number of sources). These uncertainties may limit the performance or practical application of the speaker localization algorithms. To address these issues, in this paper, we present a new indoor multi-speaker localization method in the circular harmonic domain based on the acoustic holography beamforming (AHB) technique and the Bayesian nonparametrics (BNP) method. More specifically, we use the AHB technique, which combines the delay-and-sum beamforming with acoustic-holography-based virtual sensing, to generate direction of arrival (DOA) measurements in the time-frequency (TF) domain, and then design a BNP algorithm based on the infinite Gaussian mixture model (IGMM) to estimate the DOAs of the individual sources without the prior knowledge about the number of sources. These estimates may degrade in the presence of room reverberation and background noise. To address this issue, we develop a robust TF bin selection and permutation method on the basis of mixture weights, using power, power ratio and local variance estimated at each TF bin. Experiments performed on both simulated and real-data show that our method gives significantly better performance, than four recent baseline methods, in a variety of noise and reverberation levels, in terms of the root-mean-square error (RMSE) of the DOA estimation and the source detecting success rate.

Deep Infinite Mixture Models for Fault Discovery in GPON-FTTH Networks

Fault diagnosis in telecommunication networks requires extensive expert knowledge and is key to efficient network operations and high service availability. Specifically, discovering and identifying new faults occurring in the network is a challenging task. Some dominant methods in industry are based on expert systems or Bayesian networks. Both of these methods require considerable expert knowledge and time resources to construct and maintain the diagnosis system. In this paper, we propose a data driven approach for the clustering and identification of new faults, based on existing knowledge, using neural networks and infinite mixture models. In our approach deep infinite mixture models are capable of extracting interesting features from labeled data, which are then leveraged in the clustering process to identify new relevant faults in unlabeled data. We apply our method to real operational data from Fiber-to-the Home services based on Gigabit-capable Passive Optical Networks. We show that our approach can be trained end-to-end, and allows to identify and interpret new faults.

Markov Chain Monte Carlo-Based Bayesian Inference for Learning Finite and Infinite Inverted Beta-Liouville Mixture Models

Recently Inverted Beta-Liouville mixture models have emerged as an efficient paradigm for proportional positive vectors modeling and unsupervised learning. However, little attention has been devoted to investigate these generative models within discriminative classifiers. Our aim here is to reveal the structure of non-Gaussian data by generating new probabilistic SVM kernels from inverted-Beta Liouville mixture models. The inverted Beta-Liouville has a more general covariance structure and a smaller number of parameters than the inverted Dirichlet and generalized inverted Dirichlet, respectively, which makes it more practical and useful. A principled Bayesian learning algorithm is developed to accurately estimate the model’s parameters. To cope with the problem of selecting the optimal number of components, we further propose a nonparametric Bayesian learning algorithm based on an extended infinite mixture model which may have better modelling and clustering capabilities than the finite model for some applications. Finally, the resulting generative model is exploited to build several efficient probabilistic SVM kernels in order to enhance the expected clustering and modeling performance. Through a number of experimental evaluations involving visual scenes classification, text categorization and texture images discrimination, we prove the merits of the proposed work.

Self-Paced Dynamic Infinite Mixture Model for Fatigue Evaluation of Pilots’ Brains

Current brain cognitive models are insufficient in handling outliers and dynamics of electroencephalogram (EEG) signals. This article presents a novel self-paced dynamic infinite mixture model to infer the dynamics of EEG fatigue signals. The instantaneous spectrum features provided by ensemble wavelet transform and Hilbert transform are extracted to form four fatigue indicators. The covariance of log likelihood of the complete data is proposed to accurately identify similar components and dynamics of the developed mixture model. Compared with its seven peers, the proposed model shows better performance in automatically identifying a pilot's brain workload.

Some properties of the failure rate function for mixtures of Erlang distributions

We study some properties of the failure rate for a mixture of Erlang distributions. In particular, we discuss mixtures of Erlang distributions with the same scale parameter and we describe the behavior of the failure rate. Some remarks are also given for an infinite mixture of Erlangs, in particular with regard to stochastic order properties.

Patient Triage by Topic Modeling of Referral Letters: Feasibility Study.

BackgroundMusculoskeletal conditions are managed within primary care, but patients can be referred to secondary care if a specialist opinion is required. The ever-increasing demand for health care resources emphasizes the need to streamline care pathways with the ultimate aim of ensuring that patients receive timely and optimal care. Information contained in referral letters underpins the referral decision-making process but is yet to be explored systematically for the purposes of treatment prioritization for musculoskeletal conditions.ObjectiveThis study aims to explore the feasibility of using natural language processing and machine learning to automate the triage of patients with musculoskeletal conditions by analyzing information from referral letters. Specifically, we aim to determine whether referral letters can be automatically assorted into latent topics that are clinically relevant, that is, considered relevant when prescribing treatments. Here, clinical relevance is assessed by posing 2 research questions. Can latent topics be used to automatically predict treatment? Can clinicians interpret latent topics as cohorts of patients who share common characteristics or experiences such as medical history, demographics, and possible treatments?MethodsWe used latent Dirichlet allocation to model each referral letter as a finite mixture over an underlying set of topics and model each topic as an infinite mixture over an underlying set of topic probabilities. The topic model was evaluated in the context of automating patient triage. Given a set of treatment outcomes, a binary classifier was trained for each outcome using previously extracted topics as the input features of the machine learning algorithm. In addition, a qualitative evaluation was performed to assess the human interpretability of topics.ResultsThe prediction accuracy of binary classifiers outperformed the stratified random classifier by a large margin, indicating that topic modeling could be used to predict the treatment, thus effectively supporting patient triage. The qualitative evaluation confirmed the high clinical interpretability of the topic model.ConclusionsThe results established the feasibility of using natural language processing and machine learning to automate triage of patients with knee or hip pain by analyzing information from their referral letters.

Nonparametric variational learning of multivariate beta mixture models in medical applications

AbstractClustering as an essential technique has matured into a capable solution to address the gap between the growing availability of data and deriving the knowledge from them. In this paper, we propose a novel clustering method “variational learning of infinite multivariate Beta mixture models.” The motivation behind proposing this technique is the flexibility of mixture models to fit the data. This approach has the capability to infer the model complexity and estimate model parameters from the observed data automatically. Moreover, as a label‐free method, it could also address the problem of high costs of medical data labeling, which can be undertaken just by medical experts. The performance of the model is evaluated on real medical applications and compared with other similar alternatives. We demonstrate the ability of our proposed method to outperform widely used methods in the field as it has been shown in experimental results.

Infinite mixture models for operational modal analysis: An automated and principled approach

The development of a fully automated system identifier without the need for human intervention, is a key step for real-time vibration-based Structural Health Monitoring (SHM). In this paper a novel approach based on the Dirichlet Process Gaussian Mixture Model (DP-GMM) is developed in order to analyze the stabilization diagram. The aim is to separate the true physical modes from the mathematically spurious modes in a fully automated manner, whilst eliminating the need for any manually specified parameters or thresholds. The parametric Covariance Driven Stochastic Subspace Identification (SSI-Cov) is adopted to estimate the modal parameters, and consequently establish the initial stabilization diagram. From there, the use of a two-stage algorithm involving a DP-GMM is proposed to non-parametrically perform an automated cleaning of the stabilization diagram. The contributions of the paper are five-fold: (1) A probabilistic approach based on a DP-GMM is proposed to analyze the stabilization diagram. To the best knowledge of the authors, this study presents one of the first attempts of DP-GMM for full automation of Operational Modal Analysis (OMA). The method is validated using the field test data from a large-scale operating cable-stayed bridge, which has two closely-spaced modes around 3 Hz. Not only are these two complicated scenarios consistently identified, but the performance of the method with respect to the problem of missing modes is compared against a reference method based on the conventional multi-stage clustering technique used in OMA, wherein superior performance of the proposed method is demonstrated. (2) The method does not require specification of any threshold or parameter at any stage of the algorithm for cleaning the stabilization diagram, making the approach a potential for robust and fully automated modal identification. (3) Compared to many conventional multi-stage clustering techniques, the proposed approach is computationally efficient as intelligent updates are made to the model using multiple linear algebra properties. (4) New feature vectors are developed which are justified using a combination of mathematical rigor, visual understanding, and engineering intuition. (5) Due to the probabilistic nature of the method, the identification results are accompanied with uncertainty bounds. Several mathematical proofs are presented to explain the observed behavior of the uncertainty bounds.

Infinite Mixtures of Infinite Factor Analysers

Factor-analytic Gaussian mixture models are often employed as a model-based approach to clustering high-dimensional data. Typically, the numbers of clusters and latent factors must be specified in advance of model fitting, and remain fixed. The pair which optimises some model selection criterion is then chosen. For computational reasons, models in which the number of latent factors differ across clusters are rarely considered. Here the infinite mixture of infinite factor analysers (IMIFA) model is introduced. IMIFA employs a Pitman-Yor process prior to facilitate automatic inference of the number of clusters using the stick-breaking construction and a slice sampler. Furthermore, IMIFA employs multiplicative gamma process shrinkage priors to allow cluster-specific numbers of factors, automatically inferred via an adaptive Gibbs sampler. IMIFA is presented as the flagship of a family of factor-analytic mixture models, providing flexible approaches to clustering high-dimensional data. Applications to a benchmark data set, metabolomic spectral data, and a manifold learning handwritten digit example illustrate the IMIFA model and its advantageous features. These include obviating the need for model selection criteria, reducing the computational burden associated with the search of the model space, improving clustering performance by allowing cluster-specific numbers of factors, and quantifying uncertainty in the numbers of clusters and cluster-specific factors.

Detecting Alzheimer’s Dementia Degree

The diagnosis of Alzheimer’s disease (AD) faces two important issues. They are how to extract the features of the rhythms of patients with AD and how to label them and reveal the degree of dementia in patients. This study defines 14 instantaneous power indicators of dementia judgment through Hilbert marginal spectrum (HMS) from rhythm waves. A warped infinite Gaussian mixture model (WiGMM) is proposed to learn the latent variables of these indicators to detect the degree of dementia. The experimental results show that HMS-based indicators are able to reflect the cognitive function of AD patients. This proposed method has the ability to detect brain cognitive status through a warped transform and Dirichlet process parameter prior to inference.

Background subtraction using infinite asymmetric Gaussian mixture models with simultaneous feature selection

Mixture models are broadly applied in image processing domains. Related existing challenges include failure to approximate exact data shapes, estimate correct number of components, and ignore irrelevant features. In this study, the authors develop a statistical self-refinement framework for the background subtraction task by using Dirichlet Process-based asymmetric Gaussian mixture model. The parameters of this model are learned using variational inference methods. They also incorporate feature selection simultaneously within the framework to avoid noisy influence from uninformative features. To validate the proposed framework, they report their results on background subtraction tasks on 8 different datasets for infrared and visible videos.

On a class of repulsive mixture models

Finite or infinite mixture models are routinely used in Bayesian statistical practice for tasks such as clustering or density estimation. Such models are very attractive due to their flexibility and tractability. However, a common problem in fitting these or other discrete models to data is that they tend to produce a large number of overlapping clusters. Some attention has been given in the statistical literature to models that include a repulsive feature, i.e., that encourage separation of mixture components. We study here a method that has been shown to achieve this goal without sacrificing flexibility or model fit. The model is a special case of Gibbs measures, with a parameter that controls the level of repulsion that allows construction of d-dimensional probability densities whose coordinates tend to repel each other. This approach was successfully used for density regression in Quinlan et al. (J Stat Comput Simul 88(15):2931–2947, 2018). We detail some of the global properties of the repulsive family of distributions and offer some further insight by means of a small simulation study.

Identifiability of nonparametric mixture models and Bayes optimal clustering

Motivated by problems in data clustering, we establish general conditions under which families of nonparametric mixture models are identifiable by introducing a novel framework involving clustering overfitted parametric (i.e., misspecified) mixture models. These identifiability conditions generalize existing conditions in the literature and are flexible enough to include, for example, mixtures of infinite Gaussian mixtures. In contrast to the recent literature, we allow for general nonparametric mixture components and instead impose regularity assumptions on the underlying mixing measure. As our primary application we apply these results to partition-based clustering, generalizing the notion of a Bayes optimal partition from classical parametric model-based clustering to nonparametric settings. Furthermore, this framework is constructive, so that it yields a practical algorithm for learning identified mixtures, which is illustrated through several examples on real data. The key conceptual device in the analysis is the convex, metric geometry of probability measures on metric spaces and its connection to the Wasserstein convergence of mixing measures. The result is a flexible framework for nonparametric clustering with formal consistency guarantees.