Number Of Mixing Components Research Articles

Covariate-modulated large-scale multiple testing under dependence

Large-scale multiple testing, which calls for conducting tens of thousands of hypothesis testings simultaneously, has been applied in many scientific fields. Most conventional multiple testing procedures often focused on the control of false discovery rate (FDR) and largely ignored covariate information and the dependence structure among tests. A FDR control procedure, termed as Covariate-Modulated Local Index of Significance (cmLIS) procedure, which not only takes into account local correlations among tests but also accommodates the covariate information by leveraging a covariate-modulated hidden Markov model (HMM), has been proposed. In the oracle case where all parameters of the covariate-modulated HMM are known, the cmLIS procedure is shown to be valid and optimal in some sense. According to whether the number of mixed components in the non-null distribution is known, two Bayesian sampling algorithms are provided for parameter estimation. Extensive simulations are conducted to demonstrate the effectiveness of the cmLIS procedure over state-of-the-art multiple testing procedures. Finally, the cmLIS procedure is applied to an RNA sequencing data and a schizophrenia (SCZ) data.

On posterior contraction of parameters and interpretability in Bayesian mixture modeling

We study posterior contraction behaviors for parameters of interest in the context of Bayesian mixture modeling, where the number of mixing components is unknown while the model itself may or may not be correctly specified. Two representative types of prior specification will be considered: one requires explicitly a prior distribution on the number of mixture components, while the other places a nonparametric prior on the space of mixing distributions. The former is shown to yield an optimal rate of posterior contraction on the model parameters under minimal conditions, while the latter can be utilized to consistently recover the unknown number of mixture components, with the help of a fast probabilistic post-processing procedure. We then turn the study of these Bayesian procedures to the realistic settings of model misspecification. It will be shown that the modeling choice of kernel density functions plays perhaps the most impactful roles in determining the posterior contraction rates in the misspecified situations. Drawing on concrete posterior contraction rates established in this paper we wish to highlight some aspects about the interesting tradeoffs between model expressiveness and interpretability that a statistical modeler must negotiate in the rich world of mixture modeling.

A novel mixture model using the multivariate normal mean–variance mixture of Birnbaum–Saunders distributions and its application to extrasolar planets

This paper presents a new finite mixture model based on the multivariate normal mean–variance mixture of Birnbaum–Saunders (NMVBS) distribution. We develop a computationally analytical EM algorithm for model fitting. Due to the dependence of this algorithm on initial values and the number of mixing components, a learning-based EM algorithm and an extended variant are proposed. Numerical simulations show that the proposed algorithms allow for better clustering performance and classification accuracy than some competing approaches. The effectiveness and prominence of the proposed methodology are also shown through an application to an extrasolar planet dataset.

Convergence rates of parameter estimation for some weakly identifiable finite mixtures

We establish minimax lower bounds and maximum likelihood convergence rates of parameter estimation for mean-covariance multivariate Gaussian mixtures, shape-rate Gamma mixtures and some variants of finite mixture models, including the setting where the number of mixing components is bounded but unknown. These models belong to what we call “weakly identifiable” classes, which exhibit specific interactions among mixing parameters driven by the algebraic structures of the class of kernel densities and their partial derivatives. Accordingly, both the minimax bounds and the maximum likelihood parameter estimation rates in these models, obtained under some compactness conditions on the parameter space, are shown to be typically much slower than the usual $n^{-1/2}$ or $n^{-1/4}$ rates of convergence.

A coarse-to-fine scheme for groupwise registration of multisensor images

Ensemble registration is concerned with a group of images that need to be registered simultaneously. It is challenging but important for many image analysis tasks such as vehicle detection and medical image fusion. To solve this problem effectively, a novel coarse-to-fine scheme for groupwise image registration is proposed. First, in the coarse registration step, unregistered images are divided into reference image set and float image set. The images of the two sets are registered based on segmented region matching. The coarse registration results are used as an initial solution for the next step. Then, in the fine registration step, a Gaussian mixture model with a local template is used to model the joint intensity of coarse-registered images. Meanwhile, a minimum message length criterion-based method is employed to determine the unknown number of mixing components. Based on this mixture model, a maximum likelihood framework is used to register a group of images. To evaluate the performance of the proposed approach, some representative groupwise registration approaches are compared on different image data sets. The experimental results show that the proposed approach has improved performance compared to conventional approaches.

A Thresholding Algorithm for Order Selection in Finite Mixture Models

Order selection is an important step in the application of finite mixture models. Classical methods such as AIC and BIC discourage complex models with a penalty directly proportional to the number of mixing components. In contrast, Chen and Khalili propose to link the penalty to two types of overfitting. In particular, they introduce a regularization penalty to merge similar subpopulations in a mixture model, where the shrinkage idea of regularized regression is seamlessly employed. However, the new method requires an effective and efficient algorithm. When the popular expectation-maximization (EM)-algorithm is used, we need to maximize a nonsmooth and nonconcave objective function in the M-step, which is computationally challenging. In this article, we show that such an objective function can be transformed into a sum of univariate auxiliary functions. We then design an iterative thresholding descent algorithm (ITD) to efficiently solve the associated optimization problem. Unlike many existing numerical approaches, the new algorithm leads to sparse solutions and thereby avoids undesirable ad hoc steps. We establish the convergence of the ITD and further assess its empirical performance using both simulations and real data examples.

Variable Selection for Naive Bayes Semisupervised Learning

This article deals with a semisupervised learning based on naive Bayes assumption. A univariate Gaussian mixture density is used for continuous input variables whereas a histogram type density is adopted for discrete input variables. The EM algorithm is used for the computation of maximum likelihood estimators of parameters in the model when we fix the number of mixing components for each continuous input variable. We carry out a model selection for choosing a parsimonious model among various fitted models based on an information criterion. A common density method is proposed for the selection of significant input variables. Simulated and real datasets are used to illustrate the performance of the proposed method.

Entropy of Mixing of Distinguishable Particles

The molar entropy of mixing yields values that depend only on the number of mixing components rather than on their chemical nature. To explain this phenomenon using the logic of chemistry, this article considers mixing of distinguishable particles, thus complementing the well-known approach developed for nondistinguishable particles, for example, ideal gases and solutions. The result is a simple quantitative treatment based on the Boltzmann distribution using the Gibbs paradox as a case study to define a macroscopic thermodynamic criterion of distinguishability. It is the presence of nonzero, albeit often small, ΔH(mixing) in a reversible process—due to a change in the energy of intermolecular interactions, Δε—that is responsible for the nonzero entropy. Temperature in the Clausius equation, dSsystem = đqrev/T, normalizes the heat—different in various processes—to yield the same entropy of mixing per mole, as long as this temperature, T = Θ′(mixing), that of the reversible process, is linked to the Boltzmann characteristic temperature, Θ = Δε/k = 2Θ′ ln 2. For mixing exactly the same substances, dSsystem = đqrev = 0, and thus, Θ′(mixing) is undefined as a 0/0 uncertainty; the process can occur reversibly for any amount at any temperature. Two cases of negative entropy of mixing/expansion validate the suggested approach.

Fast progressive training of mixture models for model selection

Finite mixture models (FMM) are flexible models with varying uses such as density estimation, clustering, classification, modeling heterogeneity, model averaging, and handling missing data. Expectation maximization (EM) algorithm can learn the maximum likelihood estimates for the model parameters. One of the prerequisites for using the EM algorithm is the a priori knowledge of the number of mixture components in the mixture model. However, the number of mixing components is often unknown. Therefore, determining the number of mixture components has been a central problem in mixture modelling. Thus, mixture modelling is often a two-stage process of determining the number of mixture components and then estimating the parameters of the mixture model. This paper proposes a fast training of a series of mixture models using progressive merging of mixture components to facilitate model selection algorithm to make appropriate choice of the model. The paper also proposes a data driven, fast approximation of the Kullback---Leibler (KL) divergence as a criterion to measure the similarity of the mixture components. We use the proposed methodology in mixture modelling of a synthetic dataset, a publicly available zoo dataset, and two chromosomal aberration datasets showing that model selection is efficient and effective.

Unimodal density estimation using Bernstein polynomials

The estimation of probability density functions is one of the fundamental aspects of any statistical inference. Many data analyses are based on an assumed family of parametric models, which are known to be unimodal (e.g., exponential family, etc.). Often a histogram suggests the unimodality of the underlying density function. Parametric assumptions, however, may not be adequate for many inferential problems. A flexible class of mixture of Beta densities that are constrained to be unimodal is presented. It is shown that the estimation of the mixing weights, and the number of mixing components, can be accomplished using a weighted least squares criteria subject to a set of linear inequality constraints. The mixing weights of the Beta mixture are efficiently computed using quadratic programming techniques. Three criteria for selecting the number of mixing weights are presented and compared in a small simulation study. More extensive simulation studies are conducted to demonstrate the performance of the density estimates in terms of popular functional norms (e.g., Lp norms). The true underlying densities are allowed to be unimodal symmetric and skewed, with finite, infinite or semi-finite supports. A code for an R function is provided which allows the user to input a data set and returns the estimated density, distribution, quantile, and random sample generating functions.

Stochastic modelling of multi-grain equivalent dose ( De) distributions: Implications for OSL dating of sediment mixtures

A number of recent optically stimulated luminescence (OSL) studies have cited post-depositional mixing as a dominant source of equivalent dose ( D e) scatter across a range of sedimentary environments, including those previously considered ‘best suited’ for OSL dating. The potentially insidious nature of sediment mixing means that this problem may often only be identifiable by careful statistical analysis of D e data sets. This study aims to address some of the important issues associated with the characterisation and statistical treatment of mixed D e distributions at the multi-grain scale of analysis, using simulated D e data sets produced with a simple stochastic model. Using this Monte Carlo approach we were able to generate theoretical distributions of single-grain D e values, which were then randomly mixed together to simulate multi-grain aliquot D e distributions containing a known number of mixing components and known corresponding burial doses. A range of sensitivity tests were undertaken using sediment mixtures with different aged dose components, different numbers of mixing components, and different types of dose component distributions (fully bleached, heterogeneously bleached and significantly overdispersed D e distributions). The results of our modelling simulations reveal the inherent problems encountered when dating mixed sedimentary samples with multi-grain D e estimation techniques. ‘Phantom’ dose components (i.e. discrete dose populations that do not correspond to the original single-grain mixing components) are an inevitable consequence of the ‘averaging’ effects of multi-grain D e analysis, and prevent the correct number of mixing components being identified with the finite mixture model (FMM) for all of the multi-grain mixtures tested. Our findings caution against use of the FMM for multi-grain aliquot D e data sets, even when the aliquots consist of only a few grains.

On Consistency of Bayesian Inference with Mixtures of Logistic Regression

This is a theoretical study of the consistency properties of Bayesian inference using mixtures of logistic regression models. When standard logistic regression models are combined in a mixtures-of-experts setup, a flexible model is formed to model the relationship between a binary (yes-no) response y and a vector of predictors x. Bayesian inference conditional on the observed data can then be used for regression and classification. This letter gives conditions on choosing the number of experts (i.e., number of mixing components) k or choosing a prior distribution for k, so that Bayesian inference is consistent, in the sense of often approximating the underlying true relationship between y and x. The resulting classification rule is also consistent, in the sense of having near-optimal performance in classification. We show these desirable consistency properties with a nonstochastic k growing slowly with the sample size n of the observed data, or with a random k that takes large values with nonzero but small probabilities.

INFERENCE IN DIRICHLET PROCESS MIXED GENERALIZED LINEAR MODELS BY USING MONTE CARLO EM

SummaryGeneralized linear mixed models are widely used for describing overdispersed and correlated data. Such data arise frequently in studies involving clustered and hierarchical designs. A more flexible class of models has been developed here through the Dirichlet process mixture. An additional advantage of using such mixture models is that the observations can be grouped together on the basis of the overdispersion present in the data. This paper proposes a partial empirical Bayes method for estimating all the model parameters by adopting a version of the EM algorithm. An augmented model that helps to implement an efficient Gibbs sampling scheme, under the non‐conjugate Dirichlet process generalized linear model, generates observations from the conditional predictive distribution of unobserved random effects and provides an estimate of the average number of mixing components in the Dirichlet process mixture. A simulation study has been carried out to demonstrate the consistency of the proposed method. The approach is also applied to a study on outdoor bacteria concentration in the air and to data from 14 retrospective lung‐cancer studies.

A Mixture‐of‐Genotypes Model for the Distribution of Thermostable Phenol Sulfotransferase Activity

AbstractA statistical method for parametric density estimation based upon a mixture‐of‐genotypes model is developed for the thermostable phenol sulfotransferase (SULT1A1) activity which has a putative role in modifying risk for colon and prostate cancer/polyps. The EM algorithm for the general mixture model is modified to accommodate the genetic constraints and is used to estimate genotype frequencies from the distribution of the SULT1A1 phenotype. A parametric bootstrap likelihood ratio test is considered as a testing method for the number of mixing components. The size and power of the test is then investigated and compared with the conventional chi‐squared test. The relative risk associated with genotypes defined by this model is also investigated through the generalized linear model. This analysis revealed that a genotype with the highest mean value of SULT1A1 activity has greater impact on cancer risk than others. This result suggests that the phenotype with a higher SULT1A1 activity might be important in studying the association between the cancer risk and SULT1A1 activity. (© 2004 WILEY‐VCH Verlag GmbH & Co. KGaA, Weinheim)

Bayesian mixture of splines for spatially adaptive nonparametric regression

A Bayesian approach is presented for spatially adaptive nonparametric regression where the regression function is modelled as a mixture of splines. Each component spline in the mixture has associated with it a smoothing parameter which is defined over a local region of the covariate space. These local regions overlap such that individual data points may lie simultaneously in multiple regions. Consequently each component spline has attached to it a weight at each point of the covariate space and, by allowing the weight of each component spline to vary across the covariate space, a spatially adaptive estimate of the regression function is obtained. The number of mixing components is chosen using a modification of the Bayesian information criteria. We study the procedure analytically and show by simulation that it compares favourably to three competing techniques. These techniques are the Bayesian regression splines estimator of Smith & Kohn (1996), the hybrid adaptive spline estimator of Luo & Wahba (1997) and the automatic Bayesian curve fitting estimator of Denison et al. (1998). The methodology is illustrated by modelling global air temperature anomalies. All the computations are carried out efficiently using Markov chain Monte Carlo.