Robust Mixture Model Research Articles

A Pilot Study of Nulling in 22 Pulsars Using Mixture Modeling

The phenomenon of pulsar nulling, observed as the temporary inactivity of a pulsar, remains poorly understood both observationally and theoretically. Most observational studies that quantify nulling employ a variant of Ritchings algorithm, which can suffer significant biases for pulsars where the emission is weak. Using a more robust mixture model method, we study pulsar nulling in a sample of 22 recently discovered pulsars, for which we publish the nulling fractions for the first time. These data clearly demonstrate biases of the former approach and show how an otherwise nonnulling pulsar can be classified as having significant nulls. We show that the population-wide studies that find a positive correlation of nulling with pulsar period/characteristic age can similarly be biased because of the bias in estimating the nulling fraction. We use our probabilistic approach to find the evidence for periodicity in the nulls in a subset of three pulsars in our sample. In addition, we also provide improved timing parameters for 17 of the 22 pulsars that had no prior follow-up.

Maternal serum perfluoroalkyl substance mixtures and thyroid hormone concentrations in maternal and cord sera: The HOME Study

BackgroundPer- and polyfluoroalkyl substances (PFAS) are ubiquitous. Previous studies have found associations between PFAS and thyroid hormones in maternal and cord sera, but the results are inconsistent. To further address this research question, we used mixture modeling to assess the associations with individual PFAS, interactions among PFAS chemicals, and the overall mixture. MethodsWe collected data through the Health Outcomes and Measures of the Environment (HOME) Study, a prospective cohort study that between 2003 and 2006 enrolled 468 pregnant women and their children in the greater Cincinnati, Ohio region. We assessed the associations of maternal serum PFAS concentrations measured during pregnancy with maternal (n = 185) and cord (n = 256) sera thyroid stimulating hormone (TSH), total thyroxine (TT4), total triiodothyronine (TT3), free thyroxine (FT4), and free triiodothyronine (FT3) using two mixture modeling approaches (Bayesian kernel machine regression (BKMR) and quantile g-computation) and multivariable linear regression. Additional models considered thyroid autoantibodies, other non-PFAS chemicals, and iodine deficiency as potential confounders or effect measure modifiers. ResultsPFAS, considered individually or as mixtures, were generally not associated with any thyroid hormones. A doubling of perfluorooctanesulfonic acid (PFOS) had a positive association with cord serum TSH in BKMR models but the 95% Credible Interval included the null (β = 0.09; 95% CrI: −0.08, 0.27). Using BKMR and multivariable models, we found that among children born to mothers with higher thyroid peroxidase antibody (TPOAb), perfluorooctanoic acid (PFOA), PFOS, and perfluorohexanesulfonic acid (PFHxS) were associated with decreased cord FT4 suggesting modification by maternal TPOAb status. ConclusionsThese findings suggest that maternal serum PFAS concentrations measured in the second trimester of pregnancy are not strongly associated with thyroid hormones in maternal and cord sera. Further analyses using robust mixture models in other cohorts are required to corroborate these findings.

Pseudo-stochastic EM for sub-Gaussian α-stable mixture models

Due to the non-existence of a closed-form expression for sub-Gaussian α-stable densities, the M-step of the Expectation-Maximization (EM) algorithm for the Sub-Gaussian α-Stable Mixture Models (SGαSMMs) is intractable, and the EM algorithm for SGαSMM is still an open problem. SGαSMM can model non-homogeneous Gaussian data, accommodate outliers, and high leverage data points, which are concepts of primary importance in robust mixture models. These models are robust and useful tools in modeling heterogeneous data with outlier observations, such as clustering financial or impulsive data. In this paper, a new EM algorithm based on a combination of EM (the first part) and stochastic EM (the second part) algorithms, is used to obtain the maximum likelihood estimators of the parameters of SGαSMM in the M-step. In the first part, the model parameters, except αs, are estimated from an analytical form via EM. In the second part, based on a stochastic EM, the maximum likelihood estimator of α, in each component, is calculated from pseudo-simulated data obtained by suitable rejection sampling. The efficiency of the proposed algorithm is illustrated by using both real and simulated data.

Semi-Supervised Robust Mixture Models in RKHS for Abnormality Detection in Medical Images

Abnormality detection in medical images is a one-class classification problem for which existing methods typically involve variants of kernel principal component analysis or one-class support vector machines. However, existing methods rely on highly-curated training sets with full supervision, often using heuristics for model fitting or ignore the variances of the data within principal subspaces. In contrast, we propose novel methods that can work with imperfectly curated datasets using robust statistical learning, by extending the multivariate generalized-Gaussian distribution to a reproducing kernel Hilbert space (RKHS) and employing it within a mixture model. We propose a novel semi-supervised extension of our learning scheme, showing that a small amount of expert feedback through high-quality labeled data of the outlier class can boost performance. We propose expectation maximization for our semi-supervised robust mixture-model learning in RKHS, using solely the Gram matrix and without the explicit lifting map. Our methods incorporate optimal component means, principal directions, and variances for abnormality detection. Results on four large public datasets on retinopathy and cancer, compared against a variety of contemporary methods, show that our method gives benefits over the state of the art in one-class classification for abnormality detection.

Economic Decisions through Circuit Inhibition.

Economic choices between goods are thought to rely on the orbitofrontal cortex (OFC), but the decision mechanisms remain poorly understood. To shed light on this fundamental issue, we recorded from the OFC of monkeys choosing between two juices offered sequentially. An analysis of firing rates across time windows revealed the presence of different groups of neurons similar to those previously identified under simultaneous offers. This observation suggested that economic decisions in the two modalities are formed in the same neural circuit. We then examined several hypotheses on the decision mechanisms. OFC neurons encoded good identities and values in a juice-based representation (labeled lines). Contrary to previous assessments, our data argued against the idea that decisions rely on mutual inhibition at the level of offer values. In fact, we showed that previous arguments for mutual inhibition were confounded by differences in value ranges. Instead, decisions seemed to involve mechanisms of circuit inhibition, whereby each offer value indirectly inhibited neurons encoding the opposite choice outcome. Our results reconcile a variety of previous findings and provide a general account for the neuronal underpinnings of economic choices.

Robust Mixture Modeling Based on Two-Piece Scale Mixtures of Normal Family

In this paper, we examine the finite mixture (FM) model with a flexible class of two-piece distributions based on the scale mixtures of normal (TP-SMN) family components. This family allows the development of a robust estimation of FM models. The TP-SMN is a rich class of distributions that covers symmetric/asymmetric and light/heavy tailed distributions. It represents an alternative family to the well-known scale mixtures of the skew normal (SMSN) family studied by Branco and Dey (2001). Also, the TP-SMN covers the SMN (normal, t, slash, and contaminated normal distributions) as the symmetric members and two-piece versions of them as asymmetric members. A key feature of this study is using a suitable hierarchical representation of the family to obtain maximum likelihood estimates of model parameters via an EM-type algorithm. The performances of the proposed robust model are demonstrated using simulated and real data, and then compared to other finite mixture of SMSN models.

A General Method for Robust Bayesian Modeling

Robust Bayesian models are appealing alternatives to standard models, providing protection from data that contains outliers or other departures from the model assumptions. Historically, robust models were mostly developed on a case-by-case basis; examples include robust linear regression, robust mixture models, and bursty topic models. In this paper we develop a general approach to robust Bayesian modeling. We show how to turn an existing Bayesian model into a robust model, and then develop a generic computational strategy for it. We use our method to study robust variants of several models, including linear regression, Poisson regression, logistic regression, and probabilistic topic models. We discuss the connections between our methods and existing approaches, especially empirical Bayes and James–Stein estimation.

Robust mixture modeling reveals category-free selectivity in reward region neuronal ensembles.

Classification of neurons into clusters based on their response properties is an important tool for gaining insight into neural computations. However, it remains unclear to what extent neurons fall naturally into discrete functional categories. We developed a Bayesian method that models the tuning properties of neural populations as a mixture of multiple types of task-relevant response patterns. We applied this method to data from several cortical and striatal regions in economic choice tasks. In all cases, neurons fell into only two clusters: one multiple-selectivity cluster containing all cells driven by task variables of interest and another of no selectivity for those variables. The single cluster of task-sensitive cells argues against robust categorical tuning in these areas. The no-selectivity cluster was unanticipated and raises important questions about what distinguishes these neurons and what role they play. Moreover, the ability to formally identify these nonselective cells allows for more accurate measurement of ensemble effects by excluding or appropriately down-weighting them in analysis. Our findings provide a valuable tool for analysis of neural data, challenge simple categorization schemes previously proposed for these regions, and place useful constraints on neurocomputational models of economic choice and control. NEW & NOTEWORTHY We present a Bayesian method for formally detecting whether a population of neurons can be naturally classified into clusters based on their response tuning properties. We then examine several data sets of reward system neurons for variables and find in all cases that neurons can be classified into only two categories: a functional class and a non-task-driven class. These results provide important constraints for neural models of the reward system.

EM algorithm for symmetric stable mixture model

ABSTRACTMixture models are frequently used for modeling complex data. An extension of the EM algorithm, here called ECME, is proposed to compute the maximum likelihood estimate of parameters of symmetric -stable mixture model (SSMM). Comprehensive simulation studies are performed to show the performance of the proposed ECME algorithm. The robustness of the SSMM is investigated by simulations when it is used to model data generated from mixture of exponential power and t distributions. Both proposed ECME and Bayesian approaches are applied to three sets of real data, which shows that the proposed ECME algorithm outperforms the Bayesian paradigm for all three sets. Also, the SSMM is compared with the mixture of normal, skew normal, t, and skew t distributions for modeling four sets of real data. It turns out that the SSMM works as well as or better than above models. This can be considered as SSMM capability in robust mixture modeling.

Uniform Cross-entropy Clustering

Robust mixture models approaches, which use non-normal distributions have recently been upgraded to accommodate data with fixed bounds. In this article we propose a new method based on uniform distributions and Cross-Entropy Clustering (CEC). We combine a simple density model with a clustering method which allows to treat groups separately and estimate parameters in each cluster individually. Consequently, we introduce an effective clustering algorithm which deals with non-normal data. Keywords: Clustering, Cross-entropy, Uniform distribution

General split gaussian Cross–Entropy clustering

Robust mixture models approaches, which use non-normal distributions have recently been upgraded to accommodate asymmetric data. In this article we propose a new method based on the General Split Gaussian distribution (GSG) and Cross–Entropy Clustering (CEC). The GSG is a flexible density with reasonably small number of parameters which are easy to estimate. We combine the model with a clustering method which allows to treat groups separately and estimate parameters individually in each cluster. Consequently, we introduce an effective clustering algorithm which deals with non-normal data.

Forest canopy-structure characterization: A data-driven approach

Forest canopy structure influences and partitions the energy fluxes between the atmosphere and vegetation. It serves as an indicator of a variety of biophysical variables and ecosystem goods and services. Airborne laser scanning (ALS) can simultaneously provide horizontal and vertical information on canopy structure. Existing approaches to assess canopy structure often focus on in situ collected structural variables and require a substantial set of prior information about stand characteristics. They also rely on pre-defined spatial units and are usually dependent on site-specific model calibrations. We propose a method to provide quantitative canopy-structure descriptors on different scales, retrieved from ALS data. The approach includes (i) a sensitivity assessment and a quantification of ALS-derived canopy-structure information dependent on ALS data properties, (ii) an automatic determination of the most feasible spatial unit for canopy-structure characterization, and (iii) the derivation of canopy-structure types (CSTs) using a hierarchical, multi-scale classification approach based on Bayesian robust mixture models (BRMM), satisfying structurally homogenous criteria without the use of in situ calibration information. The CSTs resulted in retrievals of canopy layering (single-, two-, and multi-layered canopies) and canopy types (deciduous or evergreen canopies). Retrievals classified seven CSTs with accuracies ranging from 52% to 82% user accuracy (canopy layering) and 89–99% user accuracy (canopy type). The method supports a data-driven approach, allowing for an efficient monitoring of canopy structure.

Outlier detection and robust mixture modeling using nonconvex penalized likelihood

Finite mixture models are widely used in a variety of statistical applications. However, the classical normal mixture model with maximum likelihood estimation is prone to the presence of only a few severe outliers. We propose a robust mixture modeling approach using a mean-shift formulation coupled with nonconvex sparsity-inducing penalization, to conduct simultaneous outlier detection and robust parameter estimation. An efficient iterative thresholding-embedded EM algorithm is developed to maximize the penalized log-likelihood. The efficacy of our proposed approach is demonstrated via simulation studies and a real application on Acidity data analysis.

Parsimonious skew mixture models for model-based clustering and classification

Robust mixture modeling approaches using skewed distributions have recently been explored to accommodate asymmetric data. Parsimonious skew-t and skew-normal analogues of the GPCM family that employ an eigenvalue decomposition of a scale matrix are introduced. The methods are compared to existing models in both unsupervised and semi-supervised classification frameworks. Parameter estimation is carried out using the expectation–maximization algorithm and models are selected using the Bayesian information criterion. The efficacy of these extensions is illustrated on simulated and real data sets.

Robust Mixture of Linear Regression Models

This article proposes a robust model for parameter estimation when modeling in the presence of latent heterogeneity of population. Mixture of regression models are used for modeling when the data are a mixture of subgroups to allow for heterogeneity of the population. Parameter estimates from standard mixture of linear regression models are sensitive to atypical observations. To study mixtures of linear regression models, we introduce a class of robust estimators, called S-estimators. We investigate their breakdown point in mixture of linear regression models. It is expected that the robust S-estimators can achieve the high breakdown point in the contaminated data from the heterogenous populations. This model presents a unified, robust framework and parameter estimation is achieved via an expectation-conditional maximization (ECM) algorithm. This new family of robust mixture models is validated through Monte Carlo simulations. The application to real life data sets has shown that use of robust S-estimators in mixture of linear regression models accommodates the outliers producing stable estimates as compared to maximum likelihood estimators from mixture of linear regression models.

Estimating the Confidence of Peptide Identifications without Decoy Databases

Using decoy databases to compute the confidence of peptide identifications has become the standard procedure for mass spectrometry driven proteomics. While decoy databases have numerous advantages, they double the run time and are not applicable to all peptide identification problems such as error-tolerant or de novo searches or the large-scale identification of cross-linked peptides. Instead, we propose a fast, simple and robust mixture modeling approach to estimate the confidence of peptide identifications without the need for decoy database searches, which automatically checks whether its underlying assumptions are fulfilled. This approach is then evaluated on 41 LC/MS data sets of varying complexity and origin. The results are very similar to those of the decoy database strategy at a negligible computational cost. Our approach is applicable not only to standard protein identification workflows, but also to proteomics problems for which meaningful decoy databases cannot be constructed.

Robust mixture modeling based on scale mixtures of skew-normal distributions

A flexible class of probability distributions, convenient for modeling data with skewness behavior, discrepant observations and population heterogeneity is presented. The elements of this family are convex linear combinations of densities that are scale mixtures of skew-normal distributions. An EM-type algorithm for maximum likelihood estimation is developed and the observed information matrix is obtained. These procedures are discussed with emphasis on finite mixtures of skew-normal, skew-t, skew-slash and skew contaminated normal distributions. In order to examine the performance of the proposed methods, some simulation studies are presented to show the advantage of this flexible class in clustering heterogeneous data and that the maximum likelihood estimates based on the EM-type algorithm do provide good asymptotic properties. A real data set is analyzed, illustrating the usefulness of the proposed methodology.

Robust mixture modeling using multivariate skew t distributions

This paper presents a robust mixture modeling framework using the multivariate skew t distributions, an extension of the multivariate Student's t family with additional shape parameters to regulate...

Robust Bayesian clustering

A new variational Bayesian learning algorithm for Student-t mixture models is introduced. This algorithm leads to (i) robust density estimation, (ii) robust clustering and (iii) robust automatic model selection. Gaussian mixture models are learning machines which are based on a divide-and-conquer approach. They are commonly used for density estimation and clustering tasks, but are sensitive to outliers. The Student-t distribution has heavier tails than the Gaussian distribution and is therefore less sensitive to any departure of the empirical distribution from Gaussianity. As a consequence, the Student-t distribution is suitable for constructing robust mixture models. In this work, we formalize the Bayesian Student-t mixture model as a latent variable model in a different way from Svensén and Bishop [Svensén, M., & Bishop, C. M. (2005). Robust Bayesian mixture modelling. Neurocomputing, 64, 235-252]. The main difference resides in the fact that it is not necessary to assume a factorized approximation of the posterior distribution on the latent indicator variables and the latent scale variables in order to obtain a tractable solution. Not neglecting the correlations between these unobserved random variables leads to a Bayesian model having an increased robustness. Furthermore, it is expected that the lower bound on the log-evidence is tighter. Based on this bound, the model complexity, i.e. the number of components in the mixture, can be inferred with a higher confidence.