Nonparametric Mixture Research Articles

Nonparametric Time-Varying Mixture Copula Models

Time-varying copula models have been shown to offer additional flexibility among multiple financial time series. In this paper we propose a new nonparametric time-varying mixture copula model, in which both weights and dependence parameters are deterministic functions of time. Asymptotic properties of the local constant estimators are established, and a revised expectation maximization (EM) algorithm is employed to reduce the computational burden. We use Monte Carlo simulation to examine the finite sample performance of the proposed estimators. In a financial example, we show that our proposed model outperforms the benchmark copula models.

Bayesian Nonparametric Mixture Estimation for Time-Indexed Functional Data inR

We present growfunctions for R that offers Bayesian nonparametric estimation models for analysis of dependent, noisy time series data indexed by a collection of domains. This data structure arises from combining periodically published government survey statistics, such as are reported in the Current Population Study (CPS). The CPS publishes monthly, by-state estimates of employment levels, where each state expresses a noisy time series. Published state-level estimates from the CPS are composed from household survey responses in a model-free manner and express high levels of volatility due to insufficient sample sizes. Existing software solutions borrow information over a modeled time-based dependence to extract a de-noised time series for each domain. These solutions, however, ignore the dependence among the domains that may be additionally leveraged to improve estimation efficiency. The growfunctions package offers two fully nonparametric mixture models that simultaneously estimate both a time and domain-indexed dependence structure for a collection of time series: (1) A Gaussian process (GP) construction, which is parameterized through the covariance matrix, estimates a latent function for each domain. The covariance parameters of the latent functions are indexed by domain under a Dirichlet process prior that permits estimation of the dependence among functions across the domains: (2) An intrinsic Gaussian Markov random field prior construction provides an alternative to the GP that expresses different computation and estimation properties. In addition to performing denoised estimation of latent functions from published domain estimates, growfunctions allows estimation of collections of functions for observation units (e.g., households), rather than aggregated domains, by accounting for an informative sampling design under which the probabilities for inclusion of observation units are related to the response variable. growfunctions includes plot functions that allow visual assessments of the fit performance and dependence structure of the estimated functions. Computational efficiency is achieved by performing the sampling for estimation functions using compiled C++.

Posterior sampling from $\varepsilon$-approximation of normalized completely random measure mixtures

This paper adopts a Bayesian nonparametric mixture model where the mixing distribution belongs to the wide class of normalized homogeneous completely random measures. We propose a truncation method for the mixing distribution by discarding the weights of the unnormalized measure smaller than a threshold. We prove convergence in law of our approximation, provide some theoretical properties, and characterize its posterior distribution so that a blocked Gibbs sampler is devised. The versatility of the approximation is illustrated by two different applications. In the first the normalized Bessel random measure, encompassing the Dirichlet process, is introduced; goodness of fit indexes show its good performances as mixing measure for density estimation. The second describes how to incorporate covariates in the support of the normalized measure, leading to a linear dependent model for regression and clustering.

Scalable approximate Bayesian inference for outlier detection under informative sampling

Government surveys of business establishments receive a large volume of submissions where a small subset contain errors. Analysts need a fast-computing algorithm to ag this subset due to a short time window between collection and reporting. We offer a computationally-scalable optimization method based on non-parametric mixtures of hierarchical Dirichlet processes that allows discovery of multiple industry-indexed local partitions linked to a set of global cluster centers. Outliers are nominated as those clusters containing few observations. We extend an existing approach with a new merge step that reduces sensitivity to hyperparameter settings. Survey data are typically acquired under an informative sampling design where the probability of inclusion depends on the surveyed response such that the distribution for the observed sample is different from the population. We extend the derivation of a penalized objective function to use a pseudo-posterior that incorporates sampling weights that undo the informative design. We provide a simulation study to demonstrate that our approach produces unbiased estimation for the outlying cluster under informative sampling. The method is applied for outlier nomination for the Current Employment Statistics survey conducted by the Bureau of Labor Statistics.

Sequential Monte Carlo methods for mixtures with normalized random measures with independent increments priors

Normalized random measures with independent increments are a general, tractable class of nonparametric prior. This paper describes sequential Monte Carlo methods for both conjugate and non-conjugate nonparametric mixture models with these priors. A simulation study is used to compare the efficiency of the different algorithms for density estimation and comparisons made with Markov chain Monte Carlo methods. The SMC methods are further illustrated by applications to dynamically fitting a nonparametric stochastic volatility model and to estimation of the marginal likelihood in a goodness-of-fit testing example.

Evolution of a Hybrid Model for an Effective Perimeter Security Device

Clustering and classification models, or hybrid models are the most widely used models that can handle the diverse nature of NIDS dataset. Dirichlet process clustering technique is a non-parametric Bayesian mixture model that considers the data distribution of the dataset for the formation of distinct clusters. The number of clusters is not known a priori and it differs across different datasets. Determining the number of clusters based on the distribution of data instances can increase the performance of the model. Naive Bayes model, a supervised learning classification technique, maintains a better computational efficiency, by reducing the training time. In this paper, we propose a hybrid model to exploit the positive aspect of proper clustering of data instances and the computational efficiency in building a NIDS. RIPPER algorithm is used to extract rules from the traffic description for updation of the rule database. Experiments were conducted in the KDD CUP’99 and SSENet-2011 datasets to study the performance of the proposed model. Also, a comparison of three hybrid methods with the proposed hybrid model was carried out. The results showed that the proposed hybrid model is superior in building a robust perimeter security device.

Network structure exploration via Bayesian nonparametric models

.Complex networks provide a powerful mathematical representation of complex systems in nature and society. To understand complex networks, it is crucial to explore their internal structures, also called structural regularities. The task of network structure exploration is to determine how many groups there are in a complex network and how to group the nodes of the network. Most existing structure exploration methods need to specify either a group number or a certain type of structure when they are applied to a network. In the real world, however, the group number and also the certain type of structure that a network has are usually unknown in advance. To explore structural regularities in complex networks automatically, without any prior knowledge of the group number or the certain type of structure, we extend a probabilistic mixture model that can handle networks with any type of structure but needs to specify a group number using Bayesian nonparametric theory. We also propose a novel Bayesian nonparametric model, called the Bayesian nonparametric mixture (BNPM) model. Experiments conducted on a large number of networks with different structures show that the BNPM model is able to explore structural regularities in networks automatically with a stable, state-of-the-art performance.

Modelling of count data using nonparametric mixtures

Nonparametric modelling of count data is partly motivated by the fact that using parametric count models not only runs the risk of model misspecification but also is rather restrictive in terms of local approximation. Accordingly, we present a framework of using nonparametric mixtures for flexible modelling of count data. We consider the use of the least squares function in nonparametric mixture modelling and provide two algorithms for least squares fitting of nonparametric mixtures. Two illustrations of the framework are given, each with a particular nonparametric mixture. One illustration is the use of the nonparametric Poisson mixture for general modelling purposes. The other illustration is concerned with modelling of count data from some decreasing distribution, in which the Poisson mixture distribution is less appropriate, for its fitted distribution might not be a decreasing distribution. We define a mixture distribution called the discrete decreasing beta mixture distribution that always has fitted probabilities conforming with the assumption of decreasing probabilities. Through numerical studies, we demonstrate the performance of nonparametric mixtures as modelling tools.

Comparing Two Mixing Densities in Nonparametric Mixture Models

In this paper we consider two nonparametric mixtures of quadratic natural exponential families with unknown mixing densities. We propose a statistic to test the equality of these mixing densities when the two natural exponential families are known. The test is based on moment characterizations of the distributions. The number of moments is retained automatically by a data driven technique. Some examples and simulations of implementation of the procedure are provided.

Time-varying extreme pattern with dynamic models

This paper is concerned with the analysis of time series data with time-varying extreme pattern. This is achieved via a model formulation that considers separately the central part and the tail of the distributions, using a two-component mixture model. Extremes beyond a threshold are assumed to follow a generalized Pareto distribution (GPD). Temporal dependence is induced by allowing the GPD parameters to vary with time. Temporal variation and dependence is introduced at a latent level via the novel use of dynamic linear models (DLM). Novelty lies in the time variation of the shape and scale parameter of the resulting distribution. These changes in limiting regimes as time changes reflect better the data behavior, with important gains in estimation and interpretation. The central part follows a nonparametric mixture approach. The uncertainty about the threshold is explicitly considered. Posterior inference is performed through Markov Chain Monte Carlo (MCMC) methods. A variety of scenarios can be entertained and include the possibility of alternation of presence and absence of a finite upper limit of the distribution for different time periods. Simulations are carried out in order to analyze the performance of our proposed model. We also apply the proposed model to financial time series: returns of Petrobras stocks and SP500 index. Results show advantage of our proposal over currently entertained models such as stochastic volatility, with improved estimation of high quantiles and extremes.

Exclusion Restrictions and Identification of Returns to Schooling

The paper presents exclusion restrictions that allow identification of returns to schooling using data from the National Survey of Young Men (1966) (NLSYM66). The approach does not use instrumental variables (IV) or control function approaches. Instead, a non-parametric finite mixture model is used to estimate the distribution of latent types. This approach requires two exclusion restrictions to identify the causal effect of attending college on wages. The model identifies two latent types. For the latent type with a low propensity for attending college, receiving 13 or more years of education leads to higher income than having 12 years or fewer years. However, the estimated returns to education for this group is approximately one half of the observed difference and one third of the IV estimates presented in Card (1995) and Kling (2001) using the same data. The difference is due to the difference in the mix of latent types associated with proximity to college and the impact of the college proximity instrument on respondents with a high propensity to attend college.

AGE-SPECIFIC PROBABILITY OF CHILDBIRTH. SMOOTHING VIA BAYESIAN NONPARAMETRIC MIXTURE OF ROUNDED KERNELS

The municipality of Milan is one of the most important areas in Italy being the center of many economic activities and the destination of strong national and international immigration. In this context, policy makers are interested in understanding socio-demographical and economical differences among the different urban areas. In this paper we concentrate in estimating differences in fertility among the nine areas of Milan. The knowledge of age-specific fertility indicators, indeed, is extremely useful in order to decide where to build a new nursery-school, where to increase obstetrics departments in hospitals, or which kind of services can be offered to families. To estimate the age-specific probabilities of child-births in the municipality of Milan, we use open-data on the births residents in Milan in 2011. It has recently been observed that the patterns of fertility of developed countries show a deviation from the classic right-skewed shape due to the fact that women tend to have children later. Also, when a large component of immigrants is present, the age-specific fertility rate exhibits an almost bimodal shape, the curve shows a little hump between 20 and 25 years of the woman, presumably due to the presence of subpopulations. To deal with this phenomena and to compare fertility between the nine urban areas of the municipality of Milan, we apply a Bayesian nonparametric mixture model which can account for skewness and multimodality and we estimate the age-specific probability of childbirth.

Noised image segmentation based on rough set and orthogonal polynomial density model

In order to segment a noised image, a method is proposed based on the rough set and orthogonal polynomial density model, in which the nonparametric mixture model can accurately fit the image gray distribution and the rough set can deal with the inaccuracy and uncertainty problems. First, the nonparametric mixture density model is constructed based on the upper and lower approximations of the rough set which can address the problem of over-relying on the prior presumption. Second, the nonparametric expectation-maximization is used to estimate the mixture model parameters. Finally, image pixels are classified according to Bayesian criterion. Experiments on different datasets show that our method is effective in solving the problem of model mismatch, restraining the noise, and preserving the boundary for the noised image segmentation.

Modeling for seasonal marked point processes: An analysis of evolving hurricane occurrences

Seasonal point processes refer to stochastic models for random events which are only observed in a given season. We develop nonparametric Bayesian methodology to study the dynamic evolution of a seasonal marked point process intensity. We assume the point process is a nonhomogeneous Poisson process and propose a nonparametric mixture of beta densities to model dynamically evolving temporal Poisson process intensities. Dependence structure is built through a dependent Dirichlet process prior for the seasonally-varying mixing distributions. We extend the nonparametric model to incorporate time-varying marks, resulting in flexible inference for both the seasonal point process intensity and for the conditional mark distribution. The motivating application involves the analysis of hurricane landfalls with reported damages along the U.S. Gulf and Atlantic coasts from 1900 to 2010. We focus on studying the evolution of the intensity of the process of hurricane landfall occurrences, and the respective maximum wind speed and associated damages. Our results indicate an increase in the number of hurricane landfall occurrences and a decrease in the median maximum wind speed at the peak of the season. Introducing standardized damage as a mark, such that reported damages are comparable both in time and space, we find that there is no significant rising trend in hurricane damages over time.

Non-parametric Bayesian mixture of sparse regressions with application towards feature selection for statistical downscaling

Abstract. Climate projections simulated by Global Climate Models (GCMs) are often used for assessing the impacts of climate change. However, the relatively coarse resolutions of GCM outputs often preclude their application to accurately assessing the effects of climate change on finer regional-scale phenomena. Downscaling of climate variables from coarser to finer regional scales using statistical methods is often performed for regional climate projections. Statistical downscaling (SD) is based on the understanding that the regional climate is influenced by two factors – the large-scale climatic state and the regional or local features. A transfer function approach of SD involves learning a regression model that relates these features (predictors) to a climatic variable of interest (predictand) based on the past observations. However, often a single regression model is not sufficient to describe complex dynamic relationships between the predictors and predictand. We focus on the covariate selection part of the transfer function approach and propose a nonparametric Bayesian mixture of sparse regression models based on Dirichlet process (DP) for simultaneous clustering and discovery of covariates within the clusters while automatically finding the number of clusters. Sparse linear models are parsimonious and hence more generalizable than non-sparse alternatives, and lend themselves to domain relevant interpretation. Applications to synthetic data demonstrate the value of the new approach and preliminary results related to feature selection for statistical downscaling show that our method can lead to new insights.

234 Kinetic analysis of dynamic 11C-verapamil PET study: Compartmental v adaptive mixture models comparison

Background: Temporal aspects of dynamic positron emission tomography (PET) generally rely on compartmental models (CM), with questionable assumptions to summarise functional kinetics of injected radio-tracers in living tissues. Material and Methods: The tracer signal time-course is a combination of vascular delivery and tissue retention effects which do not always satisfy the assumptions of tissue homogeneity and instant mixing within compartments required by the CM. This tissue activity time-course can typically be expressed as a convolution between the signal of the tracer in the arterial supply and the tissue residue function. The residue represents the amount of tracer remaining in the tissue and provides a description of the tracer kinetics measurable by the PET scan. Thus, in statistical terms, the residue can be thought of as a survival function for the residence of the tracer in the tissue which does not require the physiological constraints intrinsic in the exponential-like residue needed for CM. Accordingly more flexible novel approaches − nonparametric and later adaptive mixture models − to estimate the residue based on a piecewise linear form have been developed. The new approaches have been successfully probed on two of the most well-established PET radiotracers, O-H2O and F-fluorodeoxyglucose, used for perfusion and glucose metabolism respectively. This study shows the extension of the new approaches to the multiple-drug resistance transporter P-glycoprotein (P-gp) highly present at the blood–brain barrier (BBB) using C-Verapamil (Vp) in healthy humans before and after inhibition of the P-gp by Cyclosporine (CsA) infusion. Previous PET studies on measuring BBB activity by CsA infusion using Vp have revealed difficulties for the CM to recover metabolic information as they require a full understanding of the tracers metabolisation in the tissue, which is still not certain for Vp. This problem is especially latent when the inhibition of the P-gp allows the tracer through the BBB where it is retained once at the brain tissue level. Results: This work examines and evaluate adaptive mixture model as an alternative to conventional compartment techniques in recovering metabolic information from dynamic PET studies with less biological restrictions. Key bioparameters such as flux, flow, blood volume and volume of distribution from the two different approaches are compared. Cross-validation is used to make regional comparisons and evaluate the fit of the two different estimated residue functions to the tissue activity curves. Conclusions: The accuracy of the standard models for Vp PET studies can be questionable, likely because behaviour of Vp metabolites in tissue is still unclear. Significant statistical evidence in favour of the new adaptivemixture models has been found. Supported by Science Foundation Ireland under SFI-PI 11/27 and by the National Institute of Health (NCI) under PO1-CA-42045.

Nonparametric estimation of species richness using discrete [formula omitted]-monotone distributions

Nonparametric mixture models are commonly used to estimate the number of unobserved species for their robustness of modelling heterogeneity. In particular, Poisson mixtures are popular in this regard, but they are also known to have the boundary problem that causes unstable estimation. A family of shape-restricted distributions known as discrete k-monotone distributions is proposed for species richness estimation. These distributions are in fact also nonparametric mixtures and can thus be fitted rapidly via some algorithms that are made available recently. As shown by empirical studies, as compared with Poisson mixtures, their use avoids the boundary problem and gives more stable and more accurate estimates.

Bayesian nonparametric estimation of targeted agent effects on biomarker change to predict clinical outcome.

The effect of a targeted agent on a cancer patient's clinical outcome putatively is mediated through the agent's effect on one or more early biological events. This is motivated by pre-clinical experiments with cells or animals that identify such events, represented by binary or quantitative biomarkers. When evaluating targeted agents in humans, central questions are whether the distribution of a targeted biomarker changes following treatment, the nature and magnitude of this change, and whether it is associated with clinical outcome. Major difficulties in estimating these effects are that a biomarker's distribution may be complex, vary substantially between patients, and have complicated relationships with clinical outcomes. We present a probabilistically coherent framework for modeling and estimation in this setting, including a hierarchical Bayesian nonparametric mixture model for biomarkers that we use to define a functional profile of pre-versus-post-treatment biomarker distribution change. The functional is similar to the receiver operating characteristic used in diagnostic testing. The hierarchical model yields clusters of individual patient biomarker profile functionals, and we use the profile as a covariate in a regression model for clinical outcome. The methodology is illustrated by analysis of a dataset from a clinical trial in prostate cancer using imatinib to target platelet-derived growth factor, with the clinical aim to improve progression-free survival time.

On species sampling sequences induced by residual allocation models

We discuss fully Bayesian inference in a class of species sampling models that are induced by residual allocation (sometimes called stick-breaking) priors on almost surely discrete random measures. This class provides a generalization of the well-known Ewens sampling formula that allows for additional flexibility while retaining computational tractability. In particular, the procedure is used to derive the exchangeable predictive probability functions associated with the generalized Dirichlet process of Hjort (2000) and the probit stick-breaking prior of Chung and Dunson (2009) and Rodriguez and Dunson (2011). The procedure is illustrated with applications to genetics and nonparametric mixture modeling.

On a class of $$\sigma $$ σ -stable Poisson–Kingman models and an effective marginalized sampler

We investigate the use of a large class of discrete random probability measures, which is referred to as the class $$\mathcal {Q}$$Q, in the context of Bayesian nonparametric mixture modeling. The class $$\mathcal {Q}$$Q encompasses both the the two-parameter Poisson---Dirichlet process and the normalized generalized Gamma process, thus allowing us to comparatively study the inferential advantages of these two well-known nonparametric priors. Apart from a highly flexible parameterization, the distinguishing feature of the class $$\mathcal {Q}$$Q is the availability of a tractable posterior distribution. This feature, in turn, leads to derive an efficient marginal MCMC algorithm for posterior sampling within the framework of mixture models. We demonstrate the efficacy of our modeling framework on both one-dimensional and multi-dimensional datasets.