Mixture Of Normal Densities Research Articles

Goodness-of-fit testing for normal mixture densities

A novel goodness-of-fit test for assessing the validity of maximum likelihood estimates of normal mixture densities with known number of components is introduced. The theoretical contributions include analytic quantification of the test statistic's size and power functions under fixed and local alternatives. These are used to derive a closed-form bandwidth rule which optimizes the test's power while keeping its size constant at a given significance level, and a cut-off point suitable for finite sample implementations of the test. An extensive simulation study compares the performance of the new test to well-established tests in the literature and demonstrates the superiority of the former in all examples considered. Finally, its practical usefulness is demonstrated in the analysis of two real world datasets.

A non-parametric Bayesian approach to decompounding from high frequency data

Given a sample from a discretely observed compound Poisson process, we consider non-parametric estimation of the density f_0 of its jump sizes, as well as of its intensity lambda _0. We take a Bayesian approach to the problem and specify the prior on f_0 as the Dirichlet location mixture of normal densities. An independent prior for lambda _0 is assumed to be compactly supported and to possess a positive density with respect to the Lebesgue measure. We show that under suitable assumptions the posterior contracts around the pair (lambda _0,,f_0) at essentially (up to a logarithmic factor) the sqrt{nDelta }-rate, where n is the number of observations and Delta is the mesh size at which the process is sampled. The emphasis is on high frequency data, Delta rightarrow 0, but the obtained results are also valid for fixed Delta . In either case we assume that nDelta rightarrow infty . Our main result implies existence of Bayesian point estimates converging (in the frequentist sense, in probability) to (lambda _0,,f_0) at the same rate. We also discuss a practical implementation of our approach. The computational problem is dealt with by inclusion of auxiliary variables and we develop a Markov chain Monte Carlo algorithm that samples from the joint distribution of the unknown parameters in the mixture density and the introduced auxiliary variables. Numerical examples illustrate the feasibility of this approach.

A Framework for Compressive Sensing of Asymmetric Signals Using Normal and Skew-Normal Mixture Prior

In this work, we are interested in the compressive sensing of sparse signals whose significant coefficients are distributed asymmetrically with respect to zero. To properly address this problem, we develop a framework utilizing a two-state normal and skew normal mixture density as the prior distribution of the signal. The significant and insignificant coefficients of the signal are represented by skew normal and normal distributions, respectively. A novel approximate message passing-based algorithm is developed to estimate the signal from its compressed measurements. A fast gradient-based estimator is designed to infer the density of each state. Experiment results on simulated data and two real-world tests, i.e., multi-input multi-output (MIMO) communication system and weather sensor network, confirm that our proposed technique is powerful in exploiting asymmetrical feature, and outperforms many sophisticated methods.

Analysis of spike train data: Classification and Bayesian alignment

We analyze a data set of spike trains obtained under four different experimental conditions. We model the data curves via mixtures of normal densities. The peak locations in the fitted curves are modeled via a non-homogeneous Poisson process and classification of the spike trains into groups may be done based on the estimated spacings between peaks. We employ a Bayesian, MCMC-based registration method to align the fitted curves and summarize the data using meaningful functional statistics and posterior intervals.

Mixtures of equispaced normal distributions and their use for testing symmetry with univariate data

Given a random sample of observations, mixtures of normal densities are often used to estimate the unknown continuous distribution from which the data come. The use of this semi-parametric framework is proposed for testing symmetry about an unknown value. More precisely, it is shown how the null hypothesis of symmetry may be formulated in terms of a normal mixture model, with weights about the center of symmetry constrained to be equal one another. The resulting model is nested in a more general unconstrained one, with the same number of mixture components and free weights. Therefore, after having maximized the constrained and unconstrained log-likelihoods, by means of the Expectation–Maximization algorithm, symmetry is tested against skewness through a likelihood ratio statistic with p-value computed by using a parametric bootstrap method. The behavior of this mixture-based test is studied through a Monte Carlo simulation, where the proposed test is compared with the traditional one, based on the third standardized moment, and with the non-parametric triples test. An illustrative example is also given which is based on real data.

Iterative posterior inference for Bayesian Kriging

We propose a method for estimating the posterior distribution of a standard geostatistical model. After choosing the model formulation and specifying a prior, we use normal mixture densities to approximate the posterior distribution. The approximation is improved iteratively. Some difficulties in estimating the normal mixture densities, including determining tuning parameters concerning bandwidth and localization, are addressed. The method is applicable to other model formulations as long as all the parameters, or transforms thereof, are defined on the whole real line, $(-\infty, \infty)$. Ad hoc treatments in the posterior inference such as imposing bounds on an unbounded parameter or discretizing a continuous parameter are avoided. The method is illustrated by two examples, one using digital elevation data and the other using historical soil moisture data. The examples in particular examine convergence of the approximate posterior distributions in the iterations.

Model Misspecification: Finite Mixture or Homogeneous?

A common problem in statistical modelling is to distinguish between finite mixture distribution and a homogeneous non-mixture distribution. Finite mixture models are widely used in practice and often mixtures of normal densities are indistinguishable from homogenous non-normal densities. This paper illustrates what happens when the EM algorithm for normal mixtures is applied to a distribution that is a homogeneous non-mixture distribution. In particular, a population-based EM algorithm for finite mixtures is introduced and applied directly to density functions instead of sample data. The population-based EM algorithm is used to find finite mixture approximations to common homogeneous distributions. An example regarding the nature of a placebo response in drug treated depressed subjects is used to illustrate ideas.

Higher order values of MISE in kernel density estimation

Density estimation is the general approach adopted for the construction of an estimate of the underlying density function for independent and identically distributed random variables. There is need to reduce the error propagation in Kernel Density Estimation. Higher Order values of the exact and asymptotic mean integrated squared error of some kernels are considered. An empirical verification that EMISE is less than AMISE for higher order normal mixture densities is explored through computer algorithms. The effect of small and large values of the window width is examined over equally spaced grid of (0, 0.5]. Journal of the Nigerian Association of Mathematical Physics Vol. 9 2005: pp. 351-356

Fitting Regional Income Distributions in the European Union*

AbstractCross‐sectional distribution of per capita (log) GDP across the European Union regions from 1977 to 1996 is analysed. Kernel density estimates reveal a multimodal structure of the distribution during the 1970s and early 1980s, and a tendency towards unimodality since the mid‐1980s. The distribution is further analysed by a mixture of normal densities. A two well‐separated component mixture fits the distributions in the 1970s and early 1980s. These two clusters tend to converge, supporting the idea of a process of catching up. In the mid‐1990s, a small group of very rich regions is generated by a separated component.

Estimating the effect of price limits on limit‐hitting days

Summary This study examines whether price limits affect underlying equilibrium prices on limit-hitting days. To identify two effects—a ceiling effect and a cooling or heating effect (C-H effect)—we use the fact that equilibrium prices are reached at the day immediately after price limits are hit. We estimate the C-H effect by letting the return series be mixture normal to capture the possible 'fat tails.' We apply our models to five randomly selected Taiwanese stocks and all the continuously traded stocks in our sample period. The simple normal density which would lead one to conclude that price limits can 'cool off stock prices is soundly rejected. However, if normal mixture density is used, one would generally conclude that price limits will have no effect on the variance of stock returns.

Hysteresis modeling of Gd films and AFC thin-film recording media

This paper presents simulations of hysteresis processes in thin film media using 1D and 2D Preisach models. In the 2D version, a vector operator and superposition of angularly distributed models are used. The characteristic density of the material being modeled is reconstructed via a curve-fitting least-squares procedure that determines the parameters of a bivariate normal probability function density or a weighed mixture of normal densities based on major loop data only. The models have been identified for several samples of Gd films annealed at various temperatures and AFC thin-film recording media consisting of a hard and a soft phase antiferromagnetically coupled. The major and minor hysteresis loops calculated for all samples are in good agreement with experimental data.

Bivariate Normal Mixture Spread Option Valuation

This paper explores the properties of a European spread option valuation method for correlated assets when the marginal distribution each asset return is assumed to be a mixture of normal distributions. In this 'bivariate normal mixture' (BNM) approach no-arbitrage option values are just weighted sums of different 2GBM values based on two correlated lognormal diffusions, and likewise for their sensitivities. The main advantage of this approach is that BNM option values are consistent with the volatility smiles for each asset and an implied correlation 'frown', both of which are often observed when spread options are priced under the 2GBM assumptions. It is simple to perform an extensive consideration of model values for varying strike, and for different asset volatility and correlation structures. We compare BNM valuations with those based on the '2GBM' assumption of two correlated lognormal diffusions and explain the differences between the BNM values and the 2GBM values of spread options as a weighted sum of six second order 2GBM value sensitivities. We also investigate the BNM sensitivities and these, like the option values, can sometimes be significantly different from those obtained under the 2GBM model. Finally, we show how the correlation frown that is implied by this model is affected as we change the parameters in the bivariate normal mixture density of the asset returns.

On Practical Efficiency of Locally Parametric Nonparametric Density Estimation Based on Local Likelihood Function

This paper offers a practical comparison of efficiency between local likelihood approach and conventional kernel approach in density estimation. The local likelihood estimation procedure maximizes a kernel smoothed log-likelihood function with respect to a polynomial approximation of the log likelihood function. We use two types of data driven bandwidths for each method and compare the mean integrated squares for several densities. Numerical results reveal that local log-linear approach with simple plug-in bandwidth shows better performance comparing to the standard kernel approach in heavy tailed distribution. For normal mixture density cases, standard kernel estimator with the bandwidth in Sheather and Jones(1991) dominates the others in moderately large sample size.

Mixture density estimation with group membership functions

The mixture density model has been extensively studied in the field of statistical pattern recognition. And the EM algorithm has been well known as a convenient and efficient tool to iteratively compute the maximum likelihood estimates of mixture model parameters (except the number of components in mixture). However, it is a difficult problem to estimate the number of components in mixture. In order to solve this problem, we present a new mixture density model based on the concept of group, which is defined as a pair of different components. With the new mixture density model, it is possible for two components to have the same component parameters, so the accurate estimation of the number of components is no longer needed. Based on the new mixture density model, the mixture density estimation method with group membership functions is derived, and it is demonstrated to be an efficient method in the experiments of normal mixture density estimations.

A diagnostic tool for mixture models

Mixture models are used in a large number of applications yet there remain difficulties with maximum likelihood estimation. For instance, the likelihood surface for finite normal mixtures often has a large number of local maximizers, some of which do not give a good representation of the underlying features of the data. In this paper we present diagnostics that can be used to check the quality of an estimated mixture distribution. Particular attention is given to normal mixture models since they frequently arise in practice. We use the diagnostic tools for finite normal mixture problems and in the nonparametric setting where the difficult problem of determining a scale parameter for a normal mixture density estimate is considered. A large sample justification for the proposed methodology will be provided and we illustrate its implementation through several examples

Empirical Estimates of Effect of Price Limits on Limit-Hitting Days

In this study, we demonstrate how price limits can affect a return series on limit-hitting days. Our identification of two effects - a ceiling effect and a cooling or heating effect (C-H effect) is based on a resampling method suggested by Wei and Chiang (1999). We estimate the C-H effect by assuming that the return series will have a mixture normal density instead of a simple normal density. We apply our model to five randomly selected Taiwanese stocks as well as all the stocks that are continuously traded in our sample. The simple normal density is soundedly rejected and it would generally lead one to conclude that price limits can cool off stock prices. On the other hand, if normal mixture density is used, one would generally conclude that price limits will have no effect on the variance of stock returns.

Exact mean and mean squared error of the smoothed bootstrap mean integrated squared error estimator

New expressions are obtained for the mean and mean squared error of the smoothed bootstrap mean integrated squared error estimator in Gaussian kernel estimation of normal mixture densities. The use of such densities is in the same spirit as Marron and Wand (1992) and provides the same benefits. The resulting expressions are easily computable and describe the exact behavior of the estimator, thus complementing known asymptotic results for it. They reveal important information for small samples, not indicated by asymptotics. In particular, while asymptotics call for oversmoothing the estimator, undersmoothing may actually be more appropriate for small samples.

Kernel bandwidth selection for a first order nonparametric streamflow simulation model

A new approach for streamflow simulation using nonparametric methods was described in a recent publication (Sharma et al. 1997). Use of nonparametric methods has the advantage that they avoid the issue of selecting a probability distribution and can represent nonlinear features, such as asymmetry and bimodality that hitherto were difficult to represent, in the probability structure of hydrologic variables such as streamflow and precipitation. The nonparametric method used was kernel density estimation, which requires the selection of bandwidth (smoothing) parameters. This study documents some of the tests that were conduced to evaluate the performance of bandwidth estimation methods for kernel density estimation. Issues related to selection of optimal smoothing parameters for kernel density estimation with small samples (200 or fewer data points) are examined. Both reference to a Gaussian density and data based specifications are applied to estimate bandwidths for samples from bivariate normal mixture densities. The three data based methods studied are Maximum Likelihood Cross Validation (MLCV), Least Square Cross Validation (LSCV) and Biased Cross Validation (BCV2). Modifications for estimating optimal local bandwidths using MLCV and LSCV are also examined. We found that the use of local bandwidths does not necessarily improve the density estimate with small samples. Of the global bandwidth estimators compared, we found that MLCV and LSCV are better because they show lower variability and higher accuracy while Biased Cross Validation suffers from multiple optimal bandwidths for samples from strongly bimodal densities. These results, of particular interest in stochastic hydrology where small samples are common, may have importance in other applications of nonparametric density estimation methods with similar sample sizes and distribution shapes.

Genetic Optimization Using Derivatives

We describe a new computer program that combines evolutionary algorithm methods with a derivative-based, quasi-Newton method to solve difficult unconstrained optimization problems. The program, called GENOUD (GENetic Optimization Using Derivatives), effectively solves problems that are nonlinear or perhaps even discontinuous in the parameters of the function to be optimized. When a statistical model's estimating function (for example, a log-likelihood) is nonlinear in the model's parameters, the function to be optimized will usually not be globally concave and may contain irregularities such as saddlepoints or discontinuous jumps. Optimization methods that rely on derivatives of the objective function may be unable to find any optimum at all. Or multiple local optima may exist, so that there is no guarantee that a derivative-based method will converge to the global optimum. We discuss the theoretical basis for expecting GENOUD to have a high probability of finding global optima. We conduct Monte Carlo experiments using scalar Normal mixture densities to illustrate this capability. We also use a system of four simultaneous nonlinear equations that has many parameters and multiple local optima to compare the performance of GENOUD to that of the Gauss-Newton algorithm in SAS's PROC MODEL.

Failure of successive refinement for symmetric Gaussian mixtures

We show that information refinement fails for a continuous alphabet source whose input distribution is the symmetric mixture of normal densities with means -/spl mu/ and /spl mu/ and common variance /spl sigma//sup 2/.