Multivariate Mixture Research Articles

Using the Penalized Mutual Information Criterion in the Multivariate Edgeworth-Expanded Gaussian Mixture Density for Blind Separation of Convolutive Post-Nonlinear Mixtures

This paper proposes the blind separation of convolutive post-nonlinear (CPNL) mixtures based on the minimization of the penalized mutual information criterion. The proposed algorithm is based on the estimation score function difference (SFD) and the Newton optimization. Compared with the blind source separation of a linear mixture, the separation performance of a nonlinear mixture is strongly related to the accuracy of the score function estimation. Under this framework, the multivariate Edgeworth-expanded Gaussian mixture density is adopted to estimate the SFD, which preserves the higher-order statistical structure of the data as compared to the nonparametric density estimation. Also, the Newton optimization converges faster than the steepest descent gradient. In order to calculate the Hessian matrix, the Taylor expansion of the penalized mutual information criterion is extended to second order. The minimization of the penalized mutual information criterion ensures a priori normalization of the estimated sources, thus avoiding scale indeterminacy. The proposed algorithm has a better performance, and at the same time it speeds up the convergence. Simulations with computer-generated data and synthetic real-world data show the effectiveness of the proposed algorithm.

MULTIVARIATE MIXTURE OF NORMALS WITH UNKNOWN NUMBER OF COMPONENTS: AN APPLICATION TO CLUSTER NEOLITHIC CERAMICS FROM AEGEAN AND ASIA MINOR USING PORTABLE XRF*

Multivariate techniques and especially cluster analysis have been commonly used in archaeometry. Exploratory and model‐based techniques of clustering have been applied to geochemical (continuous) data of archaeological artefacts for provenance studies. Model‐based clustering techniques such as classification maximum likelihood and mixture maximum likelihood have been used to a lesser extent in this context and, although they seem to be suitable for such data, they either present practical difficulties—such as high dimensionality of the data—or their performance gives no evidence that they are superior to standard methods. In this paper standard statistical methods (hierarchical clustering, principal components analysis) and the recently developed model‐based multivariate mixture of normals with an unknown number of components, are applied and compared. The data set provides chemical compositions of 188 ceramic samples derived from the Aegean islands and surrounding areas.

Population-Based Reversible Jump Markov Chain Monte Carlo

We present an extension of population-based Markov chain Monte Carlo to the transdimensional case. A major challenge is that of simulating from high- and transdimensional target measures. In such cases, Markov chain Monte Carlo methods may not adequately traverse the support of the target; the simulation results will be unreliable. We develop population methods to deal with such problems, and give a result proving the uniform ergodicity of these population algorithms, under mild assumptions. This result is used to demonstrate the superiority, in terms of convergence rate, of a population transition kernel over a reversible jump sampler for a Bayesian variable selection problem. We also give an example of a population algorithm for a Bayesian multivariate mixture model with an unknown number of components. This is applied to gene expression data of 1000 data points in six dimensions and it is demonstrated that our algorithm outperforms some competing Markov chain samplers. In this example, we show how to combine the methods of parallel chains (Geyer, 1991), tempering (Geyer & Thompson, 1995), snooker algorithms (Gilks et al., 1994), constrained sampling and delayed rejection (Green & Mira, 2001).

Formula omitted] approach to intersection–union testing of hypotheses

Recent applications of statistics often lead one to encounter testing problems where the original hypothesis of interest comprises the union of several sub-hypothesis. In the framework of such intersection–union (IU) testing of hypothesis, in contrast to the usual union–intersection (UI) framework, a sub-hypothesis therein may specify a parameter or a function of some of the parameters of the underlying distribution. The parameters may even be constrained to lie on the boundary of their parameter spaces. Even large-sample tests such as the usual likelihood ratio. Lagrangian multiplier or the Wald's tests then do not apply as their usual asymptotic distribution theory remain no longer be valid. An approach based on a pivotal parametric product P 3 is enhanced here. It is shown that this approach often leads to appealing simple and elegant test statistics. The exact cut-off points and the power values can be computed by judicious use of numerical packages. L-optimality of such a test for the mixture problem is established. For multivariate multiparameter testing problems it is shown that such an approach leads to UI–IU tests. Construction of such tests are exemplified through several real-life problems as in, e.g., testing for interval specifications in acceptance sampling, for generalized variance of structured correlation matrices in generalized canonical variable, for agreement in method comparison studies, for no contamination in multiparameter multivariate mixture models, etc. It is demonstrated for a real-life data set in an acceptance sampling problem that the proposed class of P 3 tests includes the intuitive one existing in the literature.

Multivariate Poisson Markov-dependent finite mixture models for analysis of weed counts

Multivariate Poisson Markov-dependent finite mixture models for analysis of weed counts

Category Pricing with State Dependent Utility

There is a substantial literature that documents the presence of state dependent utility with packaged goods data. Typically, a form of brand loyalty is detected whereby there is a higher probability of purchasing the same brand as has been purchased in the recent past. The economic significance of the measured loyalty remains an open question. We consider the category pricing problem in the presence of loyalty and demonstrate that a retailer has an incentive to invest in building brand loyalty to maximize the present value of category profits and that this materially affects optimal pricing. Given the well-known problems with the confounding of state dependence and consumer heterogeneity, loyalty must be measured in a model which allows for an unknown and possibly highly non-normal distribution of heterogeneity. We implement a highly flexible model of heterogeneity using multivariate mixtures of normals in a hierarchical choice model. We find important deviations from the standard one component normal mixture.

Constrained monotone EM algorithms for finite mixture of multivariate Gaussians

The likelihood function for normal multivariate mixtures may present both local spurious maxima and also singularities and the latter may cause the failure of the optimization algorithms. Theoretical results assure that imposing some constraints on the eigenvalues of the covariance matrices of the multivariate normal components leads to a constrained parameter space with no singularities and at least a smaller number of local maxima of the likelihood function. Conditions assuring that an EM algorithm implementing such constraints maintains the monotonicity property of the usual EM algorithm are provided. Different approaches are presented and their performances are evaluated and compared using numerical experiments.

Mixed anxiety depression: Taxometric exploration of the validity of a diagnostic category in youth

Background Mixed anxiety depression (MAD) is a provisional diagnosis in the DSM-IV. This study determined whether MAD represents a discrete category thereby evaluating the validity of MAD as a diagnostic entity. Methods Taxometric analyses and mixture modeling were used to discern whether MAD indicators constitute a distinct psychopathological category (i.e., a taxon) in a large school-based sample of adolescents ( N = 706). Results Each taxometric procedure (MAXCOV, MAMBAC) identified a taxon with a prevalence of 13% ± 2%. A non-taxometric procedure (multivariate mixture modeling) also supported the existence of a taxon with a prevalence of 12%. Bootstrapping procedures were used to construct a measure of MAD (i.e., MAD-T). Scale construction suggested that MAD may be best represented by 12 criteria that largely overlap with the DSM-IV, though some modifications were suggested. Examination of the construct validity of the MAD taxon indicated that it is associated mood and anxiety symptoms. Taxon membership was predictive of the development of mood and anxiety disorders over a 14-month longitudinal follow-up. Limitations This category should be studied in other populations including adult samples. Conclusions MAD appears to be a viable diagnostic category for youth though it is recommended that future revisions of the DSM emphasize somewhat different criteria for this diagnosis.

Multivarible Robust Control with Time-domain Specifications: Servo and Regulator Problems

The robust design of compensators and the input scaling aiming setpoint tracking and disturbance rejection with time-domain specifications are presented from a perspective of loop-shaping. Since there is not an explicit relationship between loop-shaping and time-domain specifications, the latter are indirectly taken into account by forcing the plant output to follow the output of a reference model. The design specifications are written in form of loop-shape constraints. Hence, techniques like H∞ can be applied. In order to discuss the frequency dependence of the design constraints, an electromagnetic levitation suspension vehicle prototype and a multivariable mixture tank are considered.

Category Pricing with State-Dependent Utility

There is substantial literature documenting the presence of state-dependent utility with packaged goods data. Typically, a form of brand loyalty is detected whereby there is a higher probability of purchasing the same brand as has been purchased in the recent past. The economic significance of the measured loyalty remains an open question. We consider the category pricing problem and demonstrate that the presence of loyalty materially affects optimal pricing. The prices of higher quality products decline relative to those of lower quality when loyalty is introduced into the model. Given the well-known problems with the confounding of state dependence and consumer heterogeneity, loyalty must be measured in a model which allows for an unknown and possibly highly nonnormal distribution of heterogeneity. We implement a highly flexible model of heterogeneity using multivariate mixtures of normals in a hierarchical choice model. We use an Euler equations approach to the solution of the dynamic pricing problem which allows us to consider a very large number of consumer types.

Multivariate mixtures of normals with unknown number of components

We present full Bayesian analysis of finite mixtures of multivariate normals with unknown number of components. We adopt reversible jump Markov chain Monte Carlo and we construct, in a manner similar to that of Richardson and Green (1997), split and merge moves that produce good mixing of the Markov chains. The split moves are constructed on the space of eigenvectors and eigenvalues of the current covariance matrix so that the proposed covariance matrices are positive definite. Our proposed methodology has applications in classification and discrimination as well as heterogeneity modelling. We test our algorithm with real and simulated data.

Projeto de controladores robustos com especificações temporais

O projeto de compensadores robustos com especificações no domínio do tempo visando ao acompanhamento do sinal de referência e a rejeição de distúrbios é discutido neste artigo do ponto de vista freqüencial. As condições de projeto são obtidas em função de modelos de referência, os quais contêm as especificações temporais associadas. Mostra-se que quanto maior for a distância entre o modelo nominal da planta e o modelo de referência a ser seguido, mais restritivas serão as condições de projeto. Os problemas podem ser formulados como casos de model matching e o procedimento proposto pode reduzir o conservadorismo normalmente associado com esse tipo de projeto. Admite-se que a planta esteja sujeita a incertezas não estruturadas e as condições de projeto são escritas na forma usual de restrições sobre o diagrama de Bode. Desta forma, ferramentas de projeto como H<FONT FACE=Symbol>¥</FONT> e LQG/LTR podem ser aplicadas. Para ilustrar o uso da metodologia proposta, considera-se como exemplo um tanque de misturas multivariável.

Color texture analysis using the wavelet-based hidden Markov model

The wavelet-domain hidden Markov model (WD HMM), in particular the hidden Markov tree (HMT), has recently been proposed and applied to gray texture analysis with encouraging results. For color texture analysis, the previous WD HMM can only be used to model the different color planes individually, assuming they are independent of each other. However, this assumption in general is unrealistic. We show in this paper that the wavelet coefficients have certain inter-dependences between color planes. This paper presents a novel approach to modeling the dependences between color planes as well as the interactions across scales. In our approach, the wavelet coefficients at the same location, scale and sub-band, but different color planes are grouped into one vector. We then propose a multivariate Gaussian mixture model (MGMM) for approximating the marginal distribution of the wavelet coefficient vectors in one scale and capturing the interactions of different color planes. In addition, the statistical dependence between different scales is captured by the transition matrix of the hidden Markov tree. Using this approach, we can improve the performance of the WD HMM on the color texture classification. The experiments show that our WD HMM approach provides 85% of correct classifications (PCC) on 68 color textures from Oulu texture database and outperforms the other wavelet-based methods we study. In this paper, we also investigated the classification performance of the WD HMM methods on different color spaces.

Robust Reduced Rank Mixture Discriminant Analysis

ABSTRACT In the case of a large number of feature vector variables, using multivariate Gaussian mixture models, discrimination in a reduced subspace is studied, generalizing Hastie and Tibshirani's (1996) work, to a situation in which the outliers are present in the data. In the case of the Gaussian Mixture models, the reduced rank discriminant analysis is equivalent to the weighted rank k linear discriminant analysis (LDA). The reduced rank solution in the mixtures of multivariate Gaussian models was obtained from the full rank robust mixture solution. The classification in the new dimensions was compared with the discriminant analysis approach based on the original coordinates, using robust S-estimators. In most of the cases, the robust reduced rank mixture discriminant analysis (mda) performed better for the test data. However, for the case of common component covariance being diagonal, the robust reduced rank mixture discriminant analysis performed better than the robust full rank mixture discriminant analysis producing smaller errors in classification.

Regime Switching in the Latent Growth Curve Mixture Model

A linear latent growth curve mixture model is presented which includes switching between growth curves. Switching is accommodated by means of a Markov transition model. The model is formulated with switching as a highly constrained multivariate mixture model and is fitted using the freely available Mx program. The model is illustrated by analyzing data from the National Longitudinal Survey of Youth (NLSY97). The data concern alcohol use in a sample of 737 White youths who were assessed on 4 occasions.

Identification of Differentially Expressed Genes with Multivariate Outlier Analysis

DNA microarray offers a powerful and effective technology to monitor the changes in the gene expression levels for thousands of genes simultaneously. It is being widely applied to explore the quantitative alternation in gene regulation in response to a variety of aspects including diseases and exposure of toxicant. A common task in analyzing microarray data is to identify the differentially expressed genes under two different experimental conditions. Because of the large number of genes and small number of arrays, and higher signal-noise ratio in microarray data, many traditional approaches seem improper. In this paper, a multivariate mixture model is applied to model the expression level of replicated arrays, considering the differentially expressed genes as the outliers of the expression data. In order to detect the outliers of the multivariate mixture model, an effective and robust statistical method is first applied to microarray analysis. This method is based on the analysis of kurtosis coefficient (KC) of the projected multivariate data arising from a mixture model so as to identify the outliers. We utilize the multivariate KC algorithm to our microarray experiment with the control and toxic treatment. After the processing of data, the differential genes are successfully identified from 1824 genes on the UCLA M07 microarray chip. We also use the RT-PCR method and two robust statistical methods, minimum covariance determinant (MCD) and minimum volume ellipsoid (MVE), to verify the expression level of outlier genes identified by KC algorithm. We conclude that the robust multivariate tool is practical and effective for the detection of differentially expressed genes.

Nonlinear Prediction for Gaussian Mixture Image Models

Prediction is an essential operation in many image processing applications, such as object detection and image and video compression. When the images are modeled as Gaussian, the optimal predictor is linear and easy to obtain. However, image texture and clutter are often non-Gaussian, and, in such cases, optimal predictors are difficult to obtain. In this paper, we derive an optimal predictor for an important class of non-Gaussian image models, the block-based multivariate Gaussian mixture model. This predictor has a special nonlinear structure: it is a linear combination of the neighboring pixels, but the combination coefficients are also functions of the neighboring pixels, not constants. The efficacy of this predictor is demonstrated in object detection experiments where the prediction error image is used to identify "hidden" objects. Experimental results indicate that when the background texture is nonlinear, i.e., with fast-switching gray-level patches, it performs significantly better than the optimal linear predictor.

Performance assessment of kernel density clustering for gene expression profile data

Kernel density smoothing techniques have been used in classification or supervised learning of gene expression profile (GEP) data, but their applications to clustering or unsupervised learning of those data have not been explored and assessed. Here we report a kernel density clustering method for analysing GEP data and compare its performance with the three most widely-used clustering methods: hierarchical clustering, K-means clustering, and multivariate mixture model-based clustering. Using several methods to measure agreement, between-cluster isolation, and withincluster coherence, such as the Adjusted Rand Index, the Pseudo F test, the r2 test, and the profile plot, we have assessed the effectiveness of kernel density clustering for recovering clusters, and its robustness against noise on clustering both simulated and real GEP data. Our results show that the kernel density clustering method has excellent performance in recovering clusters from simulated data and in grouping large real expression profile data sets into compact and well-isolated clusters, and that it is the most robust clustering method for analysing noisy expression profile data compared to the other three methods assessed.

PDF optimized parametric vector quantization of speech line spectral frequencies

A computationally efficient, high quality, vector quantization scheme based on a parametric probability density function (PDF) is proposed. In this scheme, the observations are modeled as i.i.d realizations of a multivariate Gaussian mixture density. The mixture model parameters are efficiently estimated using the expectation maximization (EM) algorithm. A low complexity quantization scheme using transform coding and bit allocation techniques which allows for easy mapping from observation to quantized value is developed for both fixed rate and variable rate systems. An attractive feature of this method is that source encoding using the resultant codebook involves very few searches and its computational complexity is minimal and independent of the rate of the system. Furthermore, the proposed scheme is bit scalable and can switch seamlessly between a memoryless quantizer and a quantizer with memory. The usefulness of the approach is demonstrated for speech coding where Gaussian mixture models are used to model speech line spectral frequencies. The performance of the memoryless quantizer is 1-3 bits better than conventional quantization schemes.

Bayesian multiple testing for two-sample multivariate endpoints.

In clinical studies involving multiple variables, simultaneous tests are often considered where both the outcomes and hypotheses are correlated. This article proposes a multivariate mixture prior on treatment effects, that allows positive probability of zero effect for each hypothesis, correlations among effect sizes, correlations among binary outcomes of zero versus nonzero effect, and correlations among the observed test statistics (conditional on the effects). We develop a Bayesian multiple testing procedure, for the multivariate two-sample situation with unknown covariance structure, and obtain the posterior probabilities of no difference between treatment regimens for specific variables. Prior selection methods and robustness issues are discussed in the context of a clinical example.