Two-component Normal Mixture Research Articles

Finite Mixture Model: Prediction of Time Series Data Using Bayesian Method

The aim of this study is to measure the number of components that exhibits from the variables' series. The number of components can be affected by the time series components including trend, seasonal adjustment, and irregular changes. By using a finite mixture model, the number of components can be identifies and thereafter we can formulate a Bayesian regression equation to predict the relationship between exchange rate and international tourism expenditure in Malaysia. Identification of the number of components is an important step to weigh the probability density function for a time series data. The weight of the probability density function is then used for prediction. Besides, a Bayesian method is also used in this study to fit with the finite mixture model due to its consistency characteristic. The Bayesian parameter estimates are close to the predictive distributions because it will integrate the prior distribution with the likelihood function to produce posterior distribution. The results show that there is a two-component normal mixture model exists for the time series data. In addition, a prediction equation is obtained from the analysis.

Bimodality of Plasma Glucose Distributions in Whites: A Bootstrap Approach to Testing Mixture Models

The null distribution of the likelihood ratio test (LRT) of a onecomponent normal model versus two-component normal mixture model is unknown. In this paper, we take a bootstrap approach to the likelihood ratio test for testing bimodality of plasma glucose concentrations from Rancho Bernardo Diabetes Study. The small p-values from this approach support the hypothesis that a bimodal normal mixture model fits the data significantly better than a unimodal normal model. The size and power of the bootstrap based LRT are evaluated through simulations. The results suggest that a sample size of close to 500 would be necessary in order to attain a power of 90% for detecting the unbalanced mixtures with means and variances similar to those in the Rancho Bernardo data. Besides sample size, the power also depends on the two means and variances of the two components in the data.

A Hierarchical Bayes Unit-Level Small Area Estimation Model for Normal Mixture Populations

National statistical agencies are regularly required to produce estimates about various subpopulations, formed by demographic and/or geographic classifications, based on a limited number of samples. Traditional direct estimates computed using only sampled data from individual subpopulations are usually unreliable due to small sample sizes. Subpopulations with small samples are termed small areas or small domains. To improve on the less reliable direct estimates, model-based estimates, which borrow information from suitable auxiliary variables, have been extensively proposed in the literature. However, standard model-based estimates rely on the normality assumptions of the error terms. In this research we propose a hierarchical Bayesian (HB) method for the unit-level nested error regression model based on a normal mixture for the unit-level error distribution. Our method proposed here is applicable to model cases with unit-level error outliers as well as cases where each small area population is comprised of two subgroups, neither of which can be treated as an outlier. Our proposed method is more robust than the normality based standard HB method (Datta and Ghosh, Annals Stat. 19, 1748–1770, 1991) to handle outliers or multiple subgroups in the population. Our proposal assumes two subgroups and the two-component mixture model that has been recently proposed by Chakraborty et al. (Int. Stat. Rev. 87, 158–176, 2019) to address outliers. To implement our proposal we use a uniform prior for the regression parameters, random effects variance parameter, and the mixing proportion, and we use a partially proper non-informative prior distribution for the two unit-level error variance components in the mixture. We apply our method to two examples to predict summary characteristics of farm products at the small area level. One of the examples is prediction of twelve county-level crop areas cultivated for corn in some Iowa counties. The other example involves total cash associated in farm operations in twenty-seven farming regions in Australia. We compare predictions of small area characteristics based on the proposed method with those obtained by applying the Datta and Ghosh (Annals Stat. 19, 1748–1770, 1991) and the Chakraborty et al. (Int. Stat. Rev. 87, 158–176, 2019) HB methods. Our simulation study comparing these three Bayesian methods, when the unit-level error distribution is normal, or t, or two-component normal mixture, showed the superiority of our proposed method, measured by prediction mean squared error, coverage probabilities and lengths of credible intervals for the small area means.

Parameter estimation with profile likelihood method and penalized EM algorithm in normal mixture distributions

As we know, as the likelihood function of the normal mixture is not a bounded function on 𝚯, a global maximum likelihood estimation(MLE) can not always be found and use of an EM (Expectation-Maximization) algorithm can possible lead towards a degenerate solution.In this study, the purpose was to determine which method has a better estimate to parameters of univariate two-component normal mixture distribution, as comparing profile likelihood method and penalized EM in unequal variance and to avoid from unbounded of log-likelihood function and to maximization to likelihood function.For this purpose a simulation study was performed. Thus, we tried to determine which algorithm gives a better estimate for the parameters.

Robust normal mixtures for financial portfolio allocation

A new approach for multivariate modelling and prediction of asset returns is proposed. It is based on a two-component normal mixture, estimated using a fast new variation of the minimum covariance determinant (MCD) method made suitable for time series. It outperforms the (shrinkage-augmented) MLE in terms of out-of-sample density forecasts and portfolio performance. In addition to the usual stylized facts of skewness and leptokurtosis, the model also accommodates leverage and contagion effects, but is i.i.d., and thus does not embody, for example, a GARCH-type structure. Owing to analytic tractability of the moments and the expected shortfall, portfolio optimization is straightforward, and, for daily equity returns data, is shown to substantially outperform the equally weighted and classical long-only Markowitz framework, as well as DCC-GARCH (despite not using any kind of GARCH-type filter).

Abstract 3271: Gene expression changes underlying glioblastoma resistance to anti-angiogenic therapy

Abstract Introduction: Despite positive pre-clinical and clinical trials, treatment of glioblastoma with bevacizumab has been limited by acquired resistance and transient response. To study gene expression changes underlying tumor resistance and progression during bevacizumab therapy, we performed microarray gene expression analysis on a novel multigenerational xenograft model of acquired bevacizumab resistance. Methods: Using a two-component normal mixture model, we identified a set of genes exhibiting significant inter-generational variance. Protein-protein interaction scores among these genes were extracted from the String 10 database and used as undirected edge weights in a network representation of the significant gene set. Gene set over-representation (GSO) analysis via ConsensusPathDB of gene clusters identified biologically meaningful clusters and subnetworks mediating distinct functions and molecular pathways. Results: Gene set enrichment analysis revealed significant overexpression across generations of previously identified gene signatures of the mesenchymal subtype, as well as a tumor mesenchymal metabolic signature. Key mesenchymal markers, including putative tumor-stemness marker CD44, NT5E, SNAI2, and ZEB2, were found to be upregulated across generations. These results suggest tumor progression under bevacizumab challenge to be accompanied by a shift in gene expression towards the mesenchymal subtype, a subtype of GBM associated with enhanced invasiveness, resistance, and worse outcome. Our analysis also revealed expression changes in pathways related to angiogenesis. Pro-angiogenic genes FGF2, MMP1, HIF1A, UGCG, LPAR1, and ITGB3 were found to be upregulated. Furthermore, GSO analysis identified angiogenesis as a significantly enriched ontology within the inflammatory response and ECM remodeling subnetworks identified by spectral clustering. Angiogenesis related genes identified via GSO included highly upregulated inflammatory mediators such as COX2, IL6, IL1A, upregulated pro-angiogenic factors TGFA, WNT5A, FGF2, and a downregulated anti-angiogenic gene, SPARC. Conclusions: These results suggest a mesenchymal and pro-angiogenic response to bevacizumab treatment supported by multiple converging pathways involving inflammation, hypoxia, and ECM remodeling. Strategies preventing the evolution of these responses should be developed and tested in the context of our novel xenograft model in order to improve the durability of response to these therapies and allow them to fulfill their therapeutic promise. Citation Format: William C. Chen, Arman Jahangiri, Garima Yagnik, Maxim Sidorov, Jonathan Rick, Ruby Kuang, Michael DeLay, Manish K. Aghi. Gene expression changes underlying glioblastoma resistance to anti-angiogenic therapy. [abstract]. In: Proceedings of the 107th Annual Meeting of the American Association for Cancer Research; 2016 Apr 16-20; New Orleans, LA. Philadelphia (PA): AACR; Cancer Res 2016;76(14 Suppl):Abstract nr 3271.

A Two-Component Normal Mixture Alternative to the Fay-Herriot Model

Abstract This article considers a robust hierarchical Bayesian approach to deal with random effects of small area means when some of these effects assume extreme values, resulting in outliers. In the presence of outliers, the standard Fay-Herriot model, used for modeling area-level data, under normality assumptions of random effects may overestimate the random effects variance, thus providing less than ideal shrinkage towards the synthetic regression predictions and inhibiting the borrowing of information. Even a small number of substantive outliers of random effects results in a large estimate of the random effects variance in the Fay-Herriot model, thereby achieving little shrinkage to the synthetic part of the model or little reduction in the posterior variance associated with the regular Bayes estimator for any of the small areas. While the scale mixture of normal distributions with a known mixing distribution for the random effects has been found to be effective in the presence of outliers, the solution depends on the mixing distribution. As a possible alternative solution to the problem, a two-component normal mixture model has been proposed, based on non-informative priors on the model variance parameters, regression coefficients and the mixing probability. Data analysis and simulation studies based on real, simulated and synthetic data show an advantage of the proposed method over the standard Bayesian Fay-Herriot solution derived under normality of random effects.

Bayesian predictive inference under a Dirichlet process with sensitivity to the normal baseline

It is well known that the Dirichlet process (DP) model and Dirichlet process mixture (DPM) model are sensitive to the specifications of the baseline distribution. Given a sample from a finite population, we perform Bayesian predictive inference about a finite population quantity (e.g., mean) using a DP model. Generally, in many applications a normal distribution is used for the baseline distribution. Therefore, our main objective is empirical and we show the extent of the sensitivity of inference about the finite population mean with respect to six distributions (normal, lognormal, gamma, inverse Gaussian, a two-component normal mixture and a skewed normal). We have compared the DP model using these baselines with the Polya posterior (fully nonparametric) and the Bayesian bootstrap (sampling with a Haldane prior). We used two examples, one on income data and the other on body mass index data, to compare the performance of these three procedures. These examples show some differences among the six baseline distributions, the Polya posterior and the Bayesian bootstrap, indicating that the normal baseline model cannot be used automatically. Therefore, we consider a simulation study to assess this issue further, and we show how to solve this problem using a leave-one-out kernel baseline. Because the leave-one-out kernel baseline cannot be easily applied to the DPM, we show theoretically how one can solve the sensitivity problem for the DPM as well.

Investigation into the performance of different models for predicting stutter

In this paper we have examined five possible models for the behaviour of the stutter ratio, SR. These were two log-normal models, two gamma models, and a two-component normal mixture model. A two-component normal mixture model was chosen with different behaviours of variance; at each locus SR was described with two distributions, both with the same mean. The distributions have difference variances: one for the majority of the observations and a second for the less well-behaved ones. We apply each model to a set of known single source Identifiler™, NGM SElect™ and PowerPlex(®) 21 DNA profiles to show the applicability of our findings to different data sets. SR determined from the single source profiles were compared to the calculated SR after application of the models. The model performance was tested by calculating the log-likelihoods and comparing the difference in Akaike information criterion (AIC). The two-component normal mixture model systematically outperformed all others, despite the increase in the number of parameters. This model, as well as performing well statistically, has intuitive appeal for forensic biologists and could be implemented in an expert system with a continuous method for DNA interpretation.

Multivariate asset return prediction with mixture models

The use of mixture distributions for modeling asset returns has a long history in finance. New methods of demonstrating support for the presence of mixtures in the multivariate case are provided. The use of a two-component multivariate normal mixture distribution, coupled with shrinkage via a quasi-Bayesian prior, is motivated, and shown to be numerically simple and reliable to estimate, unlike the majority of multivariate GARCH models in existence. Equally important, it provides a clear improvement over use of GARCH models feasible for use with a large number of assets, such as constant conditional correlation, dynamic conditional correlation, and their extensions, with respect to out-of-sample density forecasting. A generalization to a mixture of multivariate Laplace distributions is motivated via univariate and multivariate analysis of the data, and an expectation–maximization algorithm is developed for its estimation in conjunction with a quasi-Bayesian prior. It is shown to deliver significantly better forecasts than the mixed normal, with fast and numerically reliable estimation. Crucially, the distribution theory required for portfolio theory and risk assessment is developed.

Multivariate Asset Return Prediction with Mixture Models

The use of mixture distributions for modeling asset returns has a long history in finance. New methods of demonstrating support for the presence of mixtures in the multivariate case are provided. The use of a two-component multivariate normal mixture distribution, coupled with shrinkage via a quasi-Bayesian prior, is motivated, and shown to be numerically simple and reliable to estimate, unlike the majority of multivariate GARCH models in existence. Equally important, it provides a clear improvement over use of GARCH models feasible for use with a large number of assets, such as CCC, DCC, and their extensions, with respect to out-of-sample density forecasting. A generalization to a mixture of multivariate Laplace distributions is motivated via univariate and multivariate analysis of the data, and an EM-algorithm is developed for its estimation in conjunction with a quasi-Bayesian prior. It is shown to deliver significantly better forecasts than the mixed normal, with fast and numerically reliable estimation. Crucially, the distribution theory required for portfolio theory and risk assessment is developed.

LARGE-SCALE SIMULTANEOUS INFERENCE WITH APPLICATIONS TO THE DETECTION OF DIFFERENTIAL EXPRESSION WITH MICROARRAY DATA

An important problem in microarray experiments is the detection of genes that are differentially expressed in agiven mumber of classes. We consider a straightforward and easily implemented method for estimating the posterior probability that an individual gene is null. The problem can be expressed in a two-component mixture framework, using an empirical Bayes approach. Current methods of implementing this approach either have some limitations due to the minimal assumptions made or with more specific assumptions are computationally intensive. By converting to a z-score the value of the test statistic used to test the significance of each gene, we can use a simple two-component normal mixture to model adequately the distribution of this score. In the context of the application of this approach to a well known breast cancer data set, we consider some of the issues associated with the problem of the detection of differential expression, including the case where there is need for the use of an empirical null distribution in place of the standard normal (the theoretical null) and the case where none of the genes might be differentially expressed. We also describe briefly some initial results on a cluster analysis approach to this problem, which attempts to model the joint distribution of the individual gene expressions. This latter approach thus has to make distributional assumptions which are note necessary with the former approach based on the z-scores. However, in the case where the distributional assumptions are valid, it has the potential to provide a more powerful analysis.

A simple implementation of a normal mixture approach to differential gene expression in multiclass microarrays

An important problem in microarray experiments is the detection of genes that are differentially expressed in a given number of classes. We provide a straightforward and easily implemented method for estimating the posterior probability that an individual gene is null. The problem can be expressed in a two-component mixture framework, using an empirical Bayes approach. Current methods of implementing this approach either have some limitations due to the minimal assumptions made or with more specific assumptions are computationally intensive. By converting to a z-score the value of the test statistic used to test the significance of each gene, we propose a simple two-component normal mixture that models adequately the distribution of this score. The usefulness of our approach is demonstrated on three real datasets.

Consistent presence of two normally distributed sperm subpopulations within normozoospermic human semen: a kinematic study

Sperm motion analysis was performed using computer-aided semen analysis (CASA) on nine normozoospermic human ejaculates following six fractionation schemes that included Percoll discontinuous gradient centrifugation, swim-up, hyper and hypo-osmotic stress in which 4600-13 400 tracks were analysed (depending on the scheme). In addition, the effect of ageing for 2 h in seminal plasma or re-addition of seminal plasma, on the kinematic properties was investigated. Analysis following sorting of the swimming velocity at 5 microm/sec intervals revealed the presence of two discrete normally distributed subpopulations selected on the basis of path velocity (VAP): these subpopulations were also present in the track speed (VCL) and progressive velocity (VSL) distributions. The validity of the two subpopulations was examined using a two-component normal mixture model and found to be highly statistically significant (p < 0.00001). The basis for the subpopulations was independent from the possession of midpiece vesicles or hyperactivation. Swim-up produced subpopulations of sperm that were significantly different from those obtained by Percoll separation. Compared with sperm in seminal plasma, swim-up and Percoll separation significantly increased the level of hyperactivated sperm at 2 h. Hyperactivation was inhibited by the re-addition of seminal plasma. Swim-up enhanced the proportion of sperm in the fast VAP subpopulation (independently of the level of hyperactivation) and whilst this altered distribution was maintained following the addition of homologous seminal plasma the mean velocity of the fast fraction was significantly decreased. Both swim-up and Percoll separation significantly increased the mean velocity of the fast subpopulation but the increase was greater with swim-up. This analysis highlights a way in which subpopulations may be detected and analysed. The procedure revealed fundamental differences in the kinematic properties of sperm processed by different methods that are used in assisted reproductive technology and showed that these were independent of hyperactivation.

Incidence and distribution of first-episode mania by age: results from a 35-year study

Few epidemiological studies have investigated incidence by age or age at onset distributions for mania or bipolar disorder. The current study aimed to determine these in a defined area in south-east London, over a 35-year period. All cases of first-episode mania presenting to psychiatric services in Camberwell, south-east London, between 1965 and 1999 were identified. Incidence rates by age, using 5-year age-at-onset bands, were estimated and the structure of the age-at-onset distribution for first-episode mania was investigated using finite mixture distributions (admixture analysis). The incidence of DSM-IV bipolar I disorder (BP I), first manic episode peaked in early adult life (16.38/100,000 population per year in the 21-25 years band) with a much smaller peak in mid-life. A two-component normal mixture distribution fitted age at onset better than either a single normal distribution or a three-component mixture, implying the existence of early and later onset subgroups. The early onset group had a stronger family history of bipolar disorder, and showed more acute, severe and atypical symptoms during their first manic episode. The incidence of mania peaks in early adult life but there is clear evidence of early and later onset subgroups which may represent different forms of disorder.

Theoretical analysis of power in a two-component normal mixture model

In a likelihood-ratio test for a two-component Normal location mixture, the natural parametrisation degenerates to non-uniqueness under the null hypothesis. One consequence of this ambiguity is that the limiting distribution of the likelihood-ratio statistic is quite irregular, being of extreme-value type rather than chi-squared. Another irregular feature is that the likelihood-ratio statistic diverges to infinity, and so limit theory is nonstandard in this respect as well. These results, in a form applying directly to the likelihood-ratio statistic rather than to an approximating stochastic process, have recently been established by Liu and Shao (2004). While they address only properties under the null hypothesis, they hint that the power of the likelihood-ratio test may be less than in more conventional settings. In this paper we show that this is indeed the case. Using a system of local alternative hypotheses we quantify the extent to which power is reduced. We show that, in a large class of circumstances, the reduction in power can be appreciated in terms of inflation (by a log–log factor) of the displacement of the closest local alternative that can just be distinguished from the null hypothesis. However, in important respects the properties of power under local alternatives are significantly more complex than this, and exhibit two types of singularity. In particular, in two quite different respects, small changes in the local alternative, in the neighbourhood of a threshold, can dramatically alter power.

Asymptotics for the likelihood ratio test in a two-component normal mixture model

This paper characterizes the asymptotic properties of the likelihood ratio test (LRT) statistic for testing homogeneity in a two-component normal mean mixture model. We justify that the LRT statistic 2 λ n is asymptotically equivalent to the square of the supremum of the stochastic process studied in Bickel and Chernoff (Statistics and Probability: A Raghu Raj Bahadur Festschrift (1993) 83). In particular, we prove that 2 λ n diverges to +∞ at a rate of log log n which confirms a conjecture of Hartigan (Proceedings of Berkeley Conference in Honor of Jerzy Neyman and Jack Kiefer (1985)). More specifically, under the null hypothesis we prove the following fact: lim n→∞ P {2λ n− log log n+ log(2π 2)⩽x}= exp(− e −x/2), x∈ R.

A LILLIEFORS TEST OF FIT TO THE TWO-COMPONENT HOMOSCEDASTIC NORMAL MIXTURE

The null distribution of a Lilliefors modification of the Kolmogorov-Smirnov test, was winvestigated as a goodness of fit test to the two-component homoscedastic normal mixture. The mixture parameters were estimated using a variable metric minimization algorithm to search for the maximum likelihood estimates. Selected percentiles of the null distribution were estimated from four sets of 2,500 pseudorandom samples, each of size 100, 150, 200, 250, 300, 350, 400, 450 and 500, with mixing proportions of p = 0.5, 0.6, 0.7, 0.8, 0.9, and 0.95, and standardized mean differences of d = 0.5, 1, 2, 3, 4, and 5 standard deviations apart (3,240,000 total samples). The results suggest that the test is not invariant to the values of the mixture parameters, p and d. The critical values appeared to be well-behaved, strictly decreasing functions of the sample size. An essentially invariant test is developed by modeling the selected critical values as nonlinear functions of p, d, and n. Power studies suggest reasonable power for several alternatives.

A hybrid EM/Gauss-Newton algorithm for maximum likelihood in mixture distributions

A faster alternative to the EM algorithm in finite mixture distributions is described, which alternates EM iterations with Gauss-Newton iterations using the observed information matrix. At the expense of modest additional analytical effort in obtaining the observed information, the hybrid algorithm reduces the computing time required and provides asymptotic standard errors at convergence. The algorithm is illustrated on the two-component normal mixture.

Correction: The Likelihood Ratio Test for the Two-Component Normal Mixture Problem: Power and Sample Size Analysis

Correction: The Likelihood Ratio Test for the Two-Component Normal Mixture Problem: Power and Sample Size Analysis