Expectation-maximization Algorithm For Parameter Estimation Research Articles

An Expectation-Maximization Algorithm for Combining a Sample of Partially Overlapping Covariance Matrices.

The generation of unprecedented amounts of data brings new challenges in data management, but also an opportunity to accelerate the identification of processes of multiple science disciplines. One of these challenges is the harmonization of high-dimensional unbalanced and heterogeneous data. In this manuscript, we propose a statistical approach to combine incomplete and partially-overlapping pieces of covariance matrices that come from independent experiments. We assume that the data are a random sample of partial covariance matrices sampled from Wishart distributions and we derive an expectation-maximization algorithm for parameter estimation. We demonstrate the properties of our method by (i) using simulation studies and (ii) using empirical datasets. In general, being able to make inferences about the covariance of variables not observed in the same experiment is a valuable tool for data analysis since covariance estimation is an important step in many statistical applications, such as multivariate analysis, principal component analysis, factor analysis, and structural equation modeling.

LIQA: long-read isoform quantification and analysis

Long-read RNA sequencing (RNA-seq) technologies can sequence full-length transcripts, facilitating the exploration of isoform-specific gene expression over short-read RNA-seq. We present LIQA to quantify isoform expression and detect differential alternative splicing (DAS) events using long-read direct mRNA sequencing or cDNA sequencing data. LIQA incorporates base pair quality score and isoform-specific read length information in a survival model to assign different weights across reads, and uses an expectation-maximization algorithm for parameter estimation. We apply LIQA to long-read RNA-seq data from the Universal Human Reference, acute myeloid leukemia, and esophageal squamous epithelial cells and demonstrate its high accuracy in profiling alternative splicing events.

Zero-inflated Poisson factor model with application to microbiome read counts.

Dimension reduction of high-dimensional microbiome data facilitates subsequent analysis such as regression and clustering. Most existing reduction methods cannot fully accommodate the special features of the data such as count-valued and excessive zero reads. We propose a zero-inflated Poisson factor analysis model in this paper. The model assumes that microbiome read counts follow zero-inflated Poisson distributions with library size as offset and Poisson rates negatively related to the inflated zero occurrences. The latent parameters of the model form a low-rank matrix consisting of interpretable loadings and low-dimensional scores that can be used for further analyses. We develop an efficient and robust expectation-maximization algorithm for parameter estimation. We demonstrate the efficacy of the proposed method using comprehensive simulation studies. The application to the Oral Infections, Glucose Intolerance, and Insulin Resistance Study provides valuable insights into the relation between subgingival microbiome and periodontaldisease.

High-dimensional unsupervised classification via parsimonious contaminated mixtures

The contaminated Gaussian distribution represents a simple heavy-tailed elliptical generalization of the Gaussian distribution; unlike the often-considered t-distribution, it also allows for automatic detection of mild outlying or “bad” points in the same way that observations are typically assigned to the groups in the finite mixture model context. Starting from this distribution, we propose the contaminated factor analysis model as a method for dimensionality reduction and detection of bad points in higher dimensions. A mixture of contaminated Gaussian factor analyzers (MCGFA) model follows therefrom, and extends the recently proposed mixture of contaminated Gaussian distributions to high-dimensional data. We introduce a family of 32 parsimonious models formed by introducing constraints on the covariance and contamination structures of the general MCGFA model. We outline a variant of the expectation-maximization algorithm for parameter estimation. Various implementation issues are discussed, and the novel family of models is compared to well-established approaches on both simulated and real data.

An approach based on expectation-maximization algorithm for parameter estimation of Lamb wave signals

Abstract In the structural health monitoring or nondestructive examination system based on the Lamb wave technology, the accurate and valid characteristics of wave packet extracted from the signal are critical factors to evaluate damage. However, the dispersion effect and the multi-mode characteristic in the elastic wave make the data extraction difficult and degrade the resolution, and therefore further prevent the effectiveness of Lamb wave for damage detection. In this study, we proposed a model-based approach for extracting effective characteristics from the noisy signals. By taking the narrow band Gabor pulse as the incident pulse and considering the general non-linear frequency dispersion (quadratic dispersion), we developed a model with five parameters to model the dispersive wave packet and obtained the parameter vector of each wave packet by the expectation-maximization (EM) algorithm. The parameters in the model present the characteristics of signals, which can be further applied to locate and evaluate the structure’s damage. To study the convergence property, synthetic signals with different sampling rates and noise intensities were considered. Furthermore the developed approach is also verified by the experimental data from an isotropic aluminum plate.

A latent-class model for estimating product-choice probabilities from clickstream data

This paper analyzes customer product-choice behavior based on the recency and frequency of each customer’s page views on e-commerce sites. Recently, we devised an optimization model for estimating product-choice probabilities that satisfy monotonicity, convexity, and concavity constraints with respect to recency and frequency. This shape-restricted model delivered high predictive performance even when there were few training samples. However, typical e-commerce sites deal in many different varieties of product, so the predictive performance of the model can be further improved by integrating such product heterogeneity. For this purpose, we develop a novel latent-class shape-restricted model for estimating product-choice probabilities for each latent class of products. We also give a tailored expectation-maximization algorithm for parameter estimation. Computational results demonstrate that higher predictive performance is achieved with our latent-class model than with the previous shape-restricted model or latent-class logistic regression.

An Online and Distributed Approach for Modulation Classification Using Wireless Sensor Networks

Based on an online and distributed implementation of the expectation-maximization (EM) algorithm, a hybrid likelihood-based modulation classifier is proposed for a sensor network subject to unknown nonidentical flat fading. The proposed algorithm compares favorably in terms of computational complexity with respect to other maximum likelihood classifiers that rely on the batch (offline) EM algorithm for parameter estimation. Upon the reception of a new sample, each sensor computes the a posteriori probability of the corresponding symbol based on the average consensus algorithm, which relies on local communications among nearby sensors and promotes scalability and power efficiency. Then, each sensor updates its statistics with the innovation obtained from the samples received during the last symbol interval and new estimates for local sensor parameters are computed. Simulation results demonstrate that the proposed online and distributed EM-based classifier can achieve performance close that of a clairvoyant classifier equipped with perfect channel state information.

Oil price volatility forecast with mixture memory GARCH

We expand the literature of volatility and Value-at-Risk forecasting of oil price returns by comparing the recently proposed Mixture Memory GARCH (MMGARCH) model to other discrete volatility models (GARCH, RiskMetrics, EGARCH, APARCH, FIGARCH, HYGARCH, and FIAPARCH). We incorporate an Expectation-Maximization algorithm for parameter estimation of the MMGARCH and find different structures in volatility level as well as shock persistence. MMGARCH is also able to cover asymmetric and long memory effects. Furthermore, a dissimilar memory structure in variance of WTI and Brent crude oil prices is observed which is supported by additional tests. Parameter estimation and comparison of the models reveal significant long memory and asymmetry in oil price returns. In regard of variance forecasting and Value-at-Risk prediction, it is shown that MMGARCH outperforms the aforementioned models due to its dynamic approach in varying the volatility level and memory of the process. We find MMGARCH superior for application in risk management as a result of its flexibility in adjusting to variance shifts and shocks.

Correlated probit analysis of repeatedly measured ordinal and continuous outcomes with application to the Health and Retirement Study.

The Health and Retirement Study was designed to evaluate changes in health and labor force participation during and after the transition from working to retirement. Every 2years, participants provided information about their self-rated health (SRH), body mass index (BMI), smoking status, and other characteristics. Our goal was to assess the effects of smoking and gender on trajectories of change in BMI and SRH over time. Joint longitudinal analysis of outcome measures is preferable to separate analyses because it allows to account for the correlation between the measures, to test the effects of predictors while controlling type I error, and potentially to improve efficiency. However, because SRH is an ordinal measure while BMI is continuous, formulating a joint model and parameter estimation is challenging. A joint correlated probit model allowed us to seamlessly account for the correlations between the measures over time. Established estimating procedures for such models are based on quasi-likelihood or numerical approximations that may be biased or fail to converge. Therefore, we proposed a novel expectation-maximization algorithm for parameter estimation and a Monte Carlo bootstrap approach for standard errors approximation. Expectation-maximization algorithms have been previously considered for combinations of binary and/or continuous repeated measures; however, modifications were needed to handle combinations of ordinal and continuous responses. A simulation study demonstrated that the algorithm converged and provided approximately unbiased estimates with sufficiently large sample sizes. In the Health and Retirement Study, male gender and smoking were independently associated with steeper deterioration in self-rated health and with lower average BMI. Copyright © 2016 John Wiley & Sons, Ltd.

Oil Price Volatility Forecast with Mixture Memory GARCH

First Version: 03/11/2015This Version: 04/01/2016We expand the literature of volatility and Value-at-Risk forecasting of oil price returns by comparing the recently proposed Mixture Memory GARCH (MMGARCH) model to other discrete volatility models (GARCH, FIGARCH, and HYGARCH). We incorporate an Expectation-Maximization algorithm for parameter estimation of the MMGARCH and find regimes that differ in volatility level as well as shock persistence. Furthermore, we observe dissimilar memory structure in variance of WTI and Brent crude oil prices which is confirmed by altering the mixture components. In regard of variance forecasting and Value-at-Risk prediction, we show that MMGARCH outperforms the aforementioned models due to its dynamic approach in varying the volatility level and memory of the process. We find MMGARCH superior for application in risk management as a result of its flexibility in adjusting to variance shifts and shocks.

A joint model for multistate disease processes and random informative observation times, with applications to electronic medical records data.

Multistate models are used to characterize individuals' natural histories through diseases with discrete states. Observational data resources based on electronic medical records pose new opportunities for studying such diseases. However, these data consist of observations of the process at discrete sampling times, which may either be pre-scheduled and non-informative, or symptom-driven and informative about an individual's underlying disease status. We have developed a novel joint observation and disease transition model for this setting. The disease process is modeled according to a latent continuous-time Markov chain; and the observation process, according to a Markov-modulated Poisson process with observation rates that depend on the individual's underlying disease status. The disease process is observed at a combination of informative and non-informative sampling times, with possible misclassification error. We demonstrate that the model is computationally tractable and devise an expectation-maximization algorithm for parameter estimation. Using simulated data, we show how estimates from our joint observation and disease transition model lead to less biased and more precise estimates of the disease rate parameters. We apply the model to a study of secondary breast cancer events, utilizing mammography and biopsy records from a sample of women with a history of primary breast cancer.

A Generative Dependency N-gram Language Model: Unsupervised Parameter Estimation and Application

We design a language model based on a generative dependency structure for sentences. The parameter of the model is the probability of a dependency N-gram, which is composed of lexical words with four types of extra tag used to model the dependency relation and valence. We further propose an unsupervised expectation-maximization algorithm for parameter estimation, in which all possible dependency structures of a sentence are considered. As the algorithm is language-independent, it can be used on a raw corpus from any language, without any part-of-speech annotation, tree-bank or trained parser. We conducted experiments using four languages, i.e., English, German, Spanish and Japanese, to illustrate the applicability and the properties of the proposed approach. We further apply the proposed approach to a Chinese microblog data set to extract and investigate Internet-based, non-standard lexical dependency features of user-generated content.

Flexible mixture modeling via the multivariate t distribution with the Box-Cox transformation: an alternative to the skew-t distribution

Cluster analysis is the automated search for groups of homogeneous observations in a data set. A popular modeling approach for clustering is based on finite normal mixture models, which assume that each cluster is modeled as a multivariate normal distribution. However, the normality assumption that each component is symmetric is often unrealistic. Furthermore, normal mixture models are not robust against outliers; they often require extra components for modeling outliers and/or give a poor representation of the data. To address these issues, we propose a new class of distributions, multivariate t distributions with the Box-Cox transformation, for mixture modeling. This class of distributions generalizes the normal distribution with the more heavy-tailed t distribution, and introduces skewness via the Box-Cox transformation. As a result, this provides a unified framework to simultaneously handle outlier identification and data transformation, two interrelated issues. We describe an Expectation-Maximization algorithm for parameter estimation along with transformation selection. We demonstrate the proposed methodology with three real data sets and simulation studies. Compared with a wealth of approaches including the skew-t mixture model, the proposed t mixture model with the Box-Cox transformation performs favorably in terms of accuracy in the assignment of observations, robustness against model misspecification, and selection of the number of components.

On the Use of Different Speech Representations for Speaker Modeling

Numerous speech representations have been reported to be useful in speaker recognition. However, there is much less agreement on which speech representation provides a perfect representation of speaker-specific information conveyed in a speech signal. Unlike previous work, we propose an alternative approach to speaker modeling by the simultaneous use of different speech representations in an optimal way. Inspired by our previous empirical studies, we present a soft competition scheme on different speech representations to exploit different speech representations in encoding speaker-specific information. On the basis of this soft competition scheme, we present a parametric statistical model, generalized Gaussian mixture model (GGMM), to characterize a speaker identity based on different speech representations. Moreover, we develop an expectation-maximization algorithm for parameter estimation in the GGMM. The proposed speaker modeling approach has been applied to text-independent speaker recognition and comparative results on the KING speech corpus demonstrate its effectiveness.

Building an identifiable latent class model with covariate effects on underlying and measured variables

In recent years, latent class models have proven useful for analyzing relationships between measured multiple indicators and covariates of interest. Such models summarize shared features of the multiple indicators as an underlying categorical variable, and the indicators' substantive associations with predictors are built directly and indirectly in unique model parameters. In this paper, we provide a detailed study on the theory and application of building models that allow mediated relationships between primary predictors and latent class membership, but that also allow direct effects of secondary covariates on the indicators themselves. Theory for model identification is developed. We detail an Expectation-Maximization algorithm for parameter estimation, standard error calculation, and convergent properties. Comparison of the proposed model with models underlying existing latent class modeling software is provided. A detailed analysis of how visual impairments affect older persons' functioning requiring distance vision is used for illustration.