Expectation Conditional Maximization Research Articles

Covariate-dependent spatio-temporal covariance models

Geostatistical regression models are widely used in environmental and geophysical sciences to characterize the mean and dependence structures for spatio-temporal data. Traditionally, these models account for covariates solely in the mean structure, neglecting their potential impact on the spatio-temporal covariance structure. This paper addresses a significant gap in the literature by proposing a novel covariate-dependent covariance model within the spatio-temporal random-effects model framework. Our approach integrates covariates into the covariance function through a Cholesky-type decomposition, ensuring compliance with the positive-definite condition. We employ maximum likelihood for parameter estimation, complemented by an efficient expectation conditional maximization algorithm. Simulation studies demonstrate the superior performance of our method compared to conventional techniques that ignore covariates in spatial covariances. We further apply our model to a PM2.5 dataset from Taiwan, highlighting wind speed’s pivotal role in influencing the spatio-temporal covariance structure. Additionally, we incorporate wind speed and sunshine duration into the covariance function for analyzing Taiwan ozone data, revealing a more intricate relationship between covariance and these meteorological variables.

A novel finite mixture model based on generalized scale mixtures of generalized normal distributions with application to stock dataset

This paper introduces a novel family of distributions known as generalized scale mixtures of generalized normal distributions (GSMGN). These distributions incorporate two additional shape parameters that serve to regulate the shape and tails of the distribution. A finite mixture model based on this family is presented to address clustering heterogeneous data in the presence of leptokurtic and heavy-tailed outcomes. The estimation of the parameters of this model are obtained by developing an ECM-PLA ensemble algorithm which combine the profile likelihood approach (PLA) and the classical Expectation Conditional Maximization (ECM) algorithm, and the observed information matrix is obtained. The applicability of this new family and the numerical performance of the proposed methodology is discussed through simulated and real data examples.

A modeling framework for detecting and leveraging node-level information in Bayesian network inference.

Bayesian graphical models are powerful tools to infer complex relationships in high dimension, yet are often fraught with computational and statistical challenges. If exploited in a principled way, the increasing information collected alongside the data of primary interest constitutes an opportunity to mitigate these difficulties by guiding the detection of dependence structures. For instance, gene network inference may be informed by the use of publicly available summary statistics on the regulation of genes by genetic variants. Here we present a novel Gaussian graphical modeling framework to identify and leverage information on the centrality of nodes in conditional independence graphs. Specifically, we consider a fully joint hierarchical model to simultaneously infer (i) sparse precision matrices and (ii) the relevance of node-level information for uncovering the sought-after network structure. We encode such information as candidate auxiliary variables using a spike-and-slab submodel on the propensity of nodes to be hubs, which allows hypothesis-free selection and interpretation of a sparse subset of relevant variables. As efficient exploration of large posterior spaces is needed for real-world applications, we develop a variational expectation conditional maximization algorithm that scales inference to hundreds of samples, nodes and auxiliary variables. We illustrate and exploit the advantages of our approach in simulations and in a gene network study which identifies hub genes involved in biological pathways relevant to immune-mediated diseases.

Analyzing method comparison data with skew-normal measurement error models: incorporating generalized scale mixtures and varying degrees of freedom

Comparing two measurement methods is vital in various fields, such as medical research, epidemiology, economics, and environmental studies, to determine whether a new measurement method can be used interchangeably with an existing one. Measurement error models (MEMs) are commonly used for this purpose, where the methods have different measuring scales. However, these models often assume normality, which can be problematic when dealing with skewed and heavy-tailed data. To address this issue, we propose the replicated measurement error model (RMEM) under scale mixtures of skew-normal (SMSN) distributions with different levels of skewness and heavy tails of the true covariate and error distributions. Our primary aim is to assess the extent of similarity and agreement between two measurement methods using this model. The expectation conditional maximization (ECM) approach is applied to fit the model. A simulation study is conducted to evaluate the effectiveness and robustness of the proposed methodology and is illustrated by analyzing systolic blood pressure data. The probability of agreement is used further to assess the agreement between the two measurement methods. The findings indicate that the proposed model effectively analyses replicated method comparison data with measurement errors, even when there are outliers, skewness, and heavy-tailedness.

A modified EM-type algorithm to estimate semi-parametric mixtures of non-parametric regressions

Semi-parametric Gaussian mixtures of non-parametric regressions (SPGMNRs) are a flexible extension of Gaussian mixtures of linear regressions (GMLRs). The model assumes that the component regression functions (CRFs) are non-parametric functions of the covariate(s) whereas the component mixing proportions and variances are constants. Unfortunately, the model cannot be reliably estimated using traditional methods. A local-likelihood approach for estimating the CRFs requires that we maximize a set of local-likelihood functions. Using the Expectation-Maximization (EM) algorithm to separately maximize each local-likelihood function may lead to label-switching. This is because the posterior probabilities calculated at the local E-step are not guaranteed to be aligned. The consequence of this label-switching is wiggly and non-smooth estimates of the CRFs. In this paper, we propose a unified approach to address label-switching and obtain sensible estimates. The proposed approach has two stages. In the first stage, we propose a model-based approach to address the label-switching problem. We first note that each local-likelihood function is a likelihood function of a Gaussian mixture model (GMM). Next, we reformulate the SPGMNRs model as a mixture of these GMMs. Lastly, using a modified version of the Expectation Conditional Maximization (ECM) algorithm, we estimate the mixture of GMMs. In addition, using the mixing weights of the local GMMs, we can automatically choose the local points where local-likelihood estimation takes place. In the second stage, we propose one-step backfitting estimates of the parametric and non-parametric terms. The effectiveness of the proposed approach is demonstrated on simulated data and real data analysis.

Constrained inference in scaled mixed effects models with applications to London Junior School Project data

Multiple outcomes arise frequently in many different settings. Mixed effect models are a useful tool for the analyses of such data. However, when outcomes are measured on different scales, analyses based on any one scale are misleading. Often parameters of the model are subject to known order restrictions. To incorporate heterogeneity across different outcomes, we propose a scaled linear mixed effect model. To estimate parameters, we propose a maximum likelihood estimation procedure based on a restricted version of the expectation-conditional maximization either algorithm. Constrained hypotheses testing procedures are developed using likelihood ratio tests. The empirical significance levels and powers are studied using simulations. This article shows that incorporating the restrictions improves the mean squared errors of the estimates and the power of the tests. The methodology is applied on the London Junior School Project data incorporating known restrictions of patterns of scores. Journal of Statistical Research 2023, Vol 57, No.1-2, pp.81-94

Time series clustering based on latent volatility mixture modeling with applications in finance

Modeling financial time series data poses a significant challenge in the realm of time series analysis. The Autoregressive Conditional Heteroskedasticity (ARCH) model stands out as a potent tool for capturing time-varying volatility and heteroskedasticity in financial data. However, conventional ARCH models display sensitivity to departures from normality, leading to the development of extensions employing more flexible distributions. In this context, we propose a robust enhancement to the mixture of ARCH (MoARCH) model by integrating normal mean–variance mixture (NMVM) distributions to model component errors. The stochastic representation of the proposed model allows for a straightforward implementation of an Expectation Conditional Maximization Either (ECME) algorithm for obtaining Maximum Penalized Likelihood estimates (MPL). To thoroughly evaluate the model, we conduct four simulation studies to explore finite-sample properties, assess MPL estimators, scrutinize model robustness, and evaluate the accuracy of our proposal in fitting, clustering, and forecasting. Practical applications further highlight the effectiveness of our methodology, showcasing successful implementations across diverse real datasets.

A novel mixture model for characterizing human aiming performance data

Fitts’ law is often employed as a predictive model for human movement, especially in the field of human-computer interaction. Models with an assumed Gaussian error structure are usually adequate when applied to data collected from controlled studies. However, observational data (often referred to as data gathered ‘in the wild’) typically display noticeable positive skewness relative to a mean trend as users do not routinely try to minimize their task completion time. As such, the exponentially modified Gaussian (EMG) regression model has been applied to aimed movements data. However, it is also of interest to reasonably characterize those regions where a user likely was not trying to minimize their task completion time. In this article, we propose a novel model with a two-component mixture structure—one Gaussian and one exponential—on the errors to identify such a region. An expectation-conditional-maximization (ECM) algorithm is developed for estimation of such a model and some properties of the algorithm are established. The efficacy of the proposed model, as well as its ability to inform model-based clustering, are addressed in this work through extensive simulations and an insightful analysis of a human aiming performance study.

Estimation for vector autoregressive model under multivariate skew-t-normal innovations

Current procedures for estimating the parameters of [Formula: see text]th order vector autoregressive (VAR [Formula: see text]) model are usually based on assuming that the ensuing error distribution is multivariate normal. But there exists large body of evidence that several data encountered in real life are skewed; thereby making estimators derived based on normality assumption not suitable in such scenarios. This prompts for the search of appropriate methods for skewed distributions. Therefore, this article proposes estimators for the mean and covariance matrices of the [Formula: see text] model under multivariate skew- [Formula: see text]-normal (MSTN) distribution. Also, estimators for the shape and skewness parameters are provided. The expectation conditional maximization (ECM) and its extension the expectation conditional maximization either (ECME) algorithms are the tools used to derive the estimators. The performance of the estimators were examined through extensive simulations, and results show that they compete favourably with other numerical methods especially when the underlying distribution is skewed. The usefulness of our estimators was illustrated using a real data set on some US economic indicators. The VAR [Formula: see text] model under MSTN distribution provides a good fit, better than [Formula: see text] model under the assumption of normality.

The generalized scale mixtures of asymmetric generalized normal distributions with application to stock data

<abstract><p>In this paper, we introduced a family of distributions with a very flexible shape named generalized scale mixtures of generalized asymmetric normal distributions (GSMAGN). We investigated the main properties of the new family including moments, skewness, kurtosis coefficients and order statistics. A variant of the expectation maximization (EM)-type algorithm was established by combining the proflie likihood approach (PLA) with the classical expectation conditional maximization (ECM) algorithm for parameter estimation of this model. This approach with analytical expressions in the E-step and tractable M-step can greatly improve the computational speed and efficiency of the algorithm. The performance of the proposed algorithm was assessed by some simulation studies. The feasibility of the proposed methodology was illustrated through two real datasets.</p></abstract>

Multivariate contaminated normal mixture regression modeling of longitudinal data based on jointmean‐covariancemodel

AbstractOutliers are common in longitudinal data analysis, and the multivariate contaminated normal (MCN) distribution in model‐based clustering is often used to detect outliers and provide robust parameter estimates in each subgroup. In this paper, we propose a method, the mixture of MCN (MCNM), based on the joint mean‐covariance model, specifically designed to analyze longitudinal data characterized by mild outliers. Our model can automatically detect outliers in longitudinal data and provide robust parameter estimates in each subgroup. We use iteratively expectation‐conditional maximization (ECM) algorithm and Aitken acceleration to estimate the model parameters, achieving both algorithm acceleration and stable convergence. Our proposed method simultaneously clusters the population, identifies progression patterns of the mean and covariance structures for different subgroups over time, and detects outliers. To demonstrate the effectiveness of our method, we conduct simulation studies under various cases involving different proportions and degrees of contamination. Additionally, we apply our method to real data on the number of people infected with AIDS in 49 countries or regions from 2001 to 2021. Results show that our proposed method effectively clusters the data based on various mean progression trajectories. In summary, our proposed MCNM based on the joint mean‐covariance model and MCD of covariance matrices provides a robust method for clustering longitudinal data with mild outliers. It effectively detects outliers and identifies progression patterns in different groups over time, making it valuable for various applications in longitudinal data analysis.

Mixtures of factor analysers with censored responses and external covariates: An application to educational data from Peru.

Analysing data from educational tests allows governments to make decisions for improving the quality of life of individuals in a society. One of the key responsibilities of statisticians is to develop models that provide decision-makers with pertinent information about the latent process that educational tests seek to represent. Mixtures of factor analysers (MtFA) have emerged as a powerful device for model-based clustering and classification of high-dimensional data containing one or several groups of observations with fatter tails or anomalous outliers. This paper considers an extension of MtFA for robust clustering of censored data, referred to as the MtFAC model, by incorporating external covariates. The enhanced flexibility of including covariates in MtFAC enables cluster-specific multivariate regression analysis of dependent variables with censored responses arising from upper and/or lower detection limits of experimental equipment. An alternating expectation conditional maximization (AECM) algorithm is developed for maximum likelihood estimation of the proposed model. Two simulation experiments are conducted to examine the effectiveness of the techniques presented. Furthermore, the proposed methodology is applied to Peruvian data from the 2007 Early Grade Reading Assessment, and the results obtained from the analysis provide new insights regarding the reading skills of Peruvian students.

Parameter estimation of the multivariate unrestricted skew-normal distribution using ECM algorithm

The probability density function of the multivariate unrestricted skew-normal (SUN) distribution, corresponding to a screened normal density, allow to modeling skewness and kurtosis in data in terms of a skewness parameter vector and a truncation parameter matrix. These parameters are related to the shape and heavy-tails of the density. In this article, we present the Expectation/Conditional Maximization (ECM) algorithm for the SUN distribution based on a hierarchical stochastic representation. In addition, behavior of ECM algorithm’s steps is measured using an information theoretic approach based on Jeffrey’s divergence and related homogeneity test. Usefulness of the proposed method is illustrated by an application to Chilean economic perception data.

Stochastic Maximum Likelihood Direction Finding in the Presence of Nonuniform Noise Fields

The maximum likelihood (ML) technique plays an important role in direction-of-arrival (DOA) estimation. In this paper, we employ and design the expectation–conditional maximization either (ECME) algorithm, a generalization of the expectation–maximization algorithm, for solving the ML direction finding problem of stochastic sources, which may be correlated, in unknown nonuniform noise. Unlike alternating maximization, the ECME algorithm updates both the source and noise covariance matrix estimates by explicit formulas, and can guarantee that both estimates are positive semi-definite and definite, respectively. Thus, the ECME algorithm is computationally efficient and operationally stable. Simulation results confirm that the ECME algorithm can efficiently obtain the ML based DOA estimate of each stochastic source.

Multivariate Contaminated Normal Censored Regression Model: Properties and Maximum Likelihood Inference

The Multivariate Contaminated Normal (MCN) distribution which contains two extra parameters with respect to parameters of the multivariate normal distribution, one for controlling the proportion of mild outliers and the other for specifying the degree of contamination, has been widely applied in robust statistics in the case of elliptically heavy-tailed empirical distributions. This article extends the MCN model to data with possibly censored values due to limits of quantification, referred to as the MCN with censoring (MCN-C) model, and further establishes the censored multivariate linear regression model where the random errors have the MCN distribution, named as the MCN censored regression (MCN-CR) model. Two computationally feasible Expectation Conditional Maximization (ECM) algorithms are developed for maximum likelihood estimation of MCN-C and MCN-CR models. An information-based method is used to approximate the standard errors of location parameters and regression coefficients. The capability and effectiveness of the MCN-C and MCN-CR models are illustrated via two real-data examples. A simulation study is conducted to investigate the superiority of the proposed models in terms of fit, accuracy of parameter estimation and censored data recovery as compared with classical approaches. Supplementary materials for this article are available online.

An expectation conditional maximization algorithm for the skew-normal based stochastic frontier model

An expectation conditional maximization algorithm for the skew-normal based stochastic frontier model

Analyzing COVID-19 data in the Canadian province of Manitoba: A new approach

The basic homogeneous SEIR (susceptible–exposed–infected–removed) model is a commonly used compartmental model for analysing infectious diseases such as influenza and COVID-19. However, in the homogeneous SEIR model, it is assumed that the population of study is homogeneous and, one cannot incorporate individual-level information (e.g., location of infected people, distance between susceptible and infected individuals, vaccination status) which may be important in predicting new disease cases. Recently, a geographically-dependent individual-level model (GD-ILM) within an SEIR framework was developed for when both regional and individual-level spatial data are available. In this paper, we propose to use an SEIR GD-ILM for each health region of Manitoba (central Canadian province) population to analyse the COVID-19 data. As different health regions of the population under study may act differently, we assume that each health region has its own corresponding parameters determined by a homogeneous SEIR model (such as contact rate, latent period, infectious period). A Monte Carlo Expectation Conditional Maximization (MCECM) algorithm is used for inference. Using estimated parameters we predict the infection rate at each health region of Manitoba over time to identify highly risk local geographical areas. Performance of the proposed approach is also evaluated through simulation studies.

Sparse and smooth functional data clustering

A new model-based procedure is developed for sparse clustering of functional data that aims to classify a sample of curves into homogeneous groups while jointly detecting the most informative portions of the domain. The proposed method is referred to as sparse and smooth functional clustering (SaS-Funclust) and relies on a general functional Gaussian mixture model whose parameters are estimated by maximizing a log-likelihood function penalized with a functional adaptive pairwise fusion penalty and a roughness penalty. The former allows identifying the noninformative portion of the domain by shrinking the means of separated clusters to some common values, whereas the latter improves the interpretability by imposing some degree of smoothing to the estimated cluster means. The model is estimated via an expectation-conditional maximization algorithm paired with a cross-validation procedure. Through a Monte Carlo simulation study, the SaS-Funclust method is shown to outperform other methods that already appeared in the literature, both in terms of clustering performance and interpretability. Finally, three real-data examples are presented to demonstrate the favourable performance of the proposed method. The SaS-Funclust method is implemented in the R package sasfunclust, available on CRAN.

Flexible modeling of multiple nonlinear longitudinal trajectories with censored and non-ignorable missing outcomes.

Multivariate nonlinear mixed-effects models (MNLMMs) have become a promising tool for analyzing multi-outcome longitudinal data following nonlinear trajectory patterns. However, such a classical analysis can be challenging due to censorship induced by detection limits of the quantification assay or non-response occurring when participants missed scheduled visits intermittently or discontinued participation. This article proposes an extension of the MNLMM approach, called the MNLMM-CM, by taking the censored and non-ignorable missing responses into account simultaneously. The non-ignorable missingness is described by the selection-modeling factorization to tackle the missing not at random mechanism. A Monte Carlo expectation conditional maximization algorithm coupled with the first-order Taylor approximation is developed for parameter estimation. The techniques for the calculation of standard errors of fixed effects, estimation of unobservable random effects, imputation of censored and missing responses and prediction of future values are also provided. The proposed methodology is motivated and illustrated by the analysis of a clinical HIV/AIDS dataset with censored RNA viral loads and the presence of missing CD4 and CD8 cell counts. The superiority of our method on the provision of more adequate estimation is validated by a simulation study.

An Improved Measurement Error Model for Analyzing Unreplicated Method Comparison Data under Asymmetric Heavy-Tailed Distributions

Method comparison studies mainly focus on determining if the two methods of measuring a continuous variable are agreeable enough to be used interchangeably. Typically, a standard mixed-effects model uses to model the method comparison data that assume normality for both random effects and errors. However, these assumptions are frequently violated in practice due to the skewness and heavy tails. In particular, the biases of the methods may vary with the extent of measurement. Thus, we propose a methodology for method comparison data to deal with these issues in the context of the measurement error model (MEM) that assumes a skew- t (ST) distribution for the true covariates and centered Student’s t (cT) distribution for the errors with known error variances, named STcT-MEM. An expectation conditional maximization (ECM) algorithm is used to compute the maximum likelihood (ML) estimates. The simulation study is performed to validate the proposed methodology. This methodology is illustrated by analyzing gold particle data and then compared with the standard measurement error model (SMEM). The likelihood ratio (LR) test is used to identify the most appropriate model among the above models. In addition, the total deviation index (TDI) and concordance correlation coefficient (CCC) were used to check the agreement between the methods. The findings suggest that our proposed framework for analyzing unreplicated method comparison data with asymmetry and heavy tails works effectively for modest and large samples.