Expectation Conditional Maximization Algorithm Research Articles

Covariate-dependent spatio-temporal covariance models

Geostatistical regression models are widely used in environmental and geophysical sciences to characterize the mean and dependence structures for spatio-temporal data. Traditionally, these models account for covariates solely in the mean structure, neglecting their potential impact on the spatio-temporal covariance structure. This paper addresses a significant gap in the literature by proposing a novel covariate-dependent covariance model within the spatio-temporal random-effects model framework. Our approach integrates covariates into the covariance function through a Cholesky-type decomposition, ensuring compliance with the positive-definite condition. We employ maximum likelihood for parameter estimation, complemented by an efficient expectation conditional maximization algorithm. Simulation studies demonstrate the superior performance of our method compared to conventional techniques that ignore covariates in spatial covariances. We further apply our model to a PM2.5 dataset from Taiwan, highlighting wind speed’s pivotal role in influencing the spatio-temporal covariance structure. Additionally, we incorporate wind speed and sunshine duration into the covariance function for analyzing Taiwan ozone data, revealing a more intricate relationship between covariance and these meteorological variables.

A novel finite mixture model based on generalized scale mixtures of generalized normal distributions with application to stock dataset

This paper introduces a novel family of distributions known as generalized scale mixtures of generalized normal distributions (GSMGN). These distributions incorporate two additional shape parameters that serve to regulate the shape and tails of the distribution. A finite mixture model based on this family is presented to address clustering heterogeneous data in the presence of leptokurtic and heavy-tailed outcomes. The estimation of the parameters of this model are obtained by developing an ECM-PLA ensemble algorithm which combine the profile likelihood approach (PLA) and the classical Expectation Conditional Maximization (ECM) algorithm, and the observed information matrix is obtained. The applicability of this new family and the numerical performance of the proposed methodology is discussed through simulated and real data examples.

A modeling framework for detecting and leveraging node-level information in Bayesian network inference.

Bayesian graphical models are powerful tools to infer complex relationships in high dimension, yet are often fraught with computational and statistical challenges. If exploited in a principled way, the increasing information collected alongside the data of primary interest constitutes an opportunity to mitigate these difficulties by guiding the detection of dependence structures. For instance, gene network inference may be informed by the use of publicly available summary statistics on the regulation of genes by genetic variants. Here we present a novel Gaussian graphical modeling framework to identify and leverage information on the centrality of nodes in conditional independence graphs. Specifically, we consider a fully joint hierarchical model to simultaneously infer (i) sparse precision matrices and (ii) the relevance of node-level information for uncovering the sought-after network structure. We encode such information as candidate auxiliary variables using a spike-and-slab submodel on the propensity of nodes to be hubs, which allows hypothesis-free selection and interpretation of a sparse subset of relevant variables. As efficient exploration of large posterior spaces is needed for real-world applications, we develop a variational expectation conditional maximization algorithm that scales inference to hundreds of samples, nodes and auxiliary variables. We illustrate and exploit the advantages of our approach in simulations and in a gene network study which identifies hub genes involved in biological pathways relevant to immune-mediated diseases.

A modified EM-type algorithm to estimate semi-parametric mixtures of non-parametric regressions

Semi-parametric Gaussian mixtures of non-parametric regressions (SPGMNRs) are a flexible extension of Gaussian mixtures of linear regressions (GMLRs). The model assumes that the component regression functions (CRFs) are non-parametric functions of the covariate(s) whereas the component mixing proportions and variances are constants. Unfortunately, the model cannot be reliably estimated using traditional methods. A local-likelihood approach for estimating the CRFs requires that we maximize a set of local-likelihood functions. Using the Expectation-Maximization (EM) algorithm to separately maximize each local-likelihood function may lead to label-switching. This is because the posterior probabilities calculated at the local E-step are not guaranteed to be aligned. The consequence of this label-switching is wiggly and non-smooth estimates of the CRFs. In this paper, we propose a unified approach to address label-switching and obtain sensible estimates. The proposed approach has two stages. In the first stage, we propose a model-based approach to address the label-switching problem. We first note that each local-likelihood function is a likelihood function of a Gaussian mixture model (GMM). Next, we reformulate the SPGMNRs model as a mixture of these GMMs. Lastly, using a modified version of the Expectation Conditional Maximization (ECM) algorithm, we estimate the mixture of GMMs. In addition, using the mixing weights of the local GMMs, we can automatically choose the local points where local-likelihood estimation takes place. In the second stage, we propose one-step backfitting estimates of the parametric and non-parametric terms. The effectiveness of the proposed approach is demonstrated on simulated data and real data analysis.

Constrained inference in scaled mixed effects models with applications to London Junior School Project data

Multiple outcomes arise frequently in many different settings. Mixed effect models are a useful tool for the analyses of such data. However, when outcomes are measured on different scales, analyses based on any one scale are misleading. Often parameters of the model are subject to known order restrictions. To incorporate heterogeneity across different outcomes, we propose a scaled linear mixed effect model. To estimate parameters, we propose a maximum likelihood estimation procedure based on a restricted version of the expectation-conditional maximization either algorithm. Constrained hypotheses testing procedures are developed using likelihood ratio tests. The empirical significance levels and powers are studied using simulations. This article shows that incorporating the restrictions improves the mean squared errors of the estimates and the power of the tests. The methodology is applied on the London Junior School Project data incorporating known restrictions of patterns of scores. Journal of Statistical Research 2023, Vol 57, No.1-2, pp.81-94

The generalized scale mixtures of asymmetric generalized normal distributions with application to stock data

<abstract><p>In this paper, we introduced a family of distributions with a very flexible shape named generalized scale mixtures of generalized asymmetric normal distributions (GSMAGN). We investigated the main properties of the new family including moments, skewness, kurtosis coefficients and order statistics. A variant of the expectation maximization (EM)-type algorithm was established by combining the proflie likihood approach (PLA) with the classical expectation conditional maximization (ECM) algorithm for parameter estimation of this model. This approach with analytical expressions in the E-step and tractable M-step can greatly improve the computational speed and efficiency of the algorithm. The performance of the proposed algorithm was assessed by some simulation studies. The feasibility of the proposed methodology was illustrated through two real datasets.</p></abstract>

Mixtures of factor analysers with censored responses and external covariates: An application to educational data from Peru.

Analysing data from educational tests allows governments to make decisions for improving the quality of life of individuals in a society. One of the key responsibilities of statisticians is to develop models that provide decision-makers with pertinent information about the latent process that educational tests seek to represent. Mixtures of factor analysers (MtFA) have emerged as a powerful device for model-based clustering and classification of high-dimensional data containing one or several groups of observations with fatter tails or anomalous outliers. This paper considers an extension of MtFA for robust clustering of censored data, referred to as the MtFAC model, by incorporating external covariates. The enhanced flexibility of including covariates in MtFAC enables cluster-specific multivariate regression analysis of dependent variables with censored responses arising from upper and/or lower detection limits of experimental equipment. An alternating expectation conditional maximization (AECM) algorithm is developed for maximum likelihood estimation of the proposed model. Two simulation experiments are conducted to examine the effectiveness of the techniques presented. Furthermore, the proposed methodology is applied to Peruvian data from the 2007 Early Grade Reading Assessment, and the results obtained from the analysis provide new insights regarding the reading skills of Peruvian students.

Parameter estimation of the multivariate unrestricted skew-normal distribution using ECM algorithm

The probability density function of the multivariate unrestricted skew-normal (SUN) distribution, corresponding to a screened normal density, allow to modeling skewness and kurtosis in data in terms of a skewness parameter vector and a truncation parameter matrix. These parameters are related to the shape and heavy-tails of the density. In this article, we present the Expectation/Conditional Maximization (ECM) algorithm for the SUN distribution based on a hierarchical stochastic representation. In addition, behavior of ECM algorithm’s steps is measured using an information theoretic approach based on Jeffrey’s divergence and related homogeneity test. Usefulness of the proposed method is illustrated by an application to Chilean economic perception data.

An expectation conditional maximization algorithm for the skew-normal based stochastic frontier model

An expectation conditional maximization algorithm for the skew-normal based stochastic frontier model

Analyzing COVID-19 data in the Canadian province of Manitoba: A new approach

The basic homogeneous SEIR (susceptible–exposed–infected–removed) model is a commonly used compartmental model for analysing infectious diseases such as influenza and COVID-19. However, in the homogeneous SEIR model, it is assumed that the population of study is homogeneous and, one cannot incorporate individual-level information (e.g., location of infected people, distance between susceptible and infected individuals, vaccination status) which may be important in predicting new disease cases. Recently, a geographically-dependent individual-level model (GD-ILM) within an SEIR framework was developed for when both regional and individual-level spatial data are available. In this paper, we propose to use an SEIR GD-ILM for each health region of Manitoba (central Canadian province) population to analyse the COVID-19 data. As different health regions of the population under study may act differently, we assume that each health region has its own corresponding parameters determined by a homogeneous SEIR model (such as contact rate, latent period, infectious period). A Monte Carlo Expectation Conditional Maximization (MCECM) algorithm is used for inference. Using estimated parameters we predict the infection rate at each health region of Manitoba over time to identify highly risk local geographical areas. Performance of the proposed approach is also evaluated through simulation studies.

Sparse and smooth functional data clustering

A new model-based procedure is developed for sparse clustering of functional data that aims to classify a sample of curves into homogeneous groups while jointly detecting the most informative portions of the domain. The proposed method is referred to as sparse and smooth functional clustering (SaS-Funclust) and relies on a general functional Gaussian mixture model whose parameters are estimated by maximizing a log-likelihood function penalized with a functional adaptive pairwise fusion penalty and a roughness penalty. The former allows identifying the noninformative portion of the domain by shrinking the means of separated clusters to some common values, whereas the latter improves the interpretability by imposing some degree of smoothing to the estimated cluster means. The model is estimated via an expectation-conditional maximization algorithm paired with a cross-validation procedure. Through a Monte Carlo simulation study, the SaS-Funclust method is shown to outperform other methods that already appeared in the literature, both in terms of clustering performance and interpretability. Finally, three real-data examples are presented to demonstrate the favourable performance of the proposed method. The SaS-Funclust method is implemented in the R package sasfunclust, available on CRAN.

Flexible modeling of multiple nonlinear longitudinal trajectories with censored and non-ignorable missing outcomes.

Multivariate nonlinear mixed-effects models (MNLMMs) have become a promising tool for analyzing multi-outcome longitudinal data following nonlinear trajectory patterns. However, such a classical analysis can be challenging due to censorship induced by detection limits of the quantification assay or non-response occurring when participants missed scheduled visits intermittently or discontinued participation. This article proposes an extension of the MNLMM approach, called the MNLMM-CM, by taking the censored and non-ignorable missing responses into account simultaneously. The non-ignorable missingness is described by the selection-modeling factorization to tackle the missing not at random mechanism. A Monte Carlo expectation conditional maximization algorithm coupled with the first-order Taylor approximation is developed for parameter estimation. The techniques for the calculation of standard errors of fixed effects, estimation of unobservable random effects, imputation of censored and missing responses and prediction of future values are also provided. The proposed methodology is motivated and illustrated by the analysis of a clinical HIV/AIDS dataset with censored RNA viral loads and the presence of missing CD4 and CD8 cell counts. The superiority of our method on the provision of more adequate estimation is validated by a simulation study.

An Improved Measurement Error Model for Analyzing Unreplicated Method Comparison Data under Asymmetric Heavy-Tailed Distributions

Method comparison studies mainly focus on determining if the two methods of measuring a continuous variable are agreeable enough to be used interchangeably. Typically, a standard mixed-effects model uses to model the method comparison data that assume normality for both random effects and errors. However, these assumptions are frequently violated in practice due to the skewness and heavy tails. In particular, the biases of the methods may vary with the extent of measurement. Thus, we propose a methodology for method comparison data to deal with these issues in the context of the measurement error model (MEM) that assumes a skew- t (ST) distribution for the true covariates and centered Student’s t (cT) distribution for the errors with known error variances, named STcT-MEM. An expectation conditional maximization (ECM) algorithm is used to compute the maximum likelihood (ML) estimates. The simulation study is performed to validate the proposed methodology. This methodology is illustrated by analyzing gold particle data and then compared with the standard measurement error model (SMEM). The likelihood ratio (LR) test is used to identify the most appropriate model among the above models. In addition, the total deviation index (TDI) and concordance correlation coefficient (CCC) were used to check the agreement between the methods. The findings suggest that our proposed framework for analyzing unreplicated method comparison data with asymmetry and heavy tails works effectively for modest and large samples.

Extending the Fellegi-Sunter record linkage model for mixed-type data with application to the French national health data system

Probabilistic record linkage is a process of combining data from different sources, when such data refer to common entities and identifying information is not available. A probabilistic record linkage framework that takes into account multiple non-identifying information that this is limited to simple binary comparison between matching variables has been previously proposed. An extension of this method is proposed for mixed-type comparison vectors. A mixture model for handling comparison values of low prevalence categorical matching variables, and a mixture of hurdle gamma distribution for handling comparison values of continuous matching variables have been developed. The parameters are estimated by means of the Expectation Conditional Maximization (ECM) algorithm. Through a Monte Carlo simulation study, both the posterior probability estimation for a record pair to be a match and the prediction of matched record pairs are evaluated. The simulation results indicate that the proposed methods outperform existing ones in most considered cases. The proposed methods are applied on a real dataset, to perform linkage between a registry of patients suffering from venous thromboembolism in the Brest district area (GETBO) and the French national health information system (SNDS).

Maneuvering extended object tracking based on constrained expectation maximization

This paper addresses the problem of maneuvering extended object tracking, in which the object extension (OE) and the turn rate are identified simultaneously. Due to its non-linearity with respect to the turn rate, the state transition is converted into a linear form of a newly defined hyper-parametric vector by parameter substitution. For the hyper-parametric equality constraints (HECs) introduced in the transformation, a constrained expectation conditional maximization algorithm is designed. The HECs are projected onto the conditional expectation function for regularization, so as to realize the iterative identification of multi-parameter (i.e., OE and turn rate). The transformation and CECM optimization bring the advantage of avoiding nonlinear model approximation, which is important for the convergence and accuracy of state estimation. Finally, simulation results demonstrate the superiority of the proposed method in terms of both estimation accuracy and identification effectiveness.

Adaptive estimation of autoregression models under long-tailed symmetric distribution

Non-normal innovations in autoregression models frequently occur in practice. In this situation, least squares (LS) estimators are known to be inefficient and non-robust, and maximum likelihood (ML) estimators need to be solved numerically, which becomes a daunting task. In the literature, the modified maximum likelihood (MML) estimation technique has been proposed to obtain the estimators of model parameters. While an explicit solution can be found via this method, the requirement of knowing the shape parameter becomes a drawback, especially in machine learning. In this study, we use the adaptive modified maximum likelihood (AMML) methodology, which combines the MML with Huber’s M-estimation so that this assumption is relaxed. The performance of the method in terms of efficiency and robustness is analyzed via simulation and compared to LS, MML and ML estimates that are obtained numerically via the Expectation Conditional Maximization (ECM) algorithm. Test statistics are proposed for the crucial parameters of the model. The results show that the AMML estimators are preferable in most of the settings according to the mean squared error (MSE) criterion and the test statistics based on AMML method are more robust than the others. Furthermore, both real life and synthetic data examples are given.

Threshold-free multi-attributes physical layer authentication based on expectation–conditional maximization channel estimation in Internet of Things

With the number of Internet of Things devices continually increasing, the endogenous security of Internet of Things communication systems is growingly critical. Physical layer authentication is a powerful means of resisting active attacks by exploiting the unique characteristics inherent in wireless signals and physical devices. Many existing physical layer authentication schemes usually assume physical layer attributes obey certain statistical distributions that are unknown to receivers. To overcome the uncertainty, machine learning–based authentication approaches have been employed to implement threshold-free authentication. In this article, we utilize an expectation–conditional maximization algorithm to provide the physical layer attribute estimates required for the authentication phase and a logistic regression model to achieve threshold-free physical layer authentication. Moreover, a Frank–Wolfe algorithm is considered to achieve fast convergence of the logistic regression parameters and multi-attributes are adopted to increase the differentiation of transmitters. Simulation results demonstrate that the obtained attribute estimates are sufficient to provide a reliable source of data for authentication and the proposed threshold-free multi-attributes physical layer authentication scheme can effectively improve authentication accuracy, with the false alarm rate P f reduced to 0.0263% and the miss detection rate P m reduced to 0.3466%.

Model-based clustering via skewed matrix-variate cluster-weighted models

Cluster-weighted models (CWMs) extend finite mixtures of regressions (FMRs) in order to allow the distribution of covariates to contribute to the clustering process. In this article, we introduce 24 matrix-variate CWMs which are obtained by allowing both the responses and covariates in each cluster to be modelled by one of four existing skewed matrix-variate distributions or the matrix-variate normal distribution. Endowed with greater flexibility, our matrix-variate CWMs are able to handle this kind of data in a more suitable manner. As a by-product, the four skewed matrix-variate FMRs are also introduced. Maximum likelihood parameter estimates are derived using an expectation-conditional maximization algorithm. Parameter recovery, classification assessment, and the capability of the Bayesian information criterion to detect the underlying groups are investigated using simulated data. Lastly, our matrix-variate CWMs, along with the matrix-variate normal CWM and matrix-variate FMRs, are applied to two real datasets for illustrative purposes.

Multivariate linear mixed models with censored and nonignorable missing outcomes, with application to AIDS studies.

The analysis of multivariate longitudinal data could encounter some complications due to censorship induced by detection limits of the assay and nonresponse occurring when participants missed scheduled visits intermittently or discontinued participation. This paper establishes a generalization of the multivariate linear mixed model that can accommodate censored responses and nonignorable missing outcomes simultaneously. To account for the nonignorable missingness, the selection approach which decomposes the joint distribution as a marginal distribution for the primary outcome variables and a model describing the missing process conditional on the hypothetical complete data is used. A computationally feasible Monte Carlo expectation conditional maximization algorithm is developed for parameter estimation with the maximum likelihood (ML) method. Furthermore, a general information-based approach is presented to assess the variability of ML estimators. The techniques for the prediction of censored responses and imputation of missing outcomes are also discussed. The methodology is motivated and exemplified by a real dataset concerning HIV-AIDS clinical trials. A simulation study is conducted to examine the performance of the proposed method compared with other traditionalapproaches.

Parsimonious hidden Markov models for matrix-variate longitudinal data

Hidden Markov models (HMMs) have been extensively used in the univariate and multivariate literature. However, there has been an increased interest in the analysis of matrix-variate data over the recent years. In this manuscript we introduce HMMs for matrix-variate balanced longitudinal data, by assuming a matrix normal distribution in each hidden state. Such data are arranged in a four-way array. To address for possible overparameterization issues, we consider the eigen decomposition of the covariance matrices, leading to a total of 98 HMMs. An expectation-conditional maximization algorithm is discussed for parameter estimation. The proposed models are firstly investigated on simulated data, in terms of parameter recovery, computational times and model selection. Then, they are fitted to a four-way real data set concerning the unemployment rates of the Italian provinces, evaluated by gender and age classes, over the last 16 years.