Expectation Maximization Research Articles

Multivariable Mendelian randomization with incomplete measurements on the exposure variables in the Hispanic Community Health Study/Study of Latinos

Multivariable Mendelian randomization allows simultaneous estimation of direct causal effects of multiple exposure variables on an outcome. When the exposure variables of interest are quantitative omic features, obtaining complete data can be economically and technically challenging: the measurement cost is high, and the measurement devices may have inherent detection limits. In this paper, we propose a valid and efficient method to handle unmeasured and undetectable values of the exposure variables in a one-sample multivariable Mendelian randomization analysis with individual-level data. We estimate the direct causal effects with maximum likelihood estimation and develop an expectation-maximization algorithm to compute the estimators. We show the advantages of the proposed method through simulation studies and provide an application to the Hispanic Community Health Study/Study of Latinos, which has a large amount of unmeasured exposure data.

X-ray spectrum estimation from the transmission method using the radioluminescence of an Al2O3:C crystal

The X-ray spectrum of X-ray tube is crucial for radiation dose calculations. Due to the high photon flux, it is difficult to directly measure the primary spectrum of X-ray tube. Transmission measurement is one of indirect method used to determine the primary spectrum of X-ray tube. Although the method is experimentally simple, X-ray spectrum estimation from transmission method is a seriously ill-conditioned problem. In this paper, a new transmission measurement instrument with an the Al2O3:C crystal detector is proposed. Real-time dose rate attenuation data from transmission method are collected by using the radioluminescence(RL) of the Al2O3:C crystal. The expectation-maximization (EM) method is applied for X-ray spectrum estimation from transmission method. The method is demonstrated on typical simulated spectra of 80,100,120,140 and 160 kV from X-ray tubes. The X-ray spectra are simulated with the Monte Carlo code Geant4. The EM method is successfully applied to seriously ill-conditioned and under determined estimation problems. The results showed that the X-ray spectrum estimation method is a robust technique for estimating the primary spectrum of X-ray tube, and that the new instrument for transmission measurements can be used to accurately estimate X-ray spectrum.

Graph learning from incomplete graph signals: From batch to online methods

Inferring graph topologies from data is crucial in many graph-related applications. Existing works typically assume that signals are observed at all nodes, which may not hold due to application-specific constraints. The problem becomes more challenging when data are sequentially available and no delay is tolerated. To address these issues, we propose an approach for learning graphs from incomplete data. First, the problem of learning graphs with missing data is formulated as maximizing the posterior distribution with hidden variables from a Bayesian perspective. Then, we propose an expectation maximization (EM) algorithm to solve the induced problem, in which graph learning and graph signal recovery are jointly performed. Furthermore, we extend the proposed EM algorithm to an online version to accommodate the delay-sensitive situations of sequential data. Theoretically, we analyze the dynamic regret of the proposed online algorithm, illustrating the effectiveness of our algorithm in tracking graphs from partial observations in an online manner. Finally, extensive experiments on synthetic and real data are conducted, and the results corroborate that our approach can learn graphs effectively from incomplete data in both batch and online situations.

Novel Method for Extraction of Ship Target with Overlaps in SAR Image via EM Algorithm

Novel Method for Extraction of Ship Target with Overlaps in SAR Image via EM Algorithm

Source shape estimation for neutron imaging systems using convolutional neural networks.

Neutron imaging systems are important diagnostic tools for characterizing the physics of inertial confinement fusion reactions at the National Ignition Facility (NIF). In particular, neutron images give diagnostic information on the size, symmetry, and shape of the fusion hot spot and surrounding cold fuel. Images are formed via collection of neutron flux from the source using a system of aperture arrays and scintillator-based detectors. Currently, reconstruction of fusion source geometry from the collected neutron images is accomplished by solving a computationally intensive maximum likelihood estimation problem via expectation maximization. In contrast, it is often useful to have simple representations of the overall source geometry that can be computed quickly. In this work, we develop convolutional neural networks (CNNs) to reconstruct the outer contours of simple source geometries. We compare the performance of the CNN for penumbral and pinhole data and provide experimental demonstrations of our methods on both non-noisy and noisy data.

Machine Learning‐Based Hydrofacies Classification: Effects of Noise and Regularization

AbstractThis study utilizes an unsupervised ML approach, the expectation‐maximization (EM) algorithm using Gaussian Mixture Models (GMM), to integrate near‐surface geophysics measurements for hydrofacies classification. We examined the impact of noise and noise estimation on classification across two synthetic models with varying lateral heterogeneity, simulating and inverting resistivity and seismic data with noise levels ranging from minimal to very high. The algorithm proved robust in accurately reconstructing hydrofacies when noise was correctly estimated or not significantly underestimated, showing minimal misclassification in shallow hydrofacies. However, severe underestimation of noise during inversion led to increased misclassifications and artifact‐laden hydrofacies models, especially in shallow regions. Higher lateral heterogeneity lessened the negative impact of noise, slightly improving algorithm performance when noise was correctly estimated. We also explored the influence of geophysical measurement uncertainties on classification uncertainty through hydrofacies probability maps, noting the greatest impact when noise was underestimated. Additionally, we investigated the effect of the regularization trade‐off parameter on the hydrofacies classification and show how the performance of the algorithm can be evaluated in the absence of ground truth data using the average silhouette score of the classification with data obtained from a basement complex field site. We found that moderate regularization (λ = 200) yielded the best hydrofacies model, as indicated by the highest average silhouette score. Our findings underscore the effectiveness of unsupervised ML for facies classification and emphasizes the critical role of accurate noise characterization in geophysical data processing for enhancing the integration of subsurface heterogeneity information into hydrological models.

Area extraction and growth monitoring of sugarcane from multi-source remote sensing images under a polarimetric SAR data compensation based on buildings

ABSTRACT Aiming at the issue of low accuracy in crop mapping and growth monitoring caused by imprecise calibration of radar time-series data, this paper proposed a Synthetic Aperture Radar (SAR) spatio-temporal error compensation method. By constructing a compensation reference image using the buildings with stable backscattering coefficients in the study area, the error correction of time-series SAR data was realized based on the compensation model. The compensated SAR data were used to establish the Sugarcane Growth Index (SGI) according to the growth curve of sugarcane. The sugarcane area was extracted after combining the compensated SGI and optical image, and the accuracy of F1 on sugarcane distribution extraction was 92.12%. There was an improvement of 4.11% compared with no compensation method. On this basis, the newly planted and ratoon sugarcane were further classified by the expectation maximization algorithm based on the compensated SAR image at the seedling stage. The newly planted sugarcane and the ratoon sugarcane could be accurately distinguished with area extraction accuracy of 82.06% and 80.59%, respectively. Moreover, the compensated SAR data of time series during the tillering to maturity stages were used to estimate the sugarcane height, yielding an R 2 value of 0.717 using a quadratic polynomial model. The Spatio-temporal error compensation method for distributed targets proposed in this study can reduce the loss of SAR data across various times and regions, and achieve accurate results of sugarcane distribution extraction and plant height estimation.

Unsupervised Liu-type shrinkage estimators for mixture of regressionmodels.

The mixture of probabilistic regression models is one of the most common techniques to incorporate the information of covariates into learning of the population heterogeneity. Despite its flexibility, unreliable estimates can occur due to multicollinearity among covariates. In this paper, we develop Liu-type shrinkage methods through an unsupervised learning approach to estimate the model coefficients in the presence of multicollinearity. We evaluate the performance of our proposed methods via classification and stochastic versions of the expectation-maximization algorithm. We show using numerical simulations that the proposed methods outperform their Ridge and maximum likelihood counterparts. Finally, we apply our methods to analyze the bone mineral data of women aged 50 and older.

Assessment of Different Methods for Estimation of Missing Rainfall Data

Missing data is a common problem encountered in various fields, including clinical research, environmental sciences and hydrology. In order to obtain reliable results from the analysis, the data inventory must be completed. This paper presents a methodology for addressing the missing data problem by examining the missing data structure and missing data techniques. Simulated datasets were created by considering the number of missing data, missing data pattern and missing data mechanism of real datasets containing missing values, which are often overlooked in hydrology. Considering the missing data pattern, the most commonly used methods for missing data analysis in hydrology and other fields were applied to the created simulated datasets. Simple imputation techniques and expectation maximization (EM) were implemented in SPSS software and machine learning techniques such as k-nearest neighbor (kNN), together with the hot-deck were implemented in the Python programming language. In the performance evaluation based on error metrics, it is concluded that the EM method is the most suitable completion method. Homogeneity analyses were performed in the Mathematica programming language to identify possible changes and inconsistencies in the completed rainfall dataset. Homogeneity analyses revealed that most of the completed rainfall datasets are homogeneous at class 1 level, consistent and reliable and do not show systematic changes in time.

Multiple-model state-space system identification with time delay using the EM algorithm

For a dynamic process identification throughout the whole operating range under diverse operating conditions, it is difficult to capture the process dynamics by a single process model in which the traditional identification method can be adopted to implement parameter estimation. By using the multiple dual-rate state-space models to approach the parameter-varying time-delay systems with different operating conditions, this paper explores an EM algorithm to simultaneously estimate the hidden variable, the parameter vector, the state variable and the time-delay by introducing hidden variables and by using a Kalman smoother.

Adversarial EM for variational deep learning: Application to semi-supervised image quality enhancement in low-dose PET and low-dose CT

In positron emission tomography (PET) and X-ray computed tomography (CT), reducing radiation dose can cause significant degradation in image quality. For image quality enhancement in low-dose PET and CT, we propose a novel theoretical adversarial and variational deep neural network (DNN) framework relying on expectation maximization (EM) based learning, termed adversarial EM (AdvEM). AdvEM proposes an encoder–decoder architecture with a multiscale latent space, and generalized-Gaussian models enabling datum-specific robust statistical modeling in latent space and image space. The model robustness is further enhanced by including adversarial learning in the training protocol. Unlike typical variational-DNN learning, AdvEM proposes latent-space sampling from the posterior distribution, and uses a Metropolis–Hastings scheme. Unlike existing schemes for PET or CT image enhancement which train using pairs of low-dose images with their corresponding normal-dose versions, we propose a semi-supervised AdvEM (ssAdvEM) framework that enables learning using a small number of normal-dose images. AdvEM and ssAdvEM enable per-pixel uncertainty estimates for their outputs. Empirical analyses on real-world PET and CT data involving many baselines, out-of-distribution data, and ablation studies show the benefits of the proposed framework.

Methods for Estimating Parameters of the Common Cause Failure Model Based on Data with Uncertainty

In this paper, we propose a new method to deal with uncertain data in the context of Common Cause Failure (CCF) analysis. Uncertain CCF data refer to the data for which the number of components involved in the failure events is not exactly known. We introduce a new formalism to describe uncertain CCF data to avoid subjective probabilities for the number of failed components in each CCF event that are used in classical methods such as the impact vector method. The parameters of the [Formula: see text]-factor model are estimated using the maximum likelihood method relying on properties of the nested Dirichlet distribution and grouped Dirichlet distribution. A data augmentation technique with an expectation-maximization algorithm is also developed for some schemes of data with uncertainty. Finally, we evaluate the performance of the proposed method through numerical simulations and illustrate its application using an example from the literature.

Analyzing risk influencing factors of ship collision accidents: A data-driven Bayesian network model integrating physical knowledge

The interaction and propagation effects among risk-influencing factors (RIFs) often lead to maritime accidents. Therefore, to support maritime risk management in decision-making, it is imperative to conduct a quantitative risk assessment (QRA) of these accidents, which requires a methodology capable of capturing causal relationships in limited cases. To achieve this goal, this study proposes a data-driven Bayesian network (BN) model that integrates physical knowledge with the QRA. Based on the collected data, a combination of domain knowledge, structure learning, and parameter learning is employed to construct the model. In the data-driven phase, three structural learning algorithms including Bayesian search (BS), Greedy thick thinning (GTT), and Peter-Clark (PC) algorithms, were used in combination with the expectation maximization algorithm of parameter learning. Then, using four indices calculated by the confusion matrix, the most fitting algorithm for structure learning was chosen, and the confusion matrix was obtained by five-fold cross-validation. Finally, network propagation impact (NPI) is introduced to prioritize RIFs. Preventive measures and emergency plans are formulated based on the network resilience metric (NRM) and network vulnerability metric (NVM). This approach leverages the objectivity of a data-driven BN and integrates domain expertise to enhance model validity. A case study of collisions was conducted to demonstrate the applicability of the model. The data were sourced from 327 collision accident reports provided by the China Maritime Safety Administration, and the PC algorithm was chosen. Findings indicate that the “Management system”, “Supervision”, and “Crew training” RIFs are prioritized for preventive measures, whereas “Communication” receives more resources for emergency handling. The results suggest the formulation of risk management strategies for practitioners to improve maritime safety.

Learning Hidden Markov Models with Structured Transition Dynamics

The hidden Markov model (HMM) provides a natural framework for modeling the dynamic evolution of latent diseases. The unknown probability matrices of HMMs can be learned through the well-known Baum–Welch algorithm, a special case of the expectation-maximization algorithm. In many disease models, the probability matrices possess nontrivial properties that may be represented through a set of linear constraints. In these cases, the traditional Baum–Welch algorithm is no longer applicable because the maximization step cannot be solved by an explicit formula. In this paper, we propose a novel approach to efficiently solve the maximization step problem under linear constraints by providing a Lagrangian dual reformulation that we solve by an accelerated gradient method. The performance of this approach critically depends on devising a fast method to compute the gradient in each iteration. For this purpose, we employ dual decomposition and derive Karush–Kuhn–Tucker conditions to reduce our problem into a set of single variable equations, solved using a simple bisection method. We apply this method to a case study on sports-related concussion and provide an extensive numerical study using simulation. We show that our approach is in orders of magnitude computationally faster and more accurate than other alternative approaches. Moreover, compared with other methods, our approach is far less sensitive with respect to increases in problem size. Overall, our contribution lies in the advancement of accurately and efficiently handling HMM parameter estimation under linear constraints, which comprises a wide range of applications in disease modeling and beyond. History: Accepted by Paul Brooks, Area Editor for Applications in Biology, Medicine, & Healthcare. Funding: This research was funded by [Grant R01DE028283] from the National Institute of Dental and Craniofacial Research, National Institutes of Health. G.-G. Garcia was also funded by the Georgia Clinical & Translational Science Alliance National Institutes of Health award [Grant UL1-TR002378]. Supplemental Material: The software that supports the findings of this study is available within the paper and its Supplemental Information ( https://pubsonline.informs.org/doi/suppl/10.1287/ijoc.2022.0342 ) as well as from the IJOC GitHub software repository ( https://github.com/INFORMSJoC/2022.0342 ). The complete IJOC Software and Data Repository is available at https://informsjoc.github.io/ .

ESTIMATION OF VARIANCE COMPONENTS FOR BODYWEIGHT OF GRASSCUTTERS USING EXPECTATION MAXIMIZATION (EM) ALGORITHM OF RESTRICTED MAXIMIMUM LIKELIHOOD (REMI)

The objective of this study was to estimate the variance components and heritability of bodyweight of grass-cutters at 4, 6 and 8 111011ths of age using EM algorithm of REML procedures. The data used for the study were obtained from the bodyweight records of grass-cutters from 4 families at 4, 6 and 8 months of age. The heritability of bodyweight of grasscutters at 4, 6 and 8 months of age were 0.14, 0.10 and 0.12 respectively. This implies that about 10 14 % of the phenotypic variability of body weight in this grasscutter population was accounted by additive genetic variance while environmental and gene combination variance made a larger contribution. The implication is that selection of grasscutters in this population should not be based on the in on the animals alone but also information from its relatives.

Semiparametric Structural Equation Models with Interval-Censored Data

Structural equation models offer a valuable tool for delineating the complicated interrelationships among multiple variables, including observed and latent variables. Over the last few decades, structural equation models have successfully analyzed complete and right-censored survival data, exemplified by wide applications in psychological, social, or genomic studies. However, the existing methodology for structural equation modeling is not concerned with interval-censored data, a type of coarse survival data arising typically from periodic examinations for the occurrence of asymptomatic disease. The present study aims to fill this gap and provide a flexible semiparametric structural equation modeling framework. A general class of factor-augmented transformation models is proposed to model the interval-censored outcome of interest in the presence of latent risk factors. An expectation–maximization algorithm is subtly designed to conduct the nonparametric maximum likelihood estimation. Furthermore, the asymptotic properties of the proposed estimators are established by leveraging the empirical process theory. The numerical results obtained from extensive simulations and an application to the Alzheimer’s disease data set demonstrate the proposed method’s empirical performance and practical utility.

A new class of zero-truncated counting models and its application

Count data is a type of data derived from the number of times an event occurs per unit of time, and zero-truncated count data refers to count data without zero, which often appears in various fields. In this paper, a new zero-truncated Bell (ZTBell) distribution is proposed on the basis of Bell distribution. We studied its statistical properties, exploring methods such as maximum likelihood estimation (MLE), expectation–maximization (EM) algorithm, and minimization–maximization (MM) algorithm for parameter estimation, as well as conducting likelihood ratio tests. In addition, we used the Bootstrap method to calculate the standard errors and confidence intervals of the parameters. The simulation results found that all of the MLE, MM algorithm and EM algorithm are effective. And, as the sample size increases, the estimates of the parameters are closer to the true values and the root mean square error is smaller. Finally, applying the model to a set of factory accident data, we found that the ZTBell distribution fits better than the other models and is close to the fitting results of the zero-truncated generalized Poisson distribution. But ZTBell distribution has only one parameter, so it’s even simpler compared to the latter. Therefore, the ZTBell distribution can be a good alternative to other zero-truncated distributions, which provides more options available for statistical analysis in this domain.

Autoregressive double latent variables probabilistic model for higher-order dynamic process monitoring

The application of multivariate statistical analysis in process monitoring has emerged as a significant research topic, with a focus on consideration of data correlations. The present study investigates an anomaly detection method based on autoregressive double latent variables probabilistic (ADLVP) model for industrial dynamic processes. Specifically, the ADLVP model incorporates two distinct types of latent variables (LVs) to capture the internal relationships within the data from both quality-correlated and uncorrelated perspectives. Moreover, the model employs autoregressive modeling to characterize the double latent variables with time-dependence, enabling them to unveil more intricate higher-order autocorrelations between industrial measurements. The model parameters and the double latent variables can be iteratively determined using the expectation maximization (EM) algorithm, upon which the statistics for process monitoring are devised. Finally, the proposed method is validated in two industrial studies, and experimental results demonstrate that the ADLVP model outperforms its counterparts in dynamic processes monitoring.

A Note on Standard Errors for Multidimensional Two-Parameter Logistic Models Using Gaussian Variational Estimation.

Accurate item parameters and standard errors (SEs) are crucial for many multidimensional item response theory (MIRT) applications. A recent study proposed the Gaussian Variational Expectation Maximization (GVEM) algorithm to improve computational efficiency and estimation accuracy (Cho et al., 2021). However, the SE estimation procedure has yet to be fully addressed. To tackle this issue, the present study proposed an updated supplemented expectation maximization (USEM) method and a bootstrap method for SE estimation. These two methods were compared in terms of SE recovery accuracy. The simulation results demonstrated that the GVEM algorithm with bootstrap and item priors (GVEM-BSP) outperformed the other methods, exhibiting less bias and relative bias for SE estimates under most conditions. Although the GVEM with USEM (GVEM-USEM) was the most computationally efficient method, it yielded an upward bias for SE estimates.

Fast maximum likelihood estimation for general hierarchical models

Hierarchical statistical models are important in applied sciences because they capture complex relationships in data, especially when variables are related by space, time, sampling unit, or other shared features. Existing methods for maximum likelihood estimation that rely on Monte Carlo integration over latent variables, such as Monte Carlo Expectation Maximization (MCEM), suffer from drawbacks in efficiency and/or generality. We harness a connection between sampling-stepping iterations for such methods and stochastic gradient descent methods for non-hierarchical models: many noisier steps can do better than few cleaner steps. We call the resulting methods Hierarchical Model Stochastic Gradient Descent (HMSGD) and show that combining efficient, adaptive step-size algorithms with HMSGD yields efficiency gains. We introduce a one-dimensional sampling-based greedy line search for step-size determination. We implement these methods and conduct numerical experiments for a Gamma-Poisson mixture model, a generalized linear mixed models (GLMMs) with single and crossed random effects, and a multi-species ecological occupancy model with over 3000 latent variables. Our experiments show that the accelerated HMSGD methods provide faster convergence than commonly used methods and are robust to reasonable choices of MCMC sample size.