Expectation Maximization Research Articles

HOW MUCH EUROPEAN YOU ARE? CULTURAL CHANGES OF EUROPEAN COUNTRIES IN THE LAST 20 YEARS WITH SURVEY DATA

This study deviates from traditional approaches by focusing on data analysis instead of historical information to comprehend the dynamics shaping societal culture. Analyzing the cultural changes in European countries over the past two decades, the study utilizes the largest dataset to date, the Integrated Values Survey. Data from four distinct time periods spanning from 2000 to 2022 have been scrutinized. Expectation Maximization method is employed to address missing data in the dataset, and factor analysis-CLARA clustering algorithm is applied for each time period. The primary objective of this academic endeavor is to understand the dynamics of cultural evolution from a data-centric perspective. This analytical approach provides a robust foundation for comprehending cultural changes and predicting potential future developments.

Estimation in shape mixtures of skew-normal linear regression models via ECM coupled with Gibbs sampling

Abstract In this paper, we study linear regression models in which the error term has shape mixtures of skew-normal distribution. This type of distribution belongs to the skew-normal (SN) distribution class that can be used for heavy tails and asymmetry data. For the first time, for the classical (non-Bayesian) estimation of the parameters of the SN family, we apply the Markov chains Monte Carlo ECM (MCMC-ECM) algorithm where the samples are generated by Gibbs sampling, denoted by Gibbs-ECM, and also, we extend two other types of the EM algorithm for the above model. Finally, the proposed method is evaluated through a simulation and compared with the Numerical Math-ECM algorithm and Monte Carlo ECM (MC-ECM) using a real data set.

A discrete approximation method for modeling interval-censored multistate data.

Many longitudinal studies are designed to monitor participants for major events related to the progression of diseases. Data arising from such longitudinal studies are usually subject to interval censoring since the events are only known to occur between two monitoring visits. In this work, we propose a new method to handle interval-censored multistate data within a proportional hazards model framework where the hazard rate of events is modeled by a nonparametric function of time and the covariates affect the hazard rate proportionally. The main idea of this method is to simplify the likelihood functions of a discrete-time multistate model through an approximation and the application of data augmentation techniques, where the assumed presence of censored information facilitates a simpler parameterization. Then the expectation-maximization algorithm is used to estimate the parameters in the model. The performance of the proposed method is evaluated by numerical studies. Finally, the method is employed to analyze a dataset on tracking the advancement of coronary allograft vasculopathy following heart transplantation.

Enhancing water quality prediction for fluctuating missing data scenarios: A dynamic Bayesian network-based processing system to monitor cyanobacteria proliferation

Tackling the impact of missing data in water management is crucial to ensure the reliability of scientific research that informs decision-making processes in public health. The goal of this study is to ascertain the root causes associated with cyanobacteria proliferation under major missing data scenarios. For this purpose, a dynamic missing data management methodology is proposed using Bayesian Machine Learning for accurate surface water quality prediction of a river from Limia basin (Spain). The methodology used entails a sequence of analytical steps, starting with data pre-processing, followed by the selection of a reliable dynamic Bayesian missing value prediction system, leading finally to a supervised analysis of the behavioral patterns exhibited by cyanobacteria. For that, a total of 2,118,844 data points were used, with 205,316 (9.69 %) missing values identified. The machine learning testing showed the iterative structural expectation maximization (SEM) as the best performing algorithm, above the dynamic imputation (DI) and entropy-based dynamic imputation methods (EBDI), enhancing in some cases the accuracy of imputations by approximately 50 % in R2, RMSE, NRMSE, and logarithmic loss values. These findings can impact how data on water quality is being processed and studied, thus, opening the door for more reliable water management strategies that better inform public health decisions.

Efficient Computation of the Quantum Rate-Distortion Function

The quantum rate-distortion function plays a fundamental role in quantum information theory, however there is currently no practical algorithm which can efficiently compute this function to high accuracy for moderate channel dimensions. In this paper, we show how symmetry reduction can significantly simplify common instances of the entanglement-assisted quantum rate-distortion problems. This allows us to better understand the properties of the quantum channels which obtain the optimal rate-distortion trade-off, while also allowing for more efficient computation of the quantum rate-distortion function regardless of the numerical algorithm being used. Additionally, we propose an inexact variant of the mirror descent algorithm to compute the quantum rate-distortion function with provable sublinear convergence rates. We show how this mirror descent algorithm is related to Blahut-Arimoto and expectation-maximization methods previously used to solve similar problems in information theory. Using these techniques, we present the first numerical experiments to compute a multi-qubit quantum rate-distortion function, and show that our proposed algorithm solves faster and to higher accuracy when compared to existing methods.

Discovering DNA shape motifs with multiple DNA shape features: generalization, methods, and validation.

DNA motifs are crucial patterns in gene regulation. DNA-binding proteins (DBPs), including transcription factors, can bind to specific DNA motifs to regulate gene expression and other cellular activities. Past studies suggest that DNA shape features could be subtly involved in DNA-DBP interactions. Therefore, the shape motif annotations based on intrinsic DNA topology can deepen theunderstanding of DNA-DBP binding. Nevertheless, high-throughput tools for DNA shape motif discovery that incorporate multiple features altogether remain insufficient. To address it, we propose a series of methods to discover non-redundant DNA shape motifs with the generalization to multiple motifs inmultiple shape features. Specifically, an existingGibbs sampling methodis generalized to multiple DNA motif discovery with multiple shape features. Meanwhile, an expectation-maximization (EM) method and a hybrid method coupling EM with Gibbs sampling are proposed anddeveloped with promisingperformance, convergence capability, and efficiency. The discovered DNA shape motif instances reveal insights intolow-signal ChIP-seq peak summits, complementing the existing sequence motif discovery works. Additionally, our modelling captures the potential interplays across multiple DNA shape features. We provide a valuable platform of tools for DNA shape motif discovery. An R package is built for open accessibility and long-lasting impact: https://zenodo.org/doi/10.5281/zenodo.10558980.

A mixture of logistic skew-normal multinomial models

The logistic normal multinomial distribution is gaining interest in modelling microbiome data. It utilizes a hierarchical structure such that the observed counts conditional on the compositions are assumed to be multinomial random variables and the log-ratio transformed compositions are assumed to be from a Gaussian distribution. While multinomial distribution accounts for the compositional nature of the data, and a Gaussian prior offers flexibility in the structure of covariance matrices, the log-ratio transformed compositions of the microbiome data can be highly skewed, especially at a lower taxonomic level. Thus, a Gaussian distribution may not be an ideal prior for the log-ratio transformed compositions. A novel mixture of logistic skew-normal multinomial (LSNM) distribution is proposed in which a multivariate skew-normal distribution is utilized as a prior for the log-ratio transformed compositions. A variational Gaussian approximation in conjunction with the EM algorithm is utilized for parameter estimation.

Estimating mixed-effects state-space models via particle filters and the EM algorithm

In this paper, we focus on studying the Mixed-Effects State-Space (MESS) models previously introduced by Liu et al. [Liu D, Lu T, Niu X-F, et al. Mixed-effects state-space models for analysis of longitudinal dynamic systems. Biometrics. 2011;67(2):476–485]. We propose an estimation method by combining the auxiliary particle learning and smoothing approach with the Expectation Maximization (EM) algorithm. First, we describe the technical details of the algorithm steps. Then, we evaluate their effectiveness and goodness of fit through a simulation study. Our method requires expressing the posterior distribution for the random effects using a sufficient statistic that can be updated recursively, thus enabling its application to various model formulations including non-Gaussian and nonlinear cases. Finally, we demonstrate the usefulness of our method and its capability to handle the missing data problem through an application to a real dataset.

Design of a novel gamma camera with large field of view for 16N diagnosis in the primary loop of nuclear reactor.

The assessment of the concentration and distribution of l6N, derived from 16O in the cooling water exposed to neutron irradiation, is essential for ensuring radiation safety during nuclear reactor operation. The imaging method allows for the visualization of the intensity distribution of these l6N by capturing gamma-rays emitted during their decay process. However, the existing gamma camera is exclusively compatible with gamma-rays below 2MeV. In this paper, a novel gamma camera featuring a thick double-conical penumbra aperture, a pixelated Lu1.8Y0.2SiO5:Ce scintillator array, and a position-sensitive photomultiplier tube is proposed to address this limitation. This innovative design offers a large field of view (FOV) and is suitable for high energy extended gamma source imaging. The optimization of key parameters of the camera was conducted, and a FOV of 60° and an angular resolution of up to 4.57° were achieved. Imaging simulations, including a simplified model of the primary loop of the pressurized-water reactor by GEANT4 code and image reconstruction using the expectation maximum algorithm, demonstrated that the proposed gamma camera could obtain a satisfactory spatial resolution for diagnosing the distribution of 16N in the primary loop of a nuclear reactor.

Gaussian Process Latent Variable Model-Based Multi-Output Modeling of Incomplete Data

The rapid development of sensor technologies allows the acquisition of high dimensional sensing data. Multi-output modeling techniques have been developed to leverage the data for decision making. However, the data often contain segments of missing values, which cause great information loss and thus affect the modeling performance. This study explores the missing pattern and the correlation structure of missing segments and maximally exploits useful information in the data to improve multi-output modeling accuracy. Specifically, a new multi-output modeling method is developed based on Gaussian Process Latent Variable Model (GPLVM). A decision score is developed to seek an optimal modeling strategy and then a tailored Expectation-Maximization (EM) algorithm based on GPLVM is designed to estimate the missing segments while optimizing model parameters. The proposed method demonstrates superior performance in both a simulation study and a case study, which makes it a powerful tool to enable process automation. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —In real-life applications, missing values are constantly present in multi-output sensing data, which greatly affects data-driven decision-making. Modeling of such data becomes more difficult when consecutive observations within or across different outputs are missing. Existing methods often discard the missing values and extract information only from the available observations. However, the pattern of missing values may contain important messages that can potentially boost the modeling performance. This research develops a new framework based on GPLVM to model multi-output data with segmented missing patterns. A tailored EM algorithm is developed to iteratively impute the missing values and optimize model parameters. In addition, a decision score that quantifies both the missing pattern and correlation is designed to determine an optimal modeling strategy. The proposed method can benefit many applications across different industries that require modeling of multi-output incomplete data, especially when the data have many segments of missing observations.

Probabilistic Graph Model Based Recommendation Algorithm for Material Selection in Self-directed Learning

Faced the vast amount of information, choosing the appropriate materials is a prerequisite for effective self-directed learning. The recommendation algorithm is a kind of intelligent technology that can accurately locate the required information which the users care about most. However, many recommendation techniques experience can not be trained adequately in scenarios with small sample data and extremely sparse ratings. Moreover, DLRAs (Deep learning based Recommendation Algorithms) require high hardware support. The probabilistic graph (PG) can effectively represent the implicit complex relations among nodes, but it still has the problem of sparse data sensitivity. Therefore, we propose a Matrix-Factorization-based Probabilistic Graph Model for Recommendation Algorithm (MF-PGMRA): By matrix-factorizing the sparse rating matrix, the users and items are mapped to the user/item spaces, respectively; We employ the inner product to data-enhance and overcome the problems of sparse data and cold start; Then, we build Probabilistic Graph to construct the “user-item” latent spaces and estimate the probability distribution based on expectation maximization (EM), so as to predict the ratings; Finally, we built a library management system with the recommendation module to highlight the benefits of MF-PGMRA for students’ subject learning. According to a questionnaire, we confirmed that the students are satisfied with the system from four aspects of speed, accuracy, usability and convenience, which can confirm that the library management system based on MF-PGMRA can efficiently and accurately recommend suitable materials for students from the huge amount of learning materials to improve students’ self-directed learning efficiency.

Modeling agricultural and methane emission data: a finite mixture regression approach

In this paper, the method for unsupervised learning of finite mixture regression (FMR) models is presented for evaluation using agricultural and emissions data sets. The FMR models can be written as problems with incomplete data, and the expectation–maximization (EM) algorithm can be used to estimate unknown variables. The goals of this research are to find the best clustering model with different sets of training and test data and examine the relationship between crop production index and methane emissions in 22 countries from 1990 to 2019 using FMR. In this study also use machine learning process for a FMR model from real world data. According to the findings, the performance of the random training data (RDM) in time series is preferable to that of the fixed training data (FXM). In addition, both RDM and FXM are capable of classifying the 22 countries into two distinct groups and constructing the parameters for the regression model. However, selecting training and test data will result in a good prediction; it is dependent on the data collected. Picking the right training and test data is crucial for accurate predictions-it all comes down to having good data in the first place.

Estimation and inference for high dimensional factor model with regime switching

This paper proposes maximum (quasi)likelihood estimation for high dimensional factor models with regime switching in the loadings. The model parameters are estimated jointly by the EM (expectation maximization) algorithm, which in the current context only requires iteratively calculating regime probabilities and principal components of the weighted sample covariance matrix. When regime dynamics are taken into account, smoothed regime probabilities are calculated using a recursive algorithm. Consistency, convergence rates and limit distributions of the estimated loadings and the estimated factors are established under weak cross-sectional and temporal dependence as well as heteroscedasticity. It is worth noting that due to high dimension, regime switching can be identified consistently after the switching point with only one observation period. Simulation results show good performance of the proposed method. An application to the FRED-MD dataset illustrates the potential of the proposed method for detection of business cycle turning points.

Automated litho-fluid and facies classification in well-logs: the Rock Physics perspective

While accurate litho-fluid and facies interpretation from wireline log data is critical for applications like joint facies and impedance inversion of seismic data, extracting this information manually is challenging due to the complexity and high dimensionality of the logs. Traditional clustering methods also struggle with litho-fluid type inference due to different depth trends in petrophysical rock properties due to compaction and diagenesis. We introduce a Rock Physics Machine Learning workflow that automates litho-fluid classification and property depth trend modeling to address these challenges. This workflow employs a maximum-likelihood approach, explicitly accounting for depth-related effects via Rock Physics models, to infer litho-fluid types from borehole data. It utilizes a robust Expectation-Maximization algorithm to associate each litho-fluid type with a specific Rock Physics model instance, constrained within physically reasonable bounds. The workflow directly outputs litho-fluid type proportions and type-specific Rock Physics models with associated uncertainties, providing essential prior information for seismic inversion.

Mapping QTL controlling count traits with excess zeros and ones using a zero-and-one-inflated generalized Poisson regression model.

The research on the quantitative trait locus (QTL) mapping of count data has aroused the wide attention of researchers. There are frequent problems in applied research that limit the application of the conventional Poisson model in the analysis of count phenotypes, which include the overdispersion and excess zeros and ones. In this article, a novel model, that is, the zero-and-one-inflated generalized Poisson (ZOIGP) model, is proposed to deal with these problems. Based on the proposed model, a score test is performed for the inflation parameter, in which the ZOIGP model with a constant proportion of excess zeros and ones is compared with a standard generalized Poisson model. To illustrate the practicability of the ZOIGP model, we extend it to the QTL interval mapping application that underpins count phenotype with excess zeros and excess ones. The genetic effects are estimated utilizing the expectation-maximization algorithm embedded with the Newton-Raphson algorithm, and the genome-wide scan and likelihood ratio test is performed to map and test the potential QTLs. The statistical properties exhibited by the proposed method are investigated through simulation. Finally, a real data analysis example is used to illustrate the utility of the proposed method for QTLmapping.

Split Bregman quantum noise removal algorithm for 3D reconstruction of neutron computed tomography image

The low intensity of the neutron source for neutron computed tomography (CT) results in a long acquisition time for a single projection, which causes the neutron projection data to contain a large amount of quantum noise. Quantum noise will degrade the quality of neutron CT reconstruction images. Therefore, an efficient quantum noise removal algorithm must be used in CT reconstruction. In this paper, an efficient quantum noise removal algorithm for neutron CT 3D image reconstruction is proposed by analysing classical image processing algorithms and quantum image processing algorithms, which employs the maximum likelihood expectation maximization to reconstruct the image and split Bregman algorithm to solve for the total variation (MLEM-SBTV). Experimental results show that MLEM-SBTV performs well in removing quantum noise and reconstructing the detailed structure of images.

A multi-rate high-order dynamic twin-latent-variable probabilistic modeling and its process monitoring application

Quality-relevant process monitoring provides important guarantees for the safety of industrial operation, which is based on the assumption that data collection is complete and low-order autocorrelated. However, real industrial processes always exhibit complex characteristics such as multi-rate sampling and high-order dynamic, which pose great challenges for process monitoring. To this end, a multi-rate high-order dynamic twin-latent-variable probabilistic (MHDTVP) model is presented in this paper to extract data correlations among multi-rate measurements from quality-relevant and irrelevant perspectives. Moreover, to reveal the dynamics in the multi-rate sampling process, an autoregressive twin-latent-variable structure is designed to extract both quality-relevant and quality-irrelevant high-order dynamic features. In the MHDTVP model, parameters are trained through an efficient expectation maximization (EM) iteration framework. Finally, the performance conclusions of MHDTVP are validated with the Tennessee Eastman process (TEP) and Thermal Power Plant (TPP). The experimental results demonstrate that the proposed model exhibits superior monitoring efficiency for multi-rate dynamic processes compared to similar approaches.

MT Method for Anomaly Detection and Classification using EM-λ Algorithm

Purpose: In this paper, we propose a method to classify and detect normal, known anomalies, and unknown anomalies by combining the expectation–maximisation (EM-λ) algorithm and the Mahalanobis–Taguchi (MT) method. Methodology/Approach: The proposed method learns normal data that are expected to be homogeneous and known abnormal data and performs classification and detection by parameter estimation using the EM-λ algorithm. Conventional methods perform analysis based on parameter estimation using the EM algorithm. However, the EM algorithm can degrade classification accuracy if it does not assume that the data fits the model's generative process. Findings: We verify the performance of the proposed method using artificially generated data and real-world bean data for classification as data that do not satisfy this assumption. The validation results show up to 6% improvement over the conventional method in classification accuracy and unknown anomaly discrimination accuracy. Research Limitation/implication: We try various patterns for the parameter of the proposed method in the verification. However, this way is computationally expensive. Originality/Value of paper: Conventional methods perform analysis based on parameter estimation using the EM algorithm. Our proposal method seeks to improve accuracy by using the EM-λ algorithm for parameter estimation, which is expected to improve classification accuracy when the data do not conform to the generative assumptions of the EM algorithm's model.

Publisher Correction: The Effectiveness of a Probabilistic Principal Component Analysis Model and Expectation Maximisation Algorithm in Treating Missing Daily Rainfall Data

Publisher Correction: The Effectiveness of a Probabilistic Principal Component Analysis Model and Expectation Maximisation Algorithm in Treating Missing Daily Rainfall Data

MbDecoda: a debiased approach to compositional data analysis for microbiome surveys.

Potentially pathogenic or probiotic microbes can be identified by comparing their abundance levels between healthy and diseased populations, or more broadly, by linking microbiome composition with clinical phenotypes or environmental factors. However, in microbiome studies, feature tables provide relative rather than absolute abundance of each feature in each sample, as the microbial loads of the samples and the ratios of sequencing depth to microbial load are both unknown and subject to considerable variation. Moreover, microbiome abundance data are count-valued, often over-dispersed and contain a substantial proportion of zeros. To carry out differential abundance analysis while addressing these challenges, we introduce mbDecoda, a model-based approach for debiased analysis of sparse compositions of microbiomes. mbDecoda employs a zero-inflated negative binomial model, linking mean abundance to the variable of interest through a log link function, and it accommodates the adjustment for confounding factors. To efficiently obtain maximum likelihood estimates of model parameters, an Expectation Maximization algorithm is developed. A minimum coverage interval approach is then proposed to rectify compositional bias, enabling accurate and reliable absolute abundance analysis. Through extensive simulation studies and analysis of real-world microbiome datasets, we demonstrate that mbDecoda compares favorably with state-of-the-art methods in terms of effectiveness, robustness and reproducibility.