Expectation Maximization Research Articles

Bayesian expectation maximization-maximization for robust estimation in proton exchange membrane fuel cells: A comparative study

In this paper, we present a novel approach based on Bayesian Expectation Maximization-Maximization (BEMM) to address this challenge. Unlike traditional optimization methods, which may struggle with high-dimensional and nonlinear optimization problems, BEMM offers a robust framework that combines the benefits of Bayesian inference with the flexibility of expectation maximization and maximization techniques. By iteratively updating parameter estimates based on observed data and maximizing the likelihood of the model, BEMM effectively navigates the solution space to converge on accurate estimates of the unspecified variables in Proton Exchange Membrane Fuel Cells (PEMFCs)models. Through extensive experimentation and comparison with other metaheuristic techniques, including Arithmetic Optimization Algorithm (AOA), Gravitational Search Algorithm (GSA), Flower Pollination Algorithm (FPA), and Biogeography-Based Optimization (BBO), we demonstrate the superior performance of our BEMM approach. Our results show that BEMM outperforms these alternative methods in terms of precision, convergence speed, and stability across various scenarios involving different numbers of unspecified variables. The implications of our findings are significant for both researchers and practitioners in the field of PEMFC modeling and optimization. By providing a more efficient and reliable method for estimating model parameters, our approach can facilitate more accurate predictions of PEMFC performance, leading to better-informed decision-making in the design, operation, and optimization of PEMFC systems. Furthermore, the robustness and versatility of BEMM make it well-suited for a wide range of optimization problems beyond PEMFC modeling, highlighting its potential impact across various domains of engineering and scientific research. In the sensitivity analysis, as the population size increases from 10 to 40, there is a significant improvement in solution quality by approximately 100 %. However, beyond a population size of 40, the marginal gains diminish, with only marginal improvements of less than 1 % observed despite further increases in population size up to 200.

Hyperspectral Image Denoising by Pixel-Wise Noise Modeling and TV-Oriented Deep Image Prior

Model-based hyperspectral image (HSI) denoising methods have attracted continuous attention in the past decades, due to their effectiveness and interpretability. In this work, we aim at advancing model-based HSI denoising, through sophisticated investigation for both the fidelity and regularization terms, or correspondingly noise and prior, by virtue of several recently developed techniques. Specifically, we formulate a novel unified probabilistic model for the HSI denoising task, within which the noise is assumed as pixel-wise non-independent and identically distributed (non-i.i.d) Gaussian predicted by a pre-trained neural network, and the prior for the HSI image is designed by incorporating the deep image prior (DIP) with total variation (TV) and spatio-spectral TV. To solve the resulted maximum a posteriori (MAP) estimation problem, we design a Monte Carlo Expectation–Maximization (MCEM) algorithm, in which the stochastic gradient Langevin dynamics (SGLD) method is used for computing the E-step, and the alternative direction method of multipliers (ADMM) is adopted for solving the optimization in the M-step. Experiments on both synthetic and real noisy HSI datasets have been conducted to verify the effectiveness of the proposed method.

Robust Estimation Method against Poisoning Attacks for Key-Value Data with Local Differential Privacy

Local differential privacy (LDP) protects user information from potential threats by randomizing data on individual devices before transmission to untrusted collectors. This method enables collectors to derive user statistics by analyzing randomized data, thereby presenting a promising avenue for privacy-preserving data collection. In the context of key–value data, in which discrete and continuous values coexist, PrivKV has been introduced as an LDP protocol to ensure secure collection. However, this framework is susceptible to poisoning attacks. To address this vulnerability, we propose an expectation maximization (EM)-based algorithm combined with a cryptographic protocol to facilitate secure random sampling. Our LDP protocol, known as emPrivKV, exhibits two key advantages: it improves the accuracy of statistical information estimation from randomized data, and enhances resilience against the manipulation of statistics, that is, poisoning attacks. These attacks involve malicious users manipulating the analysis results without detection. This study presents the empirical results of applying the emPrivKV protocol to both synthetic and open datasets, highlighting a notable improvement in the precision of statistical value estimation and robustness against poisoning attacks. As a result, emPrivKV improved the frequency and the mean gains by 17.1% and 25.9%, respectively, compared to PrivKV, with the number of fake users being 0.1 of the genuine users. Our findings contribute to the ongoing discourse on refining LDP protocols for key–value data in scenarios involving privacy-sensitive information.

Exponential Dispersion accelerated degradation modelling and reliability assessment considering initial value and processes heterogeneity

In the production and operation, inherent variability and uncertainty necessitate addressing unit-to-unit heterogeneity in initial performance values and degradation processes. This article presents a bi-stochastic exponential dispersion process (BS-ED) designed to account for heterogeneity in both initial performance values and degradation processes. First, based on the ED process, the time and acceleration covariates are introduced to form a nonlinear accelerated ED process, and a random effect coefficient associated with the accelerated stress is incorporated to consider the heterogeneity of the process. Meanwhile, through the modelling of degradation time-shift, a degradation model considering the stochastic initial value of the product performance is developed. To effectively conduct the statistic inference of the BS-ED process, an improved stochastic EM algorithm is proposed, and the information matrix and Ito calculus are combined to estimate the confidence intervals. Finally, the stability of the method is verified by simulation and analyzed by two real cases.

Bayesian multi-exposure image fusion for robust high dynamic range ptychography.

The limited dynamic range of the detector can impede coherent diffractive imaging (CDI) schemes from achieving diffraction-limited resolution. To overcome this limitation, a straightforward approach is to utilize high dynamic range (HDR) imaging through multi-exposure image fusion (MEF). This method involves capturing measurements at different exposure times, spanning from under to overexposure and fusing them into a single HDR image. The conventional MEF technique in ptychography typically involves subtracting the background noise, ignoring the saturated pixels and then merging the acquisitions. However, this approach is inadequate under conditions of low signal-to-noise ratio (SNR). Additionally, variations in illumination intensity significantly affect the phase retrieval process. To address these issues, we propose a Bayesian MEF modeling approach based on a modified Poisson distribution that takes the background and saturation into account. The expectation-maximization (EM) algorithm is employed to infer the model parameters. As demonstrated with synthetic and experimental data, our approach outperforms the conventional MEF method, offering superior phase retrieval under challenging experimental conditions. This work underscores the significance of robust multi-exposure image fusion for ptychography, particularly in imaging shot-noise-dominated weakly scattering specimens or in cases where access to HDR detectors with high SNR is limited. Furthermore, the applicability of the Bayesian MEF approach extends beyond CDI to any imaging scheme that requires HDR treatment. Given this versatility, we provide the implementation of our algorithm as a Python package.

The human mitochondrial mRNA structurome reveals mechanisms of gene expression.

The human mitochondrial genome encodes crucial oxidative phosphorylation system proteins, pivotal for aerobic energy transduction. They are translated from nine monocistronic and two bicistronic transcripts whose native structures remain unexplored, posing a gap in understanding mitochondrial gene expression. In this work, we devised the mitochondrial dimethyl sulfate mutational profiling with sequencing (mitoDMS-MaPseq) method and applied detection of RNA folding ensembles using expectation-maximization (DREEM) clustering to unravel the native mitochondrial messenger RNA (mt-mRNA) structurome in wild-type (WT) and leucine-rich pentatricopeptide repeat-containing protein (LRPPRC)-deficient cells. Our findings elucidate LRPPRC's role as a holdase contributing to maintaining mt-mRNA folding and efficient translation. mt-mRNA structural insights in WT mitochondria, coupled with metabolic labeling, unveil potential mRNA-programmed translational pausing and a distinct programmed ribosomal frameshifting mechanism. Our data define a critical layer of mitochondrial gene expression regulation. These mt-mRNA folding maps provide a reference for studying mt-mRNA structures in diverse physiological and pathological contexts.

On the Unique Channel Identification in Ambient Backscatter Communication Systems

Channel coefficient estimation is a challenging problem in ambient backscatter communication systems. In this paper, unique identification of channel coefficients using the Expectation-Maximization (EM) method is investigated. It is proven that the EM method is not able to always yield unique solutions to channel coefficients and therefore the tag symbol estimation accuracy can be compromised. A correct derivation of the hybrid Cramer-Rao Bound (HCRB) is also provided.

Identification of switched gated recurrent unit neural networks with a generalized Gaussian distribution

Due to the limitations of the model itself, the performance of switched autoregressive exogenous (SARX) models will face potential threats when modeling nonlinear hybrid dynamic systems. To address this problem, a robust identification approach of the switched gated recurrent unit (SGRU) model is developed in this paper. Firstly, all submodels of the SARX model are replaced by gated recurrent unit neural networks. The obtained SGRU model has stronger nonlinear fitting ability than the SARX model. Secondly, this paper departs from the conventional Gaussian distribution assumption for noise, opting instead for a generalized Gaussian distribution. This enables the proposed model to achieve stable prediction performance under the influence of different noises. Notably, no prior assumptions are imposed on the knowledge of operating modes in the proposed switched model. Therefore, the EM algorithm is used to solve the problem of parameter estimation with hidden variables in this paper. Finally, two simulation experiments are performed. By comparing the nonlinear fitting ability of the SGRU model with the SARX model and the prediction performance of the SGRU model under different noise distributions, the effectiveness of the proposed approach is verified.

Effect of a correlated competing risk on marginal survival estimation in an accelerated failure time model

In problems with a time to event outcome, subjects may experience competing events, which censor the outcome of interest. Cox’s partial likelihood estimator treating competing events as independent censoring is commonly used to examine group differences in clinical trials but fails to adjust for omitted covariates and can bias the assessment of marginal benefit. A bivariate normal linear model generating latent data with dependent censoring is used to assess this bias. Our R-package bnc provides maximum penalized likelihood (MPL) parameter estimation using a novel EM algorithm. Using bnc, we study the properties of such MPL estimation. Simulation results for two-sample survival comparisons of time to an event of interest, with independent censoring accompanied by censoring from a correlated competing risk, are presented. Key parameters—means, hazard ratios, and correlation—are estimated. These results demonstrated that, despite ill-conditioning in models generating correlated competing risks, estimates of marginal effects are reliable. Bivariate normal models were fitted in a trial of head and neck cancer. Model fits help with clinical interpretation while also supplementing other standard methods for follow-up that are terminated by intervening risks.

Comparison of Measurement and Prognostic Power of SUV Between High-Definition and Standard PET Imaging in Non-Small Cell Lung Cancer Patients.

This study aimed to evaluate the measurement and prognostic ability of the SUVmax of whole-body tumors (SUVmaxwb) in non-small cell lung cancer (NSCLC) patients, comparing high-definition (HD) PET imaging with standard-definition (SD) PET imaging. Methods: The study included 242 consecutive NSCLC patients who underwent baseline 18F-FDG PET/CT from April 2018 to January 2021. Two imaging techniques were used: HD PET (using ordered-subsets expectation maximization with point-spread function modeling and time-of-flight techniques and smaller voxels) and SD PET (with ordered-subsets expectation maximization and time-of-flight techniques). SUVmaxwb was determined by measuring all the tumor lesions in the whole body, and tumor-to-background ratio (TBR) was calculated using the background SUVmean of various body parts. Results: The patient cohort had an average age of 68.3 y, with 59.1% being female. During a median follow-up of 29.6 mo, 83 deaths occurred. SUVmaxwb was significantly higher in HD PET than SD PET, with respective medians of 17.4 and 11.8. The TBR of 1,125 tumoral lesions was also higher in HD PET. Univariate Cox regression analysis showed that SUVmaxwb from both HD and SD PET were significantly associated with overall survival. However, after adjusting for TNM (tumor, node, metastasis) stage, only SUVmaxwb from SD PET remained significantly associated with survival. Conclusion: HD PET imaging in NSCLC patients yields higher SUVmaxwb and TBR, enhancing tumor visibility. Despite this, its prognostic value is less significant than SD PET after adjusting clinical TNM stage. Thus, consideration should be given to using HD PET reconstruction to increase tumor visibility, and SD PET is recommended for NSCLC patient prognostication and therapeutic evaluation, as well as for the classification of lung nodules.

NQuack: An R package for predicting ploidal level from sequence data using site-based heterozygosity.

Traditional methods of ploidal-level estimation are tedious; using DNA sequence data for cytotype estimation is an ideal alternative. Multiple statistical approaches to leverage sequence data for ploidy inference based on site-based heterozygosity have been developed. However, these approaches may require high-coverage sequence data, use inappropriate probability distributions, or have additional statistical shortcomings that limit inference abilities. We introduce nQuack, an open-source R package that addresses the main shortcomings of current methods. nQuack performs model selection for improved ploidy predictions. Here, we implement expectation maximization algorithms with normal, beta, and beta-binomial distributions. Using extensive computer simulations that account for variability in sequencing depth, as well as real data sets, we demonstrate the utility and limitations of nQuack. Inferring ploidy based on site-based heterozygosity alone is difficult. Even though nQuack is more accurate than similar methods, we suggest caution when relying on any site-based heterozygosity method to infer ploidy.

Optimal life-testing plans for joint progressively Type-II censored Birnbaum-Saunders distributions with compound optimal criteria

ABSTRACT In this paper, we first derive the maximum likelihood estimates (MLEs) of the parameters of Birnbaum-Saunders (BISA) distributions based on the joint progressive Type-II censored (JPC) samples. We also discuss using the EM algorithm to obtain the MLEs of the model parameters. Then, we determine the single-objective-criterion optimal JPC schemes for BISA distributions based on the cost minimization criterion, A -optimality criterion, and D -optimality criterion by the complete search method, variable neighborhood search (VNS) algorithm and modified VNS (MVNS) algorithm. These algorithms are compared in terms of accuracy and computation time. In addition, we determine the compound-criterion optimal JPC schemes based on different optimal criteria and the reasonably efficient compound optimal JPC schemes for two competing statistical models, such as the inverse Gaussian and BISA models. The advantages of using the compound optimal scheme over the single-objective-criterion optimal scheme are demonstrated through a real-life data set on the micro-indentation test about the hardness of polymeric bone cement.

Estimating Latent-Variable Panel Data Models Using Parameter-Expanded SEM Methods

This article presents new estimation algorithms for three types of dynamic panel data models with latent variables: factor models, discrete choice models, and persistent-transitory quantile processes. The new methods combine the parameter expansion (PX) ideas of Liu, Rubin, and Wu with the stochastic expectation-maximization (SEM) algorithm in likelihood and moment-based contexts. The goal is to facilitate convergence in models with a large space of latent variables by improving algorithmic efficiency. This is achieved by specifying expanded models within the M step. Effectively, we are proposing new estimators for the pseudo-data within iterations that take into account the fact that the model of interest is misspecified for draws based on parameter values far from the truth. We establish the asymptotic equivalence of the likelihood-based PX-SEM to an alternative SEM algorithm with a smaller expected fraction of missing information compared to the standard SEM based on the original model, implying a faster global convergence rate. Finally, in simulations we show that the new algorithms significantly improve the convergence speed relative to standard SEM algorithms, sometimes dramatically so, by reducing the total computing time from hours to a few minutes.

An international relations quantitative evaluation model (IRQEM) based on an integrated method.

International relations (IR) have great uncertainty and instability. Bad IR or conflicts will bring about heavy economic losses and widespread social unrest domestically and internationally. The accurate prediction for bilateral relations can support decision making for timely responses, which will be used to find ways to maintain development in the complex international situation. An international relations quantitative evaluation model (IRQEM) is proposed by integrating a variety of research models and methods like the interpretative structural modeling method (ISM), Bayesian network (BN) model, the Bayesian search (BS), and the expectation-maximization (EM) algorithm, which is novel for IR research. Factors from several different fields are identified as BN nodes. Each node is assigned different state values. The hierarchical structure of these BN nodes is obtained by ISM. The data collection of 192 cases is used to construct the BN model by GeNIe 4.0. The IRQEM can be used to evaluate the influence of emergencies on IR. The critical factors of IR also can be explored through our proposed model. Results show that the prediction of bilateral relations under emergencies can be realized by updating the indicator set when emergencies occur. The capability to anticipate threats of IR changes is advanced by optimizing the reporting information of IR forecasting through a combination of qualitative and quantitative methods, charts, and texts. Relevant analysis results can provide support for national security decision making.

Modeling soybean growth: A mixed model approach.

The evaluation of plant and animal growth, separately for genetic and environmental effects, is necessary for genetic understanding and genetic improvement of environmental responses of plants and animals. We propose to extend an existing approach that combines nonlinear mixed-effects model (NLMEM) and the stochastic approximation of the Expectation-Maximization algorithm (SAEM) to analyze genetic and environmental effects on plant growth. These tools are widely used in many fields but very rarely in plant biology. During model formulation, a nonlinear function describes the shape of growth, and random effects describe genetic and environmental effects and their variability. Genetic relationships among the varieties were also integrated into the model using a genetic relationship matrix. The SAEM algorithm was chosen as an efficient alternative to MCMC methods, which are more commonly used in the domain. It was implemented to infer the expected growth patterns in the analyzed population and the expected curves for each variety through a maximum-likelihood and a maximum-a-posteriori approaches, respectively. The obtained estimates can be used to predict the growth curves for each variety. We illustrate the strengths of the proposed approach using simulated data and soybean plant growth data obtained from a soybean cultivation experiment conducted at the Arid Land Research Center, Tottori University. In this experiment, plant height was measured daily using drones, and the growth was monitored for approximately 200 soybean cultivars for which whole-genome sequence data were available. The NLMEM approach improved our understanding of the determinants of soybean growth and can be successfully used for the genomic prediction of growth pattern characteristics.

Interpretable and efficient RUL prediction of turbofan engines using EM-enhanced Bi-LSTM with TCN and attention mechanism

Remaining useful life (RUL) prediction for turbofan engines is important in prognostics and health management (PHM) for the maintenance and operation of critical equipment. With continuous innovations in deep learning techniques, the complexity of models continues to increase, but the interpretability and comprehensibility of the prediction results become particularly important in industrial applications. Therefore, in this study, an improved bidirectional long and short-term memory network (Bi-LSTM) based interpretable hybrid deep learning model for RUL prediction of turbofan engines is proposed, which ingeniously integrates time series convolutional networks (TCNs), expectation maximization (EM), Bi-LSTMs, and attention mechanisms. By capturing time-series features at different levels, the model adapts to the complex dynamics of turbofan engine performance evolution in an efficient and cost-effective manner. Experimental validation on the C-MAPSS dataset demonstrated that the model significantly outperforms other methods in terms of RUL prediction performance, especially in improving prediction accuracy and coping with the degradation of complex system dynamics. The largest contribution of key metrics to the model is validated through consistent results from multiple interpretable tools, providing comprehensive and consistent support for understanding and trusting prediction results in industrial applications. This study further enhances the robustness of the model and the reliability of the interpretable results by delving into the dynamic relationships between the properties of the different life stages, which not only reveal the importance of these characteristics in engine life prediction but also provide more comprehensive information about the engine performance variations by observing the dynamic relationships.

A mixed BAR(1) model driven by serially dependent innovation with application

To better analyze dependent count data with bounded support, this paper proposes a first-order mixture binomial autoregressive (BAR(1)) model, which is constructed via making use of the Pegram operator, where the innovation processes are assumed to be serially dependent to the observation rather than independent. The new model not only inherits the advantages of existing mixture BAR(1) models while also overcoming their shortcomings, but also does well in explaining binomial overdispersion and zero inflation. We infer various statistical properties of the process, including conditional moments, moments and autocovariance functions. The EM algorithm is employed to calculate the conditional maximum likelihood estimate. To evaluate the performance of obtained estimates, some simulation studies are conducted. Furthermore, the forecasting problem for the proposed model is also discussed. To show the applicability of our model, an application to proposition data set is conducted and we compare our model to some competitive integer-valued time series models with bounded support.

Different Statistical Inference Algorithms for the New Pareto Distribution Based on Type-II Progressively Censored Competing Risk Data with Applications

In this research, the statistical inference of unknown lifetime parameters is proposed in the presence of independent competing risks using a progressive Type-II censored dataset. The lifetime distribution associated with a failure mode is assumed to follow the new Pareto distribution, with consideration given to two distinct competing failure reasons. Maximum likelihood estimators (MLEs) for the unknown model parameters, as well as reliability and hazard functions, are derived, noting that they are not expressible in closed form. The Newton–Raphson, expectation maximization (EM), and stochastic expectation maximization (SEM) methods are employed to generate maximum likelihood (ML) estimations. Approximate confidence intervals for the unknown parameters, reliability, and hazard rate functions are constructed using the normal approximation of the MLEs and the normal approximation of the log-transformed MLEs. Additionally, the missing information principle is utilized to derive the closed form of the Fisher information matrix, which, in turn, is used with the delta approach to calculate confidence intervals for reliability and hazards. Bayes estimators are derived under both symmetric and asymmetric loss functions, with informative and non-informative priors considered, including independent gamma distributions for informative priors. The Monte Carlo Markov Chain sampling approach is employed to obtain the highest posterior density credible intervals and Bayesian point estimates for unknown parameters and reliability characteristics. A Monte Carlo simulation is conducted to assess the effectiveness of the proposed techniques, with the performances of the Bayes and maximum likelihood estimations examined using average values and mean squared errors as benchmarks. Interval estimations are compared in terms of average lengths and coverage probabilities. Real datasets are considered and examined for each topic to provide illustrative examples.

Fuzzy random sensitivity analysis for the overall structure reliability of reinforced concrete freezing wellbores in deep alluvium based on hidden Markov model

To address the shortcomings of traditional reliability theory in characterizing the stability of deep underground structures, the advanced first order second moment of reliability was improved to obtain fuzzy random reliability, which is more consistent with the working conditions. The traditional sensitivity analysis model was optimized using fuzzy random optimization, and an analytical calculation model of the mean and standard deviation of the fuzzy random reliability sensitivity was established. A big data hidden Markov model and expectation-maximization algorithm were used to improve the digital characteristics of fuzzy random variables. The fuzzy random sensitivity optimization model was used to confirm the effect of concrete compressive strength, thick-diameter ratio, reinforcement ratio, uncertainty coefficient of calculation model, and soil depth on the overall structural reliability of a reinforced concrete double-layer wellbore in deep alluvial soil. Through numerical calculations, these characteristics were observed to be the main influencing factors. Furthermore, while the soil depth was negatively correlated, the other influencing factors were all positively correlated with the overall reliability. This study provides an effective reference for the safe construction of deep underground structures in the future.

Expectation-Maximization enables Phylogenetic Dating under a Categorical Rate Model.

Dating phylogenetic trees to obtain branch lengths in time units is essential for many downstream applications but has remained challenging. Dating requires inferring substitution rates that can change across the tree. While we can assume to have information about a small subset of nodes from the fossil record or sampling times (for fast-evolving organisms), inferring the ages of the other nodes essentially requires extrapolation and interpolation. Assuming a distribution of branch rates, we can formulate dating as a constrained maximum likelihood (ML) estimation problem. While ML dating methods exist, their accuracy degrades in the face of model misspecification, where the assumed parametric statistical distribution of branch rates vastly differs from the true distribution. Notably, most existing methods assume rigid, often unimodal, branch rate distributions. A second challenge is that the likelihood function involves an integral over the continuous domain of the rates, often leading to difficult non-convex optimization problems. To tackle both challenges, we propose a new method called Molecular Dating using Categorical-models (MD-Cat). MD-Cat uses a categorical model of rates inspired by non-parametric statistics and can approximate a large family of models by discretizing the rate distribution into k categories. Under this model, we can use the Expectation-Maximization algorithm to co-estimate rate categories and branch lengths in time units. Our model has fewer assumptions about the true distribution of branch rates than parametric models such as Gamma or LogNormal distribution. Our results on two simulated and real datasets of Angiosperms and HIV and a wide selection of rate distributions show that MD-Cat is often more accurate than the alternatives, especially on datasets with exponential or multimodal rate distributions.