Expectation Maximization Research Articles

Performance of deconvolution codes (MAXED, GRAVEL, MLEM) for neutron spectrometry of radioactive source using plastic scintillator simulated data

Neutron spectrometry plays a crucial role in decommissioning of nuclear sites and in ensuring the safety of workers from radiation exposure. In this study, we associate a proton recoil spectrometer made by a homemade plastic scintillator with triple discrimination capabilities (fast neutrons, thermal neutrons, and gamma rays) with deconvolution methods to obtain accurate measurements of neutron spectra. The purpose of this work is to test and evaluate the performance of three distinct deconvolution codes, namely MAXED (Maximum Entropy Deconvolution), GRAVEL, and MLEM (Maximum Likelihood Expectation Maximization), using two types of input spectra (flat and Watt spectrum). These deconvolution codes are applied to simulated data using the reference MCNP6.2 Monte Carlo code. Comparing the calculated mean squared error (MSE) performed by the three unfolding methods, we find that MLEM seems to perform better than MAXED and GRAVEL. Furthermore, given the calculated MSE values, the unfolded spectrum of Cf-252 is in better agreement with the standard reference spectrum by using Watt spectrum as input spectrum (MSE<1.2 × 10−6) than using a flat spectrum (MSE<1.3 × 10−6).

Unsupervised domain adaptation with weak source domain labels via bidirectional subdomain alignment

Unsupervised domain adaptation (UDA) enables knowledge transfer from a labeled source domain to an unlabeled target domain. However, UDA performance often relies heavily on the accuracy of source domain labels, which are frequently noisy or missing in real applications. To address unreliable source labels, we propose a novel framework for extracting robust, discriminative features via iterative pseudo-labeling, queue-based clustering, and bidirectional subdomain alignment (BSA). The proposed framework begins by generating pseudo-labels for unlabeled source data and constructing codebooks via iterative clustering to obtain label-independent class centroids. Then, the proposed framework performs two main tasks: rectifying features from both domains using BSA to match subdomain distributions and enhance features; and employing a two-stage adversarial process for global feature alignment. The feature rectification is done before feature enhancement, while the global alignment is done after feature enhancement. To optimize our framework, we formulate BSA and adversarial learning as maximizing a log-likelihood function, which is implemented via the Expectation–Maximization algorithm. The proposed framework shows significant improvements compared to state-of-the-art methods on Office-31, Office-Home, and VisDA-2017 datasets, achieving average accuracies of 91.5%, 76.6%, and 87.4%, respectively. Compared to existing methods, the proposed method shows consistent superiority in unsupervised domain adaptation tasks with both fully and weakly labeled source domains.

Latent event history models for quasi-reaction systems

Various processes, such as cell differentiation and disease spreading, can be modelled as quasi-reaction systems of particles using stochastic differential equations. The existing Local Linear Approximation (LLA) method infers the parameters driving these systems from measurements of particle abundances over time. While dense observations of the process in time should in theory improve parameter estimation, LLA fails in these situations due to numerical instability. Defining a latent event history model of the underlying quasi-reaction system resolves this problem. A computationally efficient Expectation-Maximization algorithm is proposed for parameter estimation, incorporating an extended Kalman filter for evaluating the latent reactions. A simulation study demonstrates the method's performance and highlights the settings where it is particularly advantageous compared to the existing LLA approaches. An illustration of the method applied to the diffusion of COVID-19 in Italy is presented.

Automated lithofluid and facies classification in well logs: The rock-physics perspective

Although an accurate lithofluid and facies interpretation from wireline log data is critical for applications such as joint facies and impedance inversion of seismic data, extracting this information manually is challenging due to the logs’ complexity and the need to combine information from different logs. Traditional clustering methods also struggle with lithofluid type inference due to differing depth trends in petrophysical rock properties stemming from compaction and diagenesis. We introduce a rock-physics machine-learning workflow that automates lithofluid classification and property depth trend modeling to address these challenges. This workflow uses a maximum-likelihood approach, explicitly accounting for depth-related effects via rock-physics models (RPMs) to infer lithofluid types from borehole data. It uses a robust expectation-maximization algorithm to associate each lithofluid type with a specific RPM instance constrained within physically reasonable bounds. The workflow directly outputs lithofluid type proportions and type-specific RPMs with associated uncertainties, providing essential prior information for seismic inversion.

Markov-switching decision trees

Decision trees constitute a simple yet powerful and interpretable machine learning tool. While tree-based methods are designed only for cross-sectional data, we propose an approach that combines decision trees with time series modeling and thereby bridges the gap between machine learning and statistics. In particular, we combine decision trees with hidden Markov models where, for any time point, an underlying (hidden) Markov chain selects the tree that generates the corresponding observation. We propose an estimation approach that is based on the expectation-maximisation algorithm and assess its feasibility in simulation experiments. In our real-data application, we use eight seasons of National Football League (NFL) data to predict play calls conditional on covariates, such as the current quarter and the score, where the model’s states can be linked to the teams’ strategies. R code that implements the proposed method is available on GitHub.

Parameter Adaptive Non-Model-Based State Estimation Combining Attention Mechanism and LSTM

Nowadays, state estimation is widely used in fields such as autonomous driving and drone navigation. However, in practical applications, it is difficult to obtain accurate target motion models and noise covariance.This leads to a decrease in the estimation accuracy of traditional Kalman filters. To address this issue, this paper proposes an adaptive model free state estimation method based on attention parameter learning module. This method combines Transformer's encoder with Long Short Term Memory Network (LSTM), and obtains the system's operational characteristics through offline learning of measurement data without modeling the system dynamics and measurement characteristics. In addition, based on the output of the attention learning module, the expectation maximization (EM) algorithm is used to estimate the system model parameters online, and a Kalman filter is used to obtain state estimation. This paper was validated using the GPS trajectory path dataset, and the experimental results showed that the proposed parameter adaptive model free state estimation method has better estimation accuracy than other models, providing an effective method for using deep learning networks for state estimation.

Missing Data Imputation with Uncertainty-Driven Network

We study the problem of missing data imputation, which is a fundamental task in the area of data quality that aims to impute the missing data to achieve the completeness of datasets. Though the recent distribution-modeling-based techniques (e.g., distribution generation and distribution matching) can achieve state-of-the-art performance in terms of imputation accuracy, we notice that (1) they deploy a sophisticated deep learning model that tends to be overfitting for missing data imputation; (2) they directly rely on a global data distribution while overlooking the local information. Driven by the inherent variability in both missing data and missing mechanisms, in this paper, we explore the uncertain nature of this task and aim to address the limitations of existing works by proposing an u&lt;u&gt;N&lt;/u&gt;certainty-driven netw&lt;u&gt;O&lt;/u&gt;rk for &lt;u&gt;M&lt;/u&gt;issing data &lt;u&gt;I&lt;/u&gt;mputation, termed NOMI. NOMI has three key components, i.e., the retrieval module, the neural network gaussian process imputator (NNGPI) and the uncertainty-based calibration module. NOMI~ runs these components sequentially and in an iterative manner to achieve a better imputation performance. Specifically, in the retrieval module, NOMI~ retrieves local neighbors of the incomplete data samples based on the pre-defined similarity metric. Subsequently, we design NNGPI~ that merges the advantages of both the Gaussian Process and the universal approximation capacity of neural networks. NNGPI~ models the uncertainty by learning the posterior distribution over the data to impute missing values while alleviating the overfitting issue. Moreover, we further propose an uncertainty-based calibration module that utilizes the uncertainty of the imputator on its prediction to help the retrieval module obtain more reliable local information, thereby further enhancing the imputation performance. We also demonstrate that our NOMI~ can be reformulated as an instance of the well-known Expectation Maximization (EM) algorithm, highlighting the strong theoretical foundation of our proposed methods. Extensive experiments are conducted over 12 real-world datasets. The results demonstrate the excellent performance of NOMI in terms of both accuracy and efficiency.

A modified EM-type algorithm to estimate semi-parametric mixtures of non-parametric regressions

Semi-parametric Gaussian mixtures of non-parametric regressions (SPGMNRs) are a flexible extension of Gaussian mixtures of linear regressions (GMLRs). The model assumes that the component regression functions (CRFs) are non-parametric functions of the covariate(s) whereas the component mixing proportions and variances are constants. Unfortunately, the model cannot be reliably estimated using traditional methods. A local-likelihood approach for estimating the CRFs requires that we maximize a set of local-likelihood functions. Using the Expectation-Maximization (EM) algorithm to separately maximize each local-likelihood function may lead to label-switching. This is because the posterior probabilities calculated at the local E-step are not guaranteed to be aligned. The consequence of this label-switching is wiggly and non-smooth estimates of the CRFs. In this paper, we propose a unified approach to address label-switching and obtain sensible estimates. The proposed approach has two stages. In the first stage, we propose a model-based approach to address the label-switching problem. We first note that each local-likelihood function is a likelihood function of a Gaussian mixture model (GMM). Next, we reformulate the SPGMNRs model as a mixture of these GMMs. Lastly, using a modified version of the Expectation Conditional Maximization (ECM) algorithm, we estimate the mixture of GMMs. In addition, using the mixing weights of the local GMMs, we can automatically choose the local points where local-likelihood estimation takes place. In the second stage, we propose one-step backfitting estimates of the parametric and non-parametric terms. The effectiveness of the proposed approach is demonstrated on simulated data and real data analysis.

Online state and unknown inputs estimation for nonlinear systems with particle filter based recursive expectation‐maximization algorithm

AbstractThe article presents an innovative approach to simultaneously estimate states and unknown inputs (UIs) in nonlinear systems using a particle filter (PF) based recursive expectation‐maximization (EM) algorithm. This method is distinct from traditional iterative EM algorithms. During the E‐step, it calculates the Q‐function recursively within the maximum likelihood framework, while the PF estimates the system states. The M‐step involves local maximization of the recursive Q‐function to online estimate the UIs. The effectiveness of the PF‐based recursive EM algorithm is demonstrated with a numerical example, and comparisons with the augmented state PF are made to highlight its advantages. Finally, the proposed algorithm is implemented in a real application for the estimation of the continuous fermentation process.

A flexible time-varying coefficient rate model for panel count data.

Panel count regression is often required in recurrent event studies, where the interest is to model the event rate. Existing rate models are unable to handle time-varying covariate effects due to theoretical and computational difficulties. Mean models provide a viable alternative but are subject to the constraints of the monotonicity assumption, which tends to be violated when covariates fluctuate over time. In this paper, we present a new semiparametric rate model for panel count data along with related theoretical results. For model fitting, we present an efficient EM algorithm with three different methods for variance estimation. The algorithm allows us to sidestep the challenges of numerical integration and difficulties with the iterative convex minorant algorithm. We showed that the estimators are consistent and asymptotically normally distributed. Simulation studies confirmed an excellent finite sample performance. To illustrate, we analyzed data from a real clinical study of behavioral risk factors for sexually transmitted infections.

Alternative skew Laplace scale mixtures for modeling data exhibiting high-peaked and heavy-tailed traits

The search and construction of appropriate and flexible models for describing and modelling empirical data sets incongruent with normality retains a sustained interest. This paper focuses on proposing flexible skew Laplace scale mixture distributions to model these types of data sets. Each member of the collection of distributions is obtained by dividing the scale parameter of a conditional skew Laplace distribution by a purposefully chosen mixing random variable. Highly-peaked, heavy-tailed skew models with relevance and impact in different fields are obtained and investigated, and elegant sampling schemes to simulate from this collection of developed models are proposed. Finite mixtures consisting of the members of the skew Laplace scale mixture models are illustrated, further extending the flexibility of the distributions by being able to account for multimodality. The maximum likelihood estimates of the parameters for all the members of the developed models are described via a developed EM algorithm. Real-data examples highlight select models’ performance and emphasize their viability compared to other commonly considered candidates, and various goodness-of-fit measures are used to endorse the performance of the proposed models as reasonable and viable candidates for the practitioner. Finally, an outline is discussed for future work in the multivariate realm for these models.

A variational expectation-maximization framework for balanced multi-scale learning of protein and drug interactions

Protein functions are characterized by interactions with proteins, drugs, and other biomolecules. Understanding these interactions is essential for deciphering the molecular mechanisms underlying biological processes and developing new therapeutic strategies. Current computational methods mostly predict interactions based on either molecular network or structural information, without integrating them within a unified multi-scale framework. While a few multi-view learning methods are devoted to fusing the multi-scale information, these methods tend to rely intensively on a single scale and under-fitting the others, likely attributed to the imbalanced nature and inherent greediness of multi-scale learning. To alleviate the optimization imbalance, we present MUSE, a multi-scale representation learning framework based on a variant expectation maximization to optimize different scales in an alternating procedure over multiple iterations. This strategy efficiently fuses multi-scale information between atomic structure and molecular network scale through mutual supervision and iterative optimization. MUSE outperforms the current state-of-the-art models not only in molecular interaction (protein-protein, drug-protein, and drug-drug) tasks but also in protein interface prediction at the atomic structure scale. More importantly, the multi-scale learning framework shows potential for extension to other scales of computational drug discovery.

A novel robust adaptive Kalman filter with application to urban vehicle integrated navigation systems

This study addresses the formidable challenge presented by non-Gaussian, time-varying, heavy-tailed process noise in integrated vehicle navigation systems. While robust adaptive filtering methods exist, their efficacy is constrained, and certain algorithms exhibit suboptimal usability due to an excess of a priori parameters. We deduce an adaptive robust Kalman filter, characterizing process noise with a Student's t distribution and measurement noise adhering to a Gaussian distribution. To enhance robustness against unknown or varying noise statistics, our method simultaneously estimates the scale matrix, degrees of freedom parameters, state vector, and measurement noise covariance through the expectation–maximization (EM) algorithm and Bayesian variational inference theory. Rigorous simulations and empirical experiments validate its superior accuracy and stability, requiring minimal a priori parameter information. Vehicle-based experiments further demonstrate positioning error below 2.78 m, velocity error below 0.34 m/s, and heading error less than 1.28° after achieving a steady state. This approach exhibits significant potential for advancing the precision and reliability of vehicle navigation systems.

A dissolved oxygen prediction model based on GRU–N-Beats

Dissolved oxygen is one of the most important water quality parameters in aquaculture, and the level determines whether fish can grow healthily. Since there is a delay in equipment control in the aquaculture environment, dissolved oxygen prediction is needed to reduce the loss due to low dissolved oxygen. To solve the problem of insufficient accuracy and poor interpretability of traditional methods in predicting dissolved oxygen from multivariate water quality parameters, this paper proposes an improved N-Beats-based prediction network. First, the maximum expectation algorithm [expectation–maximization (EM)] was used to fill in the original data by fitting the missing values. Second, the discrete wavelet transform (DWT) was used to reduce the overall noise of the sample, then the gated recurrent unit (GRU) feature extraction network was employed to extract the water quality information from the temporal dimension, the N-Beats was utilized to predict the preprocessed data, and the residual operation through Stack was performed to obtain the prediction results. The improved algorithm overcomes the challenge of insufficient prediction accuracy of the traditional algorithm. The GRU–N-Beats network proposed in this paper can extract features from multivariate time dimensions for prediction. The values of root mean square error (RMSE), mean absolute error (MAE), mean absolute percentage error (MAPE), and R2 for the proposed algorithm were 0.171, 0.120, 0.015, and 0.97, respectively. In particular, they were 28.5%, 32.1%, 51.6%, 24.3%, 14.9%, 36.4%, and 19.3% higher than those of long short-term memory (LSTM), GRU, temporal convolutional network (TCN), LSTM–TCN, PatchTST, back-propagation neural network (BPNN), and N-Beats on RMSE, respectively.

Haplotype determination of the Bombyx mori nucleopolyhedrovirus by Nanopore sequencing and linkage of single nucleotide variants.

Naturally occurring isolates of baculoviruses, such as the Bombyx mori nucleopolyhedrovirus (BmNPV), usually consist of numerous genetically different haplotypes. Deciphering the different haplotypes of such isolates is hampered by the large size of the dsDNA genome, as well as the short read length of next generation sequencing (NGS) techniques that are widely applied for baculovirus isolate characterization. In this study, we addressed this challenge by combining the accuracy of NGS to determine single nucleotide variants (SNVs) as genetic markers with the long read length of Nanopore sequencing technique. This hybrid approach allowed the comprehensive analysis of genetically homogeneous and heterogeneous isolates of BmNPV. Specifically, this allowed the identification of two putative major haplotypes in the heterogeneous isolate BmNPV-Ja by SNV position linkage. SNV positions, which were determined based on NGS data, were linked by the long Nanopore reads in a Position Weight Matrix. Using a modified Expectation-Maximization algorithm, the Nanopore reads were assigned according to the occurrence of variable SNV positions by machine learning. The cohorts of reads were de novo assembled, which led to the identification of BmNPV haplotypes. The method demonstrated the strength of the combined approach of short- and long-read sequencing techniques to decipher the genetic diversity of baculovirus isolates.

Exploring an EM-algorithm for banded regression in computational neuroscience

Abstract Regression is a principal tool for relating brain responses to stimuli or tasks in computational neuroscience. This often involves fitting linear models with predictors that can be divided into groups, such as distinct stimulus feature subsets in encoding models or features of different neural response channels in decoding models. When fitting such models, it can be relevant to allow differential shrinkage of the different groups of regression weights. Here, we explore a framework that allows for straightforward definition and estimation of such models. We present an expectation-maximization algorithm for tuning hyperparameters that control shrinkage of groups of weights. We highlight properties, limitations, and potential use-cases of the model using simulated data. Next, we explore the model in the context of a BOLD fMRI encoding analysis and an EEG decoding analysis. Finally, we discuss cases where the model can be useful and scenarios where regularization procedures complicate model interpretation.

Identification of Wiener state–space models utilizing Gaussian sum smoothing

In this paper, we address the problem of system identification for Wiener state–space models. Our approach is based on the Maximum Likelihood method and the Expectation–Maximization algorithm. In the problem of interest, we model the output nonlinearity as a piecewise polynomial function and we jointly estimate the parameters of the linear system with the coefficients of each polynomial section. In our proposal, the computation of the cost function in the Expectation–Maximization algorithm requires the computation of the joint distribution of the state and the output of the linear system given the output of the nonlinear block. These quantities are obtained from an approximation that leads to a novel Gaussian sum smoothing algorithm. Additionally, we show that our method also addresses the identification of state–space systems in which the output is produced by a known quantizer. We present numerical examples to illustrate the benefits of the proposed identification technique.

A novel residual-based Bayesian expectation–maximization adaptive Kalman filter with inaccurate and time-varying noise covariances

In this study, we introduce a novel residual-based Bayesian expectation–maximization adaptive Kalman filter (RBEMAKF) for dynamic state estimation with inaccurate and time-varying noise covariance matrices. The proposed scheme presents a novel maximum a-posteriori (MAP) estimator for characterizing process and measurement noises, leveraging the residual information derived from the Kalman filter. Simultaneously, the MAP is addressed through the application of the expectation–maximization (EM) algorithm. Subsequently, the standard Kalman filter is executed based on the estimated posterior of process and measurement noises (PMNs) to correct the dynamic state in real-time. Additionally, RBEMAKF is extended to propose Laplacian-RBEMAKF (L-RBEMAKF) and Student’s t-RBEMAKF (ST-RBEMAKF) to accommodate outlier environments. These methods assume Laplacian and Student’s t distributions for the prior of PMNs in RBEMAKF, respectively. Extensive simulations and real-world results demonstrate the effectiveness of the proposed RBEMAKF and its extensions (L-RBEMAKF and ST-RBEMAKF) in dynamic state estimation. The availability of our codes can be found at https://github.com/Gaitxh/RBEMAKF.

Image Segmentation Based on G.O.A for Finding Deformities in Medical and Aura Images

Objectives: The research aims to develop the segmentation model to identify the deformity in the medical images as accurately as possible and plan for better medical treatment. The study is extended to identify the disease before its appearance in the human body through human aura images to support aura imaging in medical diagnosis. Methods: The study used a brain image from the UCI data set and Aura images from the Biowell data set to identify the disease. The segmentation model Bivariate Gaussian Mixture Model (B.G.M.M) was developed. Model parameters are derived using the Expectation Maximization (E.M) Algorithm. The Grasshopper optimization Algorithm (G.O.A) extracts optimal features from the images. The chosen feature is fed as input to the classification model B.G.M.M. Segmentation accuracy is measured using the quality metrics. Findings: The developed approach shows 97% accuracy in identifying the damaged tissues in MRI images and high-intensity energy zones in the aura images, indicating the potential for deformities. Novelty: This study significantly contributes to the field by offering novel solutions for precise and comprehensive image analysis in medical and aura imaging contexts. Keywords: G.O.A, segmentation, G.M.M, E.M, quality metrics, deformity identification, Hue and saturation

OcularSeg: Accurate and Efficient Multi-Modal Ocular Segmentation in Non-Constrained Scenarios

Multi-modal ocular biometrics has recently garnered significant attention due to its potential in enhancing the security and reliability of biometric identification systems in non-constrained scenarios. However, accurately and efficiently segmenting multi-modal ocular traits (periocular, sclera, iris, and pupil) remains challenging due to noise interference or environmental changes, such as specular reflection, gaze deviation, blur, occlusions from eyelid/eyelash/glasses, and illumination/spectrum/sensor variations. To address these challenges, we propose OcularSeg, a densely connected encoder–decoder model incorporating eye shape prior. The model utilizes Efficientnetv2 as a lightweight backbone in the encoder for extracting multi-level visual features while minimizing network parameters. Moreover, we introduce the Expectation–Maximization attention (EMA) unit to progressively refine the model’s attention and roughly aggregate features from each ocular modality. In the decoder, we design a bottom-up dense subtraction module (DSM) to amplify information disparity between encoder layers, facilitating the acquisition of high-level semantic detailed features at varying scales, thereby enhancing the precision of detailed ocular region prediction. Additionally, boundary- and semantic-guided eye shape priors are integrated as auxiliary supervision during training to optimize the position, shape, and internal topological structure of segmentation results. Due to the scarcity of datasets with multi-modal ocular segmentation annotations, we manually annotated three challenging eye datasets captured in near-infrared and visible light scenarios. Experimental results on newly annotated and existing datasets demonstrate that our model achieves state-of-the-art performance in intra- and cross-dataset scenarios while maintaining efficient execution.