Expectation Maximization Research Articles

Super-resolution localization and orientation estimation of multiple dipole sound sources: From a maximum likelihood framework to wind tunnel validation

This paper investigates the identification of multiple dipole sound sources using sound pressures measured from a microphone array. The problem is addressed in the maximum likelihood (ML) framework, where the locations, orientations, and powers of multiple dipole sound sources are unknown parameters to be estimated. By the consistency property of ML, the estimated parameters converge to their actual values, which implies an asymptotically perfect spatial resolution, if a sufficiently high signal-to-noise ratio can be achieved. In order to reduce the dimension of the optimization problem of ML, the contribution of each dipole source to the measured pressures is assumed to be a latent variable and the ML problem is equivalently solved via the expectation–maximization (EM) algorithm, which iteratively and sequentially updates each source contribution and the associated sound source parameters. The number of sound sources can also be determined by the model selection approaches which add a penalty of model dimension to the ML objective function. The proposed method is assessed via a laboratory experiment where the sound field is produced by dipole speakers and a wind tunnel experiment where airframe aerodynamic noise is generated at a high Reynolds number. Experimental results show that the proposed method outperforms existing approaches in the sense of higher spatial resolution, more accurate localization, and the capacity to identify the orientations of multiple dipole sound sources.

Quasi Maximum Likelihood Estimation and Inference of Large Approximate Dynamic Factor Models via the EM algorithm

We study estimation of large Dynamic Factor models implemented through the Expectation Maximization (EM) algorithm, jointly with the Kalman smoother. We prove that as both n and T diverge to infinity: (i) the estimated loadings are \sqrt{T}-consistent and asymptotically normal and equivalent to their Quasi Maximum Likelihood estimates; (ii) the estimated factors are \sqrt{n}-consistent and asymptotically normal and equivalent to their Weighted Least Squares estimates. Moreover, the estimated loadings are asymptotically as efficient as those obtained by Principal Components analysis, while the estimated factors are more efficient if the idiosyncratic covariance is sparse enough.

FusionNGFPE: An Image Fusion Approach Driven by Non-global Fuzzy Pre-enhancement Framework

The majority of prevailing image fusion methods employ a global strategy, often resulting in a reduction of contrast. This study addresses this issue by proposing a novel image fusion approach called FusionNGFPE, specifically designed for the structural characteristics of infrared (IR) imagery. The approach introduces a contrast equalization algorithm based on the Fourth-order Partial Differential Equation (FPDE) to enhance background regions effectively. Considering the inherent differences between IR and visible (VIS) images, we developed a hybrid fusion strategy that combines the Expectation Maximization (EM) algorithm and Principal Component Analysis (PCA). Comparative analysis with state-of-the-art fusion methods shows that our proposed algorithm achieves superior performance in both qualitative and quantitative evaluations. To further demonstrate the practical significance of FusionNGFPE, we integrated this fusion framework into the RGBT target tracking task using the VOT-RGBT and OTCBVS datasets. Extensive comparative experiments confirm that the FusionNGFPE framework integrates seamlessly with the tracking task, significantly improving tracking accuracy across diverse scenarios.

Self-Organizing a Latent Hierarchy of Sketch Patterns for Controllable Sketch Synthesis.

Encoding sketches as Gaussian mixture model (GMM)-distributed latent codes is an effective way to control sketch synthesis. Each Gaussian component represents a specific sketch pattern, and a code randomly sampled from the Gaussian can be decoded to synthesize a sketch with the target pattern. However, existing methods treat the Gaussians as individual clusters, which neglects the relationships between them. For example, the giraffe and horse sketches heading left are related to each other by their face orientation. The relationships between sketch patterns are important messages to reveal cognitive knowledge in sketch data. Thus, it is promising to learn accurate sketch representations by modeling the pattern relationships into a latent structure. In this article, we construct a tree-structured taxonomic hierarchy over the clusters of sketch codes. The clusters with the more specific descriptions of sketch patterns are placed at the lower levels, while the ones with the more general patterns are ranked at the higher levels. The clusters at the same rank relate to each other through the inheritance of features from common ancestors. We propose a hierarchical expectation-maximization (EM)-like algorithm to explicitly learn the hierarchy, jointly with the training of encoder-decoder network. Moreover, the learned latent hierarchy is utilized to regularize sketch codes with structural constraints. Experimental results show that our method significantly improves controllable synthesis performance and obtains effective sketch analogy results.

Fault diagnosis based on incomplete sensor variables with a hierarchical semi-supervised Gaussian mixture classifier

We propose a hierarchical semi-supervised variational semi-Bayesian Gaussian mixture classifier based on the partially incomplete and unlabeled samples for the fault diagnosis of mechanical and electrical systems. These systems are typically complex in structure and are monitored by multiple sensors simultaneously. Some sensor variables in the collected data may be incomplete due to sensor malfunctions or transmission errors. Additionally, labeling the data can be a time-consuming and labor-intensive task, resulting in many unlabeled samples. To address these challenges, the missing sensor variables and the unavailable labels are treated as hidden values and handled within the framework of the Expectation Maximization algorithm. We employ a semi-Bayesian technique with variational inference to estimate the model parameters. Specifically, we introduce prior distribution to the mean and covariance matrix to address the possible singularity of the empirical covariance matrix. The weighting coefficients are left without putting prior distribution so that their values can decay to zero, effectively removing redundant components of the mixture model. The factorized distribution is utilized to approximate the posterior distribution of the model parameters, as well as the missing labels and other latent variables. Numerical and real case studies are carried out to verify the effectiveness of the proposed technique.

PDE-regularised spatial quantile regression

We consider the problem of estimating the conditional quantiles of an unknown distribution from data gathered on a spatial domain. We propose a spatial quantile regression model with differential regularisation. The penalisation involves a partial differential equation defined over the considered spatial domain, that can display a complex geometry. Such regularisation permits, on one hand, to model complex anisotropy and non-stationarity patterns, possibly on the basis of problem-specific knowledge, and, on the other hand, to comply with the complex conformation of the spatial domain. We define an innovative functional Expectation-Maximisation algorithm, to estimate the unknown quantile surface. We moreover describe a suitable discretisation of the estimation problem, and investigate the theoretical properties of the resulting estimator. The performance of the proposed method is assessed by simulation studies, comparing with state-of-the-art techniques for spatial quantile regression. Finally, the considered model is applied to two real data analyses, the first concerning rainfall measurements in Switzerland and the second concerning sea surface conductivity data in the Gulf of Mexico.

Reliability estimation and statistical inference under joint progressively Type-II right-censored sampling for certain lifetime distributions

In this article, the parameter estimation of several commonly used two-parameter lifetime distributions, including the Weibull, inverse Gaussian, and Birnbaum–Saunders distributions, based on joint progressively Type-II right-censored sample is studied. Different numerical methods and algorithms are used to compute the maximum likelihood estimates of the unknown model parameters. These methods include the Newton–Raphson method, the stochastic expectation–maximization (SEM) algorithm, and the dual annealing (DA) algorithm. These estimation methods are compared in terms of accuracy (e.g. the bias and mean squared error), computational time and effort (e.g. the required number of iterations), the ability to obtain the largest value of the likelihood, and convergence issues by means of a Monte Carlo simulation study. Recommendations are made based on the simulated results. A real data set is analyzed for illustrative purposes. These methods are implemented in Python, and the computer programs are available from the authors upon request.

CT image reconstruction: integrating iterative methods with Ml-EM algorithm and deep learning models

Computed Tomography (CT) imaging faces limitations, including low spatial resolution and noise, particularly in low-radiation-dose imaging. To address these challenges, researchers are exploring CT image reconstruction from sinogram data. Sinograms represent X-ray absorption throughout the body, and sophisticated image reconstruction methods, including machine learning algorithms and generative adversarial networks (GANs), can improve precision and resolution without increasing patient radiation exposure. This study proposes an iterative reconstruction approach that combines filters from deep learning models (Convolutional Neural Networks and UNet) with the Maximum Likelihood Expectation Maximization (ML-EM) algorithm and the Enhanced Super-Resolution Generative Adversarial Networks (ESRGAN) model. Our method aims to enhance image quality and reconstruction speed. Experimental results show significant improvements in image quality and resolution, with the proposed method (DL-MLEM-IR-UNET-ESRGAN) achieving an average SSIM of 0.9980 and PSNR of 53.2119, outperforming other methods. Additionally, our method reduces reconstruction time, with an average runtime of 131 seconds.

Image deconvolution using hybrid threshold based on modified L1-clipped penalty in EM framework

Image deconvolution remains a challenging task due to its inherent ill-posedness. While existing algorithms show strong numerical performance, their complexity often complicates analysis and implementation. This paper introduces a computationally efficient image deconvolution method within the expectation maximization (EM) framework. The proposed algorithm alternates between an E-step leveraging the fast Fourier transform (FFT) and an M-step utilizing the discrete wavelet transform (DWT). In the M-step, we introduce a novel L1-clipped penalty to compute the maximum a posteriori (MAP) estimate, resulting in a hybrid threshold that combines the strengths of soft and hard thresholding. This hybrid threshold is mathematically derived, overcoming the high variance of hard-thresholding and the high bias of soft-thresholding, thus optimizing the trade-off between variance and bias. Extensive experiments demonstrate that our method significantly outperforms state-of-the-art techniques in terms of improved signal-to-noise ratio (ISNR) and peak signal-to-noise ratio (PSNR), as well as visual quality. Notably, the proposed method shows average PSNR improvements of 3.49 dB, 4.23 dB, and 1.44 dB for uniform blur and 0.76 dB, 3.57 dB, and 0.66 dB for Gaussian blur on the Set12, BSD68, and Set14 datasets, respectively.

Estimating genetic parameters and trends in growth curve traits of Zandi sheep using the SAEM algorithm.

The objectives of this study were to evaluate the influence of environmental effects on growth curve traits of Zandi lambs and estimate their genetic parameters with the best-fit animal model. For this purpose, live body weight (BW) records (n = 10,607) of 2,519 individuals (which were progeny of 278 rams and 1,485 ewes) were used to estimate genetic effects on growth curve traits from birth to yearling age of Zandi lambs. Using the Stochastic Approximation Expectation Maximization (SAEM) algorithm the growth curve parameters of five different mixed functions (i.e., Brody, Richards, Von Bertalanffy, Gompertz and Logistic) were obtained, then for the most appropriate model the genetic parameters were estimated using a Bayesian approach fitted multivariate animal model and ignoring or including maternal genetic effect. Except Richards model, all other mixed functions used here closely fitted actual BW records (R2 > 0.96). However, the Logistic function provided the best fit in every type. So, studied growth curve traits were estimated asymptotic weight which considered as mature weight (a), rate parameter (b), rate of maturing (k), and age (Ai)/weight (Wi) at the point of inflection. Of the fixed effects studied (i.e., gender, birth type, dam age, season and year of birth), the only non-significant relationship was the effect dam age on b and Ai. Based on the best-fitted model, posterior means of heritability estimates for a, b, k, Wi and Ai were 0.142 ± 0.036, 0.094 ± 0.029, 0.143 ± 0.063, 0.149 ± 0.039 and 0.029 ± 0.013, respectively. Posterior means of genetic correlations between mentioned traits ranged from -0.018 ± 0.069 (b-k) to 0.959 ± 0.029 (a-b), whereas the phenotypic correlation varied from -0.047 ± 0.014 (b-k) to 0.836 ± 0.007 (a-b). It was concluded that the model including only direct additive effect was sufficient to explain the variation in all investigated growth traits of Zandi lambs, selection for these traits results in slow genetic gain (due to the lack of sufficient genetic variation), but it would not be difficult to improve their mature weight and rate of maturing jointly. The results indicate that although the rate of genetic change for mature weight has been small (0.008 ± 0.003 kgyear-1; P < 0.05) but in the favorable direction for this breed.

Remote Dance Action Correction Based on EM and Min-min Algorithms

Remote Dance Action Correction Based on EM and Min-min Algorithms

TabMoE: A General Framework for Diverse Table-Based Reasoning with Mixture-of-Experts

Tables serve as a widely adopted data format, attracting considerable academic interest concerning semantic understanding and logical inference of tables. In recent years, the prevailing paradigm of pre-training and fine-tuning on tabular data has become increasingly prominent in research on table understanding. However, existing table-based pre-training methods frequently exhibit constraints, supporting only single tasks while requiring substantial computational resources, which hinders their efficiency and applicability. In this paper, we introduce the TabMoE, a novel framework based on mixture-of-experts, designed to handle a wide range of tasks involving logical reasoning over tabular data. Each expert within the model specializes in a distinct logical function and is trained through the utilization of a hard Expectation–Maximization algorithm. Remarkably, this framework eliminates the necessity of dependency on tabular pre-training, instead exclusively employing limited task-specific data to significantly enhance models’ inferential capabilities. We conduct empirical experiments across three typical tasks related to tabular data: table-based question answering, table-based fact verification, and table-to-text generation. The experimental results underscore the innovation and feasibility of our framework.

Knowledge discovery of patients reviews on breast cancer drugs: Segmentation of side effects using machine learning techniques

Breast cancer stands as the most frequently diagnosed life-threatening cancer among women worldwide. Understanding patients' drug experiences is essential to improving treatment strategies and outcomes. In this research, we conduct knowledge discovery on breast cancer drugs using patients' reviews. A new machine learning approach is developed by employing clustering, text mining and regression techniques. We first use Latent Dirichlet Allocation (LDA) technique to discover the main aspects of patients' experiences from the patients' reviews on breast cancer drugs. We also use Expectation-Maximization (EM) algorithm to segment the data based on patients' overall satisfaction. We then use the Forward Entry Regression technique to find the relationship between aspects of patients' experiences and drug's effectiveness in each segment. The textual reviews analysis on breast cancer drugs found 8 main side effects: Musculoskeletal Effects, Menopausal Effects, Dermatological Effects, Metabolic Effects, Gastrointestinal Effects, Neurological and Cognitive Effects, Respiratory Effects and Cardiovascular. The results are provided and discussed. The findings of this study are expected to offer valuable insights and practical guidance for prospective patients, aiding them in making informed decisions regarding breast cancer drug consumption.

Accuracy of holmium-166 SPECT/CT quantification over a large range of activities

BackgroundQuantitative imaging is a crucial step for dosimetry in radionuclide therapies. Traditionally, SPECT/CT imaging is quantified based on scanner-specific conversion factors or self-calibration, but recently absolute quantification methods have been introduced in commercial SPECT reconstruction software (Broad Quantification, Siemens Healthineers). In this phantom study we investigate the accuracy of three quantification methods for holmium-166 SPECT/CT imaging, and provide recommendations for clinical dosimetry.MethodsOne cylindrical phantom, filled with a homogeneous holmium-166-chloride activity concentration solution, was imaged at one time point to determine a scanner-specific conversion factor, and to characterize the spatial dependency of the activity concentration recovery. One Jaszczak phantom with six fillable spheres, 10:1 sphere-to-background ratio, was imaged over a large range of holmium-166 activities (61-3130 MBq). The images were reconstructed with either an ordered subset expectation maximization (OSEM, Flash3D-reconstruction; scanner-specific quantification or self-calibration quantification) or an ordered subset conjugate gradient (OSCG, xSPECT-reconstruction; Broad Quantification) algorithm. These three quantification methods were compared for the data of the Jaszczak phantom and evaluated based on whole phantom recovered activity, activity concentration recovery coefficients (ACRC), and recovery curves.ResultsThe activity recovery in the Jaszczak phantom was 28–115% for the scanner-specific, and 57–97% for the Broad Quantification quantification methods, respectively. The self-calibration-based activity recovery is inherently always 100%. The ACRC for the largest sphere (Ø60 mm, ~ 113 mL) ranged over (depending on the activity level) 0.22–0.89, 0.76–0.86, 0.39–0.72 for scanner-specific, self-calibration and Broad Quantification, respectively.ConclusionOf the three investigated quantification methods, the self-calibration technique produces quantitative SPECT images with the highest accuracy in the investigated holmium-166 activity range.

Statistical inference of Poisson censored δ-shock model

In this article, the sufficient statistics, complete statistics, and a uniformly minimum variance unbiased estimate (UMVUE) of the failure parameters of the Poisson censored δ -shock model are obtained, which are based on the sample data of shock arrival time. The maximum likelihood estimates of the shock parameters are also obtained by using the EM algorithm, which is based on the total number of observed shock samples. Finally, the estimation errors and convergence rate of the EM iteration are simulated numerically using Matlab R2016b.

A 3D imaging system for dosimetry in FLASH radiotherapy

This study examines the feasibility of reconstructing the spatial dose distribution delivered by a 9 MeV electron beam within a plastic scintillator block using optical tomography. The system is designed for use in 3D-dosimetry for clinical applications in quality assurance and beam characterization, especially with FLASH-beam sources. The proposed detector reconstructs the 3D dose distribution map employing a monolith with 10 cm side, imaged in four different projections by scientific cameras. The 3D dose distribution simulated with Geant4 was reconstructed using a maximum likelihood expectation maximization algorithm applied to the lateral projection, resulting in a longitudinal image with a standard deviation of 0.2 Gy. Furthermore, a preliminary experimental test has been performed.

A Highly Efficient Compressive Sensing Algorithm Based on Root-Sparse Bayesian Learning for RFPA Radar

Off-grid issues and high computational complexity are two major challenges faced by sparse Bayesian learning (SBL)-based compressive sensing (CS) algorithms used for random frequency pulse interval agile (RFPA) radar. Therefore, this paper proposes an off-grid CS algorithm for RFPA radar based on Root-SBL to address these issues. To effectively cope with off-grid issues, this paper derives a root-solving formula inspired by the Root-SBL algorithm for velocity parameters applicable to RFPA radar, thus enabling the proposed algorithm to directly solve the velocity parameters of targets during the fine search stage. Meanwhile, to ensure computational feasibility, the proposed algorithm utilizes a simple single-level hierarchical prior distribution model and employs the derived root-solving formula to avoid the refinement of velocity grids. Moreover, during the fine search stage, the proposed algorithm combines the fixed-point strategy with the Expectation-Maximization algorithm to update the hyperparameters, further reducing computational complexity. In terms of implementation, the proposed algorithm updates hyperparameters based on the single-level prior distribution to approximate values for the range and velocity parameters during the coarse search stage. Subsequently, in the fine search stage, the proposed algorithm performs a grid search only in the range dimension and uses the derived root-solving formula to directly solve for the target velocity parameters. Simulation results demonstrate that the proposed algorithm maintains low computational complexity while exhibiting stable performance for parameter estimation in various multi-target off-grid scenarios.

Improving community detection in blockmodel by distance-based observation selection

Community detection is an important research topic in complex systems and has plenty of applications in real-world networks. Probabilistic methods, such as the Expectation–Maximization (EM), are developed to classify nodes that have similar connection patterns in a network based on blockmodels. However, the detection procedures in these models are typically started from randomly generated initial community distributions without prior knowledge. In biological and social networks, there are practical measures to obtain prior knowledge for a subset of nodes, such as local observations. These facts lead us to question how we can select a subset of nodes with known community labels to enhance the accuracy of the EM method. The current selection methods lack the relationship between detection accuracy and structural characteristics and most approaches consider the nodes as the center of communities, which is not suitable for block models. In this paper, we first study the relationships between the structural distance and detection accuracy without prior knowledge. Then we propose a distance-based indicator to describe the performance of the observation node set in the EM method. Finally, we introduce a scoring method based on the indicator to select a partial observation set, improving the accuracy of community detection using the EM method. Empirical results from synthetic and real-world networks corroborate that the proposed indicator could contribute to a better performance in kinds of scenarios.

Short-term prediction of horizontal winds in the mesosphere and lower thermosphere over coastal Peru using a hybrid model

The mesosphere and lower thermosphere (MLT) are transitional regions between the lower and upper atmosphere. The MLT dynamics can be investigated using wind measurements conducted with meteor radars. Predicting MLT winds could help forecast ionospheric parameters, which has many implications for global communications and geo-location applications. Several literature sources have developed and compared predictive models for wind speed estimation. However, in recent years, hybrid models have been developed that significantly improve the accuracy of the estimates. These integrate time series decomposition and machine learning techniques to achieve more accurate short-term predictions. This research evaluates a hybrid model that is capable of making a short-term prediction of the horizontal winds between 80 and 95 km altitudes on the coast of Peru at two locations: Lima (12°S, 77°W) and Piura (5°S, 80°W). The model takes a window of 56 data points as input (corresponding to 7 days) and predicts 16 data points as output (corresponding to 2 days). First, the missing data problem was analyzed using the Expectation Maximization algorithm (EM). Then, variational mode decomposition (VMD) separates the components that dominate the winds. Each resulting component is processed separately in a Long short-term memory (LSTM) neural network whose hyperparameters were optimized using the Optuna tool. Then, the final prediction is the sum of the predicted components. The efficiency of the hybrid model is evaluated at different altitudes using the root mean square error (RMSE) and Spearman’s correlation (r). The hybrid model performed better compared to two other models: the persistence model and the dominant harmonics model. The RMSE ranged from 10.79 to 27.04 m s−1, and the correlation ranged from 0.55 to 0.94. In addition, it is observed that the prediction quality decreases as the prediction time increases. The RMSE at the first step reached 6.04 m s−1 with a correlation of 0.99, while at the sixteenth step, the RMSE increased up to 30.84 m s−1 with a correlation of 0.5.

Unsupervised machine learning with different sampling strategies and topographic factors for distinguishing between landslide source and runout areas to improve landslide inventory production

This study derived 12 topographical and hydrological factors related to landslides from a 10-m digital elevation model. Three unsupervised machine learning algorithms were employed to distinguish between the features of landslide sources and runout areas for Typhoon Morakot. Two sampling strategies were designed in this study. The first strategy involved creating a data set by pairing landslide sources and runout areas on the basis of the size of polygonal areas recorded in the inventory and then exploring the effectiveness of feature separation with k-means++ as the primary method and EM (expectation maximization) and DBSCAN (Density-Based Spatial Clustering of Applications with Noise) as supplementary methods. Because of practical challenges associated with the inability to determine whether remote sensing results correspond to landslide sources or runout areas, the first sampling strategy was used to test the feasibility of the adopted algorithms. In the second sampling strategy, the effectiveness of feature separation was tested by dividing polygon samples into several intervals on the basis of their sizes, thereby verifying the practicability of the adopted algorithms in real-world operations. This study verified that combining unsupervised machine learning algorithms with topographic factors facilitates the effective separation of landslide sources and runout areas, thereby enhancing the quality of landslide inventories and reducing the cost of landslide inventory creation. Moreover, it indicated that meter-scale topographic data yield classification results similar to those obtained from manually created landslide inventories based on centimeter-scale stereo aerial photos. The proposed method effectively balances the cost and quality of landslide inventories.