Expectation-maximization Framework Research Articles

Identifying the Most Probable Transition Path with Constant Advance Replicas.

Locating plausible transition paths and enhanced sampling of rare events are fundamental to understanding the functional dynamics of biomolecules. Here, a constraint-based constant advance replicas (CAR) formalism of reaction paths is reported for identifying the most probable transition path (MPTP) between two given states. We derive the temporal-integrated effective dynamics governing the projected subsystem under the holonomic CAR path constraints and show that a dynamical action functional can be defined and used for optimizing the MPTP. We further demonstrate how the CAR MPTP can be located by asymptotically minimizing an upper bound of the CAR action functional using a variational expectation-maximization framework. Essential thermodynamics and kinetic observables are retrieved by integrating the boxed molecular dynamics on the CAR MPTP using a newly proposed adaptive reflecting boundary condition. The efficiency of the proposed method is demonstrated for the Müller potential, the alanine dipeptide isomerization, and the DNA base pairing transition (Watson-Crick to Hoogsteen) in explicit solvent. The CAR representation of transition paths constitutes a robust and extensible platform that can be combined with diverse enhanced sampling methods to aid future flexible and reliable biomolecular simulations.

A unified EM framework for estimation and inference of normal ogive item response models.

Normal ogive (NO) models have contributed substantially to the advancement of item response theory (IRT) and have become popular educational and psychological measurement models. However, estimating NO models remains computationally challenging. The purpose of this paper is to propose an efficient and reliable computational method for fitting NO models. Specifically, we introduce a novel and unified expectation-maximization (EM) algorithm for estimating NO models, including two-parameter, three-parameter, and four-parameter NO models. A key improvement in our EM algorithm lies in augmenting the NO model to be a complete data model within the exponential family, thereby substantially streamlining the implementation of the EM iteration and avoiding the numerical optimization computation in the M-step. Additionally, we propose a two-step expectation procedure for implementing the E-step, which reduces the dimensionality of the integration and effectively enables numerical integration. Moreover, we develop a computing procedure for estimating the standard errors (SEs) of the estimated parameters. Simulation results demonstrate the superior performance of our algorithm in terms of its recovery accuracy, robustness, and computational efficiency. To further validate our methods, we apply them to real data from the Programme for International Student Assessment (PISA). The results affirm the reliability of the parameter estimates obtained using our method.

Image deconvolution using hybrid threshold based on modified L1-clipped penalty in EM framework

Image deconvolution remains a challenging task due to its inherent ill-posedness. While existing algorithms show strong numerical performance, their complexity often complicates analysis and implementation. This paper introduces a computationally efficient image deconvolution method within the expectation maximization (EM) framework. The proposed algorithm alternates between an E-step leveraging the fast Fourier transform (FFT) and an M-step utilizing the discrete wavelet transform (DWT). In the M-step, we introduce a novel L1-clipped penalty to compute the maximum a posteriori (MAP) estimate, resulting in a hybrid threshold that combines the strengths of soft and hard thresholding. This hybrid threshold is mathematically derived, overcoming the high variance of hard-thresholding and the high bias of soft-thresholding, thus optimizing the trade-off between variance and bias. Extensive experiments demonstrate that our method significantly outperforms state-of-the-art techniques in terms of improved signal-to-noise ratio (ISNR) and peak signal-to-noise ratio (PSNR), as well as visual quality. Notably, the proposed method shows average PSNR improvements of 3.49 dB, 4.23 dB, and 1.44 dB for uniform blur and 0.76 dB, 3.57 dB, and 0.66 dB for Gaussian blur on the Set12, BSD68, and Set14 datasets, respectively.

An Expectation-Maximization framework for Personalized Itinerary Recommendation with POI Categories and Must-see POIs

In this paper, we introduce a novel deterministic method based on Expectation Maximization (EM) to solve the rather complex problem of designing a tourist trip or Personalized Itinerary Recommendation (PIR). PIR objective is to recommend a personalized tour consisting of successive Points of Interest (POIs), which maximizes user satisfaction and respects user time-frame constraints. On top of that, the POIs are divided into categories, in order for travelers to be able to set limits on the maximum (and minimum) number of POIs that belong to one category and are included in the itinerary. In the proposed framework, emphasis is given on the POIs sequence selection, which exploits the customized POI recommendations offered by a recommender system. Additionally, the proposed methodology with POIs categories is able to solve the TourMustSee problem, so that the tour includes a set of POIs that must be visited. The proposed system has been successfully incorporated into a mobile app, offering a complete tourist trip design. The high performance, resilience, and computational efficiency of the proposed framework are demonstrated by experimental findings and comparisons to existing approaches on numerous synthetic and real datasets.

Semi-Supervised Mixture Learning for Graph Neural Networks With Neighbor Dependence.

A graph neural network (GNN) is a powerful architecture for semi-supervised learning (SSL). However, the data-driven mode of GNNs raises some challenging problems. In particular, these models suffer from the limitations of incomplete attribute learning, insufficient structure capture, and the inability to distinguish between node attribute and graph structure, especially on label-scarce or attribute-missing data. In this article, we propose a novel framework, called graph coneighbor neural network (GCoNN), for node classification. It is composed of two modules: GCoNN Γ and GCoNN Γ° . GCoNN Γ is trained to establish the fundamental prototype for attribute learning on labeled data, while GCoNN Γ° learns neighbor dependence on transductive data through pseudolabels generated by GCoNN Γ . Next, GCoNN Γ is retrained to improve integration of node attribute and neighbor structure through feedback from GCoNN Γ° . GCoNN tends to convergence iteratively using such an approach. From a theoretical perspective, we analyze this iteration process from a generalized expectation-maximization (GEM) framework perspective which optimizes an evidence lower bound (ELBO) by amortized variational inference. Empirical evidence demonstrates that the state-of-the-art performance of the proposed approach outperforms other methods. We also apply GCoNN to brain functional networks, the results of which reveal response features across the brain which are physiologically plausible with respect to known language and visual functions.

Likelihood-based inference for semi-parametric transformation cure models with interval censored data

A simple yet effective way of modeling survival data with cure fraction is by considering Box-Cox transformation cure model (BCTM) that unifies mixture and promotion time cure models. In this article, we numerically study the statistical properties of the BCTM when applied to interval censored data. Time-to-events associated with susceptible subjects are modeled through proportional hazards structure that allows for non-homogeneity across subjects, where the baseline hazard function is estimated by distribution-free piecewise linear function with varied degrees of non-parametricity. Due to missing cured statuses for right censored subjects, maximum likelihood estimates of model parameters are obtained by developing an expectation-maximization (EM) algorithm. Under the EM framework, the conditional expectation of the complete data log-likelihood function is maximized by considering all parameters (including the Box-Cox transformation parameter α) simultaneously, in contrast to conventional profile-likelihood technique of estimating α. The robustness and accuracy of the model and estimation method are established through a detailed simulation study under various parameter settings, and an analysis of real-life data obtained from a smoking cessation study.

A variational expectation-maximization framework for balanced multi-scale learning of protein and drug interactions

Protein functions are characterized by interactions with proteins, drugs, and other biomolecules. Understanding these interactions is essential for deciphering the molecular mechanisms underlying biological processes and developing new therapeutic strategies. Current computational methods mostly predict interactions based on either molecular network or structural information, without integrating them within a unified multi-scale framework. While a few multi-view learning methods are devoted to fusing the multi-scale information, these methods tend to rely intensively on a single scale and under-fitting the others, likely attributed to the imbalanced nature and inherent greediness of multi-scale learning. To alleviate the optimization imbalance, we present MUSE, a multi-scale representation learning framework based on a variant expectation maximization to optimize different scales in an alternating procedure over multiple iterations. This strategy efficiently fuses multi-scale information between atomic structure and molecular network scale through mutual supervision and iterative optimization. MUSE outperforms the current state-of-the-art models not only in molecular interaction (protein-protein, drug-protein, and drug-drug) tasks but also in protein interface prediction at the atomic structure scale. More importantly, the multi-scale learning framework shows potential for extension to other scales of computational drug discovery.

Hybrid Thresholding for Image Deconvolution in Expectation Maximization framework

ABSTRACT In this study, we proposed an image deconvolution method in the expectation maximization (EM) framework. This method involves two steps: (i) E-step: utilizing the fast Fourier transform (FFT) for computationally efficient inversion of the convolution operator and (ii) M-step: employing the discrete wavelet transform (DWT) for estimating the original image from the image obtained in the E-step. In M-step, we proposed a modified L1-clipped penalty resulting in a hybrid thresholding scheme that integrates conventional hard and soft thresholds. This hybrid threshold ameliorates the inherent bias-variance trade-offs associated with traditional hard and soft thresholding schemes. The mathematical expressions for the risk, bias, and variance of the proposed hybrid threshold are derived and the performance is evaluated through simulation. Experimental results show that the proposed method achieves optimal values for variance and bias, thereby minimizing the risk. Moreover, the proposed method outperforms state-of-the-art methods in terms of performance metrics: PSNR, ISNR, and SSIM.

A Nonlinear Quantile Regression for Accelerated Destructive Degradation Testing Data

Traditional regression approaches to accelerated destructive degradation test (ADDT) data have modeled the mean curve as representative. However, maximum likelihood estimates (MLEs) of the mean model are likely to be biased when the data are non-Gaussian or highly skewed. The median model can be an alternative for skewed degradation data. In this work, we introduce a nonlinear quantile regression (QR) approach for estimating quantile curves of ADDT data. We propose an iterative QR algorithm that uses the generalized expectation-maximization (GEM) framework to estimate the parameters of the nonlinear QR ADDT model, based on the asymmetric Laplace distribution to accommodate non-Gaussian and skewed errors. Using the asymptotic properties of the QR parameter estimates, we estimate variance-covariance matrix for the τ th QR parameters using order statistics and bootstrap methods. We propose a new prediction method of the quantile of the failure-time distribution in the normal use condition. Confidence intervals for the quantiles of the failure-time distribution are constructed using the parametric bootstrap method. The proposed model is illustrated using an industrial application and compared with the existing model. Various quantile curve estimates derived using the QR ADDT model provide a more flexible modeling framework than the traditional mean ADDT modeling approach.

Formula: see text]-[Formula: see text]: A Nonparametric Model for Neural Power Spectra Decomposition.

The power spectra estimated from the brain recordings are the mixed representation of aperiodic transient activity and periodic oscillations, i.e., aperiodic component (AC) and periodic component (PC). Quantitative neurophysiology requires precise decomposition preceding parameterizing each component. However, the shape, statistical distribution, scale, and mixing mechanism of AC and PCs are unclear, challenging the effectiveness of current popular parametric models such as FOOOF, IRASA, BOSC, etc. Here, ξ- π was proposed to decompose the neural spectra by embedding the nonparametric spectra estimation with penalized Whittle likelihood and the shape language modeling into the expectation maximization framework. ξ- π was validated on the synthesized spectra with loss statistics and on the sleep EEG and the large sample iEEG with evaluation metrics and neurophysiological evidence. Compared to FOOOF, both the simulation presenting shape irregularities and the batch simulation with multiple isolated peaks indicated that ξ- π improved the fit of AC and PCs with less loss and higher F1-score in recognizing the centering frequencies and the number of peaks; the sleep EEG revealed that ξ- π produced more distinguishable AC exponents and improved the sleep state classification accuracy; the iEEG showed that ξ- π approached the clinical findings in peak discovery. Overall, ξ- π offered good performance in the spectra decomposition, which allows flexible parameterization using descriptive statistics or kernel functions. ξ- π is a seminal tool for brain signal decoding in fields such as cognitive neuroscience, brain-computer interface, neurofeedback, and brain diseases.

Clustering-Guided Twin Contrastive Learning for Endomicroscopy Image Classification.

Learning better representations is essential in medical image analysis for computer-aided diagnosis. However, learning discriminative semantic features is a major challenge due to the lack of large-scale well-annotated datasets. Thus, how can we learn a well-structured categorizable embedding space in limited-scale and unlabeled datasets? In this paper, we proposed a novel clustering-guided twin-contrastive learning framework (CTCL) that learns the discriminative representations of probe-based confocal laser endomicroscopy (pCLE) images for gastrointestinal (GI) tumor classification. Compared with traditional contrastive learning, in which only two randomly augmented views of the same instance are considered, the proposed CTCL aligns more semantically related and class-consistent samples by clustering, which improved intra-class tightness and inter-class variability to produce more informative representations. Furthermore, based on the inherent properties of CLE (geometric invariance and intrinsic noise), we proposed to regard CLE images with any angle rotation and CLE images with different noises as the same instance, respectively, for increased variability and diversity of samples. By optimizing CTCL in an end-to-end expectation-maximization framework, comprehensive experimental results demonstrated that CTCL-based visual representations achieved competitive performance on each downstream task as well as more robustness and transferability compared with existing state-of-the-art SSL and supervised methods. Notably, CTCL achieved 75.60%/78.45% and 64.12%/77.37% top-1 accuracy on the linear evaluation protocol and few-shot classification downstream tasks, respectively, which outperformed the previous best results by 1.27%/1.63% and 0.5%/3%, respectively. The proposed method holds great potential to assist pathologists in achieving an automated, fast, and high-precision diagnosis of GI tumors and accurately determining different stages of tumor development based on CLE images.

A Bayesian Approach Toward Robust Multidimensional Ellipsoid-Specific Fitting.

This work presents a novel and effective method for fitting multidimensional ellipsoids (i.e., ellipsoids embedded in [Formula: see text]) to scattered data in the contamination of noise and outliers. Unlike conventional algebraic or geometric fitting paradigms that assume each measurement point is a noisy version of its nearest point on the ellipsoid, we approach the problem as a Bayesian parameter estimate process and maximize the posterior probability of a certain ellipsoidal solution given the data. We establish a more robust correlation between these points based on the predictive distribution within the Bayesian framework, i.e., considering each model point as a potential source for generating each measurement. Concretely, we incorporate a uniform prior distribution to constrain the search for primitive parameters within an ellipsoidal domain, ensuring ellipsoid-specific results regardless of inputs. We then establish the connection between measurement point and model data via Bayes' rule to enhance the method's robustness against noise. Due to independent of spatial dimensions, the proposed method not only delivers high-quality fittings to challenging elongated ellipsoids but also generalizes well to multidimensional spaces. To address outlier disturbances, often overlooked by previous approaches, we further introduce a uniform distribution on top of the predictive distribution to significantly enhance the algorithm's robustness against outliers. Thanks to the uniform prior, our maximum a posterior probability coincides with a more tractable maximum likelihood estimation problem, which is subsequently solved by a numerically stable Expectation Maximization (EM) framework. Moreover, we introduce an ε-accelerated technique to expedite the convergence of EM considerably. We also investigate the relationship between our algorithm and conventional least-squares-based ones, during which we theoretically prove our method's superior robustness. To the best of our knowledge, this is the first comprehensive method capable of performing multidimensional ellipsoid-specific fitting within the Bayesian optimization paradigm under diverse disturbances. We evaluate it across lower and higher dimensional spaces in the presence of heavy noise, outliers, and substantial variations in axis ratios. Also, we apply it to a wide range of practical applications such as microscopy cell counting, 3D reconstruction, geometric shape approximation, and magnetometer calibration tasks. In all these test contexts, our method consistently delivers flexible, robust, ellipsoid-specific performance, and achieves the state-of-the-art results.

Bayesian Inference of Hidden Cognitive Performance and Arousal States in Presence of Music.

Goal: Poor arousal management may lead to reduced cognitive performance. Specifying a model and decoder to infer the cognitive arousal and performance contributes to arousal regulation via non-invasive actuators such as music. Methods: We employ a Bayesian filtering approach within an expectation-maximization framework to track the hidden states during the [Formula: see text]-back task in the presence of calming and exciting music. We decode the arousal and performance states from the skin conductance and behavioral signals, respectively. We derive an arousal-performance model based on the Yerkes-Dodson law. We design a performance-based arousal decoder by considering the corresponding performance and skin conductance as the observation. Results: The quantified arousal and performance are presented. The existence of Yerkes-Dodson law can be interpreted from the arousal-performance relationship. Findings display higher matrices of performance within the exciting music. Conclusions: The performance-based arousal decoder has a better agreement with the Yerkes-Dodson law. Our study can be implemented in designing non-invasive closed-loop systems.

Graph Decoupling Attention Markov Networks for Semisupervised Graph Node Classification.

Graph neural networks (GNNs) have been ubiquitous in graph node classification tasks. Most GNN methods update the node embedding iteratively by aggregating its neighbors' information. However, they often suffer from negative disturbances, due to edges connecting nodes with different labels. One approach to alleviate this negative disturbance is to use attention to learn the weights of aggregation, but current attention-based GNNs only consider feature similarity and suffer from the lack of supervision. In this article, we consider label dependency of graph nodes and propose a decoupling attention mechanism to learn both hard and soft attention. The hard attention is learned on labels for a refined graph structure with fewer interclass edges so that the aggregation's negative disturbance can be reduced. The soft attention aims to learn the aggregation weights based on features over the refined graph structure to enhance information gains during message passing. Particularly, we formulate our model under the expectation-maximization (EM) framework, and the learned attention is used to guide label propagation in the M-step and feature propagation in the E-step, respectively. Extensive experiments are performed on six well-known benchmark graph datasets to verify the effectiveness of the proposed method.

A Neural Expectation-Maximization Framework for Noisy Multi-Label Text Classification

Multi-label text classification (MLTC) has a wide range of real-world applications. Neural networks recently promoted the performance of MLTC models. Training these neural-network models relies on sufficient accurately labelled data. However, manually annotating large-scale multi-label text classification datasets is expensive and impractical for many applications. Weak supervision techniques have thus been developed to reduce the cost of annotating text corpus. However, these techniques introduce noisy labels into the training data and may degrade the model performance. This paper aims to deal with such noise-label problems in MLTC in both single-instance and multi-instance settings. We build a novel Neural Expectation-Maximization Framework (nEM) that combines neural networks with probabilistic modelling. The nEM framework produces text representations using neural-network text encoders and is optimized with the Expectation-Maximization algorithm. It naturally considers the noisy labels during learning by iteratively updating the model parameters and estimating the distribution of the ground-truth labels. We evaluate our nEM framework in multi-instance noisy MLTC on a benchmark relation extraction dataset constructed by distant supervision and in single-instance noisy MLTC on synthetic noisy datasets constructed by keywords supervision and label flipping. The experimental results demonstrate that nEM significantly improves upon baseline models in both single-instance and multi-instance noisy MLTC tasks. The experiment analysis suggests that our nEM framework efficiently reduces the noisy labels in MLTC datasets and significantly improves model performance.

Laplace Distribution Based Online Identification of Linear Systems With Robust Recursive Expectation–Maximization Algorithm

The robust online identification problem of linear systems is considered in this paper using a faster robust recursive expectation-maximization (RREM) framework. To improve the convergence rate, the outliers, which would deteriorate the identified models, are accommodated with a Laplace distribution instead of Student's t-distribution. Then, the recursive transformation of the maximum likelihood function is realized with a recursive Q-function. The extensively recognized autoregressive exogenous (ARX) models are used for the description of general linear systems. As a result, the unknown parameters, including the regression coefficient vector of the ARX models, the variance of the noise without outliers, and the scale parameter of the Laplace distribution, are determined in a recursive manner. The performance of the proposed approach is tested with a simulated continuous fermentation reactor (CFR) system example and a coupled-tank experiment.

Phasor data correction and transmission system state estimation under spoofing attacks

Cyberinfrastructure (e.g., sensors, actuators and the associated communication network) has become an integral part of our modern power grid. While these cyber technologies enhance situational awareness and operational efficiency, they also expose the physical system to cyber-attacks. In this paper, we consider the problem of transmission system state estimation based on measurements from a number of PMUs. In this context, two PMU data integrity attacks namely, Time Synchronization Attack (TSA) and Man-in-the-Middle (MitM) attacks that can potentially cause a severe impact on the grid, are analyzed. Specifically, we propose a novel method based on an alternate expectation–maximization framework to mitigate the effects of these attacks on the state estimation process. Numerical tests are conducted on IEEE-14, 30 and 118 bus systems with different attack scenarios to validate the developed method. Unlike existing works, the proposed algorithm provides accurate state estimates without any prior knowledge of the location of the attack, the number of meters being attacked, or the magnitude of the attack parameter.

A Point-Process Approach for Tracking Valence using a Respiration Belt.

Emotional valence is difficult to be inferred since it is related to several psychological factors and is affected by inter- and intra-subject variability. Changes in emotional valence have been found to cause a physiological response in respiration signals. In this study, we propose a state-space model and decode the valence by analyzing a person's respiration pattern. Particularly, we generate a binary point process based on features that are indicative of changes in respiration pattern as a result of an emotional valence response. High valence is typically associated with faster and deeper breathing. As a result, (i)depth of breath, (ii)rate of respiration, and (iii) breathing cycle time are indicators of high valence and used to generate the binary point process representing underlying neural stimuli associated with changes in valence. We utilize an expectation-maximization (EM) framework to decode a hidden valence state and the associated valence index. This predicted valence state is compared to self-reported valence ratings to optimize the parameters and determine the accuracy of the model. The accuracy of the model in predicting high and low valence events is found to be 77% and 73%, respectively. Our study can be applied towards the long term analysis of valence. Additionally, it has applications in a closed-loop system procedures and wearable design paradigm to track and regulate the emotional valence.

Channel Tracking and AoA Acquisition in Quantized Millimeter Wave MIMO Systems

This paper considers the channel tracking and angle of arrival (AoA) acquisition for millimeter wave (mmWave) massive multiple-input multiple-output (MIMO) systems, in which each antenna at the base station is equipped with low-resolution analog-to-digital converters (ADCs) to quantize the received signals. We utilize the time-varying model to capture the sparsity and temporal correlation of the mmWave channel, and an offgrid model is incorporated for an accurate AoA acquisition. Essentially, the beamspace channel tracking is a quantized sparse Bayesian learning problem, which is solved under the expectation maximization (EM) framework. We first employ the Bussgang decomposition to linearize quantized signals, based on which, the Kalman filtering (KF) is adapted for the statistics' update in the expectation step. In this way, we propose a Bussgang joint channel tracking and data detection (BJ-CTDD) algorithm, in which the detected data symbols are reused to enhance the tracking without extra pilot overhead. To further reduce the computational burden caused by the KF, a variational inference joint channel tracking and data detection (VIJ-CTDD) algorithm is proposed. Finally, extensive simulations validate the superiority of the proposed BJ-CTDD and VIJ-CTDD algorithms over several existing works.

GMDNet: A Graph-Based Mixture Density Network for Estimating Packages’ Multimodal Travel Time Distribution

In the logistics network, accurately estimating packages' Travel Time Distribution (TTD) given the routes greatly benefits both consumers and platforms. Although recent works perform well in predicting an expected time or a time distribution in a road network, they could not be well applied to estimate TTD in logistics networks. Because TTD prediction in the logistics network requires modeling packages' multimodal TTD (MTTD, i.e., there can be more than one likely output with a given input) while leveraging the complex correlations in the logistics network. To this end, this work opens appealing research opportunities in studying MTTD learning conditioned on graph-structure data by investigating packages' travel time distribution in the logistics network. We propose a Graph-based Mixture Density Network, named GMDNet, which takes the benefits of both graph neural network and mixture density network for estimating MTTD conditioned on graph-structure data (i.e., the logistics network). Furthermore, we adopt the Expectation-Maximization (EM) framework in the training process to guarantee local convergence and thus obtain more stable results than gradient descent. Extensive experiments on two real-world datasets demonstrate the superiority of our proposed model.