EM Gradient Research Articles

Note on the Equivalence of Orthogonalizing EM and Proximal Gradient Descent

Xiong et al. (2016) develop a method called orthogonalizing EM (OEM) to solve penalized regression problems for tall data. While OEM is developed in the context of the EM algorithm, we show that it is, in fact, an instance of proximal gradient descent, a popular first-order convex optimization algorithm.

Construction of an epileptic seizure prediction model using a semi-supervised method of generative adversarial and long short term memory network combined with Stockwell transform

To propose a semi-supervised epileptic seizure prediction model (ST-WGAN-GP-Bi-LSTM) to enhance the prediction performance by improving time-frequency analysis of electroencephalogram (EEG) signals, enhancing the stability of the unsupervised feature learning model and improving the design of back-end classifier. Stockwell transform (ST) of the epileptic EEG signals was performed to locate the time-frequency information by adaptive adjustment of the resolution and retaining the absolute phase to obtain the time-frequency inputs. When there was no overlap between the generated data distribution and the real EEG data distribution, to avoid failure of feature learning due to a constant JS divergence, Wasserstein GAN was used as a feature learning model, and the cost function based on EM distance and gradient penalty strategy was adopted to constrain the unsupervised training process to allow the generation of a high-order feature extractor. A temporal prediction model was finally constructed based on a bi-directional long short term memory network (Bi-LSTM), and the classification performance was improved by obtaining the temporal correlation between high-order time-frequency features. The CHB-MIT scalp EEG dataset was used to validate the proposed patient-specific seizure prediction method. The AUC, sensitivity, and specificity of the proposed method reached 90.40%, 83.62%, and 86.69%, respectively. Compared with the existing semi-supervised methods, the propose method improved the original performance by 17.77%, 15.41%, and 53.66%. The performance of this method was comparable to that of a supervised prediction model based on CNN. The utilization of ST, WGAN-GP, and Bi-LSTM effectively improves the prediction performance of the semi-supervised deep learning model, which can be used for optimization of unsupervised feature extraction in epileptic seizure prediction.

Optical multipolar torque in structured electromagnetic fields: On helicity gradient torque, quadrupolar torque, and spin of the field gradient

Structured light mechanically interacts with matter via optical forces and torques. Optical torque is traditionally calculated via the flux of total angular momentum (AM) into a volume enclosing an object. In the work published in [Phys. Rev. A 92, 043843 (2015)], a powerful method was suggested to calculate the optical torque separately from the flux of the spin and the orbital parts of optical AM, rather than the total, providing useful physical insight. However, the method predicted a new type of dipolar torque dependent on the gradient of the helicity density of the optical beam, inconsistent with prior torque calculations. In this work, we first intend to clarify this discrepancy and clear up the confusion. We rederive, from first principles and with detailed derivations, both the traditional dipolar torque using total AM flux, and the spin and orbital torque components based on the corresponding AM contributions, ensuring that their sum agrees with the total torque. We also test our derived analytical expressions against numerical integration, with the exact agreement. We find that ``helicity gradient'' torque terms indeed exist in the spin and orbital components separately, but we present corrected prefactors, such that upon adding them, they cancel out, and the helicity gradient term vanishes from the total dipolar torque, reconciling literature results. In the second part of the work, we derive the analytical expression of the quadrupolar torque, showing that it is proportional to the spin of the EM field gradient, rather than the local EM field spin, as sometimes wrongly assumed in the literature. We provide examples of counter-intuitive situations where the spin of the EM field gradient behaves very differently from the local EM spin. Naively using the local EM field spin leads to wrong predictions of the torque on large particles with strong contributions of quadrupole and higher-order multipoles, especially in a structured incident field.

An Improved Parameter-Estimating Method in Bayesian Networks Applied for Cognitive Diagnosis Assessment.

Bayesian networks (BNs) can be employed to cognitive diagnostic assessment (CDA). Most of the existing researches on the BNs for CDA utilized the MCMC algorithm to estimate parameters of BNs. When EM algorithm and gradient descending (GD) learning method are adopted to estimate the parameters of BNs, some challenges may emerge in educational assessment due to the monotonic constraints (greater skill should lead to better item performance) cannot be satisfied in the above two methods. This paper proposed to train the BN first based on the ideal response pattern data contained in every CDA and continue to estimate the parameters of BN based on the EM or the GD algorithm regarding the parameters based on the IRP training method as informative priors. Both the simulation study and realistic data analysis demonstrated the validity and feasibility of the new method. The BN based on the new parameter estimating method exhibits promising statistical classification performance and even outperforms the G-DINA model in some conditions.

Automatic Chinese Font Generation System Reflecting Emotions Based on Generative Adversarial Network

Manual font design is difficult and requires professional knowledge and skills to perform. Therefore, how to automatically generate the required fonts is a very challenging research task. On the other hand, there are few people who have studied the relationship between fonts and emotions, and common fonts generally cannot reflect emotional information. This paper proposes an Emotional Guidance GAN: an automatic Chinese font generation framework based on Generative Adversarial Network (GAN), which enables the generated fonts to reflect human emotional information. First, an elaborated questionnaire system was developed from Tencent company, which aims to quantitatively figure out the relationship between fonts and emotions. A visual expression recognition part is designed based on the trained model to provide a font generation module with conditional information. Moreover, the Emotional Guidance GAN (EG-GAN) with EM Distance and Gradient Penalty, as well as classification strategies, is proposed to generate new fonts with combined multiple styles that infer by an expression recognition module. The results of the evaluation experiments and the resolution of the synthesized font characters show the credibility of our model.

Left truncated and right censored Weibull data and likelihood inference with an illustration

The Weibull distribution is a very popular distribution for modeling lifetime data. Left truncation and right censoring are often observed in lifetime data. Here, the EM algorithm is applied to estimate the model parameters of the Weibull distribution fitted to data containing left truncation and right censoring. The maximization part of the EM algorithm is carried out using the EM gradient algorithm (Lange, 1995). The Weibull distribution is also fitted using the Newton–Raphson (NR) method. The two methods of estimation are then compared through an extensive Monte Carlo simulation study. The asymptotic variance–covariance matrix of the MLEs under the EM framework is obtained through the missing information principle (Louis, 1982), and asymptotic confidence intervals for the parameters are then constructed. The asymptotic confidence intervals corresponding to the missing information principle and the observed information matrix are compared in terms of coverage probabilities, through a simulation study. Finally, all the methods of inference discussed here are illustrated through some numerical examples.

Likelihood inference for lognormal data with left truncation and right censoring with an illustration

The lognormal distribution is quite commonly used as a lifetime distribution. Data arising from life-testing and reliability studies are often left truncated and right censored. Here, the EM algorithm is used to estimate the parameters of the lognormal model based on left truncated and right censored data. The maximization step of the algorithm is carried out by two alternative methods, with one involving approximation using Taylor series expansion (leading to approximate maximum likelihood estimate) and the other based on the EM gradient algorithm ( Lange, 1995). These two methods are compared based on Monte Carlo simulations. The Fisher scoring method for obtaining the maximum likelihood estimates shows a problem of convergence under this setup, except when the truncation percentage is small. The asymptotic variance–covariance matrix of the MLEs is derived by using the missing information principle ( Louis, 1982), and then the asymptotic confidence intervals for scale and shape parameters are obtained and compared with corresponding bootstrap confidence intervals. Finally, some numerical examples are given to illustrate all the methods of inference developed here.

Maximum Likelihood Estimation for Spatial GLM Models

Spatial generalized linear mixed models are usually used for modelling non-Gaussian and discrete spatial responses. In these models, spatial correlation of the data is usually modelled by spatial latent variables. Although, it is a standard assumption that the latent variables have normal distribution, in practice this assumption may not be valid. The first purpose of this paper is to use a closed skew normal distribution for the spatial latent variables which is more flexible distribution and also includes normal and skew normal distributions. The second is to develop Monte Carlo EM gradient algorithm for maximum likelihood estimation of the model parameters. Then, the performance of the proposed model is illustrated through a simulation study. Finally, the model and algorithm are applied to a case study on a temperature data.

On Estimation and Prediction for Spatial Generalized Linear Mixed Models

We use spatial generalized linear mixed models (GLMM) to model non-Gaussian spatial variables that are observed at sampling locations in a continuous area. In many applications, prediction of random effects in a spatial GLMM is of great practical interest. We show that the minimum mean-squared error (MMSE) prediction can be done in a linear fashion in spatial GLMMs analogous to linear kriging. We develop a Monte Carlo version of the EM gradient algorithm for maximum likelihood estimation of model parameters. A by-product of this approach is that it also produces the MMSE estimates for the realized random effects at the sampled sites. This method is illustrated through a simulation study and is also applied to a real data set on plant root diseases to obtain a map of disease severity that can facilitate the practice of precision agriculture.

Constrained maximum likelihood estimation of two-level covariance structure model via EM type algorithms

In this paper, the constrained maximum likelihood estimation of a two-level covariance structure model with unbalanced designs is considered. The two-level model is reformulated as a single-level model by treating the group level latent random vectors as hypothetical missing-data. Then, the popular EM algorithm is extended to obtain the constrained maximum likelihood estimates. For general nonlinear constraints, the multiplier method is used at theM-step to find the constrained minimum of the conditional expectation. An accelerated EM gradient procedure is derived to handle linear constraints. The empirical performance of the proposed EM type algorithms is illustrated by some artifical and real examples.

4. Estimating Two-Sided Logit Models

Logan (1996a) introduced a new micro-behavioral model of employment opportunity and choice, and a multivariate statistical method based on the micro-behavioral model. This article considers the connection between the behavioral model and the two-sided logit (TSL) statistical method in more detail than the original paper, discussing issues of parameter identification, model constraints, data reduction, and practical estimation. The article compares the EM gradient algorithm used in Logan (1996a) with an accelerated EM gradient algorithm and with a public-domain quasi-Newton algorithm. The latter two algorithms, now incorporated in a single program, greatly enhance the practicality of TSL modeling. Strategies for further development of TSL methods are also considered.

Latent Variable Modeling of Diagnostic Accuracy

Latent class analysis has been applied in medical research to assessing the sensitivity and specificity of diagnostic tests/diagnosticians. In these applications, a dichotomous latent variable corresponding to the unobserved true disease status of the patients is assumed. Associations among multiple diagnostic tests are attributed to the unobserved heterogeneity induced by the latent variable, and inferences for the sensitivities and specificities of the diagnostic tests are made possible even though the true disease status is unknown. However, a shortcoming of this approach to analyses of diagnostic tests is that the standard assumption of conditional independence among the diagnostic tests given a latent class is contraindicated by the data in some applications. In the present paper, models incorporating dependence among the diagnostic tests given a latent class are proposed. The models are parameterized so that the sensitivities and specificities of the diagnostic tests are simple functions of model parameters, and the usual latent class model obtains as a special case. Marginal models are used to account for the dependencies within each latent class. An accelerated EM gradient algorithm is demonstrated to obtain maximum likelihood estimates of the parameters of interest, as well as estimates of the precision of the estimates.

A Gradient Algorithm Locally Equivalent to the Em Algorithm

SUMMARY In many problems of maximum likelihood estimation, it is impossible to carry out either the E-step or the M-step of the EM algorithm. The present paper introduces a gradient algorithm that is closely related to the EM algorithm. This EM gradient algorithm approximately solves the M-step of the EM algorithm by one iteration of Newton's method. Since Newton's method converges quickly, the local properties of the EM gradient algorithm are almost identical with those of the EM algorithm. Any strict local maximum point of the observed likelihood locally attracts the EM and EM gradient algorithm at the same rate of convergence, and near the maximum point the EM gradient algorithm always produces an increase in the likelihood. With proper modification the EM gradient algorithm also exhibits global convergence properties that are similar to those of the EM algorithm. Our proof of global convergence applies and improves existing theory for the EM algorithm. These theoretical points are reinforced by a discussion of three realistic examples illustrating how the EM gradient algorithm can succeed where the EM algorithm is intractable.

Multiphoton ionization and EM field gradient forces

Measurements are reported of the energy of electrons ejected by multiphoton ionization of N 2 and Ar. Electrons with energy > 160 eV were found and the angular distribution using plane polarized light was anisotropic, at least for low electron energies. The data is interpreted in terms of multiphoton ionization followed by acceleration of free electrons by an EM field gradient force.

Trapping of electrons in a spatially inhomogeneous laser beam

Measurements are reported of the energy of electrons ejected by multiphoton ionization of N2 and Ar. Electrons with energy > 160 eV were found and the angular distribution using plane polarized light was anisotropic, at least for low electron energies. The data is interpreted in terms of multiphoton ionization followed by acceleration of free electrons by an EM field gradient force.