Abstract
The Expectation-Maximization Algorithm (EM) is a widely used method allowing to estimate the maximum likelihood of models involving latent variables. When the Expectation step cannot be computed easily, one can use stochastic versions of the EM such as the Stochastic Approximation EM. This algorithm, however, has the drawback to require the joint likelihood to belong to the curved exponential family. To overcome this problem, [16] introduced a rewriting of the model which “exponentializes” it by considering the parameter as an additional latent variable following a Normal distribution centered on the newly defined parameters and with fixed variance. The likelihood of this new exponentialized model now belongs to the curved exponential family. Although often used, there is no guarantee that the estimated mean is close to the maximum likelihood estimate of the initial model. In this paper, we quantify the error done in this estimation while considering the exponentialized model instead of the initial one. By verifying those results on an example, we see that a trade-off must be made between the speed of convergence and the tolerated error. Finally, we propose a new algorithm allowing a better estimation of the parameter in a reasonable computation time to reduce the bias.
Highlights
With the increase of data, parametric statistical models have become a crucial tool for data analysis and understanding
It is interesting to notice that, even if this proof is done in the context of the Stochastic Approximation Expectation Maximization (SAEM) algorithm, the same results can be obtained for the MCMC-SAEM [15] as well as for the Approximate SAEM [2]
We have proved that the exponentialization process does not converge in general towards the maximum likelihood of the initial model using the SAEM or SAEM-MCMC algorithm
Summary
With the increase of data, parametric statistical models have become a crucial tool for data analysis and understanding. The Expectation Maximization (EM) algorithm provides a numerical process to answer this problem by computing iteratively a sequence of estimates (θn)n∈N which, under several conditions (see [12, 34]), converges towards the maximum likelihood estimate It proceeds in two steps for each iteration k. One needs to compute the gradient descent steps and compute the stochastic approximation of the complete likelihood while this function may not have a simple form To solve this problem, [16] propose to transform the initial model to make it curved exponential. For σ too small, despite the theoretical guarantees, the numerical convergence is difficult to obtain To overcome this problem, we will present a new algorithm allowing a better estimation of the initial parameter θ in a reasonable computation time.
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
