Maximum Likelihood Estimation of Latent Variable Models by SMC with Marginalization and Data Cloning

Abstract

A data-cloning SMC2 maximum likelihood estimation algorithm is proposed as a general-purpose optimization routine for models with latent variables. Our algorithm first marginalizes out latent variables by applying one layer of SMC at a fixed parameter value, and then estimates the model parameters by another layer of SMC utilizing density-tempering. Data-cloning is employed to effectively reduce Monte Carlo errors inherent in the SMC2 algorithm, and also to address multi-modality present in typical latent variable models. This new method has wide applicability and can be massively parallelized to take advantage of typical computers today.