Parameter estimation of Gaussian mixture models (GMM) with expectation maximization (EM) algorithm

Abstract

Bayesian Network (BN) is a model that applies Bayes principle with assumption that input variables can be interdependent. BN is described as a graph consisting of nodes and arcs. Node shows variables while arc shows relationship between nodes. Combined probability distribution between nodes in BN is built using gaussian mixture models (GMM) which is a type of density model consisting of components of Gaussian functions. There are 3 of mixture models, probability mixture model, parametric mixture model and continuous mixture. GMM parameters can be estimated using expectation maximization (EM) algorithm. EM algorithm is an iterative method that involves expectation (E-step) and maximization (M-step) and is often used to find estimated value of Maximum Likelihood (ML) of parameters in a probabilistic model, where the model also depends on unknown latent variables. E-step is calculating the expectation value of the log-likelihood function, while M-step maximizes the expected value of the log-likelihood function. Advantage of EM algorithm is that it can solve mixed function parameter estimation problems as well as parameters from incomplete data. EM algorithm can solve the log-likelihood function problem which is difficult to solve by simple analysis by assuming the existence of a value for an additional but hidden parameter.