Comparison of approximation methods to Kullback–Leibler divergence between Gaussian mixture models for satellite image retrieval

Abstract

ABSTRACTAs a probabilistic distance between two probability density functions, Kullback–Leibler divergence is widely used in many applications, such as image retrieval and change detection. Unfortunately, for some models, e.g., Gaussian Mixture Models (GMMs), Kullback–Leibler divergence is not analytically tractable. One has to resort to approximation methods. A number of methods have been proposed to address this issue. In this article, we compare seven methods, namely Monte Carlo method, matched bound approximation, product of Gaussians, variational method, unscented transformation, Gaussian approximation and min-Gaussian approximation, for approximating the Kullback–Leibler divergence between two Gaussian mixture models for satellite image retrieval. Two experiments using two public data sets have been performed. The comparison is carried out in terms of retrieval accuracy and computational time.