Bayesian Parameter Estimation in LDA

Abstract

Latent Dirichlet Allocation (LDA) probabilistic topic model is widely used in text mining, natural language processing and so on. But LDA's mathematical theory is particularly complex, thus it is very difficult to understand LDA for a novice. In order to more quickly and easily learn LDA, and further promote its application, this paper will deeply analyze LDA from the perspective of Bayesian parameter estimation. At first we explain the advantage of Bayesian parameter estimation by an instance, and then introduce a simple Bayesian Unigram model. Next based on the simple Bayesian Unigram model and PLSA model, a full Bayesian probabilistic topic model—LDA is presented. In order to more quickly understand LDA model, and further promote its application, this paper will start from Bayesian parameter estimation, and then analyze the mathematical theory of LDA model from the perspective of Bayesian parameter estimation. This paper is organized as follows. In Section 2 we introduce Bayesian parameter estimation based on an illustrative example. Compared to related PLSA model, LDA model with Bayesian parameter estimation is presented in Section 3. Finally, Section 4 presents summary.