A Method of Adaptively Selecting Best LDA Model Based on Density

Abstract

Topic models have been successfully used to information classification and retrieval.These models can capture word correlations in a collection of textual documents with a low-dimensional set of multinomial distribution,called "topics".It is important but difficult to select an appropriate number of topics for a specific dataset.This paper proposes a theorem that the model reaches optimum as the average similarity among topics reaches minimum,and based on this theorem,proposes a method of adaptively selecting the best LDA model based on density.Experiments show that the proposed method can achieve performance matching the best of LDA without manually tuning the number of topics.