Entropy as a lens into LDA model understanding

Abstract

Latent Dirichlet Allocation (LDA) has seen increasing application to source code and its related artifacts in part because of its impressive modeling power. However, this expressive power comes at a cost: the technique includes several tuning parameters whose impact on the resulting LDA model must be carefully considered. The goal of this work is to highlight the initial promise of combined entropy as a technique for better understanding the tuning parameters' impact. Doing so aims to improve the effectiveness of both researchers seeking to exploit the power of LDA in their work and tool users who must interpret the output of LDA-using tools. Initial results obtained using seven production systems find that combined entropy shows very little variation from system to system indicating that it has strong external validity. These results also highlight how effective combined entropy can be as a lens for understanding LDA's potential value to downstream applications.