Comparison of Neural Network Models for LDA Inferring

Abstract

One of the most reliable methods to find topics for a document is Latent Dirichlet Allocation (LDA) which is a generative statistical model, but with the growing amount of data, this method can be time consuming. This problem can be solved by utilizing neural network to learn from LDA and train model for faster processing time. This study aims to inspect further on how accurate different neural network models can be when learn from LDA. The neural network models that are used to compare in this work are dense neural network (DNN), recurrent neural network (RNN), long short-term memory (LSTM), gated recurrent unit (GRU), bidirectional LSTM (BiLSTM), and bidirectional GRU (BiGRU). From the experiment, it shows that BiGRU and RNN are good alternative to learn from LDA when compare to DNN, RNN has the best test accuracy on 15 topics at 0.8833, comparing to Dense 3 at 0.8807, Dense 2 at 0.8798, and BiGRU at 0.8767, while BiGRU has the best test accuracy on 20 topics at 0.8727, comparing to Dense 2 at 0.8704, RNN at 0.8664, and Dense 3 at 0.8642. If the topic is more than 35 topics, Dense 2 outperform other techniques including Dense 3 as well.