Abstract
Topic models are widely explored for summarizing a corpus of documents. Recent advances in Variational AutoEncoder (VAE) have enabled the development of black-box inference methods for topic modeling in order to alleviate the drawbacks of classical statistical inference. Most existing VAE based approaches assume a unimodal Gaussian distribution for the approximate posterior of latent variables, which limits the flexibility in encoding the latent space. In addition, the unsupervised architecture hinders the incorporation of extra label information, which is ubiquitous in many applications. In this paper, we propose a semi-supervised topic model under the VAE framework. We assume that a document is modeled as a mixture of classes, and a class is modeled as a mixture of latent topics. A multimodal Gaussian mixture model is adopted for latent space. The parameters of the components and the mixing weights are encoded separately. These weights, together with partially labeled data, also contribute to the training of a classifier. The objective is derived under the Gaussian mixture assumption and the semi-supervised VAE framework. Modules of the proposed framework are appropriately designated. Experiments performed on three benchmark datasets demonstrate the effectiveness of our method, comparing to several competitive baselines.
Highlights
Topic models [1], [2] provide us with methods to discover abstract word and phrase patterns that best summarize and characterize a corpus of documents
Variational AutoEncoder (VAE) [16] provides us a framework to alleviate the above-mentioned limitations, by training an inference network that maps the representations of documents to an approximate posterior distribution directly
We study topic modeling under the framework of VAE
Summary
Topic models [1], [2] provide us with methods to discover abstract word and phrase patterns that best summarize and characterize a corpus of documents. Variational AutoEncoder (VAE) [16] provides us a framework to alleviate the above-mentioned limitations, by training an inference network that maps the representations of documents to an approximate posterior distribution directly. We propose a Semi-supervised Variational AutoEncoder with Gaussian Mixture posteriors (S-VAE-GM) to address the above challenges in topic modeling. Each class label, which can be observed for a subset of the data, corresponds to a Gaussian with its specific parameters. This assumption means that topics weigh differently for different classes. The Gaussian mixture model for latent space is rational for the assumption of the class-topic hierarchy in a document, and it alleviates the unimodal limitation. Experiments performed on three standard datasets demonstrate the effectiveness of our model
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.