A neural model that implements probabilistic topics

Abstract

We present a neural network model that can execute some of the procedures used in the information sciences literature. In particular we offer a simplified notion of topic and how to implement it using neural networks that use the Kronecker tensor product. We show that the topic detecting mechanism is related to Naive Bayes statistical classifiers, and that it is able to disambiguate the meaning of polysemous words. We evaluate our network in a text categorization task, resulting in performance levels comparable to Naive Bayes classifiers, as expected. Hence, we propose a simple scalable neural model capable of dealing with machine learning tasks, while retaining biological plausibility and probabilistic transparency.