Generative and Discriminative Fuzzy Restricted Boltzmann Machine Learning for Text and Image Classification.

Abstract

The restricted Boltzmann machine (RBM) is an excellent generative learning model for feature extraction. By extending its parameters from real numbers to fuzzy ones, we have developed the fuzzy RBM (FRBM) which is demonstrated to possess better generative capability than RBM. In this paper, we first propose a generative model named Gaussian FRBM (GFRBM) to deal with real-valued inputs. Then, motivated by the fact that the discriminative variant of RBM can provide a self-contained framework for classification with competitive performance compared with some traditional classifiers, we establish the discriminative FRBM (DFRBM) and discriminative GFRBM (DGFRBM) that combine both the generative and discriminative facility by adding extra neurons next to the input units. Specifically, they can be trained into excellent stand-alone classifiers and retain outstanding generative capability simultaneously. The experimental results including text and image (both clean and noisy) classification indicate that DFRBM and DGFRBM outperform discriminative RBM models in terms of reconstruction and classification accuracy, and they behave more stable when encountering noisy data. Moreover, the proposed learning models show some promising advantages over other standard classifiers.