Abstract
Generative models can be used for a wide range of tasks, and have the appealing ability to learn from both labelled and unlabelled data. In contrast, discriminative models cannot learn from unlabelled data, but tend to outperform their generative counterparts in supervised tasks. We develop a framework to jointly train deep generative and discriminative models, enjoying the benefits of both. The framework allows models to learn from labelled and unlabelled data, as well as naturally account for uncertainty in predictive distributions, providing the first Bayesian approach to semi-supervised learning with deep generative models. We demonstrate that our blended discriminative and generative models outperform purely generative models in both predictive performance and uncertainty calibration in a number of semi-supervised learning tasks.
Highlights
Much of the recent work in machine learning has focused on the development of neural network based models
Our contributions can be summarized as follows: (i) we propose a general purpose framework for combining deep generative and discriminative models in a principled manner, (ii) we provide the first semi-supervised learning method with DGMs that incorporates model uncertainty in its predictions, and (iii) we demonstrate that jointly trained deep generative and discriminative models outperform their generative counterparts in both predictive performance and uncertainty calibration in a number of semi-supervised learning tasks
Semi-supervised learning with generative adversarial networks. Another line of research that is similar in spirit to ours and has shown empirical success is that of semi-supervised learning with Generative Adverserial Networks (GANs; [13])
Summary
Much of the recent work in machine learning has focused on the development of neural network based models. Supervised training of neural networks typically (i) assumes access to a dataset D = {xn, yn}Nn=1 of input-output pairs, (ii) models the conditional distribution p(y|xn) of the outputs given the inputs with a neural network with parameters θ, and (iii) optimizes the likelihood of the data with respect to θ This approach has resulted in an astonishing array of successes on AI and pattern recognition tasks such as object recognition [17], remote sensing [39], finegrained action segmentation [10], and game-playing [45]. Achieve efficient and useful approximations to the parameter posterior distributions in discriminative networks [2,18,19,41] Another drawback of discriminative neural networks is that they require massive labelled data sets for training. Our contributions can be summarized as follows: (i) we propose a general purpose framework for combining deep generative and discriminative models in a principled manner, (ii) we provide the first (to the best of our knowledge) semi-supervised learning method with DGMs that incorporates model uncertainty in its predictions, and (iii) we demonstrate that jointly trained deep generative and discriminative models outperform their generative counterparts in both predictive performance and uncertainty calibration in a number of semi-supervised learning tasks
Sign-in/Register to view highlights & summary 
Published Version (
Free)
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call