Denoising Deep Boltzmann Machines: Compression for Deep Learning

Abstract

This work is to improve deep learning training performances via compression-based denoising algorithms. The deep learning models we focus on are Deep Boltzmann Machines (DBM) including Restricted Boltzmann Machines (RBM), Bernoulli DBM, Gaussian Bernoulli RBM, and Gaussian Bernoulli DBM. In theory, DBMs are universal appropximators, however, there is a gap between DBMs' promised theoretical properties and their practical performances. In this paper, we propose a post-hoc signal processing technique, i.e., denoise, to trained DBM. The proposed technique denoises DBMs, and thus the resulting models, denoising DBMs, remain more information about the training data. The proposed denoising DBM has its roots in compression-based denoising techniques, and is unlike other denoising techniques widely used in deep learning areas.