Fine-Tuning Infinity Restricted Boltzmann Machines

Abstract

Restricted Boltzmann Machines (RBMs) have received special attention in the last decade due to their outstanding results in number of applications, such as face and human motion recognition, and collaborative filtering, among others. However, one of the main concerns about RBMs is related to the number of hidden units, which is application-dependent. Infinite RBM (iRBM) was proposed as an alternative to the regular RBM, where the number of units in the hidden layer grows as long as it is necessary, dropping out the need for selecting a proper number of hidden units. However, a less sensitive regularization parameter is introduced as well. This paper proposes to fine-tune iRBM hyper-parameters by means of meta-heuristic techniques such as Particle Swarm Optimization, Bat Algorithm, Cuckoo Search, and the Firefly Algorithm. The proposed approach is validated in the context of binary image reconstruction over two well-known datasets. Furthermore, the experimental results compare the robustness of the iRBM against the RBM and Ordered RBM (oRBM) using two different learning algorithms, showing the suitability in using meta-heuristics for hyper-parameter fine-tuning in RBM-based models.