Analysis of Different Sparsity Methods in Constrained RBM for Sparse Representation in Cognitive Robotic Perception

Abstract

Cognitive robotic systems nowadays are intensively involving learning algorithms to achieve highly adaptive and intelligent behaviors, including actuation, sensing, perception and adaptive control. Deep learning has emerged as an effective approach in image-based robotic perception and actions. Towards cognitive robotic perception based on deep learning, this paper focuses the Constrained Restricted Boltzmann Machine (RBM) on visual images for sparse feature representation. Inspired by sparse coding, the sparse constraints are performed on the hidden layer of RBM to obtain sparse and effective feature representation from perceived visual images. The RBM with Sparse Constraint (RBMSC) is proposed with a generalized optimization problem, where the constraints are applied on the probability density of hidden units directly to obtain more sparse representation. This paper presents three novel RBM variants, namely L1-RBM, L2-RBM, and L1/2-RBM constrained by L1-norm, L2-norm, and L1/2-norm on RBM, respectively. A Deep Belief Network with two hidden layers is built for comparison between each RBM variants. The experiments on MNIST database (Mixed National Institute of Standards and Technology database) show that the L1/2-RBM can obtain more sparse representation than RBM, L1-RBM, L2-RBM, and Sparse-RBM (SRBM) in terms of sparseness metric. For further verification, the proposed methods are still tested on MNIST Variations dataset. The recognition results from perceived images in MNIST and MNIST Variations demonstrate that our proposed constrained RBM variants are feasible for object cognitive and perception, and the proposed L1/2-RBM and L1-RBM outperforms RBM and SRBM in terms of object recognition.