Abstract
Deep belief networks hold great promise for the simulation of human cognition because they show how structured and abstract representations may emerge from probabilistic unsupervised learning. These networks build a hierarchy of progressively more complex distributed representations of the sensory data by fitting a hierarchical generative model. However, learning in deep networks typically requires big datasets and it can involve millions of connection weights, which implies that simulations on standard computers are unfeasible. Developing realistic, medium-to-large-scale learning models of cognition would therefore seem to require expertise in programing parallel-computing hardware, and this might explain why the use of this promising approach is still largely confined to the machine learning community. Here we show how simulations of deep unsupervised learning can be easily performed on a desktop PC by exploiting the processors of low cost graphic cards (graphic processor units) without any specific programing effort, thanks to the use of high-level programming routines (available in MATLAB or Python). We also show that even an entry-level graphic card can outperform a small high-performance computing cluster in terms of learning time and with no loss of learning quality. We therefore conclude that graphic card implementations pave the way for a widespread use of deep learning among cognitive scientists for modeling cognition and behavior.
Highlights
Unsupervised learning in neural network models has provided important insights into how sensory information can be efficiently encoded in neural systems in a way that strikingly mirrors singlecell recoding data (Olshausen and Field, 1996; Rao and Ballard, 1999)
We present a straightforward graphic processor units (GPUs) implementation of deep belief networks trained with contrastive divergence learning (Hinton and Salakhutdinov, 2006) that is based on high-level programming routines and can run on a common desktop computer, provided that it has a recent NVIDIA graphic card
We found that even an entry-level GPU significantly outperforms the cluster with respect to the computational time required for deep unsupervised learning, with no cost on the quality of learning
Summary
Unsupervised learning in neural network models has provided important insights into how sensory information can be efficiently encoded in neural systems in a way that strikingly mirrors singlecell recoding data (Olshausen and Field, 1996; Rao and Ballard, 1999). Learning in a deep belief network can be seen as fitting a hierarchical generative model to the sensory data, where learning aims at reconstructing the input data from the internal representations and can be performed locally at each level in an unsupervised fashion This represents a novelty in training multilayer neural networks, because it demonstrates how probabilistic learning can be performed using a mechanism that is both efficient (Hinton and Osindero, 2006) and neurally plausible (O’Reilly, 1998). Building higher-level abstractions can reveal causal features that are only www.frontiersin.org
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
