Abstract

One of the central problems in computational neuroscience is to understand how the object-recognition pathway of the cortex learns a deep hierarchy of nonlinear feature detectors. Recent progress in machine learning shows that it is possible to learn deep hierarchies without requiring any labelled data. The feature detectors are learned one layer at a time and the goal of the learning procedure is to form a good generative model of images, not to predict the class of each image. The learning procedure only requires the pairwise correlations between the activations of neuron-like processing units in adjacent layers. The original version of the learning procedure is derived from a quadratic ‘energy’ function but it can be extended to allow third-order, multiplicative interactions in which neurons gate the pairwise interactions between other neurons. A technique for factoring the third-order interactions leads to a learning module that again has a simple learning rule based on pairwise correlations. This module looks remarkably like modules that have been proposed by both biologists trying to explain the responses of neurons and engineers trying to create systems that can recognize objects.

Highlights

  • The obvious way to falsify a theory of how the human cortex learns to interpret the visual input is to show that its predictions disagree with experimental data

  • This paper describes a historical sequence of progressively more powerful learning rules that have emerged from computer simulations

  • Learning procedures that are inspired by biology but evaluated by their computational performance have become much more sophisticated over the last few decades

Read more

Summary

INTRODUCTION

The obvious way to falsify a theory of how the human cortex learns to interpret the visual input is to show that its predictions disagree with experimental data. Single cell recordings in the visual systems of mammals (Felleman & Van Essen 1991) are consistent with this model and show that the individual feature detectors become progressively more tolerant to variations in retinal position, orientation and scale as we ascend the hierarchy. This raises the question of how such a hierarchy could be learned. Back propagation is a method for computing how to change the connection weights in a feed-forward neural network composed of multiple layers of artificial neurons. The idea of learning connection weights by following the gradient of some objective function is very powerful, but classification error is not a good objective function for learning a visual system

UNSUPERVISED LEARNING OF FEATURE DETECTORS
LEARNING WITH INCORRECT INFERENCE
A STACKABLE LEARNING MODULE
ASSOCIATING FEATURE VECTORS WITH LABELS
A BETTER MODULE FOR DEEP LEARNING
Findings
CONCLUSION

Talk to us

Join us for a 30 min session where you can share your feedback and ask us any queries you have

Schedule a call

Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.