Reinforcement Learning of Linking and Tracing Contours in Recurrent Neural Networks.

Matthias Bethge,Tobias Brosch,Pieter R Roelfsema,Heiko Neumann

doi:10.1371/journal.pcbi.1004489

Abstract

The processing of a visual stimulus can be subdivided into a number of stages. Upon stimulus presentation there is an early phase of feedforward processing where the visual information is propagated from lower to higher visual areas for the extraction of basic and complex stimulus features. This is followed by a later phase where horizontal connections within areas and feedback connections from higher areas back to lower areas come into play. In this later phase, image elements that are behaviorally relevant are grouped by Gestalt grouping rules and are labeled in the cortex with enhanced neuronal activity (object-based attention in psychology). Recent neurophysiological studies revealed that reward-based learning influences these recurrent grouping processes, but it is not well understood how rewards train recurrent circuits for perceptual organization. This paper examines the mechanisms for reward-based learning of new grouping rules. We derive a learning rule that can explain how rewards influence the information flow through feedforward, horizontal and feedback connections. We illustrate the efficiency with two tasks that have been used to study the neuronal correlates of perceptual organization in early visual cortex. The first task is called contour-integration and demands the integration of collinear contour elements into an elongated curve. We show how reward-based learning causes an enhancement of the representation of the to-be-grouped elements at early levels of a recurrent neural network, just as is observed in the visual cortex of monkeys. The second task is curve-tracing where the aim is to determine the endpoint of an elongated curve composed of connected image elements. If trained with the new learning rule, neural networks learn to propagate enhanced activity over the curve, in accordance with neurophysiological data. We close the paper with a number of model predictions that can be tested in future neurophysiological and computational studies.

Highlights

Visual perception appears to be remarkable effortless and automatic
Our results demonstrate that (1) the behavior of the neural networks during learning and the pattern of errors is similar to that of monkeys that are trained in these tasks [14, 40], (2) the model reproduces the changes in neuronal firing rates and the emergence of incremental grouping during the learning process, (3) that the labeling with enhanced neuronal activity for grouping is an efficient code that can be used to guide behavior, and (4) that RELEARNN is a comprehensive and powerful learning scheme, which captures fundamental aspects of synaptic plasticity in a recurrent neural network
The analysis described above establishes that RELEARNN can train a fully recurrent network to adjust its fixed point and to thereby improve its estimate of the value of a selected action

Summary

Introduction

Visual perception appears to be remarkable effortless and automatic. The visual system extracts many elementary features such as colors, local orientations, contrasts, motion directions in low level areas and more complex features such as shape properties (curvature, corners) in higher areas [8] This early processing phase thereby produces a pattern of activity across the various areas of the visual cortex that has been called “base representation” [8]. This contextual effect even occurs if the information in the V1 receptive field is held constant, which implies that it depends on the lateral influences of V1 neurons with different receptive fields and/or on feedback from higher visual areas where receptive fields are larger [8] In accordance with this view, the effect of grouping on V1 activity does not occur during the initial visual response but at an additional delay (Fig 1A)

Objectives

Results

Discussion

Conclusion