Temporal Credit Assignment Problem Research Articles

Models of birdsong learning

The study of birdsong learning and the passerine song system represents an excellent opportunity to understand how complex acoustic perceptions are represented in the brain, and how these interact with motor systems to produce learned behavior. Recent advances suggesting specific functional roles for different nuclei within the song system help to constrain proposed models of the sensorimotor phase of learning. In particular, a descending motor pathway organized in a temporal hierarchy is involved in motor programming whereas an anterior forebrain basal ganglialike pathway may provide auditory and motor feedback-mediated error signals to adjust the motor pathway. A separate pathway outside of the song system may be devoted to perception of conspecific songs. Central to the problem of vocal learning is the time delay between motor command and auditory feedback. Both real time online models and offline models have been proposed as solutions to the temporal credit assignment problem. The models differ in the types of synaptic mechanisms envisioned that involve forward predictions, and the relative importance of state dependency of song system activity and song learning. Recent data will be given that describe the insight into neural solutions to these problems.

LEARNING TO GENERATE ARTIFICIAL FOVEA TRAJECTORIES FOR TARGET DETECTION

This paper shows how ‘static’ neural approaches to adaptive target detection can be replaced by a more efficient and more sequential alternative. The latter is inspired by the observation that biological systems employ sequential eye movements for pattern recognition. A system is described, which builds an adaptive model of the time-varying inputs of an artificial fovea controlled by an adaptive neural controller. The controller uses the adaptive model for learning the sequential generation of fovea trajectories causing the fovea to move to a target in a visual scene. The system also learns to track moving targets. No teacher provides the desired activations of ‘eye muscles’ at various times. The only goal information is the shape of the target. Since the task is a ‘reward-only-at-goal’ task, it involves a complex temporal credit assignment problem. Some implications for adaptive attentive systems in general are discussed.