Neural dynamics of sequence generation: concepts and robotic demonstrations. A Dynamic Field Theory approach.

Abstract

Event Abstract Back to Event Neural dynamics of sequence generation: concepts and robotic demonstrations. A Dynamic Field Theory approach. Yulia Sandamirskaya1* and Gregor Schoner1 1 Ruhr-Universität Bochum, Institut für Neuroninformatik, Germany Serial order is central to behavior of both human and artificial agents. The neural mechanisms for storing, processing and producing sequences of neural states are widely discussed in the literature [1,2]. Although multiple models fit the behavioral data on serial order (error patterns, reaction times, storage capacity), issues of embodiment have not been seriously examined. When a sequence of neural events is coupled to the actuators of a real-world agent, whose behavior is controlled by unreliable sensor information and is executed in a partially unknown and time variable environment, additional constraints on sequence generation arise, which are not addressed by most contemporary models for serial order: (1) Flexible timing: The duration of each action may vary in unpredictable ways on the time scale of behavior, which is much longer than that of the underlying neuronal processes. (2) Stability: Actions must be generated autonomously based on graded, time-varying sensory information. Recently we introduced a Dynamic Fields Theory (DFT) model for sequence learning and production, which addresses these issues [3]. Here, actions are represented as stable states of the dynamics of a collection of neural fields. This enables stability and gradedness of action representations and thus makes it possible to use graded low-level sensory information to guide cognitive behavior. The sequential transitions between the stable states are controlled by a neural representation of a condition of satisfaction that signals successful accomplishment of an action. This first model required all actions to be embedded in a uniform dimension represented by the neuronal fields. Here, we revise and extend this model, to accommodate increased flexibility and, in particular, to enable sequences that contain different categories of actions and perceptual representations. The dynamical representation of serial order is projected onto a distributed multimodal neural representation of actions and perceptual states. The agent acquires a sequence of actions perceived with low-level sensors, and then acts out that sequence in new, unknown environments. The concept of a condition of satisfaction, represented in a dynamic neural field that reflects the convergence of expected and real sensory information, is central to controlling the switch of attractor states that occurs at every step of a sequence. The model is implemented on an autonomous robot and its functionality - flexible timing and coupling to real behavior - is demonstrated in a simple robotic scenario. Although the model does not address statistics of experimental data on sequential behavior, it demonstrates neuronal principles of sequential action and their functional implications. In particular, the model shows how the problems of stability and flexible timing of sequence generation can be solved.