An actor-critic architecture and simulator for goal-directed brain-machine interfaces

Abstract

The Perception-Action Cycle (PAC) is a central component of goal-directed behavior because it links internal percepts with external outcomes in the environment. Using inspiration from the PAC, we are developing a Brain-Machine Interface control architecture that utilizes both motor commands and goal information directly from the brain to navigate to novel targets in an environment. An Actor-Critic algorithm was selected for decoding the neural motor commands because it is a PAC-based computational framework where the perception component is implemented in the critic structure and the actor is responsible for taking actions. We develop in this work a biologically realistic simulator to analyze the performance of the decoder in terms of convergence and target acquisition. Experience from the simulator will guide parameter selection and assist in understanding the architecture before animal experiments. By varying the signal to noise ratio of the neural input and error signal, we were able to demonstrate how the learning rate and initial conditions affect a motor control target selection task. In this framework, the naïve decoder was able to reach targets in the presence of noise in the error signal and neural motor command with 98% accuracy.