A Biologically Constrained Cerebellar Model With Reinforcement Learning for Robotic Limb Control

Abstract

The cerebellum is known to be critical for accurate adaptive control and motor learning. It has long been recognized that the cerebellum acts as a supervised learning machine. However, recent evidence shows that cerebellum is integral to reinforcement learning. This paper proposes a biologically plausible cerebellar model with reinforcement learning based on the cerebellar neural circuitry to eliminate the need for explicit teacher signals. The learning capacity of cerebellar reinforcement learning is first demonstrated by constructing a simulated cerebellar neural network agent and a detailed model of the human arm and muscle system in the Emergent virtual environment. Next, the cerebellar model is incorporated in both a simulated arm and a Geomagic Touch device to further verify the effectiveness of the cerebellar model in reaching tasks. Results from these experiments indicate that the cerebellar simulation is capable of driving the “arm plant” to arrive at the target positions accurately. Moreover, by examining the effect of the number of basic units, we find the results are consistent with previous findings that the central nervous system may recruit the muscle synergies to realize motor control. The study described here prompts several hypotheses about the relationship between motor control and learning and may be useful in the development of general-purpose motor learning systems for machines.