Computational role of exploration noise in error-based de novo motor learning

Abstract

The redundancy inherent to the human body is a central problem that must be solved by the brain when acquiring new motor skills. The problem of redundancy becomes particularly critical when learning a new motor policy from scratch in a novel environment and task (i.e., de novo learning). It has been proposed that motor variability could be leveraged to explore and identify task-potent motor commands, and recent results indicated a possible role of motor exploration in error-based motor learning, including in de novo learning tasks. However, the precise computational mechanisms underlying this role remain poorly understood. A new controller in a de novo motor task can potentially be learned by first using motor exploration to learn a sensitivity derivative, which can transform observed task errors into motor corrections, enabling the error-based learning of the controller. Although this approach has been discussed, the computational properties of exploration and how this mechanism can explain recent reports of motor exploration in error-based de-novo learning have not been thoroughly examined. Here, we used this approach to simulate the tasks used in several recent studies of human motor learning tasks in which motor exploration was observed, and replicating their main results. Analyses of the proposed learning mechanism using equations and simulations suggested that exploring the entire motor command space leads to the training of an efficient sensitivity derivative, enabling rapid learning of the controller, in visuomotor adaptation and de novo tasks. The successful replication of previous experimental results elucidated the role of motor exploration in motor learning.