Motor learning and control based on the internal model

Abstract

The present paper proposes a learning control method for the musculoskeletal system of arm based on the reinforcement learning with the internal model. In general, it is not easy to apply the reinforcement learning to the motor control because of higher dimensional search domain and non-Markov properties. The proposed scheme consists of musculoskeletal system, actor-critic network and neural internal model. Neural internal model is employed to compensate for the non-Markov property. In addition, it is designed that the viscoelastic parameters are preset to be larger in the early stages of learning in order to increase the robustness of the internal model. To hold the viscoelasticity high at first, the constraint for searching noise is introduced, which decreases the search domain. The viscoelasticity results in an optimal level as the learning progresses by the relaxation of the constraint. The effectiveness and the biological plausibility of the proposed model is demonstrated by computer simulation.