Learning Coupled Parameters with Continuous Domains for Human-powered Lower Exoskeleton

Abstract

Human-powered lower exoskeletons are concerned by academia and industry in areas of human locomotion and strength augmentation. With the development of technology, machine learning is used to improve the control performance of exoskeleton system, in which reinforcement learning is used to adapt changing with different pilots and walking patterns. To combine reinforcement learning with real-life applications, the continuous observation spaces must be discretized and the discrete low-dimensional observation spaces can only be handled in traditional strategies. In real-life applications, Almost all tasks of interest and most notably physical control tasks have continuous and high-dimensional observation spaces. The continuous states can characterize the state of exoskeleton system and the continuous actions can control the exoskeleton system accurately. Therefore, this paper proposes an Inter-active Learning based on Actor-Critic (ILAC) to solve the problems with continuous high-dimensional observation spaces. In proposed ILAC, the actor-critic algorithm that can learn optimal policies in high-dimensional continuous domains is used to learn the controller’s coefficients of exoskeleton system, in which the sensitivity factors of Sensitivity Amplification Control (SAC) and the coefficients of the compensation controller are learned at the same time. The experiments on both single Degree Of Freedom (DOF) exoskeleton and HUman-powered Augmentation Lower EXoskeleton (HUALEX) is shown. The experimental results show that the proposed ILAC has better performance in interactive learning.