Optimal balancing control of bipedal robots using reinforcement learning

Abstract

The balance control a bipedal robot in the presence of external disturbances is still a challenge. In this paper, a novel optimal ankle stiffness regulation for humanoid robot balancing problem is proposed. The presented techniques are developed based on the integral reinforcement learning (IRL) algorithm, which is designed for unknown continuous-time systems using only partial knowledge of the system dynamics. A linear mathematical model of an inverted pendulum model (IPM) is employed to study robot balance, and optimization is achieved by using linear quadratic regulator (LQR). Because the internal system dynamics is nonlinear and time-varying, IRL algorithm is proposed to solve the algebraic Riccati equation online without knowledge of the internal system dynamics. Nonlinear stiffness stabilizer based on both IRL and the fixed stiffness are studied in the simulation, the comparative results demonstrate the superior balance ability of the proposed method. In addition, dynamics changing is also discussed in the simulation to test robustness of the proposed method.