Neuroevolution of Inverted Pendulum Control: A Comparative Study of Simulation Techniques

Abstract

The inverted pendulum control problem is a classical benchmark in control theory. Amongst the approaches to developing control programs for an inverted pendulum, the evolution of Artificial Neural Network (ANN) based controllers has received some attention. The authors have previously shown that Evolutionary Robotics (ER) can successfully be used to evolve inverted pendulum stabilization controllers in simulation and that these controllers can transfer successfully from simulation to real-world robotic hardware. During this process, use was made of robotic simulators constructed from empirically-collected data and based on ANNs. The current work aims to compare this method of simulator construction with the more traditional method of building robotic simulators based on physics equations governing the robotic system under consideration. In order to compare ANN-based and physics-based simulators in the evolution of inverted pendulum controllers, a real-world wheeled inverted pendulum robot was considered. Simulators based on ANNs as well as on a system of ordinary differential equations describing the dynamics of the robot were developed. These two simulation techniques were then compared by using each in the simulation-based evolution of controllers. During the evolution process, the effects of injecting different levels of noise into the simulation was furthermore studied. Encouraging results were obtained, with controllers evolved using ANN-based simulators and realistic levels of noise outperforming those evolved using the physics-based simulators.