Three-Dimensional Path-Following Control of a Robotic Airship with Reinforcement Learning

Chunyu Nie,Ming Zhu,Zewei Zheng

doi:10.1155/2019/7854173

Chunyu Nie, Ming Zhu + Show 1 more

Open Access

https://doi.org/10.1155/2019/7854173

Copy DOI

Abstract

This paper proposed an adaptive three-dimensional (3D) path-following control design for a robotic airship based on reinforcement learning. The airship 3D path-following control is decomposed into the altitude control and the planar path-following control, and the Markov decision process (MDP) models of the control problems are established, in which the scale of the state space is reduced by parameter simplification and coordinate transformation. To ensure the control adaptability without dependence on an accurate airship dynamic model, a Q-Learning algorithm is directly adopted for learning the action policy of actuator commands, and the controller is trained online based on actual motion. A cerebellar model articulation controller (CMAC) neural network is employed for experience generalization to accelerate the training process. Simulation results demonstrate that the proposed controllers can achieve comparable performance to the well-tuned proportion integral differential (PID) controllers and have a more intelligent decision-making ability.

Highlights

The lift of an airship mainly comes from the buoyancy of the gas, and it does not have to do continuous movement to balance gravity, which makes it a promising platform for long-endurance and low-energy consumption
In each round of learning, the controller selects the action based on policy π periodically before reaching the target or boundary, and the fitness coefficient and the weights of the cerebellar model articulation controller (CMAC) neural network are updated at the same time
A reinforcement learning airship 3D path-following control is proposed, which is robust against dynamic model changes

Summary

Introduction

The lift of an airship mainly comes from the buoyancy of the gas, and it does not have to do continuous movement to balance gravity, which makes it a promising platform for long-endurance and low-energy consumption. In contrast to the previous reinforcement learning airship control without dependence on an accurate dynamic model, for the first time, this study proposed an airship control strategy through autonomous online training in 3D space (2) To deal with the “curse of dimensionality” problem in the airship planar path-following control, a novel coordinate frame is proposed to describe the relative state between the airship and the target. This coordinate transformation form can reduce the scale of the state space and generalize the experience by rotation and makes it possible to learn the control strategy online (3) In the proposed reinforcement learning airship control strategy, a CMAC neural network is employed to generalize the experience in a local neighborhood, which effectively accelerates the training process.

Problem Formulation and the MDP Model of the Airship Control

Path-Following Control Design

Simulation

Conclusions