Feature Extraction for Controller Design by Deep Auto-Encoder Neural Network and Least squares Policy Iteration

Abstract

Due to the extensively existing complexity and uncertainty of systems, feature extraction based on samples is an important task in controller design. As one of the research hotspots, deep auto-encoder neural network can be used to extract features from raw data. This paper proposed a modified deep auto-encoder neural network (MDAENN). An accelerated proximal gradient (APG) method is proposed in this method. MDAENN has lower computational complexity, easier parameters tuning and better convergence than traditional neural network methods, such as RBF, in feature extraction and reconstruction. Based on the feature extraction, least squares policy iteration (LSPI) is used to design the optimal controller. When the dimension of state space is large or even continuous, value function approximation (VFA) method is used instead of value function. Experimental results show that the proposed method can successfully deal with feature extraction and learn control policies with low computational complexity.