Model compression optimized neural network controller for nonlinear systems

Abstract

Neural network-based controllers are widely used within the domain of robotic control systems. A network controller with more neurons typically achieves better performance, but an excessive number of neurons may make the model computationally intensive, resulting in slow dynamic responses in real-world environments. This paper reports a network compression method that integrates knowledge distillation technology for the development of concise neural network-based controllers to achieve a balance between the control performance and computational costs. The method first trains a full-size teacher model, which is then pruned, leading to a concise network with a minimum compromise of performance. From in this study, the resulting concise network is considered to be the prototype of a student model, which is further trained by a knowledge distillation process. The proposed compression method was applied to three classical networks, and the resultant compact controllers were tested on a robot manipulator for efficacy and potential demonstration. The experimental results from a comparative study confirm that the student models with fewer neurons resulting from the proposed model compression approach can achieve similar performance to that of the teacher models for intelligent dynamic control but with faster convergence speed.