A new lightweight framework based on knowledge distillation for reducing the complexity of multi-modal solar irradiance prediction model

Abstract

The inherent uncertainty of solar energy brings great difficulties to the grid connection and short-term energy planning and dispatching. Deep learning method makes it possible to predict the short-term solar energy with its powerful learning ability, but its complex model structure and huge trainable parameters bring great difficulties to the practical deployment. Therefore, this paper proposes a lightweight framework based on knowledge distillation strategy, which greatly reduces the complexity of multi-modal solar irradiance prediction model meanwhile ensuring an acceptable accuracy, facilitating the practical deployment. Firstly, a teacher model with multi-modal structure and good accuracy is built based on ResNet18-Informer. Then, the lightweight model is obtained by the proposed lightweight framework depending on the knowledge of teacher model. The comparisons of various models and the optimal settings of knowledge distillation are analyzed. Results show that the lightweight model can reduce the trainable parameters, inference time, and GPU memory by 97.7%, 52.5%, and 36.3%, respectively. The normalized root mean square error is reduced by 24.87% compared with the same structure model but without knowledge distillation, verifying the superiority of the proposed framework. The soft loss using the light loss with the ratio of 0.3 can obtain the best training results for the lightweight model. The structure with 3 residual blocks and 3 LSTM layers is proved to be the best for the lightweight model in the solar irradiance prediction task.