A multi-head attention mechanism aided hybrid network for identifying batteries’ state of charge

Abstract

A Convolution Neural Network and a Bidirectional Long Short-Term Memory network connected architecture with a Multi-Head Attention mechanism (CNN-BiLSTM-MHA) is studied for predicting state of charge (SOC) of lithium-ion batteries (LIBs). Firstly, an adaptive noise based complete ensemble empirical mode decomposition (AN-CEEMD) algorithm is adopted to catch the intrinsic features of measured battery signals by adding white noises. Secondly, a CNN-BiLSTM model with the MHA mechanism is developed to learn the mapping between processed input signals and battery SOC, it has three parts: 1) the CNN extracts features of the processed data, mine the relation among input signals, and promote estimation precision; 2) the BiLSTM has memory ability to catch battery dynamics, and the Swish activation function in the BiLSTM ensures unsaturated and reduces over-fitting due to its upper unbound and lower bound; 3) The multi-head attention mechanism uses several independent self-attention layers to associate input information and variables, extracts more associate information by adding weights; it reduces the overfitting risk of a single attention head, and improves the model generalization performance through joint learning of multiple heads. Finally, experiments and simulations are implemented under four operating conditions at five different temperatures, and the presented method is verified effective.