Accurate Prediction of Workloads and Resources With Multi-Head Attention and Hybrid LSTM for Cloud Data Centers

Abstract

Currently, cloud computing service providers face big challenges in predicting large-scale workload and resource usage time series. Due to the difficulty in capturing nonlinear features, traditional forecasting methods usually fail to achieve high prediction performance for resource usage and workload sequences. Besides, there is much noise in original time series of resources and workloads. If these time series are not de-noised by smoothing algorithms, the prediction results can fail to meet the providers' requirements. To do so, this work proposes a hybrid prediction model named VAMBiG that integrates <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">V</u> ariational mode decomposition, an <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">A</u> daptive Savitzky-Golay (SG) filter, a <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">M</u> ulti-head attention mechanism, <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Bi</u> directional and <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">G</u> rid versions of Long and Short Term Memory (LSTM) networks. VAMBiG adopts a signal decomposition method named variational mode decomposition to decompose complex and non-linear original time series into low-frequency intrinsic mode functions. Then, it adopts an adaptive SG filter as a data pre-processing tool to eliminate noise and extreme points in such functions. Afterwards, it adopts bidirectional and grid LSTM networks to capture bidirectional features and dimension ones, respectively. Finally, it adopts a multi-head attention mechanism to explore importance of different data dimensions. VAMBiG aims to predict resource usage and workloads in highly variable traces in clouds. Extensive experimental results demonstrate that it achieves higher-accuracy prediction than several advanced prediction approaches with datasets from Google and Alibaba cluster traces.