DLSTM-MSF: Distributed LSTM Models for Multimedia Streaming Workload Forecasting Based on Kafka Environment

Abstract

This paper introduces DLSTM-MSF, a distributed approach designed to address the challenge of demand forecasting in multimedia streaming workloads. DLSTM-MSF leverages the power of multi-LSTM networks, each tailored to predict data demand for a specific type of multimedia streaming workload. The central problem addressed in this research is the accurate prediction of workload demand in a dynamic and diverse multimedia streaming environment. To achieve specialization, the training time series set for each LSTM network comprises examples with targets belonging exclusively to the workload type it is designed to predict. This specialization ensures that each LSTM network becomes proficient at capturing the unique demand patterns associated with its designated workload category. The methodology of the proposed approach is based on building the best forecasting model for each multimedia streaming workload type by exploring various combinations of LSTM hyper-parameters using the grid search method. This enables the proposed approach to effectively capture nonlinear patterns in time series data. Furthermore, the implementation of DLSTM-MSF incorporates Apache Kafka for online demand prediction, utilizing the best-developed model for each workload type. Experimental evaluations of DLSTM-MSF compare the performance of two ensemble-learning LSTM models (Ensemble V1 and Ensemble V2) with a single LSTM model. The results unequivocally highlight the superiority of Ensemble V1, with reductions of 71.85% and 74.88% in RMSE and MAE values, respectively, compared to the single LSTM model. Index Terms— Multimedia streaming, LSTM, Ensemble learning, Forecasting, Workload Demand, Big data.