Optimization of the Use of Cloud Computing Resources Using Exploratory Data Analysis and Machine Learning

Abstract

Abstract Rapid growth in the popularity of cloud computing has been largely caused by increasing demand for scalable IT solutions, which could provide a cost-effective way to manage the software development process and meet business objectives. Optimization of cloud resource usage remains a key issue given its potential to significantly increase efficiency and flexibility, minimize costs, ensure security, and maintain high availability of services. This paper presents a novel concept of a Cloud Computing Resource Prediction and Optimization System, which is based on exploratory data analysis that acknowledges, among others, the information value of outliers and dynamic feature selection. The optimization of cloud resource usage relies on long-term forecasting, which is considered a dynamic and proactive optimization category. The analysis presented here focuses on the applicability of classical statistical models, XGBoost, neural networks and Transformer. Experimental results reveal that machine learning methods are highly effective in long-term forecasting. Particularly promising results – in the context of potential prediction-based dynamic resource reservations – have been yielded by prediction methods based on the BiGRU neural network and the Temporal Fusion Transformer.