SQLR: Short-Term Memory Q-Learning for Elastic Provisioning

Abstract

As a growing number of service and application providers choose cloud networks to deliver their services on a software-as-a-service (SaaS) basis, cloud providers need to make their provisioning systems agile enough to meet service level agreements (SLAs). At the same time, they should guard against over-provisioning, which limits their capacity to accommodate more tenants. To this end, we propose Shortterm memory Q-Learning pRovisioning (SQLR, pronounced as “scaler”), a system employing a customized variant of the modelfree reinforcement learning algorithm. It can reuse contextual knowledge learned from one workload to optimize the number of virtual machines (resources) allocated to serve other workload patterns. With minimal overhead, SQLR achieves comparable results to systems where resources are unconstrained. Our experiments show that we can reduce the amount of provisioned resources by about 20% with less than 1% overall service unavailability (due to blocking), while delivering similar response times to those of an over-provisioned system.