A Self-Learning Scheduling in Cloud Software Defined Block Storage

Abstract

Software Defined Storage (SDS) separates the control layer from the data layer allowing the automation of data management and deployment of Commercial Off-The-Shelf (COTS) storage media rather than expensive traditional hardware-based solutions. Cloud block storage services lack an SDS framework that allows customization of block storage, policy enforcement, automate provisioning, and storage management. SDS decreases the human intervention and improves the resource utilization. Moreover, SDS allows cloud tenants to define customized functionalities based on their needs with guaranteed performance and high availability that meets Service Level Agreements (SLAs). However, maintaining SLAs requirements in cloud block storage is challenging due to the storage cluster features, the workload interference, the workload characteristics and other indirect related latent variables. To address the mentioned issues, cloud providers often over-provision the storage resources. Moving towards SDS, we initiate a framework for cloud block storage as an active storage system. Our framework provides customization of block storage services and optimized scheduling decisions based on the workload characteristics and performance of the underlying data layer leveraging a self-learning scheduler. The proposed scheduler treats the storage backend nodes as a black box and requires zero knowledge of their internal states. We showcase a practical application of the proposed scheduler in our private OpenStack deployment.