OMBM: Optimized Memory Bandwidth Management for Ensuring QoS and High Server Utilization

Abstract

Latency-critical workloads such as web search engines, social networks and finance market applications are sensitive to tail latencies for meeting Service Level Objectives (SLOs). Since unexpected tail latencies are caused by sharing hardware resources with other co-executing workloads, a service provider executes the latency-critical workload alone. Thus, the data center for the latency-critical workloads has exceedingly low hardware resource utilization. For improving hardware resource utilization, the service provider has to co-locate the latency-critical workloads and other batch processing workloads. However, because memory bandwidth cannot be provided in isolation unlike cores and cache memory, the latency-critical workloads experience poor performance isolation even though the core and the cache memory are allocated in isolation to the workloads. To solve this problem, we propose an optimized memory bandwidth management approach for ensuring Quality of Service (QoS) and high server utilization. By providing isolated shared resources including memory bandwidth to the latencycritical workload and co-executing batch processing ones, our proposed approach guarantees SLOs and improves hardware resource utilization. Firstly, we predict the size of the memory bandwidth to meet the SLO for all Queries Per Seconds (QPSs) while executing the latency-critical workload with minimal preprofiling. Then, our approach allocates the amount of the isolated memory bandwidth that guarantees the SLO to the latencycritical workload and the rest of the memory bandwidth to coexecuting batch processing workloads. As a result, our proposed approach can achieve up to 99% SLO assurance and improve server utilization up to 6.5x.