Reducing Tail Latency of Interactive Multi-tier Workloads in the Cloud

Abstract

Reducing tail latency becomes increasingly important to improve user-perceived service experience. User-facing latency-sensitive cloud applications typically contain multiple interactive tiers running in different virtual machines (VMs) with complex interaction patterns. Consolidation of those applications is a challenge. In this paper we study the consolidation of multi-tier interactive workloads from a new perspective of user-perceived tail latency. We propose a novel profiling-based consolidation methodology. The objective is to satisfy tail latency while reducing the number of physical machines. We consider two key factors that affecting the tail latency of multi-tier workloads: interference with neighboring VMs and interaction between different tiers. We model the consolidation of multi-tier workloads as an optimization problem with different objectives and constraints. We implement and evaluate the proposed models, as well as comparing with other methods (i.e., without profiling or without considering interaction influence). Experimental results show that the proposed method is able to greatly reduce the tail latency compared with the traditional consolidation method.