OLTP In Real Life: A Large-scale Study of Database Behavior in Modern Online Retail

Abstract

E-commerce is a multi-trillion US dollar industry and a major user of online transaction processing (OLTP) systems. Yet, we do not have a good understanding of the real-world properties that characterize hardware usage in database systems serving online retail. One key reason is the difficulty of obtaining monitoring logs from production retail datacenters, as large retailers typically consider such data to be highly sensitive. Additionally, resource-usage patterns in OLTP systems and their correlations with the workload are generally challenging to model (not just in retail), due to high levels of transaction concurrency and complexity in OLTP applications. Relying on synthetic benchmarks alone to simulate and study these systems is not sufficient. Instead, this paper takes an empirical approach.We present the first large-scale study of OLTP systems in real life, using traces from two large retailers. The first trace represents six months worth of logs from a large online retailer in Latin America, B2W Digital (B2W). The second trace represents 8 days of hardware logs from Alibaba, the largest online retailer in China. Our analysis reveals important characteristics of real systems serving retail. First, we identify the statistical distributions that best fit hardware-usage data and find that system behavior in modern retail workloads departs from certain conventional assumptions in the literature, particularly the assumption that resource utilization follows a ‘heavy-tailed’ distribution. Next, we examine how these resource-usage patterns change with different retail seasons and correlate with workload demand and transaction type. Finally, we discuss practical implications for OLTP systems management, specifically in hardware provisioning and capacity planning.