Improving NoSQL's Performance Metrics via Machine Learning

Abstract

Listen

Translate article

As the NoSQLs are becoming more and more popular in the big data era, a few auto-tuning tools have been developed to optimize its application performance. However, the majority of them to date has focused on improving the performance of a single objective-throughput. As such, these tools cannot cover the diverse optimization requirements because that many applications need to make a trade-off between the response time(latency) and the throughput. In this paper, we propose a novel approach, called Multiple Objective Optimization for NoSQL (MOTuner) applications from the angle of parameter tuning. The key is an accurate performance model and a search algorithm along with a searching direction, which determined by the weights of optimization objectives. The model takes configuration parameters as input and outputs performance metrics (the throughput and latency). The Generic Algorithm (GA) takes this model as input, along the specified direction determined by the weights, search the unique optimal configuration in the huge configuration space for a given application. We validate MOTuner in a HBase cluster with 10 nodes by using 5 typical applications from YCSB. The experimental results show that MOTuner can improve throughput by 48% on average and up to 89% compared to the default configurations with different weight pairs of optimization objectives. At the same time, the latency of HBase operations is reduced by 13.1% on average and up to 52.4%. MOTuner is a powerful tool for operators to make a trade-off between the throughput and latency.