Automatic index selection with learned cost estimator

Abstract

Choosing useful indexes for a relational database is important for efficient query optimization. However, current methods tend to use the cost estimated by the optimizer in the database management system (DBMS) to measure the benefits of the index and, due to inaccurate cost estimates, do not bring to the most optimal solution. In addition, existing reinforcement learning methods treat the creation of different indexes as independent actions, ignoring the relationships among indexes, which may bring to unnecessary training costs. To address these problems, we propose DeepIndex, an automatic index selector with a learning-based cost estimator to improve the quality of index selection. To accurately estimate the benefit of an index, we design a learning-based cost estimator to predict the execution time of queries on certain indexes. In particular, we treat query plans as graphs and develop a model based on graph convolutional network (GCN) to learn features from queries and indexes. After that, we design a reinforcement-learning-based index selection model considering the relationships among indexes and combine our cost estimator to select indexes. Extensive experiments on two benchmark workloads JOB and TPC-DS show that our model performs better than state-of-the-art models with the lowest storage costs, and our lowest relative execution cost outperforms the baselines by 38.59% on JOB and 10.42% on TPC-DS.