Abstract
Cardinality estimation is a fundamental task in database query processing and optimization. As shown in recent papers, machine learning (ML)-based approaches may deliver more accurate cardinality estimations than traditional approaches. However, a lot of training queries have to be executed during the model training phase to learn a data-dependent ML model making it very time-consuming. Many of those training or example queries use the same base data, have the same query structure, and only differ in their selective predicates. To speed up the model training phase, our core idea is to determine a predicate-independent pre-aggregation of the base data and to execute the example queries over this pre-aggregated data. Based on this idea, we present a specific aggregate-based training phase for ML-based cardinality estimation approaches in this paper. As we are going to show with different workloads in our evaluation, we are able to achieve an average speedup of 90 with our aggregate-based training phase and thus outperform indexes.
Highlights
Due to skew and correlation in data managed by database systems (DBMS), query optimization is still an important challenge
We propose a novel training phase based on pre-aggregated data for machine learning (ML)-based cardinality estimation approaches
We made the case for cardinality estimation as a candidate for database support of machine learning for DBMS
Summary
Due to skew and correlation in data managed by database systems (DBMS), query optimization is still an important challenge. As shown in recent papers [9, 10], including our own work [11], machine learning-based cardinality estimation approaches are able to meet higher accuracy requirements, especially for highly correlated data. To overcome these shortcomings, we propose a novel training phase based on pre-aggregated data for ML-based cardinality estimation approaches. We propose a novel training phase based on pre-aggregated data for ML-based cardinality estimation approaches This is an extended version of previous work [12]. Based on this discussion, we introduce our general solution approach of an aggregated-based training phase by pre-aggregating the base data using the data cube concept and executing the example queries over this preaggregated data. 4. we present experimental evaluation results for four different workloads for the training phase of ML-based cardinality estimation in Sect.
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
