Join cardinality estimation by combining operator-level deep neural networks

Abstract

Join cardinality estimation is fundamental to cost-based query optimizers. The state-of-the-art deep learning based join cardinality estimation methods do not fully take the structure of a query plan into account and lack of data information about joins, causing significant performance degradation when the number of joins in a query increases. In this paper, we propose CAPE, a join cardinality estimation method combining operator-level deep neural networks. CAPE introduces two operator-level deep neural networks for selection operators and join operators, as well as an output deep neural network that maps the intermediate representations to join cardinality estimates. Given the query plan rooted at a join operator, CAPE generates a plan-level deep neural network by combining operator-level deep neural networks. In this way, CAPE can handle arbitrary query plans with simple operator-level deep neural networks rather than a single complicated model. In addition, we introduce join-crossing sampling information to detect join-crossing correlations. Experiments are conducted on a dataset constructed from the IMDb dataset, and the experimental results show that CAPE is significantly better than the state-of-the-art methods.