Qualitative measures for ad hoc table retrieval

Abstract

The focus of our work is the ad hoc table retrieval task, which aims to rank a list of structured tabular objects in response to a user query. Given the importance of this task, various methods have already been proposed in the literature that focus on syntactic, semantic and neural representations of tables for determining table relevance. However, recent works have highlighted queries that are consistently difficult for baseline methods to satisfy, referred to as hard queries. For this reason, the objectives of this paper include: (1) effectively satisfying hard queries by proposing three classes of qualitative measures, namely coherence, interpretability and exactness, (2) offering a systematic approach to interpolate these three classes of measures with each other and with baseline table retrieval methods, and (3) performing extensive experiments using a range of baseline retrieval methods to show the feasibility of the proposed measures for hard queries. We demonstrate that the consideration of the proposed qualitative measures will lead to improved performance for hard queries on a range of state-of-the-art ad hoc table retrieval baselines. We further show that our proposed measures are synergistic and will lead to even higher performance improvements over the baselines when interpolated with each other. The improvements measure up to 22.94% on the Semantic Table Retrieval (STR) method with an NDCG@20 of 0.5, which is superior to the performance of any state-of-the-art baseline for hard queries in the ad hoc table retrieval task.