Promoting convergence and efficacy of open‐domain question answering via unsupervised clustering

Abstract

Listen

Translate article

AbstractOpen‐Domain Question Answering (ODQA) has attracted increasing interests due to its extensive applications in search engines and smart robots. In the experiments, it is observed that the convergence of the method has a huge effect on the generalizability performance. Motivated by this observation, an unsupervised clustering technique (namely, ClusSampling) is proposed to promote both the convergence and efficacy of existing ODQA methods via unsupervised clustering. Specifically, unsupervised clustering is first conducted and then negative samples are selected for higher similarity to the questions. In addition, the authors propose to use gap statistics to determine the optimal number of clusters. Experimental results show that the method achieves notable speedup during training and produces accuracy gains of 5.3% and 2.2 on two widely used benchmarks.