Abstract
AbstractAddressing Answer Selection (AS) tasks with complex neural networks typically requires a large amount of annotated data to increase the accuracy of the models. In this work, we are interested in simple models that can potentially give good performance on datasets with no or few annotations. First, we propose new unsupervised baselines that leverage distributed word and sentence representations. Second, we compare the ability of our neural architectures to learn from few annotated examples in a weakly supervised scheme and we demonstrate how these methods can benefit from a pre-training on an external dataset. With an emphasis on results reproducibility, we show that our simple methods can reach or approach state-of-the-art performances on four common AS datasets.
Highlights
Large-scale Question Answering (QA) tasks have recently received a substantial amount of attention
We present the unsupervised methods as well as the neural models trained on an external dataset and deployed on the target dataset without any further fine-tuning
The distantly supervised approach provides state-ofthe-art results on the WikiQA dataset, which is a remarkable result, given that no annotation were used on the target dataset
Summary
Large-scale Question Answering (QA) tasks have recently received a substantial amount of attention. We present a domain adaptation approach, where we pre-train our models on a large external QA dataset and fine-tune the models on the target dataset, similar to the work by Min et al [9]. We assess the behaviour of our model when varying the amount of training data and observe the ability of some architecture to achieve low-shot learning. We show that our methods can reach approach state-of-the-art performance when all the training data is used. We make a major effort to have fully reproducible results from the initial dataset, as it is established that the deep learning community is affected by a reproducibility crisis due to poorly controlled environment settings [14]. We demonstrate on four publicly available datasets that our methods reach near state-of-the-art results. The source code to reproduce our experiments is available 1
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
