Bilingual Question Answering System Using Bidirectional Encoder Representations from Transformers and Best Matching Method

Abstract

Question answering (QA) system is built to answer asked queries based on an unstructured collection of documents in natural language. The implementation of the QA system makes QA more efficient because the system can answer similar questions automatically. However, similarity queries based on questions or answers alone fail to retrieve documents relevant to the query in some cases because the word choice used in the query is different from the word choice in the QA database even though the context is the same. The same context can be seen from the list of references used by a QA. Therefore, it is necessary to measure the similarity of the query that does not only take into account the question and answer but also the reference. In this paper, we propose to build a bilingual QA system that answers Indonesian questions based on the combination of query similarities among question, answer, and external reference in Arabic using Bidirectional Encoder Representation from Transformers (BERT) and Best Matching (BM25) method. The similarity between query and reference are able to help to recognize a QA that uses reference with similar context. Based on the experimental result, the combination parameter of query-Question followed by query-Answer achieves the highest evaluation score with the Mean Average Precision (MAP) score of 0.988 and the Mean Reciprocal Rank (MRR) score of 1.000.