Case based Indonesian closed domain question answering system with real world questions

Abstract

Number of people having expertise in a certain domain is less than people who need information in that domain. In this situation, an automatic question answering (QA) system is necessary. Observing available manual QA sites on internet, the real world question that people usually ask have different expected answer type (EAT) compared to a common automatic QA. Addressing a case study of a religion domain which makes it a closed domain QA, we proposed the EAT into 6 types: LAW, DEFINITION, COMPARISON, METHOD, TIME and PERSON. Different with common QA approach, we built the QA system using case based approach which consists of two main components: Question Analyzer and Case Retriever. Related with the case based reasoning (CBR) framework, these two main components act as the Retrieve and Reuse process while the Revise and Retain process is handle by Case Retainer component. The QA system was built using available Indonesian Natural Language Processing (NLP) tools and FreeCBR as the CBR library. The experiments were done to calculate the accuracy and testing the system with unknown case. By using 77 cases collected from internet with assumption that all answers are available, the experiments achieved 97% accuracy. And by using 10 test cases for the unknown case, the similarity score calculated by the system showed that the test questions have no answer in the available case base.