Arabic Question Answering System Research Articles

Challenges and opportunities for Arabic question-answering systems: current techniques and future directions

Artificial intelligence-based question-answering (QA) systems can expedite the performance of various tasks. These systems either read passages and answer questions given in natural languages or if a question is given, they extract the most accurate answer from documents retrieved from the internet. Arabic is spoken by Arabs and Muslims and is located in the middle of the Arab world, which encompasses the Middle East and North Africa. It is difficult to use natural language processing techniques to process modern Arabic owing to the language’s complex morphology, orthographic ambiguity, regional variations in spoken Arabic, and limited linguistic and technological resources. Only a few Arabic QA experiments and systems have been designed on small datasets, some of which are yet to be made available. Although several reviews of Arabic QA studies have been conducted, the number of studies covered has been limited and recent trends have not been included. To the best of our knowledge, only two systematic reviews focused on Arabic QA have been published to date. One covered only 26 primary studies without considering recent techniques, while the other covered only nine studies conducted for Holy Qur’an QA systems. Here, the included studies were analyzed in terms of the datasets used, domains covered, types of Arabic questions asked, information retrieved, the mechanism used to extract answers, and the techniques used. Based on the results of the analysis, several limitations, concerns, and recommendations for future research were identified. Additionally, a novel taxonomy was developed to categorize the techniques used based on the domains and approaches of the QA system.

DAQAS: Deep Arabic Question Answering System based on duplicate question detection and machine reading comprehension

As of late, various deep learning techniques and methods have shown their superiority to feature-based and shallow learning techniques in the field of open-domain question–answering systems (OpenQAS). However, only a few works adopted these techniques to build Arabic OpenQAS that can extract exact answers from large information sources (e.g., Wikipedia). In addition, no available Arabic OpenQAS integrated a module to identify duplicate questions to accelerate response time and reduce computation cost. In this paper, we propose an Arabic OpenQAS (named DAQAS) based on deep learning methods. It consists of three components: (1) Dense Duplicate Question Detection which returns answers to questions that already have been answered; (2) Retriever based on BM25 and Query Expansion by neural text generation; and (3) Reader able to extract exact answers given a question and the retrieved passages that probably contains the answer. All components of our system integrate deep learning models, specially transformers-based techniques, which have scored state-of-the-art in different NLP fields. We performed several experiments with publicly available question answering datasets to show the effectiveness of our system. DAQAS obtained promising results and scored 21.77% Exact Match and 54.71% F1 score when using only top 5 retrieved passages.

So2al-wa-Gwab: A new Arabic Question-Answering Dataset Trained on Answer Extraction Models

Question answering (QA) is the task of responding to questions posed by users automatically. A question-answering system is divided into three main components: question analysis, information retrieval, and answer extraction; where this paper has focused only on the answer extraction part. In the past couple of years, many QA systems have been developed and become mature and ready for use in different languages. Nevertheless, the advancement of Arabic QA systems still faces different obstacles and a lack of relevant resources and tools for researchers. This paper presents the So2al-wa-Gwab dataset since the publicly available datasets include various faults, such as the use of machine translation to build the data, a short context size, and a small number of question-answer pairings. Thus, this new dataset avoids the aforementioned drawbacks. Furthermore, in this paper, we have trained three deep learning models, namely, Bi-Directional flow network (BiDAF), QA Network (QANet), and BERT model; and tested them on seven different datasets, thus providing a comprehensive comparison between existing Arabic QA datasets. The obtained results emphasize that machine-translated datasets fall back when compared with human-annotated data. Also, the QA task becomes harder as the context, from which to extract the answer, becomes larger.

A comprehensive survey of techniques for developing an Arabic question answering system.

The question-answering system (QAS) aims to produce a response to a query using information from a text corpus. Arabic is a complex language. However, it has more than 450 million native speakers across the globe. The Saudi Arabian government encourages organizations to automate their routine activities to provide adequate services to their stakeholders. The performance of current Arabic QASs is limited to the specific domain. An effective QAS retrieves relevant responses from structured and unstructured data based on the user query. Many QAS studies categorized QASs according to factors, including user queries, dataset characteristics, and the nature of the responses. A more comprehensive examination of QASs is required to improve the QAS development according to the present QAS requirements. The current literature presents the features and classifications of the Arabic QAS. There is a lack of studies to report the techniques of Arabic QAS development. Thus, this study suggests a systematic literature review of strategies for developing Arabic QAS. A total of 617 articles were collected, and 40 papers were included in the proposed review. The outcome reveals the importance of the dataset and the deep learning techniques used to improve the performance of the QAS. The existing systems depend on supervised learning methods that lower QAS performance. In addition, the recent development of machine learning techniques encourages researchers to develop unsupervised QAS.

Arabic question answering system: a survey

Arabic question answering system: a survey

Arabic Question Answering Systems: Gap Analysis

Question-answering (QA) systems aim to provide answers for given questions. The answers can be extracted or generated from either unstructured or structured text. Therefore, QA is considered an important field that can be used to evaluate machine text understanding. Arabic is a challenging language for many reasons; although it is spoken by more than 330 million native speakers, research on this language is limited. A few QA systems created for Arabic text are available. They were created to experiment on small datasets, some of which are unavailable. The research on QA systems can be expanded into different components of QA systems, such as question analysis, information retrieval, and answer extraction. The objective of this research is to analyze the QA systems created for Arabic text by reviewing, categorizing, and analyzing the gaps by providing advice to those who would like to work in this field. Six benchmark datasets are available for testing and evaluating Arabic QA systems, and 26 selected Arabic QA systems are analyzed and discussed in this research.

Arabic factoid Question-Answering system for Islamic sciences using normalized corpora

Factoid Question-Answering (QA) Systems were developed to provide an accurate answer to a factoid question expressed in a natural language. The prime knowledge resources for most of factoid QA systems are online databases. However, the unstructured information in these resources rises the complexity of the information retrieval task. In this paper, we aim to develop an Arabic QA system for factoid questions specialized in Islamic sciences as prophetic tradition (Hadith), Hadith narrator and Quran interpretation (Tafsir). In fact, many questions in the Islamic research fields are focusing on Tafsir and Hadith text. In addition, a number of those interrogations focus on the chain of narrators who transmitted the Hadith. Besides, analyzing the narrator profile is considered an important and enquired task in hadith science. Furthermore, the proposed QA system is based on a normalized database specified in Text Encoding Initiative (TEI) standard. To achieve this, we propose a method composed of three phases to retrieve an accurate answer for the user question. The first phase is the question analysis. The second is the information search. The third phase is the answer processing. Besides, we implant the proposed QA system with a graphic interface allowing the interaction with the user. Finally, we experiment our prototype on 100 questions in the theme of Hadith, narrator and Tafsir text. The QA system succeeds to generate accurate responses for 92% of the entered questions.

A novel semantic and logical-based approach integrating RTE technique in the Arabic question–answering

In this paper, we propose a new approach for the Arabic question-answering system. This approach is based on the automatic understanding of Arabic texts to convert them into semantic and logical representations. Our approach is designed to determine the relation of the textual entailment between the logical representations of the question and the text passage in order to select the text passage that contains an answer to the question and find the precise answer. For a logical representation, we referred to a semantic representation. The idea is to convert the Arabic texts into conceptual graphs that allow the modelling of textual information by concepts and relations. From these graphs, we proposed an algorithm that transforms each conceptual graph to a logical representation. Finally, we extracted three features and combined them to determine the textual entailment relation between two logical representations of the question and its passage answer. Our approach has been validated through a question-answering system, called NArQAS. For the evaluation, we used questions and text passages from our corpus of questions-texts, AQA-WebCorp. The performance of our approach has reached an accuracy of 74%.

Pattern-Matching Based for Arabic Question Answering: A Challenge Perspective

In the 21st century, the Arabic language is amongst the most spoken language of all time, having about 300 million speakers in the globe. Thus, Arabic question answering systems are becoming highly needful for the intellectual benefit internet users. Contrary to the need of Arabic Question Answering, there are only a few reports concerning it. In view of the aforementioned hence the need for more information in this regard. Question classification covers tasks, which identifies a response in the file. It is the basic and important module of a question-answering task to assign one or several classes to a given question. Errors in question classification will result in failure to answer in essence the needed question. In this survey, we focus on the problem of classifying users’ questions and methods to enhance meanings to questions in order to get correct answers that are commensurate. Categorizing users’ questions are daunting due to the Natural Language flexibility issues, in which the questions can be written in divers’ forms and the little information available is not sufficient to base questions. Little research has been focused on the classification for Arabic question answering. Previous Arabic research has used hand-crafted rules and keywords matching that cannot be adopted in a new domain and is not suitable for a new language. Major challenges of the Arabic language are explained in this paper. This study highlighted the Arabic Question Answering Systems as answer identification of the Arabic language domain in the Hadith context.

Toward an Arabic Question Answering System over Linked Data

Toward an Arabic Question Answering System over Linked Data

A logical representation of Arabic questions toward automatic passage extraction from the Web

With the expanding growth of Arabic electronic data on the web, extracting information, which is actually one of the major challenges of the question-answering, is essentially used for building corpus of documents. In fact, building a corpus is a research topic that is currently referred to among some other major themes of conferences, in Natural Language Processing (NLP), such as, Information Retrieval (IR), Question-Answering (QA), Automatic Summary (AS), etc. Generally, a question-answering system provides various passages to answer the user questions. To make these passages truly informative, this system needs access to an underlying knowledge base; this requires the construction of a corpus. The aim of our research is to build an Arabic question-answering system. In addition, analyzing the question must be the first step. Next, it is essential to retrieve a passage from the web that can serve as an appropriate answer. In this paper, we propose a method to analysis the question and retrieve the passage answer in the Arabic language. For the question analysis, five factual question types are processed. Additionally, our purpose is to experiment with the generation of a logic representation from the declarative form of each question. Several studies, deal with the logic approaches in question-answering, are discussed in other languages than the Arabic language. This representation is very promising because it helps us later in the selection of a justifiable answer. The accuracy of questions that are correctly analyzed and translated into the logic form achieved 64%. And then, the results of passages of texts that are automatically generated achieved an 87% score for accuracy and a 98% score for c@1.

A Review and Future Perspectives of Arabic Question Answering Systems

Question Answering Systems (QASs) have emerged as a good alternative for information seekers to retrieve precise information over the Internet. A good amount of research has been done to improve the performance of QASs across several languages, including European and Asian languages. However, Arabic, a morphologically rich Semitic language spoken by over 422 million people, has not seen similar development in the field of question answering. This article reviews the developments taking place in Arabic QASs as well as the challenges faced by researchers in developing Arabic QASs. After conducting an extensive literature survey of a number of English and Arabic QASs, this article classifies them according to several criteria. The most commonly used architecture for the development of an Arabic QAS, known as pipeline architecture, has been presented. In order to encourage and support the new researchers and scholars in conducting research in Arabic QASs, a list of techniques, tools, and computational linguistic resources, required to implement the components of the presented pipelined architecture, are described in this article in a simple and persuasive manner. Finally, the gap analysis between the research in Arabic and English QASs has been performed and accordingly, some future directions for research in Arabic QASs have been proposed.

Answering Arabic Why-Questions

A Question Answering (QA) system is concerned with building a system that automatically answer questions posed by humans in a natural language. Compared to other languages, little effort was directed towards QA systems for Arabic. Due to the difficulty of handling why -questions, most Arabic QA systems tend to ignore it. In this article, we specifically address the why -question for Arabic using two different approaches and compare their performance and the quality of their answer. The first is the baseline approach, a generic method that is used to answer all types of questions, including factoid; and for the second approach, we use Rhetorical Structure Theory (RST). We evaluate both schemes using a corpus of 700 textual documents in different genres collected from Open Source Arabic Corpora (OSAC), and a set of 100 question-answer pairs. Overall, the performance measures of recall, precision, and c@1 was 68% (all three measures) for the baseline approach, and 71%, 78%, and 77.4%, respectively, for the RST-based approach. The recently introduced extension of the accuracy, the c@1 measure, rewards unanswered questions over those wrongly answered.

Arabic Question Answering: Systems, Resources, Tools, and Future Trends

Arabic is the 6th most wide-spread natural language in the world with more than 350 million native speakers. Arabic question answering systems are gaining great importance due to the increasing amounts of Arabic content on the Internet and the increasing demand for information that regular information retrieval techniques cannot satisfy. In spite of the importance of Arabic question answering, there is no review that covers Arabic question answering systems, tools, resources, and test-sets so far, which was the motivation for this work. In this survey, different Arabic question answering systems are demonstrated and analyzed and the main question answering tasks like question analysis, passage retrieval, and answer extraction are explored. The main difficulties of modern standard Arabic and how these difficulties are tamed and classified are also explained. Arabic question answering evaluation metrics, test-sets, and language resources are reviewed, and future trends are also highlighted to guide new research in this area. This survey provides guidance for new research in Arabic question answering to get up-to-date knowledge about the state-of-the-art approaches in this area. It also demonstrates the tools created and used by researchers to build an Arabic question answering system.

Construction of an ontology for intelligent Arabic QA systems leveraging the Conceptual Graphs representation

The last decade had known a great interest in Arabic Natural Language Processing (NLP) applications. This interest is due to the prominent importance of this 6th most wide-spread language in the world with more than 350 million native speakers. Currently, some basic Arabic language challenges related to the high inflection and derivation, Part-of-Speech (PoS) tagging, and diacritical ambiguity of Arabic text are practically tamed to a great extent. However, the development of high level and intelligent applications such as Question Answering (QA) systems is still obstructed by the lacks in terms of ontologies and other semantic resources. In this paper, we present the construction of a new Arabic ontology leveraging the contents of Arabic WordNet (AWN) and Arabic VerbNet (AVN). This new resource presents the advantage to combine the high lexical coverage and semantic relations between words existing in AWN together with the formal representation of syntactic and semantic frames corresponding to verbs in AVN. The Conceptual Graphs representation was adopted in the framework of a multi-layer platform dedicated to the development of intelligent and multi-agents systems. The built ontology is used to represent key concepts in questions and documents for further semantic comparison. Experiments conducted in the context of the QA task show a promising coverage with respect to the processed questions and passages. The obtained results also highlight an improvement in the performance of Arabic QA regarding the [email protected] measure.

Development of Yes/No Arabic Question Answering System

Developing Question Answering systems has been one of the important research issues because it requires insights from a variety of disciplines, including, Artificial Intelligence, Information Retrieval, Information Extraction, Natural Language Processing, and Psychology. In this paper we realize a formal model for a lightweight semantic–based open domain yes/no Arabic question answering system based on paragraph retrieval (with variable length). We propose a constrained semantic representation. Using an explicit unification framework based on semantic similarities and query expansion (synonyms and antonyms). This frequently improves the precision of the system. Employing the passage retrieval system achieves a better precision by retrieving more paragraphs that contain relevant answers to the question; It significantly reduces the amount of text to be processed by the system.

Towards an Arabic Question Answering system on the web

Towards an Arabic Question Answering system on the web

A knowledge based Arabic question answering system (AQAS)

This system accepts an arabic sentence (declarative statements or questions to be answered) and generates the appropriate output to the user. The proposed system could be considered an Arabic implementation to the general natural language processing system. It uses frames technique to represent the knowledge base of a radiation domain.