Abstract
In the era of Big Knowledge Graphs, Question Answering (QA) systems have reached a milestone in their performance and feasibility. However, their applicability, particularly in specific domains such as the biomedical domain, has not gained wide acceptance due to their “black box” nature, which hinders transparency, fairness, and accountability of QA systems. Therefore, users are unable to understand how and why particular questions have been answered, whereas some others fail. To address this challenge, in this paper, we develop an automatic approach for generating explanations during various stages of a pipeline-based QA system. Our approach is a supervised and automatic approach which considers three classes (i.e., success, no answer, and wrong answer) for annotating the output of involved QA components. Upon our prediction, a template explanation is chosen and integrated into the output of the corresponding component. To measure the effectiveness of the approach, we conducted a user survey as to how non-expert users perceive our generated explanations. The results of our study show a significant increase in the four dimensions of the human factor from the Human-computer interaction community.
Highlights
The recent advances of Question Answering (QA) technologies mostly rely on (i) the advantages of Big Knowledge Graphs which augment the semantics, structure, and accessibility of data, e.g., Web of Data has published around 150B triples from a variety of domains1, and (ii) the competency of contemporary AI approaches which train sophisticated learning models (statistical models (Shekarpour et al, 2015, 2013), neural networks (Lukovnikov et al, 2017), and attention models (Liu, 2019)) on a large size of training data, and given a variety of1http://lodstats.aksw.org/novel features captured from semantics, structure, and context of the background data
We focus on the challenge of explainable QA systems
Our primary aim is to take the initial steps to break down the full black-box QA systems
Summary
The recent advances of Question Answering (QA) technologies mostly rely on (i) the advantages of Big Knowledge Graphs which augment the semantics, structure, and accessibility of data, e.g., Web of Data has published around 150B triples from a variety of domains, and (ii) the competency of contemporary AI approaches which train sophisticated learning models (statistical models (Shekarpour et al, 2015, 2013), neural networks (Lukovnikov et al, 2017), and attention models (Liu, 2019)) on a large size of training data, and given a variety of1http://lodstats.aksw.org/novel features captured from semantics, structure, and context of the background data. Similar to other branches of AI applications, the state of the art of QA systems are “black boxes” that fail to provide transparent explanations about why a particular answer is generated. This black box behavior diminishes the confidence and trust of the user and hinders the reliance and acceptance of the black-box systems, especially in critical domains such as healthcare, biomedical, life-science, and self-driving cars (Samek et al, 2017; Miller, 2018). In other words, when the background data is flawed or outdated, it undermines the humanlikeness and acceptance of the QA systems if no explanation is provided, especially for non-expert users.
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
