Question Answering Challenge Research Articles

Introduction to “This is Watson”

In 2007, IBM Research took on the grand challenge of building a computer system that could compete with champions at the game of Jeopardy!™. In 2011, the open-domain question-answering (QA) system, dubbed Watson, beat the two highest ranked players in a nationally televised two-game Jeopardy! match. This paper provides a brief history of the events and ideas that positioned our team to take on the Jeopardy! challenge, build Watson, IBM Watson™, and ultimately triumph. It describes both the nature of the QA challenge represented by Jeopardy! and our overarching technical approach. The main body of this paper provides a narrative of the DeepQA processing pipeline to introduce the articles in this special issue and put them in context of the overall system. Finally, this paper summarizes our main results, describing how the system, as a holistic combination of many diverse algorithmic techniques, performed at champion levels, and it briefly discusses the team's future research plans.

Use of Multiple Documents as Evidence with Decreased Adding in a Japanese Question-answering System

We propose a new method of using multiple documents as evidence with decreased adding to improve the performance of question-answering systems.Sometimes, the answer to a question may be found in multiple documents.In such cases, using multiple documents to predict answers may generate better answers than using a single document.Our method therefore uses information from multiple documents, adding the scores of candidate answers extracted from various documents.However, because simply adding the scores can degrade the performance of question-answering systems, we add the scores with progressively decreasing weights to reduce the negative effect of simple adding.We carried out experiments using the Question-Answering Challenge (QAC) test collection.The results showed that our method produced a statistically significant improvement, with the degree of improvement ranging fro 0.05 to 0.14.These results, and the fact that our method is simple and easy to use, indicate its potential feasibility and utility in question-answering systems.Experiments comparing our decreased adding method with several previously proposed methods that use multiple documents showed that our method was more effective than these other methods.

An evaluation of question answering challenge (QAC-1) at the NTCIR workshop 3

The Question Answering Challenge (QAC) was carried out as the first evaluation task on question answering of the NTCIR Workshop 3 [Fukumoto2002] [NTCIR]. Question answering in an open domain is a task for obtaining appropriate answers to given domain independent questions written in natural language from a large corpus. The purpose of the QAC was to develop practical QA systems in an open domain focusing on research of user interaction and information extraction. A further objective was to develop an evaluation method for the question answering system and information resources for evaluation. To evaluate QA technologies, there are several technical aspects to consider for the extraction of answer expressions from knowledge sources. Question type is one aspect of the QA system evaluation. In QAC-1, question types are defined as a noun or noun phrase which indicates names of persons, organizations, and various artifacts and facts, such as money, size, date and so on. Moreover, information related to these can also be considered as answer candidates: for example, names of persons, their affiliations, age and status can be an answer; and for names of organizations, their annual profit, year of establishment and so on. Another aspect to consider is how many answer expressions exist in the knowledge sources. In QAC-1, there may be multiple answers or no answers to questions in general. This aspect makes development of a QA system difficult because, QA system has to check all answer candidates very carefully. User interaction technology requires actual interaction between the computer and person. In actual QA between people, there are typically several interactions which take place in order to confirm the intention of the questions and so on. In QAC-1, we gave one follow-up question for the first question. It will be necessary to resolve an ellipsis in the follow-up question which is frequently occurs in Japanese sentences.

Discovery of inference rules for question-answering

One of the main challenges in question-answering is the potential mismatch between the expressions in questions and the expressions in texts. While humans appear to use inference rules such as ‘X writes Y’ implies ‘X is the author of Y’ in answering questions, such rules are generally unavailable to question-answering systems due to the inherent difficulty in constructing them. In this paper, we present an unsupervised algorithm for discovering inference rules from text. Our algorithm is based on an extended version of Harris’ Distributional Hypothesis, which states that words that occurred in the same contexts tend to be similar. Instead of using this hypothesis on words, we apply it to paths in the dependency trees of a parsed corpus. Essentially, if two paths tend to link the same set of words, we hypothesize that their meanings are similar. We use examples to show that our system discovers many inference rules easily missed by humans.