Question Answering Systems With Deep Learning-Based Symbolic Processing

Hiroshi Honda,Masafumi Hagiwara

doi:10.1109/access.2019.2948081

Hiroshi Honda, Masafumi Hagiwara

Open Access

https://doi.org/10.1109/access.2019.2948081

Copy DOI

Abstract

The authors propose methods to learn symbolic processing with deep learning and to build question answering systems by means of learned models. Symbolic processing, performed by the Prolog processing systems which execute unification, resolution, and list operations, is learned by a combination of deep learning models, Neural Machine Translation (NMT) and Word2Vec training. To our knowledge, the implementation of a Prolog-like processing system using deep learning is a new experiment that has not been conducted in the past. The results of their experiments revealed that the proposed methods are superior to the conventional methods because symbolic processing (1) has rich representations, (2) can interpret inputs even if they include unknown symbols, and (3) can be learned with a small amount of training data. In particular (2), handling of unknown data, which is a major task in artificial intelligence research, is solved using Word2Vec. Furthermore, question answering systems can be built from knowledge bases written in Prolog with learned symbolic processing, which, with conventional methods, is extremely difficult to accomplish. Their proposed systems can not only answer questions through powerful inferences by utilizing facts that harbor unknown data but also have the potential to build knowledge bases from a large amount of data, including unknown data, on the Web. The proposed systems are a completely new trial, there is no state-of-the-art methods in the sense of “newest”. Therefore, to evaluate their efficiency, they are compared with the most traditional and robust system i.e., the Prolog system. This is new research that encompasses the subjects of conventional artificial intelligence and neural network, and their systems have higher potential to build applications such as FAQ chatbots, decision support systems and energy-efficient estimation using a large amount of information on the Web. Mining hidden information through these applications will provide great value.

Highlights

In artificial intelligence research, a wide range of studies on inferences using symbolic processing exist
DEEP LEARNING-BASED SYMBOLIC PROCESSING We propose methods to learn matching, resolution and membership relations described in section II by combining Neural Machine Translation (NMT) [41]–[43] and
EVALUATION EXPERIMENTS Using knowledge bases described in Prolog, we trained models, built question answering systems, and evaluated their performance

Summary

Introduction

A wide range of studies on inferences using symbolic processing exist. Expert systems needed to strictly build knowledge bases by hand. Studies were conducted to generate rules where the knowledge base was rendered incomplete [2], [3] and to obtain inferences based on hypotheses when there was a shortage of data [4]–[6], which, helped to improve these expert systems. In the 1990s, connectionists studied symbolic processing by using multilayered neural networks [7], [8]. Owing to limitations of the hardware at that time and learning ability of the multilayered neural networks, the connectionists were limited to propose methods but were unable to build practical systems

Objectives

Results

Conclusion