Semantic Question Answering Research Articles

Efficient Machine Reading Comprehension for Health Care Applications: Algorithm Development and Validation of a Context Extraction Approach.

Extractive methods for machine reading comprehension (MRC) tasks have achieved comparable or better accuracy than human performance on benchmark data sets. However, such models are not as successful when adapted to complex domains such as health care. One of the main reasons is that the context that the MRC model needs to process when operating in a complex domain can be much larger compared with an average open-domain context. This causes the MRC model to make less accurate and slower predictions. A potential solution to this problem is to reduce the input context of the MRC model by extracting only the necessary parts from the original context. This study aims to develop a method for extracting useful contexts from long articles as an additional component to the question answering task, enabling the MRC model to work more efficiently and accurately. Existing approaches to context extraction in MRC are based on sentence selection strategies, in which the models are trained to find the sentences containing the answer. We found that using only the sentences containing the answer was insufficient for the MRC model to predict correctly. We conducted a series of empirical studies and observed a strong relationship between the usefulness of the context and the confidence score output of the MRC model. Our investigation showed that a precise input context can boost the prediction correctness of the MRC and greatly reduce inference time. We proposed a method to estimate the utility of each sentence in a context in answering the question and then extract a new, shorter context according to these estimations. We generated a data set to train 2 models for estimating sentence utility, based on which we selected more precise contexts that improved the MRC model's performance. We demonstrated our approach on the Question Answering Data Set for COVID-19 and Biomedical Semantic Indexing and Question Answering data sets and showed that the approach benefits the downstream MRC model. First, the method substantially reduced the inference time of the entire question answering system by 6 to 7 times. Second, our approach helped the MRC model predict the answer more correctly compared with using the original context (F1-score increased from 0.724 to 0.744 for the Question Answering Data Set for COVID-19 and from 0.651 to 0.704 for the Biomedical Semantic Indexing and Question Answering). We also found a potential problem where extractive transformer MRC models predict poorly despite being given a more precise context in some cases. The proposed context extraction method allows the MRC model to achieve improved prediction correctness and a significantly reduced MRC inference time. This approach works technically with any MRC model and has potential in tasks involving processing long texts.

Discovery and recognition of formula concepts using machine learning

Citation-based Information Retrieval (IR) methods for scientific documents have proven effective for IR applications, such as Plagiarism Detection or Literature Recommender Systems in academic disciplines that use many references. In science, technology, engineering, and mathematics, researchers often employ mathematical concepts through formula notation to refer to prior knowledge. Our long-term goal is to generalize citation-based IR methods and apply this generalized method to both classical references and mathematical concepts. In this paper, we suggest how mathematical formulas could be cited and define a Formula Concept Retrieval task with two subtasks: Formula Concept Discovery (FCD) and Formula Concept Recognition (FCR). While FCD aims at the definition and exploration of a ‘Formula Concept’ that names bundled equivalent representations of a formula, FCR is designed to match a given formula to a prior assigned unique mathematical concept identifier. We present machine learning-based approaches to address the FCD and FCR tasks. We then evaluate these approaches on a standardized test collection (NTCIR arXiv dataset). Our FCD approach yields a precision of 68% for retrieving equivalent representations of frequent formulas and a recall of 72% for extracting the formula name from the surrounding text. FCD and FCR enable the citation of formulas within mathematical documents and facilitate semantic search and question answering, as well as document similarity assessments for plagiarism detection or recommender systems.

A Question-Answering Model Based on Knowledge Graphs for the General Provisions of Equipment Purchase Orders for Steel Plants Maintenance

Recently, equipment replacement and maintenance repair and operation (MRO) optimization have substantially increased owing to the aging and deterioration of industrial plants, such as steel-making factories in Korea. Therefore, plant owners are required to quickly review equipment supply contracts, i.e., purchase order (PO) documents, with suppliers and vendors. Currently, there is inconsistency in the time and quality required for the PO document review process by engineers, depending on their manual skills and practice. This study developed a general provisions question-answering model (GPQAM) by combining knowledge graph (KG) and question-answering (QA) techniques to search for semantically connected contract clauses through the definition of relationships between entities during the review of equipment purchase contracts. The PO documents analyzed in this case study were based on one steel-making company’s general provisions (GP). GPQAM is a machine learning (ML)-based model with two sub-models (i.e., KG and QA) that automatically generates the most relevant answers to semantic search questions through a cypher query statement in GP for the PO engineers. First, based on the developed GP lexicon and its classifying taxonomy to be stored in the Neo4j graph database (GDB), the KG sub-model finds the corresponding synonyms and consequently shows GP-related information in a graphic form. Second, the QA sub-model is a function to find and answer contract information within the KG and applies pattern-matching technology based on the Aho–Corasick (AC) algorithm. Third, nodes with the meaning most similar to the question are selected using similarity measurement if a response cannot be extracted through the pattern-matching process. Forty-five pilot test questions were created and applied to the GPQAM model evaluation. The F1 score was 82.8%, indicating that the unsupervised training methods developed in this study could be better applied to a semantic QA process in plant engineering documents, where sufficient training data are limited and bargained. An expert survey of PO practitioners confirmed that the semantic QA capability of GPQAM might be efficient and useful for their work. As the first case of applying KG technology to semantic QA for plant equipment PO contracts, this study might be a meaningful contribution to the steel plant industry and, therefore, extended to construction and engineering contract applications.

Developing a Semantic Question Answering System for E-learning Environments using Linguistic Resources

The Question answering (QA) system plays a basic role in the acquisition of information and the e-learning environment is considered to be the field that is most in need of the question-answering system to help learners ask questions in natural language and get answers in short periods of time. The main problem in this context is how to understand the questions without any doubts in meaning and how to provide the most relevant answers to the questions. In this study, a question-answering system for specific courses has been developed to support the learning environment. The research outcomes indicate that the proposed method helps to solve the problem of ambiguities in meaning through the integration of natural language processing tools and semantic resources that can help to overcome several problems related to the natural language structure. This method also helps improve the capability to understand students’ needs and, consequently, to retrieve the most suitable answers.

Discerning appropriate reviews based on hierarchical deep neural network for answering product-related queries

In recent years, E-Commerce is globally increasing among online purchaser, in which customer post product related queries for finding the best product in online shopping. Manually answering the product related queries in real-time, cause online traffic and practically not possible. So, automatic answering system is helpful for answering product related queries. But, the product queries are always in product-explicit, so discovering related product queries and recovering its responds is distinctly be impractical. Accordingly, we propose Hierarchical Deep Neural Network (HiDeNN) model using MOQA framework to discern the appropriate reviews based on Mixtures of Opinions Question Answering (MOQA). The Hierarchical Deep Neural Network provides discerning the most relevant review for queries and it also provides the relevant answer for specific product category queries. The proposed method is executed on Python and it provides 9.594% and 7.574% higher accuracy value for Discerning Appropriate Reviews compared with the existing method like Relevant Reviews for Answering Product-related Queries (MOQA-BERTQA+FLTR+PT) and IQA: Interactive Query Construction on Semantic Question Answering Systems (IQC-SQA). The experimental result indicates that the proposed MOQA- HiDeNN method can efficiently and accurately get the optimal global solutions for recognizing the appropriate discerning of most relevant review for queries and it also provides the relevant answer for specific product category queries.

A Question Answering Framework Based on Hybridization of Deep Learning and Semantic Web Techniques

The question answering (QA) system has been existing for several years. QA systems are divided into different processes such as question processing, document processing, paragraph extraction, answer extraction, question analysis, phrase mapping, disambiguation, query construction, querying the Knowledge Base (KB), and result ordering on user response respectively. Based on these processes, many models have been developed using approaches ranging from linguistic, statistical, and pattern matching. Popular models are Feedback, Refinement and Extended VocabularY Aggregation (FREyA), PowerAqua, SemSek, Semantic Interpretation of User Queries for QA on Interlinked Data (SINA), DEep Answers for maNy Naturally Asked questions (DEANNA), gAnswer, SemGraph, OKBQA (Open Knowledge Base and Question Answering) and Semantic Question Answering (SQA), for performance evaluation, these mostly focus on higher precision, recall, and/or F-measure. However, most of these models are constrained in the following operations: the combination of knowledge bases from different sources, formalism for knowledge representation to support interoperability, optimization of query construction and generation, ranking of responses, and support for set operations (union, sorting, comparison, and aggregation on user query) during query generation. This research proposes a hybrid of recurrent neural network and semantic-web-based question answering as the (RNNSQA) framework that combines heterogeneous knowledge sources and improves on state-of-the-art query generation mechanisms to allow for integration of comprehensive question-type operations, and set-based operations listed above. First, there would be a combination of techniques that is Natural Language Processing (NLP) and Recurrent Neural Network (RNN) techniques in classifying questions into types. Secondly, the semantic web technique is then employed in generating (SPARQL Protocol and RDF (Resource Description Framework) Query Language) SPARQL-based candidate queries. The result of this study is an enhanced QA framework with improved query translation and construction capability.

Literature Review on Trends of Comprehension Instruction for Elementary School Students

Abstract: The objective of this article is to construct evidence-based arguments on what comprehension is and what are various instructional approaches to improve comprehension post the National Reading Panel’s (NRP) report (2000). The NRP (2000) reported seven text comprehension instructions that have scientific evidences in improving reading comprehension. Those instructions are: (1) comprehension monitoring, (2) cooperative learning, (3) graphic and semantic organizers, (4) question answering, (5) question generation, (6) summarization, and (7) multiple strategies. It has been more that 20 years since the NRP reviewed empirical evidences of those text comprehension instructions. What have we learnt since then? Having reviewed 29 studies on comprehension instruction published in the past twenty years, we found top three comprehension instructions that are used in elementary classroom. They are: (1) differentiated/individualized reading instruction, (2) vocabulary-based reading instruction, and (3) transactional approach. These findings informed us that further research on elementary reading comprehension instruction needed. Also, the NRP categorization of comprehension instruction (2000) needs modification.

데이터 증강 기법을 이용한 한글 개체명 인식

Named entity recognition (NER) is the task of natural language processing that recognizes a predefined entity name such as a person, place name, or organization into a token in a sentence. The NER is important because it significantly affects the performance of subsequent analyses including semantic search, question answering, and machine translation. The performance of NER for English has been greatly improved with an advent of deep learning techniques with a large dataset of English. However, only few studies have been conducted for languages spoken by ethnic minorities, such as Korean (i.e., Hangul), because an appropriate dataset for NER is difficult to obtain. In this study, we propose using various data augmentation techniques to improve the performance NER for Hangul datasets. Our methods can be applied without pre-trained models or external pre-building. We demonstrated the usefulness of the presented data augmentation techniques using a Changwon University-Naver Challenge dataset and found that even a small dataset can achieve a satisfactory performance for Hangul NER.

A survey on semantic question answering systems

Abstract Recently, many question answering systems that derive answers from linked data repositories have been developed. The purpose of this survey is to identify the common features and approaches of the semantic question answering (SQA) systems, although many different and prototype systems have been designed. The SQA systems use a formal query language like SPARQL and knowledge of a specific vocabulary. This paper analyses different frameworks, architectures, or systems that perform SQA and classifies SQA systems based on different criteria.

COVID-19 information retrieval with deep-learning based semantic search, question answering, and abstractive summarization

The COVID-19 global pandemic has resulted in international efforts to understand, track, and mitigate the disease, yielding a significant corpus of COVID-19 and SARS-CoV-2-related publications across scientific disciplines. Throughout 2020, over 400,000 coronavirus-related publications have been collected through the COVID-19 Open Research Dataset. Here, we present CO-Search, a semantic, multi-stage, search engine designed to handle complex queries over the COVID-19 literature, potentially aiding overburdened health workers in finding scientific answers and avoiding misinformation during a time of crisis. CO-Search is built from two sequential parts: a hybrid semantic-keyword retriever, which takes an input query and returns a sorted list of the 1000 most relevant documents, and a re-ranker, which further orders them by relevance. The retriever is composed of a deep learning model (Siamese-BERT) that encodes query-level meaning, along with two keyword-based models (BM25, TF-IDF) that emphasize the most important words of a query. The re-ranker assigns a relevance score to each document, computed from the outputs of (1) a question–answering module which gauges how much each document answers the query, and (2) an abstractive summarization module which determines how well a query matches a generated summary of the document. To account for the relatively limited dataset, we develop a text augmentation technique which splits the documents into pairs of paragraphs and the citations contained in them, creating millions of (citation title, paragraph) tuples for training the retriever. We evaluate our system (http://einstein.ai/covid) on the data of the TREC-COVID information retrieval challenge, obtaining strong performance across multiple key information retrieval metrics.

Review and Analysis of Different Approaches to Semantic Level Question Answering and Information Retrieval

Review and Analysis of Different Approaches to Semantic Level Question Answering and Information Retrieval

Richpedia: A Large-Scale, Comprehensive Multi-Modal Knowledge Graph

Large-scale knowledge graphs such as Wikidata and DBpedia have become a powerful asset for semantic search and question answering. However, most of the knowledge graph construction works focus on organizing and discovering textual knowledge in a structured representation, while paying little attention to the proliferation of visual resources on the Web. To consolidate this recent trend, in this paper, we present Richpedia, aiming to provide a comprehensive multi-modal knowledge graph by distributing sufficient and diverse images to textual entities in Wikidata. We also set Resource Description Framework links (visual semantic relations) between image entities based on the hyperlinks and descriptions in Wikipedia. The Richpedia resource is accessible on the Web via a faceted query endpoint, which provides a pathway for knowledge graph and computer vision tasks, such as link prediction and visual relation detection.

A comparative review on deep learning models for text classification

<p>Text classification is a fundamental task in several areas of natural language processing (NLP), including words semantic classification, sentiment analysis, question answering, or dialog management. This paper investigates three basic architectures of deep learning models for the tasks of text classification: Deep Belief Neural (DBN), Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN), these three main types of deep learning architectures, are largely explored to handled various classification tasks. DBN have excellent learning capabilities to extracts highly distinguishable features and good for general purpose. CNN have supposed to be better at extracting the position of various related features while RNN is modeling in sequential of long-term dependencies. This paper work shows the systematic comparison of DBN, CNN, and RNN on text classification tasks. Finally, we show the results of deep models by research experiment. The aim of this paper to provides basic guidance about the deep learning models that which models are best for the task of text classification.</p>

IQA: Interactive query construction in semantic question answering systems

Semantic Question Answering (SQA) systems automatically interpret user questions expressed in a natural language in terms of semantic queries. This process involves uncertainty, such that the resulting queries do not always accurately match the user intent, especially for more complex and less common questions. In this article, we aim to empower users in guiding SQA systems towards the intended semantic queries through interaction. We introduce IQA — an interaction scheme for SQA pipelines. This scheme facilitates seamless integration of user feedback in the question answering process and relies on Option Gain — a novel metric that enables efficient and intuitive user interaction. Our evaluation shows that using the proposed scheme, even a small number of user interactions can lead to significant improvements in the performance of SQA systems.

Math-word embedding in math search and semantic extraction

Word embedding, which represents individual words with semantically fixed-length vectors, has made it possible to successfully apply deep learning to natural language processing tasks such as semantic role-modeling, question answering, and machine translation. As math text consists of natural text, as well as math expressions that similarly exhibit linear correlation and contextual characteristics, word embedding techniques can also be applied to math documents. However, while mathematics is a precise and accurate science, it is usually expressed through imprecise and less accurate descriptions, contributing to the relative dearth of machine learning applications for information retrieval in this domain. Generally, mathematical documents communicate their knowledge with an ambiguous, context-dependent, and non-formal language. Given recent advances in word embedding, it is worthwhile to explore their use and effectiveness in math information retrieval tasks, such as math language processing and semantic knowledge extraction. In this paper, we explore math embedding by testing it on several different scenarios, namely, (1) math-term similarity, (2) analogy, (3) numerical concept-modeling based on the centroid of the keywords that characterize a concept, (4) math search using query expansions, and (5) semantic extraction, i.e., extracting descriptive phrases for math expressions. Due to the lack of benchmarks, our investigations were performed using the arXiv collection of STEM documents and carefully selected illustrations on the Digital Library of Mathematical Functions (DLMF: NIST digital library of mathematical functions. Release 1.0.20 of 2018-09-1, 2018). Our results show that math embedding holds much promise for similarity, analogy, and search tasks. However, we also observed the need for more robust math embedding approaches. Moreover, we explore and discuss fundamental issues that we believe thwart the progress in mathematical information retrieval in the direction of machine learning.

A Bilinear Ranking SVM for Knowledge Based Relation Prediction and Classification

As an important and challenging problem, knowledge representation and inference are typically carried out in a knowledge embedding framework over a multi-relational knowledge graph, and thus have a wide range of applications such as semantic retrieval and question answering. In this paper, we propose a bilinear learning framework which performs cross-entity knowledge relation analysis in the continuous vector space (derived from knowledge embedding). In the framework, we effectively model the intrinsic correlations among different types of knowledge relations within a max-margin multi-relational ranking scheme, which jointly optimizes the tasks of entity embedding and cross-entity relation prediction in terms of multi-relational structures of the knowledge graph. Specifically, we devise a bilinear scoring function that aims to evaluate the confidence degree of semantic relation prediction for entity pairs through a multi-relational learning-to-rank pipeline. In essence, the pipeline formulates the problem of relation prediction for entity pairs as that of learning relation-specific ranking functions by max-margin optimization. Experimental results demonstrate the effectiveness of the proposed framework on two common benchmark datasets.

SemBioNLQA: A semantic biomedical question answering system for retrieving exact and ideal answers to natural language questions.

Question answering (QA), the identification of short accurate answers to users questions written in natural language expressions, is a longstanding issue widely studied over the last decades in the open-domain. However, it still remains a real challenge in the biomedical domain as the most of the existing systems support a limited amount of question and answer types as well as still require further efforts in order to improve their performance in terms of precision for the supported questions. Here, we present a semantic biomedical QA system named SemBioNLQA which has the ability to handle the kinds of yes/no, factoid, list, and summary natural language questions. This paper describes the system architecture and an evaluation of the developed end-to-end biomedical QA system named SemBioNLQA, which consists of question classification, document retrieval, passage retrieval and answer extraction modules. It takes natural language questions as input, and outputs both short precise answers and summaries as results. The SemBioNLQA system, dealing with four types of questions, is based on (1) handcrafted lexico-syntactic patterns and a machine learning algorithm for question classification, (2) PubMed search engine and UMLS similarity for document retrieval, (3) the BM25 model, stemmed words and UMLS concepts for passage retrieval, and (4) UMLS metathesaurus, BioPortal synonyms, sentiment analysis and term frequency metric for answer extraction. Compared with the current state-of-the-art biomedical QA systems, SemBioNLQA, a fully automated system, has the potential to deal with a large amount of question and answer types. SemBioNLQA retrieves quickly users' information needs by returning exact answers (e.g., "yes", "no", a biomedical entity name, etc.) and ideal answers (i.e., paragraph-sized summaries of relevant information) for yes/no, factoid and list questions, whereas it provides only the ideal answers for summary questions. Moreover, experimental evaluations performed on biomedical questions and answers provided by the BioASQ challenge especially in 2015, 2016 and 2017 (as part of our participation), show that SemBioNLQA achieves good performances compared with the most current state-of-the-art systems and allows a practical and competitive alternative to help information seekers find exact and ideal answers to their biomedical questions. The SemBioNLQA source code is publicly available at https://github.com/sarrouti/sembionlqa.

Learning soft domain constraints in a factor graph model for template-based information extraction

The ability to accurately extract key information from textual documents is necessary in several downstream applications e.g., automatic knowledge base population from text, semantic information retrieval, question answering, or text summarization. However, information extraction (IE) systems are far from being errorless and in some cases commit errors that seem obvious to a human expert as they violate common sense or domain knowledge.Towards improving the performance of IE systems, we focus on the question of how domain knowledge can be incorporated into IE models to reduce the number of spurious extractions. Starting from the assumption that such domain knowledge cannot be incorporated explicitly and manually by domain experts due to the amount of effort and technical complexities involved, we propose a machine learning approach in which domain constraints are acquired as a byproduct of learning a model that learns to extract key information in a supervised setting. We frame the task as a template-based information extraction problem in which several dependent slots need to be automatically filled and propose a factor graph based approach to model the joint distribution of slot assignments given a text. Beyond using standard textual features in factors that score the compatibility of slot fillers in relation to the text, we use additional features that are text-independent and capture soft domain constraints. During the training process, these constraints receive a weight as part of the parameter learning process indicating how strongly a constraint should be enforced. These domain constraints are thus ‘soft’ in the sense that they can be violated, but the system learns to penalize solutions that violate them. The soft constraints we introduce come in two flavors: on the one hand we incorporate information about the mean of numerical attributes and use features that indicate how far a certain value is from the mean. We call these features single slot soft constraints. On the other hand, we model the pairwise compatibility between slot filler assignments independent of the textual context, thus modeling the (domain) compatibility of the slot assignments. We call the latter ones pairwise slot soft constraints. As main result of our work, we show that learning pairwise slot soft constraints improves the performance of our extraction model compared to single slot soft constraints by up to 6 points in F1, leading to an F1 score of 0.91 for individual template types. Further, the human readable output format of our model enables the extraction and interpretation of the learned soft constraints. Based on this, we show in an evaluation by domain experts that more than 68% of the learned soft constraints are regarded as plausible.

Improving Natural Language Inference Using External Knowledge in the Science Questions Domain

Natural Language Inference (NLI) is fundamental to many Natural Language Processing (NLP) applications including semantic search and question answering. The NLI problem has gained significant attention due to the release of large scale, challenging datasets. Present approaches to the problem largely focus on learning-based methods that use only textual information in order to classify whether a given premise entails, contradicts, or is neutral with respect to a given hypothesis. Surprisingly, the use of methods based on structured knowledge – a central topic in artificial intelligence – has not received much attention vis-a-vis the NLI problem. While there are many open knowledge bases that contain various types of reasoning information, their use for NLI has not been well explored. To address this, we present a combination of techniques that harness external knowledge to improve performance on the NLI problem in the science questions domain. We present the results of applying our techniques on text, graph, and text-and-graph based models; and discuss the implications of using external knowledge to solve the NLI problem. Our model achieves close to state-of-the-art performance for NLI on the SciTail science questions dataset.

New instances classification framework on Quran ontology applied to question answering system

Instances classification with the small dataset for Quran ontology is the current research problem which appears in Quran ontology development. The existing classification approach used machine learning: Backpropagation Neural Network. However, this method has a drawback; if the training set amount is small, then the classifier accuracy could decline. Unfortunately, Holy Quran has a small corpus. Based on this problem, our study aims to formulate new instances classification framework for small training corpus applied to semantic question answering system. As a result, the instances classification framework consists of several essential components: pre-processing, morphology analysis, semantic analysis, feature extraction, instances classification with Radial Basis Function Networks algorithm, and the transformation module. This algorithm is chosen since it robustness to noisy data and has an excellent achievement to handle small dataset. Furthermore, document processing module on question answering system is used to access instances classification result in Quran ontology.